Identifying elements having different states in automated e2e tests

I was recently writing an automated end-to-end (e2e) test that was expanding a section then taking an action within the expanded content.

A very simplified example is the details html tag that expands to show content:

When do you start on red and stop on green?

When you’re eating a watermelon!

I initially wrote the test so that it clicked the details element each time the test ran, something like:

async expandJoke() {
	return await this.driver.findElement( by.css( '#joke' ) ).click();
}

The problem with this straightforward approach is that if the element is already open the click will still perform and the test will continue and then subsequently fail when the test tries to access the content within the expanded section which is now collapsed 😾

I wanted to make sure the test was as resilient and consistent as possible, so instead of just assuming the section was already collapsed I then wrote a function like this to expand the element if it wasn’t expanded and then continue the test:

async expandJokeIfNecessary() {
	const open = await this.driver.findElement( by.css( '#joke' ) ).getAttribute( 'open' );
	if (!open) {
		this.expandJoke()
	}
}

The benefits of this are the test is the most resilient, since it caters for whether the UI is open or closed by checking the open attribute and acting accordingly. I realised that the problem with this approach was that our user experience expects this section to be closed on page load (the punchline is hidden on page load in our example) so if we introduced a bug where we immediately displayed the punchline our test would completely miss it since it just skips expanding the section!

Both these approaches use the same selector to refer to a web element which can have two entirely different states: open and not open.

Keeping this in mind the best solution I could come up with was a selector that combines the element and the state, so the test will fail to click the element to expand the section if it can’t be found, including if the element is already expanded. This gives us simple code that is deterministic and fails at the appropriate time:

async assertAndExpandJoke() {
	return await this.driver.findElement( by.css( '#joke:not([open])' ) ).click();
}

What’s your preferred approach here? How do you identify elements in different states?

All demo code is available here: #

Adventures of end-to-end automated web testing in Node.js

This is a presentation I gave at ATTAC in Melbourne on Friday 24th May 2019. The full Google Slides are available here.

Good morning everyone and thanks for having me along to speak.
My name is Alister Scott and I’m from Brisbane. I work as a QA generalist at a software company in Brisbane called Console, and I write a blog called WatirMelon. Today I’m going to be sharing my knowledge of end to end automated testing in Node.js. I’ve created very simple but working demos of the main tools I’ll be discussing today and I’ve put these on GitHub as separate repositories you can easily clone and play around with: github.com/alisterscott

Outside of work I enjoy hiking, often to the summits of mountains, and spending time with my wife and our three young kids.
In 2015 I started a paid trial at Automattic. My trial project – on which I would be assessed to gain full time employment – was to establish an automated e2e testing framework for WordPress.com – the first of its kind for Automattic. I quickly spun up a framework using Watir in Ruby – because that’s what I knew – and it worked. However I quickly gained some feedback that of the hundred+ developers at Automattic almost none knew Ruby and creating shared ownership for e2e tests would be a key measure of success so I had to rethink. At the time Automattic was moving from primarily developing in PHP towards Node.js meaning full stack JavaScript and it made sense that the e2e tests were also developed in Node.js.

This of course made my trial project a lot more difficult than I had originally thought as I had to teach myself Node.js and understand what testing tools existed in this space.

Fast forward to 2019 and earlier this year I decided to change jobs as I could no longer travel for work. When I started looking at job advertisements for testing and QE positions in Brisbane I noticed just how many mentioned Node.js as their technology stack of choice: in the 3.5 years at Automattic Node.js had become increasingly popular at other companies as well.
But automated e2e testing in Node.js was and is really hard. Much much harder than I was used to in Ruby and Watir where things just worked.

It’s slightly better in 2019 than 2015, however there are some reasons why it’s hard.
There’s also the paradox of choice when it comes to tooling specifically for e2e web testing. Searching for selenium and WebDriver on NPM provides a list of dozens of libraries – and the official Selenium bindings (WebDriverJs) don’t appear when searching for WebDriver. It’s all very confusing.

My aim today is to create some clarity in this space.
I believe there’s more e2e test tools in Node.js that something like Java or C#. I’ll talk about the four most popular/mature ones.
Cypress.io is better suited to component testing than true end-to-end testing. Cypress.io also doesn’t support cross-domain stuff – so beware if you’re doing anything like that.
You’ll need your own test runner and assertion library.
I’ve distilled these four tools down into an easy to read visual.
And an even easier to understand flow chart.

AMA: Protractor for e2e Testing?

John X asks…

 I have dodged the AngularJS /Protractor bullet for many years now. May last foray, some 5 years ago, was a cluster to put it mildly! Cucumberjs/Angularjs/Protractor/chai/mocha/ the stack was in its infancy and failed miserably!! Many cycles were spent pulling fixes from our own repos instead of waiting for PRs to get done. This was in stark contrast to the ease and stability of the automation I wrote using Watir-Webdriver and eventually Watir.

I am now faced with automating the regression test cases for an Angularjs App.

Question: Do I finally jump back into the using stack that almost caused me to lose my mind, or is it possible to use Watir/Selenium to build out meaningful e2e UI automation for an angularjs app as we dawn on 2019?

My response…

It’s still my opinion in 2018 that writing e2e tests in Node using either Protractor or WebDriverJs is still more difficult than using Watir in Ruby.

Sure using async functions with await commands makes things easier than before (see examples in our project), when you would have first come across Protractor, but I still feel like there’s a lot of catching up to do to get to the stability and ease-of-use of Watir.

My decision would come down to whether others on your project are going to be comfortable maintaining tests in ruby – if they are I’d use Ruby and Watir, otherwise I’d revisit Protractor if they really want to use Node.

Using async/await with WebDriverJs

We’ve been using WebDriverJs for a number of years and the control flow promise manager that it offers to make writing WebDriverJs commands in a synchronous blocking way a bit easier, particularly when using promises.

The problem with the promise manager is that it is hard to understand its magic as sometimes it just works, and other times it was very confusing and not very predictable. It was also harder to develop and support by the Selenium project so it’s being deprecated later this year.

Fortunately recent versions of Node.js support asynchronous functions and use of the await command which makes writing WebDriverJs tests so much easier and understandable.

I’ve recently updated my WebDriverJs demo project to use async/await so I’ll use that project as examples to explain what is involved.

WebDriverJs would allow you to write consecutive statements like this without worrying about waiting for each statement to finish – note the use of test.it instead of the usual mocha it function:

test.it( 'can wait for an element to appear', function() {
	const page = new WebDriverJsDemoPage( driver, true );
	page.waitForChildElementToAppear();
	page.childElementPresent().then( ( present ) => {
		assert( present, 'The child element is not present' );
	} );
} );

When you were waiting on the return value from a promise you could use a .then function to wait for the value as shown above.

This is quite a simple example and this could get complicated pretty quickly.

Since the promise manager is being removed, we need to update our tests so they continue to execute in the correct order. We can make the test function asynchronous by adding the async prefix, remove the test. prefix on the it block, and add await statements every time we expect a statement to finish before continuing:

it( 'can wait for an element to appear', async function() {
	const page = new WebDriverJsDemoPage( driver, true );
	await page.waitForChildElementToAppear();
	assert( await page.childElementPresent(), 'The child element is not present' );
} );

I personally find this much easier to read and understand, less ‘magic’, but the one bit that stands out is visiting the page and creating the new page object. The code in the constructor for this page, and other pages, is asynchronous as well, however we can’t have an async constructor!

export default class BasePage {
	constructor( driver, expectedElementSelector, visit = false, url = null ) {
		this.explicitWaitMS = config.get( 'explicitWaitMS' );
		this.driver = driver;
		this.expectedElementSelector = expectedElementSelector;
		this.url = url;

		if ( visit ) this.driver.get( this.url );

		this.driver.wait( until.elementLocated( this.expectedElementSelector ), this.explicitWaitMS );
	}
}

How we can get around this is to define a static async function that acts as a constructor and returns our new page object for us.

So, our BasePage now looks like:

export default class BasePage {
	constructor( driver, expectedElementSelector, url = null ) {
		this.explicitWaitMS = config.get( 'explicitWaitMS' );
		this.driver = driver;
		this.expectedElementSelector = expectedElementSelector;
		this.url = url;
	}

	static async Expect( driver ) {
		const page = new this( driver );
		await page.driver.wait( until.elementLocated( page.expectedElementSelector ), page.explicitWaitMS );
		return page;
	}

	static async Visit( driver, url ) {
		const page = new this( driver, url );
		if ( ! page.url ) {
			throw new Error( `URL is required to visit the ${ page.name }` );
		}
		await page.driver.get( page.url );
		await page.driver.wait( until.elementLocated( page.expectedElementSelector ), page.explicitWaitMS );
		return page;
	}
}

In our Expect and Visit functions we call new this( driver ) which creates an instance of the child class which suits our purposes. So, this means our spec now looks like:

it( 'can wait for an element to appear', async function() {
	const page = await WebDriverJsDemoPage.Visit( driver );
	await page.waitForChildElementToAppear();
	assert( await page.childElementPresent(), 'The child element is not present' );
} );

which means we can await visiting and creating our page objects and we don’t have any asynchronous code in our constructors for our pages. Nice.

Once we’re ready to not use the promise manager we can set SELENIUM_PROMISE_MANAGER to 0 and it won’t use it any more.

Summary

The promise manager is being removed in WebDriverJs but using await in async functions is a much nicer solution anyway, so now is the time to make the move, what are you awaiting for? 😊

Check out the full demo code at https://github.com/alisterscott/webdriver-js-demo

→ How Canaries Help Us Merge Good Pull Requests

I recently published an article on the WordPress.com Developer’s Blog about how we run automated canary tests on pull requests to give us confidence to release frequent changes without breaking things. Feel free to check it out.

AMA: Difference between explicit and fluent wait

Anonymous asks…

What is the difference between Explicit wait and Fluent wait?

My response…

I hadn’t heard of fluent waiting before, only explicit and implicit waiting.

From my post about Waiting in C# WebDriver:

Implicit Waiting

Implicit, or implied waiting involves setting a configuration timeout on the driver object where it will automatically wait up to this amount of time before throwing a NoSuchElementException.

The benefit of implicit waiting is that you don’t need to write code in multiple places to wait as it does it automatically for you.

The downsides to implicit waiting include unnecessary waiting when doing negative existence assertions and having tests that are slow to fail when a true failure occurs (opposite of ‘fail fast’).

Explicit Waiting

Explicit waiting involves putting explicit waiting code in your tests in areas where you know that it will take some time for an element to appear/disappear or change.

The most basic form of explicit waiting is putting a sleep statement in your WebDriver code. This should be avoided at all costs as it will always sleep and easily blow out test execution times.

WebDriver provides a WebDriverWait class which allows you to wait for an element in your code.

As for fluent waits, according to this page it’s a type of explicit wait with more limited conditions on it. I don’t believe WebDriverJs supports fluent waits.

AMA: Moving automated tests from Java to JavaScript

Anonymous asks…

I am currently using a BDD framework with Cucumber, Selenium and Java for automating a web application. I used page factory to store the objects and using them in java methods I wanted to replace the java piece of code with javaScript like mocha or webdriverio. could you share your thoughts on this? can I still use page factory to maintain objects and use them in js files

My response…

What’s the reasoning for moving to JavaScript from Java? Despite having common names, there’s very little otherwise in common (Car is to Carpet as Java is to JavaScript.)

I wouldn’t move for moving sake since I see no benefit in writing BDD style web tests in JavaScript, if anything, e2e automated tests are much harder to write in JavaScript/Node because everything is asynchronous and so you have to deal with promises etc. which is much harder to do than just using Java (or Ruby).

Aside: I still dream of writing e2e tests in Ruby: it’s just so pleasant. But our new user interface is written extensively in JavaScript (React) so it makes sense from a sustainability point of view to use JS over Ruby.