AMA: Async JavaScript Tests?

David asks…

Now that you work with javascript and asynchronous code. It would be interesting to get your thoughts on asynchronous REST/HTTP calls in javascript with respect to testing, if you ever get around to that. I’m familiar with using it in web UIs on client side, but haven’t wrapped my head around that on the test/server side in node.js. I have so far opted to use a synchronous HTTP request library in node.js when I can.

My response…

This is an interesting question David. Whilst WebDriverJs and Mocha etc. are asynchronous on Node, we aren’t using any of that asynchronicity, but dealing with it using promises to resolve to a value we are looking for. It would be very cool to be able to have every WebDriver test we write truly asynchronous to spin up a browser do some testing and wait for any results all whilst running other asynchronous tests, but I believe to do this would be very complicated and pushing the limits of current technology/browser process threads etc. It would be possible if you were writing pure node/ES unit tests though.

So to answer your question we have a bunch of e2e tests written in Mocha that run sequentially one after another in a synchronous manner (using promises), even though they utilise asynchronous libraries.

AMA: bdd frameworks, api test tools & unit testing responsibility

Swapnil Waghmare asks…

Are BDD frameworks like Specflow, Cucumber better for E2E tests?

My response…

There’s two ways I can interpret this question (my additions are in bold).

Firstly:

Are BDD frameworks that use ‘Given/When/Then’ feature files like Specflow, Cucumber  better than frameworks that use ‘describe/it’ blocks like RSpec and Mocha for e2e tests?

As I previously explained, Automattic’s unit tests are written in Mocha, so that was a logical choice for writing e2e tests as there is a lot of familiarity of it within Automattic, which will hopefully mean more developers are interested in the e2e tests we are writing using Mocha/WebDriverJs.

There are some challenges with writing end-to-end tests with Mocha (mainly that Mocha tests are all independent so will continue to run if a previous step in the scenarios fails) so I haven’t completely ruled out investigating a move to Cucumber at some point for the e2e tests.

Secondly:

Are BDD frameworks like Specflow, Cucumber better for E2E tests than using them for automated integration, component or unit tests?

I think you could write integration tests or even unit tests in Given/When/Then format as most unit tests follow the same arrange/act/assert pattern anyhow which is exactly what Given/When/Then is.

Keep in mind there is overhead in maintaining all the step definitions and feature files for Cucumber/Specflow that give you non technical readability so if you don’t require that readability it is probably overkill. But a personal preference nonetheless.

Swapnil Waghmare also asks…

Which tools have you used for API testing, which ones would you recommend?

My response…

At the moment my main focus is on automated unit test and automated e2e test coverage in JavaScript.

I try to keep things simple, so in the past when I’ve written REST integration tests I’ve just called them using in built libraries, and have used Postman quite a lot for manual testing and debugging.

Swapnil Waghmare finally asks…

What is the reason behind writing E2E tests using JavaScript? Isn’t any Object oriented language a better choice for writing E2E tests? Should QA’s write Unit tests?

My response…

 

Firstly, JavaScript is actually object oriented, it’s just not class-based object oriented like Java, C# or Ruby. The newer versions of JavaScript, called ECMAScript, or ESScript are more class-based object oriented.

I’ve actually already answered why I chose JavaScript in a previous answer, so I’ll summarise that here and you can read the rest in that answer.

WordPress.com built an entirely new UI for managing sites using 100% JavaScript with React for the main UI components. I am responsible for e2e automated tests across this UI, and whilst I originally contemplated, and trialled even, using Ruby, this didn’t make long term sense for WordPress.com where the original WordPress developers are mostly PHP and the newer UI developers are all JavaScript.

As for whether QA’s should write unit tests? No, I don’t think so, as I believe unit tests should drive software development, and writing code by writing unit tests is much easier than trying to add unit tests after by someone else, as the original code will not likely be very testable as it wasn’t written with testability in mind. One benefit of code written with unit tests at the time is that it will mostly be better code as the tests are consuming your API that you are developing.

ES2015 Page Classes

As mentioned yesterday, I am updating my WebDriverJs and Mocha demo to use new ES2015 features.

ES2015 supports classes more elegantly than using prototype based ones in older versions of JavaScript.

For example, this old class:

var webdriver = require('selenium-webdriver');
var until = webdriver.until;
var config = require('config');

RalphSaysPage = function RalphSaysPage(driver, visit) {
	this.driver = driver;
	this.url = config.get('url');
	this.explicitWaitMS = config.get('explicitWaitMS');
	this.quoteSelector = webdriver.By.id('quote');
	if (visit === true) {
		this.driver.get(this.url);
	}
	this.driver.wait(until.elementLocated(this.quoteSelector), this.explicitWaitMS);
};

RalphSaysPage.prototype.quoteContainerPresent = function() {
	var d = webdriver.promise.defer();
	this.driver.isElementPresent(this.quoteSelector).then(function(present) {
		d.fulfill(present);
	});
	return d.promise;
};

RalphSaysPage.prototype.quoteTextDisplayed = function() {
	var d = webdriver.promise.defer();
	this.driver.findElement(this.quoteSelector).getText().then(function(text) {
		d.fulfill(text);
	});
	return d.promise;
};

module.exports = RalphSaysPage;

can be better written as a ES2015 class:

import webdriver from 'selenium-webdriver';
import config from 'config';

export default class RalphSaysPage {
	constructor( driver, visit = false ) {
		this.driver = driver;
		this.url = config.get('ralphURL');
		this.explicitWaitMS = config.get('explicitWaitMS');
		this.quoteSelector = webdriver.By.id('quote');

		if (visit) this.driver.get(this.url);

		this.driver.wait(webdriver.until.elementLocated(this.quoteSelector), this.explicitWaitMS);
	}
	quoteContainerPresent() {
		return this.driver.isElementPresent(this.quoteSelector);
	}
	quoteTextDisplayed() {
		return this.driver.findElement(this.quoteSelector).getText();
	}
}

You can see I have also simplified the functions quoteContainerPresent() and quoteTextDisplayed() to directly return the webDriver promise instead of creating our own which is unnecessary.

When I introduce another page class:

import webdriver from 'selenium-webdriver';
import config from 'config';

const by = webdriver.By;
const until = webdriver.until;

export default class WebDriverJsDemoPage {
	constructor( driver, visit = false ) {
		this.driver = driver;
		this.url = config.get('demoURL');
		this.explicitWaitMS = config.get('explicitWaitMS');
		this.expectedElementSelector = by.id('elementappearsparent');

		if (visit) this.driver.get(this.url);

		this.driver.wait(webdriver.until.elementLocated(this.expectedElementSelector), this.explicitWaitMS);
	}
	waitForChildElementToAppear() {
		return this.driver.wait(until.elementLocated(by.id('elementappearschild')), this.explicitWaitMS, 'Could not locate the child element within the time specified');
	}
	childElementPresent() {
		return this.driver.isElementPresent(by.id('elementappearschild'));
	}
}

you can see I have duplicated some common functionality such as the navigation and page waiting across these two classes. This is where we can an ES2015 parent (or base) page class to inherit from.

Our base class might look something like:

export default class BasePage {
	constructor( driver, expectedElementSelector, visit = false, url = null ) {
		this.explicitWaitMS = config.get('explicitWaitMS');
		this.driver = driver;
		this.expectedElementSelector = expectedElementSelector;
		this.url = url;

		if (visit) this.driver.get(this.url);

		this.driver.wait(webdriver.until.elementLocated(this.expectedElementSelector), this.explicitWaitMS);
	}
}

which means our page classes are much nicer now:

export default class RalphSaysPage extends BasePage {
	constructor( driver, visit = false ) {
		const quoteSelector = webdriver.By.id('quote');
		super(driver, quoteSelector, visit, config.get('ralphURL'));
		this.quoteSelector = quoteSelector;
	}
	quoteContainerPresent() {
		return this.driver.isElementPresent(this.quoteSelector);
	}
	quoteTextDisplayed() {
		return this.driver.findElement(this.quoteSelector).getText();
	}
}

export default class WebDriverJsDemoPage extends BasePage {
	constructor( driver, visit = false ) {
		super(driver, by.id('elementappearsparent'), visit, config.get('demoURL'));
	}
	waitForChildElementToAppear() {
		return this.driver.wait(until.elementLocated(by.id('elementappearschild')), this.explicitWaitMS, 'Could not locate the child element within the time specified');
	}
	childElementPresent() {
		return this.driver.isElementPresent(by.id('elementappearschild'));
	}
}

This gives us the ability to add any common functionality across pages (such as checking the page title) quickly and easily without duplication.