Bailing with Mocha e2e Tests

At Automattic we use Mocha to write our end-to-end (e2e) automated tests in JavaScript/Node.js. One issue with Mocha is that it’s not really a tool suited to writing e2e tests where one test step can rely on a previous test step – for example our sign up process is a series of pages/steps which rely on the previous step passing. Mocha is primarily a unit testing tool and it’s bad practice for one unit test to depend on another, so that is why Mocha doesn’t support this.

A more simplified example of this is shown in my webdriver-js-demo project:

describe( 'Ralph Says', function() {
	this.timeout( mochaTimeoutMS );

	before( async function() {
		const builder = new webdriver.Builder().withCapabilities( webdriver.Capabilities.chrome() );
		driver = await builder.build();
	} );

	it( 'Visit the page', async function() {
		page = await RalphSaysPage.Visit( driver );
	} );

	it( 'shows a quote container', async function() {
		assert( await page.quoteContainerPresent(), 'Quote container not displayed' );
	} );

	it( 'shows a non-empty quote', async function() {
		assert.notEqual( await page.quoteTextDisplayed(), '', 'Quote is empty' );
	} );

	afterEach( async function() { await driver.manage().deleteAllCookies(); } );

	after( async function() { await driver.quit(); } );
} );

Continue reading “Bailing with Mocha e2e Tests”

Using async/await with WebDriverJs

We’ve been using WebDriverJs for a number of years and the control flow promise manager that it offers to make writing WebDriverJs commands in a synchronous blocking way a bit easier, particularly when using promises.

The problem with the promise manager is that it is hard to understand its magic as sometimes it just works, and other times it was very confusing and not very predictable. It was also harder to develop and support by the Selenium project so it’s being deprecated later this year.

Fortunately recent versions of Node.js support asynchronous functions and use of the await command which makes writing WebDriverJs tests so much easier and understandable.

I’ve recently updated my WebDriverJs demo project to use async/await so I’ll use that project as examples to explain what is involved.

WebDriverJs would allow you to write consecutive statements like this without worrying about waiting for each statement to finish – note the use of test.it instead of the usual mocha it function:

test.it( 'can wait for an element to appear', function() {
	const page = new WebDriverJsDemoPage( driver, true );
	page.waitForChildElementToAppear();
	page.childElementPresent().then( ( present ) => {
		assert( present, 'The child element is not present' );
	} );
} );

When you were waiting on the return value from a promise you could use a .then function to wait for the value as shown above.

This is quite a simple example and this could get complicated pretty quickly.

Since the promise manager is being removed, we need to update our tests so they continue to execute in the correct order. We can make the test function asynchronous by adding the async prefix, remove the test. prefix on the it block, and add await statements every time we expect a statement to finish before continuing:

it( 'can wait for an element to appear', async function() {
	const page = new WebDriverJsDemoPage( driver, true );
	await page.waitForChildElementToAppear();
	assert( await page.childElementPresent(), 'The child element is not present' );
} );

I personally find this much easier to read and understand, less ‘magic’, but the one bit that stands out is visiting the page and creating the new page object. The code in the constructor for this page, and other pages, is asynchronous as well, however we can’t have an async constructor!

export default class BasePage {
	constructor( driver, expectedElementSelector, visit = false, url = null ) {
		this.explicitWaitMS = config.get( 'explicitWaitMS' );
		this.driver = driver;
		this.expectedElementSelector = expectedElementSelector;
		this.url = url;

		if ( visit ) this.driver.get( this.url );

		this.driver.wait( until.elementLocated( this.expectedElementSelector ), this.explicitWaitMS );
	}
}

How we can get around this is to define a static async function that acts as a constructor and returns our new page object for us.

So, our BasePage now looks like:

export default class BasePage {
	constructor( driver, expectedElementSelector, url = null ) {
		this.explicitWaitMS = config.get( 'explicitWaitMS' );
		this.driver = driver;
		this.expectedElementSelector = expectedElementSelector;
		this.url = url;
	}

	static async Expect( driver ) {
		const page = new this( driver );
		await page.driver.wait( until.elementLocated( page.expectedElementSelector ), page.explicitWaitMS );
		return page;
	}

	static async Visit( driver, url ) {
		const page = new this( driver, url );
		if ( ! page.url ) {
			throw new Error( `URL is required to visit the ${ page.name }` );
		}
		await page.driver.get( page.url );
		await page.driver.wait( until.elementLocated( page.expectedElementSelector ), page.explicitWaitMS );
		return page;
	}
}

In our Expect and Visit functions we call new this( driver ) which creates an instance of the child class which suits our purposes. So, this means our spec now looks like:

it( 'can wait for an element to appear', async function() {
	const page = await WebDriverJsDemoPage.Visit( driver );
	await page.waitForChildElementToAppear();
	assert( await page.childElementPresent(), 'The child element is not present' );
} );

which means we can await visiting and creating our page objects and we don’t have any asynchronous code in our constructors for our pages. Nice.

Once we’re ready to not use the promise manager we can set SELENIUM_PROMISE_MANAGER to 0 and it won’t use it any more.

Summary

The promise manager is being removed in WebDriverJs but using await in async functions is a much nicer solution anyway, so now is the time to make the move, what are you awaiting for? 😊

Check out the full demo code at https://github.com/alisterscott/webdriver-js-demo

Executing JS in IE11 using WebDriverJs

We write our e2e tests in JavaScript running on Node.js which allows us to use newer JavaScript/ECMAScript features like template literals.

We have a subset of our e2e tests – mainly signing up as a new customer – which we run a few times a day against Internet Explorer 11: our lowest supported IE version.

I recently added a function that sets a cookie to set the currency for a customer:

setCurrencyForPayments( currency ) {
  const setCookieCode = function( currencyValue ) {
    window.document.cookie = `landingpage_currency=${ currencyValue };domain=.wordpress.com`;
  }
return this.driver.executeScript( setCookieCode, currency );
}

This code works perfectly when executing against Chrome or Firefox, but when it came to executing against IE11 I would see the following (rather unhelpful) error:

Uncaught JavascriptError: JavaScript error (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 69 milliseconds

I couldn’t work out what was causing this so I decided to take a break. On my break I realised that WebDriverJs is trying to execute a new JavaScript feature (template literals) against an older browser that doesn’t support it! Eureka!

So all I had to do was update our code to:

setCurrencyForPayments( currency ) {
  const setCookieCode = function( currencyValue ) {
    window.document.cookie = 'landingpage_currency=' + currencyValue + ';domain=.wordpress.com';
  }
return this.driver.executeScript( setCookieCode, currency );
}

and all our tests were happy against IE11 again 😊

Having not found a lot about this error or the cause I am hoping this blog post can help someone else out if they encounter this issue also.

AMA: Difference between explicit and fluent wait

Anonymous asks…

What is the difference between Explicit wait and Fluent wait?

My response…

I hadn’t heard of fluent waiting before, only explicit and implicit waiting.

From my post about Waiting in C# WebDriver:

Implicit Waiting

Implicit, or implied waiting involves setting a configuration timeout on the driver object where it will automatically wait up to this amount of time before throwing a NoSuchElementException.

The benefit of implicit waiting is that you don’t need to write code in multiple places to wait as it does it automatically for you.

The downsides to implicit waiting include unnecessary waiting when doing negative existence assertions and having tests that are slow to fail when a true failure occurs (opposite of ‘fail fast’).

Explicit Waiting

Explicit waiting involves putting explicit waiting code in your tests in areas where you know that it will take some time for an element to appear/disappear or change.

The most basic form of explicit waiting is putting a sleep statement in your WebDriver code. This should be avoided at all costs as it will always sleep and easily blow out test execution times.

WebDriver provides a WebDriverWait class which allows you to wait for an element in your code.

As for fluent waits, according to this page it’s a type of explicit wait with more limited conditions on it. I don’t believe WebDriverJs supports fluent waits.

Why you should use CSS selectors for your WebDriver tests

I didn’t used to be a fan of CSS selectors for automated web tests, but I changed my mind.

The reason I didn’t use to be a fan of CSS selectors is that historically they weren’t really encouraged by Watir, since the Watir API was designed to find elements by type and attribute, so the Watir API would look something like:

browser.div(:class => 'highlighted')

where the same CSS selector would look like:

div.highlighted

Since WebDriver doesn’t use the same element type/attribute API and just uses findElement with a By selector, CSS selectors make the most sense since they’re powerful and self-contained.

The the best thing about using CSS selectors, in my opinion, is the Chrome Dev Tools allows you to search the DOM using a CSS selector (and XPath selectors, but please don’t use XPath), using Command/Control & F:

chrome css selectors
Using CSS selectors to find elements in Chrome Dev Tools

So you can ‘test’ your CSS in a live browser window before deciding to use it in your WebDriver test.

The downside of using CSS selectors are they’re a bit less self explanatory than explicitly using by.className or by.id.

But CSS selectors are pretty powerful: especially pseudo selectors like nth-of-type and I’ve found the only thing you can’t really do in CSS is select by text value, which you probably shouldn’t be doing anyway as text values are more likely to change (since they’re copy often changed by your business) and can be localised in which case your tests won’t run across different cultures.

The most powerful usage of CSS selectors is where you add your own data attributes to elements in your application and use these to select elements: straightforward, efficient and less brittle than other approaches. For example:

a[data-e2e-value="free"]

How do you identify elements in your WebDriver automated tests?

Save password prompts in Chrome 57 with WebDriver

When running Selenium WebDriver scripts against the latest version of Chrome (57) it shows a save password prompt that hasn’t appeared previously whilst using Chromedriver, as far as I know.

chrome 57 save password prompt Continue reading “Save password prompts in Chrome 57 with WebDriver”

Upgrading WebdriverJs to Selenium 3

Yes, I know that Selenium 3 has been out for a while, but I’ve finally got around at looking at updating our end-to-end tests to use it. Newer versions of Firefox require Geckodriver which require Selenium 3.3+ so it’s a forced upgrade of sorts.

Continue reading “Upgrading WebdriverJs to Selenium 3”

Feature Toggles for Automated e2e Tests

Feature toggles aren’t just for production code. Feature toggles are also a powerful technique to change the behaviour of your automated end-to-end tests without changing code.

Continue reading “Feature Toggles for Automated e2e Tests”

Using WebDriver to automatically check for JavaScript errors on every page (2016 edition)

Back in 2012 I wrote about how to use WebDriver to automatically check for JavaScript console errors on every page. The solution I proposed involved adding some common JavaScript to every page in your app and then checking that errors object when using WebDriver page object classes.

Fortunately since then the WebDriver project now supports checking for these errors without making any changes to your app, so if this has been stopping you doing this you can now do it quite easily.

Continue reading “Using WebDriver to automatically check for JavaScript errors on every page (2016 edition)”

AMA: C# WebDriver Questions

omayer asks…

how to use test data from xlsx in c# webdriver, thank you in advance.

My response…

I haven’t done this as I’ve managed data/scenarios using SpecFlow, but there’s this post that should help you.

omayer also asks…

Handling popup Windows in C# WebDriver – Closing the popup window , finding elements on popup window, locating element inside popup window and frameset

My response…

Stackoverflow is a good resource for these types of questions; there’s some examples on how to do this here.

pallavi asks…

Which tool can we use for report generation in selenium with C#

My response…

This very much depends on which framework you are using to manage your tests. I have used SpecFlow which allows a formatted HTML report of test results.

pallavi also asks…

Can we use selenium with coded ui in visual studio? If I use with coded ui then what template it will follow

My response…

My experience with CodedUI tests in C# has only been negative. When I looked at it, the approach Microsoft used was to generate a huge number of lines of CodedUI test code which I couldn’t understand or make into reusable objects. I much prefer Selenium/WebDriver for any web based tests, and to use White to test any non-web based UIs. I am not familiar with CodedUI templates since it has been a long time since I looked at it.

 

AMA: reusing the same browser for different automated tests

Ankitha asks…

Let’s say we have two tests files inside specs folder
1) test1.js — has 3 tests
2)test2.js– has 3 tests

When I do npm test, a seperate browser is opened for running test1.js and second browser is opened up for running test2.js.
Whereas I want only one browser to be opened.
Like perform test1.js tests close the browser, For test2.js is now open browser again run test2.js. I would like to see tests running sequentially. How can this be achieved?

My response…

You may have seen that we recently open-sourced our automated e2e tests for WordPress.com.

One the huge number of benefits this brings if you can see how we do things :)

If you have a look at driverManager.startBrowser, which is what we call from a before hook in each test, you’ll see how we reuse the browser across different tests.

There’s one additional thing you need to do which is close the browser down at the very end. We use a separate file after.js to do this, which we pass to Mocha when running any test.

AMA: JS vs Ruby

Butch Mayhew asks…

I have noticed you blogging more about JS frameworks. How do these compare to Watir/Ruby? Would you recommend one over the other?

My response…

I had a discussion recently with Chuck van der Linden about this same topic as he has a lot of experience with Watir and is now looking at JavaScript testing frameworks like I have done.

Some Background

WordPress.com built an entirely new UI for managing sites using 100% JavaScript with React for the main UI components. I am responsible for e2e automated tests across this UI, and whilst I originally contemplated, and trialled even, using Ruby, this didn’t make long term sense for WordPress.com where the original WordPress developers are mostly PHP and the newer UI developers are all JavaScript.

Whilst I see merit in both views: I still think having your automated acceptance tests in the same language as your application leads to better maintainability and adoptability.

I still think writing automated acceptance tests in Ruby is much cleaner and nicer than JavaScript Node tests, particularly as Ruby allows meta-programming which means page objects can be implemented really neatly.

The JavaScript/NodeJS landscape is still very immature where people are using various tools/frameworks/compilers and certain patterns or de facto standards haven’t really emerged yet. The whole ES6/ES2015/ES2016 thing is very confusing to newcomers like me, especially on NodeJS where some ES6+ features are supported, but others require something like Babel to compile your code.

But generally with the direction ES is going, writing page objects as classes is much nicer than using functions for everything as in ES5.

Whilst there’s nothing I have found that is better (or even as good) in JavaScript/Mocha/WebDriverJS than Ruby/RSpec/Watir-WebDriver, I still think it’s a better long term decision for WordPress.com to use the JavaScript NodeJS stack for our e2e tests.

Checking an image is actually visible in WebDriverJs

I recently discovered a gap in one of my e2e automated tests where I was checking the existence of an uploaded image in the DOM, but not that the image was actually displayed.

driver.isElementPresent( By.css( `img[alt='upload.jpg']` ) ).then( function( present ) {
  assert.equal( present, true, 'Image not displayed' );
} );

If the DOM has a reference to the image, but it isn’t actually rendered this test will pass. This isn’t ideal.

I remembered my post about how to check that an image is actually rendered using WebDriver in C# and so I used the same JavaScript script which WebDriverJs sends to the driver:

driver.findElement( By.css( `img[alt='upload.jpg']` ) ).then( function( element ) {
  driver.executeScript( 'return (typeof arguments[0].naturalWidth!=\"undefined\" && arguments[0].naturalWidth>0)', element ).then( function( present ) {
    assert.equal( present, true, 'Image not displayed' );
  } );
} );

This works a treat. I’ve moved it into a helper function so I can use this anywhere without repeating it also.

Testing end-to-end with Mocha

As part of my excellent Excellence Wrangler role at Automattic, one of my key tasks has been establishing some end-to-end tests for WordPress.com using Mocha with WebDriverJs. Our testing pyramid doesn’t look much like a pyramid:

wordpress.com test pyramid.png

We’ve got lots of React unit tests at the bottom: these are to speed development.

We’re intentionally missing a middle: the REST API we consume has its own unit tests, we don’t need integration tests for it. We don’t have detailed full stack acceptance tests of our UI: these are too slow and brittle.

We have a handful of e2e flow tests at the top, these are to protect the user experience, we run these on every deployment and frequently in production. These can be brittle on such a fast moving code base, but we limit their number (depth) so they still give us good confidence everything is working well together but limiting our overhead.

So what do these end-to-end tests look like?

I hadn’t used Mocha before and I was used to writing end-to-end tests in Gherkin format in tools like Cucumber and Specflow so I initially began writing end-to-end tests that looked like this:

test.describe('WordPress.com Sign Up', function() {
  test.beforeEach(function() {
    driver.manage().deleteAllCookies();
  });

  test.it('Can Create A Free Blog', function() {
    var signupFlow = new SignUpFlow( driver, 'desktop' );
    signupFlow.createFreeBlog( 'en' );
  });

  test.it('Can Create A New Site With a Paid Domain Upgrade', function() {
    var signupFlow = new SignUpFlow( driver, 'desktop' );
    signupFlow.CreateSiteWithDomainPaidByCreditCard( 'en' );
  });
});

I was pushing the code down into flow classes which I have used before, but the issue with this was the output I was getting from Mocha wasn’t very rich:

WordPress.com Sign Up
      ✓ Can Create A Free Blog
      ✓ Can Create A New Site With a Paid Domain Upgrade

I then realized by looking at an end-to-end test written by another developer that you can nest describe and it statements to give you much more expressive end-to-end tests.

test.describe( `Sign Up (${screenSize})`, function() {

  test.describe( 'Free Site:', function() {
    test.before( 'Delete Cookies and Local Storage', function() {
      driverManager.clearCookiesAndDeleteLocalStorage( driver );
    } );

    test.describe( 'Sign up for a free site', function() {

      test.describe( 'Step One: Themes', function() {
        test.before( 'Can see the choose a theme page as the starting page', function() {
          this.chooseAThemePage = new ChooseAThemePage( driver, { visit: true } );
          return this.chooseAThemePage.displayed().then( ( displayed ) => {
            assert.equal( displayed, true, 'The choose a theme start page is not displayed' );
          } );
        } );

        test.it( 'Can select the first theme', function() {
          return this.chooseAThemePage.selectFirstTheme();
        } );
      } );

      test.describe( 'Step Two: Domains', function() {
        test.before( 'Can then see the domains page ', function() {
          this.findADomainComponent = new FindADomainComponent( driver );
          return this.findADomainComponent.displayed().then( ( displayed ) => {
            assert.equal( displayed, true, 'The choose a domain page is not displayed' );
          } );
        } );

        test.it( 'Can search for a blog name', function() {
          return this.findADomainComponent.searchForBlogNameAndWaitForResults( blogName );
        } );

        test.it( 'Can see a free WordPress.com blog address in results ', function() {
          return this.findADomainComponent.freeBlogAddress().then( ( actualAddress ) => {
            assert.equal( actualAddress, expectedBlogAddress, 'The expected free address is not shown' )
          } );
        } );

        test.it( 'Can select the free address', function() {
          return this.findADomainComponent.selectFreeAddress();
        } );
      } );

This gives us rich feedback:

Sign Up (desktop)
    Free Site:
      Sign up for a free site
        Step One: Themes
          ✓ Can see the choose a theme page as the starting page
          ✓ Can select the first theme
        Step Two: Domains
          ✓ Can then see the domains page
          ✓ Can search for a blog name
          ✓ Can see a free WordPress.com blog address in results
          ✓ Can select the free address

The mistake I had made which I didn’t realize was not creating enough nesting, instead of having Step One, Step Two etc. next to one another, they should be nested within each other. This is because if they’re next to one another, Mocha will run the Step Two, Step Three etc. even if Step One has failed, which is not what we want in an end-to-end scenario where each step is dependent on the previous one.

So, it now looks something like this:

test.describe( 'Free Site:', function() {
    test.before( 'Delete Cookies and Local Storage', function() {
      driverManager.clearCookiesAndDeleteLocalStorage( driver );
    } );

    test.describe( 'Sign up for a free site', function() {
      test.describe( 'Step One: Themes', function() {
        test.before( 'Can see the choose a theme page as the starting page', function() {
          this.chooseAThemePage = new ChooseAThemePage( driver, { visit: true } );
          return this.chooseAThemePage.displayed().then( ( displayed ) => {
            assert.equal( displayed, true, 'The choose a theme start page is not displayed' );
          } );
        } );

        test.it( 'Can select the first theme', function() {
          return this.chooseAThemePage.selectFirstTheme();
        } );

        test.describe( 'Step Two: Domains', function() {
          test.before( 'Can then see the domains page ', function() {
            this.findADomainComponent = new FindADomainComponent( driver );
            return this.findADomainComponent.displayed().then( ( displayed ) => {
              assert.equal( displayed, true, 'The choose a domain page is not displayed' );
            } );
          } );

          test.it( 'Can search for a blog name', function() {
            return this.findADomainComponent.searchForBlogNameAndWaitForResults( blogName );
          } );

          test.it( 'Can see a free WordPress.com blog address in results ', function() {
            return this.findADomainComponent.freeBlogAddress().then( ( actualAddress ) => {
              assert.equal( actualAddress, expectedBlogAddress, 'The expected free address is not shown' )
            } );
          } );

          test.it( 'Can select the free address', function() {
            return this.findADomainComponent.selectFreeAddress();
          } );

          test.describe( 'Step Three: Plans', function() {

which means the output is slightly different but still very useful:

Sign Up (mobile)
  Free Site:
    Sign up for a free site
      Step One: Themes
        ✓ Can select the first theme
        Step Two: Domains
          ✓ Can search for a blog name
          ✓ Can see a free WordPress.com blog address in results
          ✓ Can select the free address
          Step Three: Plans
            ✓ Can select the free plan

These tests are much better written this way. The only issue I am left facing with Mocha is when a before hook fails (such as logging in) the generic afterEach hook we have to take screenshots is not triggered (this is only triggered when an it block is run.

Running Automated Tests with A/B Testing

Like a lot of modern, data driven sites, WordPress.com uses A/B testing extensively to introduce new features. These tests may be as simple as a label change or as complex as changing the entire sign up flow, for example by offering a free trial.

Since I have been working on a set of automated end-to-end tests for WordPress.com, I have found A/B testing to be problematic for automated testing on this very fast moving codebase, namely:

  1. Automated tests need to be deterministic: having a randomised experiment as an A/B test means the first test run may get an entirely different sign up flow than a second test run which is very hard to automate; and
  2. Automated tests need to know which experiments are running otherwise they may encounter unexpected behaviour randomly.

What we need is two methods to deal with A/B tests when running automated tests:

  1. We need to be able to see which A/B tests are active and compare this to a known list of expected A/B tests – so that we don’t suddenly encounter some unexpected/random behaviour for some of our test runs
  2. We need to be able to set the desired behaviour to the control group so that are our tests are deterministic.

Different sites conduct A/B testing using different tools and approaches, WordPress.com uses HTML5 local storage to set which A/B tests are active and which group the user belongs to.

Luckily it’s easy to read and update local storage using WebDriver and JavaScript. This means our approach is to:

  1. Each time a page object is initialised, there is a call on the base page model that checks the A/B tests that are active using something like return window.localStorage.ABTests; and then compares this to the known list of A/B tests which are checked in as a config item. This fails the test if there’s a new A/B test introduced that isn’t in the list of known tests. This is better than not knowing about the A/B test and failing based upon some non-deterministic behaviour.
  2. When a new A/B test is introduced and we wish to ensure our automated tests always use the control group, we can set this using a similar method window.localStorage.setItem('ABTests','{"flow":"default"}'); and refresh the page.

Ideally it would be good to know and plan every A/B test for our automated e2e tests, but since this isn’t possible, checking against known A/B tests and ensuring control groups are set means our automated tests are at least more consistent and deterministic, and fail a lot faster and more consistently when a new A/B test has been introduced.

How do you deal with non-determinism with A/B tests?

WebDriverJs & Mocha in ES2015

A friend of mine, Mark Ryall, recently created a fork of my WebDriverJs and Mocha example project and updated it to use ES2015. I’ve made some further changes and merged these in, and would like to share these.

Background

JavaScript is an implementation of the ECMAScript scripting language standard.

The latest version of ECMAScript, known as ES2015, ES6, ES6 Harmony, ECMAScript 2015, or ECMAScript 6, has some neat features which are handy to use for our WebDriverJs & Mocha tests I have previously written about.

It seems that there will be yearly releases of the ECMAScript standard from 2015 onwards, and the most common way to refer to these will be as ES2015, ES2016 etc.

Enabling ES2015 Support for our Example Tests

There is a node tool called Babel which is a JavaScript compiler that allows you to use new ECMAScript features and compile these into JavaScript. This requires two node packages which we add to our package.json file:

"babel-core": "^6.3.13",
"babel-preset-es2015": "^6.3.13"

This means we have a babel compiler and a babel library to transform ES2015.

The second thing we need to do is add a plugin to actually tell babel to transform ES2015.

We add a .babelrc file to our project with the following content:

{
"presets": ["es2015"]
}

Running our Specs using Babel

Once we’ve done this, we can use Mocha and WebDriverJs with ES2015. Instead of calling mocha specs we now need to use babel like:
mocha --compilers js:babel-core/register specs.

This isn’t as nice, so we can update our package.json file so our test command is set to the longer babel command, and we just need to call npm test to run our Mocha specs.

Updating our code to use ES2015

The great thing about ES2015 is it is backwards compatible, so we don’t need to update all our code at once, we can made gradual changes to use new features available to us.

Mark made changes to the spec and the page object to use some of the pretty ES2015 features:

Import Statements

This:

var assert = require('assert');
var webdriver = require('selenium-webdriver');
var test = require('selenium-webdriver/testing');
var config = require('config');
var RalphSaysPage = require('../lib/ralph-says-page.js');

Becomes:

import assert from 'assert';
import webdriver from 'selenium-webdriver';
import test from 'selenium-webdriver/testing';
import config from 'config';
import { ralphSays } from '../lib/pages.js';

Using let instead of var

let is block scoped so this is better to use.

This:

var driver;

Becomes:

let driver = null;

Arrow functions

The arrow functions make the clean up hooks simpler to read:

From this:

test.afterEach(function() {
  driver.manage().deleteAllCookies();
});

To this:

test.afterEach(() => driver.manage().deleteAllCookies());

Summary

Moving to use ES2015 wasn’t as daunting as I initially thought as once you add support for it using Babel, you can gradually start using the new features.

Real vs Headless Browsers for Automated Acceptance Tests

When I was comparing/evaluating JavaScript browser automation tools for Automattic, I noticed that a lot of the tools were headless only: either they use PhantomJS/SlimerJS or have their own headless browser engine (like Zombie.js).

I must admit I am not a fan of headless browsers for a number of reasons:
Continue reading “Real vs Headless Browsers for Automated Acceptance Tests”