Handling JavaScript alerts when leaving a page with WebDriver

You’ve most probably seen the sometimes-useful-but-often-annoying browser alerts when navigating away from a page:JavaScript onbeforeunload alert

How do we deal with these using WebDriver?

Continue reading “Handling JavaScript alerts when leaving a page with WebDriver”

Using WebDriver to automatically check for JavaScript errors on every page (2016 edition)

Back in 2012 I wrote about how to use WebDriver to automatically check for JavaScript console errors on every page. The solution I proposed involved adding some common JavaScript to every page in your app and then checking that errors object when using WebDriver page object classes.

Fortunately since then the WebDriver project now supports checking for these errors without making any changes to your app, so if this has been stopping you doing this you can now do it quite easily.

Continue reading “Using WebDriver to automatically check for JavaScript errors on every page (2016 edition)”

AMA: JS vs Ruby

Butch Mayhew asks…

I have noticed you blogging more about JS frameworks. How do these compare to Watir/Ruby? Would you recommend one over the other?

My response…

I had a discussion recently with Chuck van der Linden about this same topic as he has a lot of experience with Watir and is now looking at JavaScript testing frameworks like I have done.

Some Background

WordPress.com built an entirely new UI for managing sites using 100% JavaScript with React for the main UI components. I am responsible for e2e automated tests across this UI, and whilst I originally contemplated, and trialled even, using Ruby, this didn’t make long term sense for WordPress.com where the original WordPress developers are mostly PHP and the newer UI developers are all JavaScript.

Whilst I see merit in both views: I still think having your automated acceptance tests in the same language as your application leads to better maintainability and adoptability.

I still think writing automated acceptance tests in Ruby is much cleaner and nicer than JavaScript Node tests, particularly as Ruby allows meta-programming which means page objects can be implemented really neatly.

The JavaScript/NodeJS landscape is still very immature where people are using various tools/frameworks/compilers and certain patterns or de facto standards haven’t really emerged yet. The whole ES6/ES2015/ES2016 thing is very confusing to newcomers like me, especially on NodeJS where some ES6+ features are supported, but others require something like Babel to compile your code.

But generally with the direction ES is going, writing page objects as classes is much nicer than using functions for everything as in ES5.

Whilst there’s nothing I have found that is better (or even as good) in JavaScript/Mocha/WebDriverJS than Ruby/RSpec/Watir-WebDriver, I still think it’s a better long term decision for WordPress.com to use the JavaScript NodeJS stack for our e2e tests.

AMA: JavaScript & Mobile App Automation Tools

Justin Watts asks…

Have you looked at Chimp.js ? Any thoughts? I was very closely following your posts on picking a new framework for Automattic and I was quite surprised when you chose an async tool and stepped away from Cucumber. As a ruby dev looking to get comfortable with testing web-apps in JavaScript, I am torn on what library to dig into.

My response…

I had a quick look at Chimp.js when I was evaluating tools, it looks quite impressive, but quite opinionated and integrated, so I find these types of libraries are good to start with, but quickly become frustrating when you want to start doing advanced things (like the custom Slack reporter we just wrote that pings slack when tests fail with screenshots). I am curious about how Chimp.js is synchronous when the underlying WebDriver.IO tool uses promises?

I think the asynchronous thing is less of a big deal than most people make out. Once you get used to using promises whenever querying a value from the browser, then you just do that. I have set up my own es6 page objects so these take away most of this complication anyway.

The one synchronous tool I did seriously contemplate was webdriver-sync but since it wraps the underlying WebDriver Java driver, it was quite heavyweight as you need Java, the node-java bridge, and I couldn’t get this working on CircleCI which is what Automattic’s uses as its CI platform.

As for stepping away from Cucumber. Automattic’s unit tests are written in Mocha, so that was a logical choice as there is a lot of familiarity of it within Automattic, which will hopefully mean more developers are interested in the e2e tests we are writing using Mocha/WebDriverJs.

There are some challenges with writing end-to-end tests with Mocha (mainly that Mocha tests are all independent so will continue to run if a previous step in the scenarios fails) so I haven’t completely ruled out investigating a move to Cucumber at some point for the e2e tests.

Justin Watts also asks…

Appium seems to be getting more unstable as time goes on. Early Grey caught my eye. Do you have any thoughts / outlooks on the mobile app testing landscape?

My response…

I must say I haven’t had a lot of recent experience with mobile UI automation tools. A couple of years back we started using Appium but abandoned this effort as the tests were so fragile we could never get green builds, so we focused on writing unit tests for the specific mobile code, testing webviews from webdriver, and doing human end to end testing on real devices. This worked well, and unless I worked on complex native apps I would probably end up doing something similar again should this arise.

 

 

Checking an image is actually visible in WebDriverJs

I recently discovered a gap in one of my e2e automated tests where I was checking the existence of an uploaded image in the DOM, but not that the image was actually displayed.

driver.isElementPresent( By.css( `img[alt='upload.jpg']` ) ).then( function( present ) {
  assert.equal( present, true, 'Image not displayed' );
} );

If the DOM has a reference to the image, but it isn’t actually rendered this test will pass. This isn’t ideal.

I remembered my post about how to check that an image is actually rendered using WebDriver in C# and so I used the same JavaScript script which WebDriverJs sends to the driver:

driver.findElement( By.css( `img[alt='upload.jpg']` ) ).then( function( element ) {
  driver.executeScript( 'return (typeof arguments[0].naturalWidth!=\"undefined\" && arguments[0].naturalWidth>0)', element ).then( function( present ) {
    assert.equal( present, true, 'Image not displayed' );
  } );
} );

This works a treat. I’ve moved it into a helper function so I can use this anywhere without repeating it also.

WebDriverJs & Mocha in ES2015

A friend of mine, Mark Ryall, recently created a fork of my WebDriverJs and Mocha example project and updated it to use ES2015. I’ve made some further changes and merged these in, and would like to share these.

Background

JavaScript is an implementation of the ECMAScript scripting language standard.

The latest version of ECMAScript, known as ES2015, ES6, ES6 Harmony, ECMAScript 2015, or ECMAScript 6, has some neat features which are handy to use for our WebDriverJs & Mocha tests I have previously written about.

It seems that there will be yearly releases of the ECMAScript standard from 2015 onwards, and the most common way to refer to these will be as ES2015, ES2016 etc.

Enabling ES2015 Support for our Example Tests

There is a node tool called Babel which is a JavaScript compiler that allows you to use new ECMAScript features and compile these into JavaScript. This requires two node packages which we add to our package.json file:

"babel-core": "^6.3.13",
"babel-preset-es2015": "^6.3.13"

This means we have a babel compiler and a babel library to transform ES2015.

The second thing we need to do is add a plugin to actually tell babel to transform ES2015.

We add a .babelrc file to our project with the following content:

{
"presets": ["es2015"]
}

Running our Specs using Babel

Once we’ve done this, we can use Mocha and WebDriverJs with ES2015. Instead of calling mocha specs we now need to use babel like:
mocha --compilers js:babel-core/register specs.

This isn’t as nice, so we can update our package.json file so our test command is set to the longer babel command, and we just need to call npm test to run our Mocha specs.

Updating our code to use ES2015

The great thing about ES2015 is it is backwards compatible, so we don’t need to update all our code at once, we can made gradual changes to use new features available to us.

Mark made changes to the spec and the page object to use some of the pretty ES2015 features:

Import Statements

This:

var assert = require('assert');
var webdriver = require('selenium-webdriver');
var test = require('selenium-webdriver/testing');
var config = require('config');
var RalphSaysPage = require('../lib/ralph-says-page.js');

Becomes:

import assert from 'assert';
import webdriver from 'selenium-webdriver';
import test from 'selenium-webdriver/testing';
import config from 'config';
import { ralphSays } from '../lib/pages.js';

Using let instead of var

let is block scoped so this is better to use.

This:

var driver;

Becomes:

let driver = null;

Arrow functions

The arrow functions make the clean up hooks simpler to read:

From this:

test.afterEach(function() {
  driver.manage().deleteAllCookies();
});

To this:

test.afterEach(() => driver.manage().deleteAllCookies());

Summary

Moving to use ES2015 wasn’t as daunting as I initially thought as once you add support for it using Babel, you can gradually start using the new features.

Comparison of JavaScript browser automation and test specification libraries

As part of my trial for my current role at Automattic, I was tasked with implementing some e2e acceptance tests using my choice of library/framework/language.

I very much recommend writing automated acceptance tests in the same language as your app, even though I have described some benefits of using a different language, and since WordPress is moving towards JavaScript from PHP, JavaScript seems the most suitable language for Automattic.

Continue reading “Comparison of JavaScript browser automation and test specification libraries”