WebDriverJS & Mocha Part 3: Page Objects

I recently shared how to get started with WebDriverJS and Mocha, and how to use hooks.

This post continues on from there: I will share how to set up our own page objects to model our application for maintainability.

Continue reading “WebDriverJS & Mocha Part 3: Page Objects”

Why I don’t like Jasmine JavaScript unit tests (for MVC web apps)

Update 11-Feb-2013: After some feedback, I would like to clarify what I am talking about here: I am talking from experience in applications not purely written in JavaScript: for example a MVC application written with say C# with JavaScript added to pages.

I’m really not a big fan of Jasmine JavaScript unit tests for applications not written purely in JavaScript. Here’s why:

jasmine logo

Most modern web applications use JavaScript as a way to provide client side validation and interaction. Most of this JavaScript manipulates the Document Object Model (DOM) of the browser, therefore the JavaScript is coupled to the DOM using ids or classes of elements.

As most web pages are dynamically rendered by the server, Jasmine, the JavaScript unit test framework, can’t run against the ‘real’ DOM as it appears in the application, so what it does is duplicates parts of the DOM to test the JavaScript is working properly based upon this duplication.

As there’s no reliance on the real application or its DOM, these tests run super fast, you can run thousands of JavaScript tests in seconds which is pretty amazing.

The problem becomes that over time the real DOM in the application is changed but the Jasmine tests and its own duplicated DOM is not! This means that often I find that all Jasmine tests pass (as they have to for a new version to be deployed to QA) but the JavaScript doesn’t work in the application because an id or class name has been changed and someone ‘forgot’ to update the Jasmine tests.

Tests that don’t pick up valid errors are worthless, and I continually find myself getting annoyed that all the Jasmine tests have passed, but JavaScript is broken and our application doesn’t work.

Another downside is that Jasmine tests are only as good as the JavaScript engine they are run against. We run our Jasmine tests in a headless PhantomJS browser, which is fast, but most of our JavaScript bugs are found in Internet Explorer, particularly earlier versions of it, so Jasmine tests again are worthless.

I don’t know of a way to overcome these downsides to Jasmine that I have mentioned. What we do have is a suite of end to end acceptance tests that run in 5 different ‘real’ browsers that check the JavaScript works correctly. Whilst these take 8 minutes to run per browser, they pick up a lot more JavaScript issues than Jasmine.

I am not still not sure whether it is worth investing the time and effort in Jasmine tests when they don’t provide sufficient confidence that our application ‘works’.

What have others experiences been with Jasmine JavaScript unit testing?

Using WebDriver to automatically check for JavaScript errors on every page

Update: please see my newer post on this that doesn’t require changes to your application.


One of the benefits of using a page-object model is that you can perform certain actions on every page in your application that you visit, such as checking for accessibility.

One such check is automatically checking for JavaScript errors on page load. There’s a couple of approaches out there, one involves copying and pasting a small snippet of JavaScript into each page. Since our application we are working on uses a standard template for every page, we simply add some JavaScript to the common page header that is the first thing to load on the page, and catches any JavaScript errors that occur:

define(["amdUtils/string/interpolate"], function(interpolate) {
    window.jsErrors = [];

    window.onerror = function (errorMessage, url, lineNumber) {
        var message = interpolate("Error: [{{0}}], url: [{{1}}], line: [{{2}}]", [errorMessage, url, lineNumber]);
        window.jsErrors.push(message);
        return false;
    };
});

It’s then a matter of checking the jsErrors each time we visit a page, this is example C# code we use in the base page class for every page.

var js = driver as IJavaScriptExecutor;
ICollection javascriptErrors = null;
for (var i = 0; i < 20; i++)
{
  javascriptErrors = js.ExecuteScript("return window.jsErrors") as ICollection;
  if (javascriptErrors != null) break;
  System.Threading.Thread.Sleep(1000);
}
Assert.IsNotNull(javascriptErrors, "Can't seem to load JavaScript on the page to find JavaScript errors. Check that JavaScript is enabled.");
var javaScriptErrorsAsString = javascriptErrors.Cast<string>().Aggregate("", (current, error) => current + (error + ", "));
Assert.AreEqual("", javaScriptErrorsAsString, "Found JavaScript errors on page load: " + javaScriptErrorsAsString);

This code waits until it can read the jsErrors on the page. If it can’t, it means that JavaScript didn’t load and this is an error. Once it gets the jsErrors, it checks that there are none.

This code has been very useful for us. It has caught a number of JavaScript errors, and is especially great for finding cross browser JavaScript issues, as we run our acceptance tests in 5 different browsers.