Feature Toggles for Automated e2e Tests

Feature toggles aren’t just for production code. Feature toggles are also a powerful technique to change the behaviour of your automated end-to-end tests without changing code.

Continue reading “Feature Toggles for Automated e2e Tests”

Using WebDriver to automatically check for JavaScript errors on every page (2016 edition)

Back in 2012 I wrote about how to use WebDriver to automatically check for JavaScript console errors on every page. The solution I proposed involved adding some common JavaScript to every page in your app and then checking that errors object when using WebDriver page object classes.

Fortunately since then the WebDriver project now supports checking for these errors without making any changes to your app, so if this has been stopping you doing this you can now do it quite easily.

Continue reading “Using WebDriver to automatically check for JavaScript errors on every page (2016 edition)”

AMA: C# WebDriver Questions

omayer asks…

how to use test data from xlsx in c# webdriver, thank you in advance.

My response…

I haven’t done this as I’ve managed data/scenarios using SpecFlow, but there’s this post that should help you.

omayer also asks…

Handling popup Windows in C# WebDriver – Closing the popup window , finding elements on popup window, locating element inside popup window and frameset

My response…

Stackoverflow is a good resource for these types of questions; there’s some examples on how to do this here.

pallavi asks…

Which tool can we use for report generation in selenium with C#

My response…

This very much depends on which framework you are using to manage your tests. I have used SpecFlow which allows a formatted HTML report of test results.

pallavi also asks…

Can we use selenium with coded ui in visual studio? If I use with coded ui then what template it will follow

My response…

My experience with CodedUI tests in C# has only been negative. When I looked at it, the approach Microsoft used was to generate a huge number of lines of CodedUI test code which I couldn’t understand or make into reusable objects. I much prefer Selenium/WebDriver for any web based tests, and to use White to test any non-web based UIs. I am not familiar with CodedUI templates since it has been a long time since I looked at it.


AMA: reusing the same browser for different automated tests

Ankitha asks…

Let’s say we have two tests files inside specs folder
1) test1.js — has 3 tests
2)test2.js– has 3 tests

When I do npm test, a seperate browser is opened for running test1.js and second browser is opened up for running test2.js.
Whereas I want only one browser to be opened.
Like perform test1.js tests close the browser, For test2.js is now open browser again run test2.js. I would like to see tests running sequentially. How can this be achieved?

My response…

You may have seen that we recently open-sourced our automated e2e tests for WordPress.com.

One the huge number of benefits this brings if you can see how we do things :)

If you have a look at driverManager.startBrowser, which is what we call from a before hook in each test, you’ll see how we reuse the browser across different tests.

There’s one additional thing you need to do which is close the browser down at the very end. We use a separate file after.js to do this, which we pass to Mocha when running any test.

AMA: JS vs Ruby

Butch Mayhew asks…

I have noticed you blogging more about JS frameworks. How do these compare to Watir/Ruby? Would you recommend one over the other?

My response…

I had a discussion recently with Chuck van der Linden about this same topic as he has a lot of experience with Watir and is now looking at JavaScript testing frameworks like I have done.

Some Background

WordPress.com built an entirely new UI for managing sites using 100% JavaScript with React for the main UI components. I am responsible for e2e automated tests across this UI, and whilst I originally contemplated, and trialled even, using Ruby, this didn’t make long term sense for WordPress.com where the original WordPress developers are mostly PHP and the newer UI developers are all JavaScript.

Whilst I see merit in both views: I still think having your automated acceptance tests in the same language as your application leads to better maintainability and adoptability.

I still think writing automated acceptance tests in Ruby is much cleaner and nicer than JavaScript Node tests, particularly as Ruby allows meta-programming which means page objects can be implemented really neatly.

The JavaScript/NodeJS landscape is still very immature where people are using various tools/frameworks/compilers and certain patterns or de facto standards haven’t really emerged yet. The whole ES6/ES2015/ES2016 thing is very confusing to newcomers like me, especially on NodeJS where some ES6+ features are supported, but others require something like Babel to compile your code.

But generally with the direction ES is going, writing page objects as classes is much nicer than using functions for everything as in ES5.

Whilst there’s nothing I have found that is better (or even as good) in JavaScript/Mocha/WebDriverJS than Ruby/RSpec/Watir-WebDriver, I still think it’s a better long term decision for WordPress.com to use the JavaScript NodeJS stack for our e2e tests.

Checking an image is actually visible in WebDriverJs

I recently discovered a gap in one of my e2e automated tests where I was checking the existence of an uploaded image in the DOM, but not that the image was actually displayed.

driver.isElementPresent( By.css( `img[alt='upload.jpg']` ) ).then( function( present ) {
  assert.equal( present, true, 'Image not displayed' );
} );

If the DOM has a reference to the image, but it isn’t actually rendered this test will pass. This isn’t ideal.

I remembered my post about how to check that an image is actually rendered using WebDriver in C# and so I used the same JavaScript script which WebDriverJs sends to the driver:

driver.findElement( By.css( `img[alt='upload.jpg']` ) ).then( function( element ) {
  driver.executeScript( 'return (typeof arguments[0].naturalWidth!=\"undefined\" && arguments[0].naturalWidth>0)', element ).then( function( present ) {
    assert.equal( present, true, 'Image not displayed' );
  } );
} );

This works a treat. I’ve moved it into a helper function so I can use this anywhere without repeating it also.