AMA: JavaScript in Test Automation

Max asks…

First of all thanks for sharing all your insights on this blog. I regularly try to come back to your blog and I helped me grow as a QA Engineer quite a lot.
I wanted to ask you on your opinion on a seemingly new generation of test tools for the web that are written in JavaScript to help deal with all the asynchronicity on modern front end frameworks. Our company is in the process of redesigning our website and it seems that it just gets harder and harder to deal with all the JavaScript in test automation. I recently started looking into some other tool like, but as of now they still seem quite immature.
Could you help me and point me in the right direction?

My response…

I think there’s two parts to the JavaScript asynchronicity issue which are not necessarily related.

Modern JavaScript front-end web frameworks (like React and Angular) are designed to be asynchronous and fast and therefore this can cause issues when trying to write automated tests against them. I don’t believe writing automated tests in an asynchronous way actually makes the tests easier to write, or more maintainable or robust, but just a result of writing these tests in the same way the web frameworks work.

You can write synchronous tests (like using Watir/Ruby) against asynchronous web interfaces, you just need to use the waiting/polling mechanisms (or write/extend your own) – the same as you need to do in asynchronous test automation tools.

We choose WebDriverJS for automated end-to-end testing of our React application as it was the official WebDriver for JavaScript project and seems to be a good choice at the time. I somewhat regret that decision now as using a synchronous third party implementation like seems like an easier way to write and maintain tests.

I have tried to use but the way it controls sites (through proxies) has (current) limitations like not working on iFrames and cross-domains which are deal-breakers for our end-to-end testing needs at present.

If you don’t need to write your end-to-end tests in JavaScript I’d say avoid it unless absolutely necessary and stick to another non-asynchronous programming language.

I’m glad I’ve been able to help you grow over the years.

AMA: Clicking a non-visible element in Watir

Mike asks…

Is there any way to make Watir click a link/button that is not visible? I wanted to switch from Capybara/CapybaraWebKit (which allows clicking non-visible elements) but I am stuck since Watir always times out on the click attempt.

My response…

The easiest way to do this is just to execute a click in JavaScript on the object itself.

In Watir, this looks something like:

browser.execute_script( "return arguments[0].click();", => 'blah')

Hope this helps!

AMA: Misc

Amruth asks…

Hi Scott, I am Amruth and I am from INDIA. I would like to automate my project by using Watir with Ruby. Plz brief me about Watir and Watir-Webdriver.

My response…

I suggest you visit and follow the guides. Have fun.

Victor Hugo dos Santos asks…

How we can run our tests parallel when we use Cucumber-JVM?

My response…

I haven’t done this personally, but these instructions look helpful.

Viktoriya Musiy asks…

Just to add: I’ve tried your instruction Unfortunately it does not work on my macbook. I get the error message: You don’t have write permissions for the /usr/bin directory. Viktoriyas-MBP:~ viktoriyamusiy$ sudo gem install rspec –no-rdoc –no-ri Password: ERROR: While executing gem … (Gem::FilePermissionError) You don’t have write permissions for the /usr/bin directory. Viktoriyas-MBP:~ viktoriyamusiy$

My response..

That post is over seven years old so I am not surprised it no longer works. Please try for up to date instructions.

Niopvief asks…

Food of fried tomatoes like absolutely everything. Guests usually require recipe are surprised mastery hostess. Even guys with pleasure eat fried tomato in breadcrumbs. Tomatoes very good combined are combined with many products. Garlic adds sharpness. Cheese brings ruddy crust. Italian herbs turn roasted tomatoes into a dish from a restaurant. Omelette with tomatoes – easy and nutritious dish for breakfast. Make including sandwiches with fried tomatoes. Cooking tomatoes simply. For cooking use butter. In olive oil – fewer calories. It is also combined mixed with fried cubes tomatoes.

My response…

I am not a big tomato fan but my wife likes them. Thanks for the tips – Merry Christmas

AMA: Difference between explicit and fluent wait

Anonymous asks…

What is the difference between Explicit wait and Fluent wait?

My response…

I hadn’t heard of fluent waiting before, only explicit and implicit waiting.

From my post about Waiting in C# WebDriver:

Implicit Waiting

Implicit, or implied waiting involves setting a configuration timeout on the driver object where it will automatically wait up to this amount of time before throwing a NoSuchElementException.

The benefit of implicit waiting is that you don’t need to write code in multiple places to wait as it does it automatically for you.

The downsides to implicit waiting include unnecessary waiting when doing negative existence assertions and having tests that are slow to fail when a true failure occurs (opposite of ‘fail fast’).

Explicit Waiting

Explicit waiting involves putting explicit waiting code in your tests in areas where you know that it will take some time for an element to appear/disappear or change.

The most basic form of explicit waiting is putting a sleep statement in your WebDriver code. This should be avoided at all costs as it will always sleep and easily blow out test execution times.

WebDriver provides a WebDriverWait class which allows you to wait for an element in your code.

As for fluent waits, according to this page it’s a type of explicit wait with more limited conditions on it. I don’t believe WebDriverJs supports fluent waits.

AMA: Dealing with Docker

Howdy! First, thanks to Alister for asking me to guest-post on his blog. I’m always excited to talk about Docker and its potential to solve all the world’s problems 😉.  He asked me to take on the following question from his AMA:

Alister, What are your thoughts on how containerization should fit into a great development and testing workflow? Have you got behind using Docker in your day to day? Thanks!

One of the oldest problems in software development and testing is that a developer writes code on their desktop, where everything works flawlessly, but when it’s shipped to the test or production environments it mysteriously breaks. Maybe their desktop was running a different version of a specific library, or they had unique file permissions enabled. When used correctly, Docker eliminates the “works on my machine” concern. By packaging the runtime environment configuration along with the source code you ensure that the application executes the same in every instance. And just as important, changes to that configuration are logged and can be easily reverted.

Another concern is how to test specific behavior that only gets executed when your application is running in the production environment. By putting all of your application and test servers in individual containers, you can easily connect them on their own private network and just tell the application that it’s running in production. Obviously this is application unique, and building a copy of the production environment presents its own challenges, but at the core it’s definitely doable within a Docker infrastructure. The important thing is that a network of containers is isolated, so you can do things like set machine hostnames to exactly match their production counterparts without worrying about conflicts.

It all just boils down to consistency…if you can ensure that your developer is writing code against the same configuration as your test environment, which is the same configuration is production – everybody wins.

The second part of the question is a little trickier – Have you got behind using Docker in your day to day?

In some ways the answer is yes. The main application that we test is the Calypso front-end to, which itself is built and runs inside Docker. Our core end-to-end tests also run in a custom Docker container on CircleCI 2.0, so we can define exactly what version of NodeJS and Chrome we’re using to test with. However, some of our other test sets (such as certain WooCommerce and Jetpack tests) still run using the default CircleCI container. And as far as I know nobody on our team actually uses that container for developing tests locally, we typically just run directly on our laptops. The CI server is the first place that actually executes via Docker.

The other piece that’s missing for a full Dockerization of the our test setup is that our Canary tests run against the custom setup ( rather than directly building/running Calypso side by side in a container. It’s something I’d like to pursue updating at some point, but in the interim the existing setup works great…and most importantly it’s already built and working, allowing us to focus on other things.

So the long story short here is that containerization is a great technology, and has a ton of potential for solving problems in the dev/test world. We’re just scratching the surface of that potential at Automattic, but even the limited use we’re giving it right now is beneficial and I plan on continuing to dig deeper.

AMA: Moving automated tests from Java to JavaScript

Anonymous asks…

I am currently using a BDD framework with Cucumber, Selenium and Java for automating a web application. I used page factory to store the objects and using them in java methods I wanted to replace the java piece of code with javaScript like mocha or webdriverio. could you share your thoughts on this? can I still use page factory to maintain objects and use them in js files

My response…

What’s the reasoning for moving to JavaScript from Java? Despite having common names, there’s very little otherwise in common (Car is to Carpet as Java is to JavaScript.)

I wouldn’t move for moving sake since I see no benefit in writing BDD style web tests in JavaScript, if anything, e2e automated tests are much harder to write in JavaScript/Node because everything is asynchronous and so you have to deal with promises etc. which is much harder to do than just using Java (or Ruby).

Aside: I still dream of writing e2e tests in Ruby: it’s just so pleasant. But our new user interface is written extensively in JavaScript (React) so it makes sense from a sustainability point of view to use JS over Ruby.


AMA: Trunk Guardian Service?

Sue asks…

I read a LinkedIn blog post from 2015 by Keqiu Hu from LinkedIn about flaky UI tests. He explains how they fixed their flaky UI tests for the LinkedIn app. Among other things they implemented what they called the “Trunk Guardian service” which runs automated UI tests on the last known good build twice and if the test passes on the first run but fails on the second it is marked as ‘flaky’ and disabled and the owner is notified to fix it or get rid of it. I wondered what your thoughts were on such a “Trunk Guardian service” – if the culture / process was in place to solve the other issues that create flaky tests, could such a thing be worth the effort to implement? Article: Test Stability – How We Make UI Tests Stable

My response…

Continue reading “AMA: Trunk Guardian Service?”