Adventures of end-to-end automated web testing in Node.js

This is a presentation I gave at ATTAC in Melbourne on Friday 24th May 2019. The full Google Slides are available here.

Good morning everyone and thanks for having me along to speak.
My name is Alister Scott and I’m from Brisbane. I work as a QA generalist at a software company in Brisbane called Console, and I write a blog called WatirMelon. Today I’m going to be sharing my knowledge of end to end automated testing in Node.js. I’ve created very simple but working demos of the main tools I’ll be discussing today and I’ve put these on GitHub as separate repositories you can easily clone and play around with: github.com/alisterscott

Outside of work I enjoy hiking, often to the summits of mountains, and spending time with my wife and our three young kids.
In 2015 I started a paid trial at Automattic. My trial project – on which I would be assessed to gain full time employment – was to establish an automated e2e testing framework for WordPress.com – the first of its kind for Automattic. I quickly spun up a framework using Watir in Ruby – because that’s what I knew – and it worked. However I quickly gained some feedback that of the hundred+ developers at Automattic almost none knew Ruby and creating shared ownership for e2e tests would be a key measure of success so I had to rethink. At the time Automattic was moving from primarily developing in PHP towards Node.js meaning full stack JavaScript and it made sense that the e2e tests were also developed in Node.js.

This of course made my trial project a lot more difficult than I had originally thought as I had to teach myself Node.js and understand what testing tools existed in this space.

Fast forward to 2019 and earlier this year I decided to change jobs as I could no longer travel for work. When I started looking at job advertisements for testing and QE positions in Brisbane I noticed just how many mentioned Node.js as their technology stack of choice: in the 3.5 years at Automattic Node.js had become increasingly popular at other companies as well.
But automated e2e testing in Node.js was and is really hard. Much much harder than I was used to in Ruby and Watir where things just worked.

It’s slightly better in 2019 than 2015, however there are some reasons why it’s hard.
There’s also the paradox of choice when it comes to tooling specifically for e2e web testing. Searching for selenium and WebDriver on NPM provides a list of dozens of libraries – and the official Selenium bindings (WebDriverJs) don’t appear when searching for WebDriver. It’s all very confusing.

My aim today is to create some clarity in this space.
I believe there’s more e2e test tools in Node.js that something like Java or C#. I’ll talk about the four most popular/mature ones.
Cypress.io is better suited to component testing than true end-to-end testing. Cypress.io also doesn’t support cross-domain stuff – so beware if you’re doing anything like that.
You’ll need your own test runner and assertion library.
I’ve distilled these four tools down into an easy to read visual.
And an even easier to understand flow chart.

Where is the ‘story’ in user stories?

There’s an old Jerry Weinberg quote:

“no matter what the problem is, it’s always a people problem”

which pretty much describes every project I’ve worked on over the years. Lately, I’ve particularly noticed that most of the tough problems evident on projects are more people or business problems than technology problems, which makes me think it’s worthwhile for me to continue my exploration of the business/user end of my list of software development roles.

BA = Business Analyst
UX = User Experience
ET = Exploratory Tester
TT = Technical Tester
Dev = Software Developer

In this vein, I’ve recently been trying to understand how to better articulate user stories, in that one day I’d love to work as a business analyst.

Most nights I read stories to my (almost) three year-old son as a nice way to end the day. Lately I have been making up my own impromptu stories to keep things interesting. I have really enjoyed making up stories on the spot; I think I’d be a good BA.

But thinking about user stories along with bedtime stories immediately raises a question: where is the ‘story’ in user stories?

Most user stories at work sound something like this: “As a user, I want to log onto the system, so that I can access my information”. What a shitty story! Sorry, but seriously, if I told this story to my two year old son, he’d die of boredom!

I’ve spent a fair amount of time reading about user stories but I still can’t find out why they’re actually called stories, because I don’t think they are actual stories:

sto·ry/ˈstôrē/
Noun:
An account of imaginary or real people and events told for entertainment: “an adventure story”.

The closest thing I have found to actual user stories is the concept of ‘soap opera‘ testing scenarios which outline implausible yet possible scenarios:

“A man (Chris Patterson) and his wife (Chris Patterson) want to take their kids (from previous marriages, Chris Patterson, a boy, and Chris Patterson, a girl) on a flight from San Francisco to Glasgow to San Jose (Costa Rica) to San Jose (California) back to San Francisco. He searches for flights by schedule. He’s a Super-Elite-Premium frequent flyer, but he doesn’t want the upgrade that the system automatically grants him so that he can sit with his wife and kids in economy class. He requires a kosher meal, his wife is halal, the boy is a vegetarian, and the girl is allergic to wheat. He has four pieces of luggage per person (including two pairs of skis, three sets of golf clubs, two 120 lb. dogs, and three overweight suitcases), where his frequent flyer plan allows him (but only him) to take up to four checked items, but the others can only take two each. He gets to the last step on the payment page before he realizes that he has confused San Jose (California) for San Jose (Costa Rica), so the order of the itinerary is wrong. The airline cancels the flight after it has accepted his bags, and reroutes him on a partner. The partner cancels the flight (after it has accepted the bags) to San Jose (California) so it reroutes him to another competitor, who cancels the flight (after accepting the bags) to San Jose (Costa Rica) and reroutes him to another partner, who goes bankrupt after it has accepted the bags for the San Francisco flight.”

~ Michael Bolton

Now that’s a real user story!

So, I think we have two choices on the user stories front. We can either make our user stories actually like real juicy stories, or at least start calling them something else!