The blurry line between test and development

One of the themes I talked about during my presentation in Wellington was the blurry line between test and development in a distributed environment like Automattic.

I was recently having trouble with a complex method in our e2e test page objects, so I used my skills as a developer and wrote a change to our user interface which adds a data attribute to the HTML element.

This meant our page object method immediately went from this:

Continue reading “The blurry line between test and development”

AMA: how to cope as a solo tester

Bodda asks…

hey, i like your blog, i’m reading your blog on a daily basis, and i just want to ask you what i can i do to enhance my skills, knowledge and to be a be good in functional testing (manual and automation) IF i’m the ONLY tester in my current company (performing all testing activities), but i feel that i have a mess in my head and lots to learn to be up to date with the last trends.
my question now “What to learn, When(everyday?) and How in case your are the only tester in your company”.

i hope to answer my question soon to make my head calm down

My response…

I have fond memories of a project where I worked as the solo tester on a software delivery team; we had something like 8 developers (including one lead who took on iteration management tasks) and me as a tester. That was it. I loved it because we established really good unhindered rapport with our business stakeholders, and I was always busy!

But getting back to your question: when I was working in that situation it was vital that we had developers working on as much test automation as possible, alongside functional development, since there was just too much functional testing to do for a single person also doing test automation. I found myself spending about 80% of my time just testing new functionality and spot-testing different browsers/devices etc. The remaining 20% I spent ensuring we had good regression test coverage through the automated tests that developers were writing and I was helping maintain. So first and foremost I would strongly encourage you to get the developers you work with to have as much responsibility as possible for automated tests.

If you have this in place you should be able to take a deep breath and spend more time doing quality testing. I have found the best way to learn is on the job, you’ll be in a good position to do this, and often you’ll learn best by making your own mistakes.

learning - 1
via The New Yorker

If you want to know more about how to become better at what you do, I’ve shared some tips in a previous answer. There’s also my Pride & Paradev book which I wrote whilst working as a solo tester on a team.

All the best.

Test your web apps in production? Stylebot can help.

I test in production way too much for my liking (more details in an upcoming blog post).


Testing in production is risky, especially because I test in a lot of different environments and they all look the same. I found the only way I could tell which environment I was in was by looking closely at the URL. This was problematic as it led to doing things in a production environment thinking I was using a pre-production or test environment – oops.

I initially thought about putting some environment specific code/CSS into our apps that made the background colour different for each environment, but the solution was complex and it still couldn’t tell me I was using production from a glance.

I recently found the Stylebot extension for Chrome that allows you to locally tweak styles on any websites you visit. I loaded this extension and added our production sites with the background colour set to bright red, so now I immediately know I am using production as it’s bright red, be extra careful.

Stylebot Example

I’ve also set some other environments to be contrasting bright colours (purple, yellow etc.) so I am know from a quick glance what environment I am using.

I like this solution as I haven’t had to change any of our apps at all and it works in all environments: which is just what I needed.

Do you do something similar? Leave a comment below.

Checking IS testing

The ‘testing vs checking’ topic has been in discussion for many years in the software testing community. Two very vocal participants are James Bach[1] and Michael Bolton[2].

“…we distinguish between aspects of the testing process that machines can do versus those that only skilled humans can do. We have done this linguistically by adapting the ordinary English word “checking” to refer to what tools can do.”

“One common problem in our industry is that checking is confused with testing.”

~ James Bach & Michael Bolton [1]

The issue I have with the checking vs testing topic is that it is dogmatic in implying that almost everyone around the world confuses checking with testing. Apparently unit testing is actually unit checking, the test pyramid is a check pyramid, test driven development is check driven development, and there is no such thing as automated testing, only automated fact checking.

“The “testing pyramid” is a simple heuristic that has little to do with testing. It’s called the testing pyramid because whomever created it probably confuses testing with checking. That’s a very common problem and we as an industry should clean up our language.”

~ James Bach [3]

We don’t need to clean up our language: we need to adapt, invent new language and move on.

The meaning of words aren’t static. ‘Literally’ originally meant in a literal way or sense but many people now use it to stress a point[4].  ‘Awful’ used to mean inspiring wonder but now has strong negative connotations[4]. Testing now means checking. Checking now means testing.

So perhaps instead of accusing everyone of confusing  ‘testing’ and ‘checking’, we move on, accept people call checking ‘testing’, and come up with another term to describe the value added human stuff we do on projects: you know, the questioning, studying, exploring, evaluating etc.

It’ll be much easier to educate everyone on some new terminology for pure human testing  exploratory testing based on intuition, instead of trying to get them to split their current view of testing in half and admit confusion on their behalf.

[1] Testing and Checking Refined: James Bach – 26 March 2013
[2] On Testing and Checking Refined: Michael Bolton – 29 March 2013
[3] Disruptive Testing Part 1: James Bach – 6 Jan 2014
[4] From abandon to nice… Words that have literally changed meaning through the years

The new QA: the Quality Advocate

“Quality is not an act, it is a habit.”
~ Aristotle

When I was recently writing about Quality Guardians/Gatekeepers for Pride and Paradev, I asked myself what does the term Quality Assurance actually mean?

In the world of software testing, there’s a lot of contention over the use of the term QA instead of testing. Often testers consider themselves to be Quality Assurance (QA) even though they don’t ensure quality, they just test it.

It actually makes more sense to call independent software testing after development Quality Control (QC) as it is simply verifying quality rather than ensuring it.

As previously stated, I believe that is is no longer sufficient to test quality in; agile software delivery teams need to build it in. A person performing a Quality Assurance role in an agile team can work closely with the team to strive to build quality in, but still can’t ensure quality actually exists (programmers can still check in bad code).

So, I propose we rename QA to mean Quality Advocate.

A Quality Advocate (QA) in an agile team advocates quality. Whilst their responsibilities include testing, they aren’t limited to just that. They work closely with other team members to build quality in, whether that be though clear, consistent acceptance criteria, ensuring adequate automated test coverage at the unit/integration level, asking for dev box walkthrough, or encouraging collaboration/discussion about doing better testing within the team.

Whilst the Quality Advocate promotes quality as part of their role: quality is everyone’s responsibility.

The benefit of doing this is that testing becomes more efficient for the Quality Advocate and a better product is produced overall as quality has been built in from the beginning.

“Quality means doing it right when no one is looking.”
~ Henry Ford

Addendum: Whilst writing this post, a colleague of mine suggested I use the term Quality Activist instead of Advocate. Whilst I like the sound of it, I think it’s a little too extreme: I want to promote quality, just not tie myself to office furniture over it.

Are software testers the gatekeepers or guardians of quality?

This post is part of the Pride & Paradev series.

Are software testers the gatekeepers or guardians of quality?

Software Testers are the Guardians and Gatekeepers of Quality

“Quality is value to some person.”

~ Jerry Weinberg

Working as a sole tester in a small agile team, you are the guardian of quality. You care about quality and it’s your job to fight the right fight to ensure it prevails. As Jerry Weinberg says: quality is value to some person, you need to ensure that value is realized.

User stories aren’t ‘done’ until you’ve tested each of them, which means you get to provide information to the Product Owner about each of them. You define the quality bar and you work closely with your team and product owner to strive for it.

You’ll soon realize that’s it’s better to start building quality in than testing it in, so you can make sure there are clearly defined acceptance criteria which have been marked as completed so that testing is more focused: efficient and effective. You’ll work with the programmers to make sure that as many acceptance tests are automated along side the code so that the regression testing burden is lessened each time a story is delivered.

One way to build quality into your development process to introduce some humorous passive-aggressive yet lighthearted signage around your story wall:

Tick the Acceptance Criteria

Whilst your Product Owner ultimately wants a great product, you’re working with the programmers closely in your team to ensure this happens on a day to day basis. You’re the guardian of quality and they’ll respect you for making them look good.

Software Testers aren’t the Guardians and Gatekeepers of Quality

Whilst you think you may define the quality of the system, it’s actually the development team as a whole that does that. Developers write the good/poor quality code.

Whilst you can provide information and suggestions about problems: product owners can and should overrule you: it’s their product for their business that you’re building: you can’t always get what you consider to be important, often business decisions trump technical ones.

You’re not perfect. Everyone is under pressure to deliver and if you act like an unreasonable gatekeeper of quality, you’ll quickly gain enemies or have people simply go around or above you.

And that’s no fun.