Soap Opera Testing
A couple of years back I read about soap opera testing, a derivative of scenario testing that involves complicating a test scenario to the point of it resembling a soap opera plot. Most of the material on soap opera testing is now unavailable, but Eric Petersen wrote about a real life example of a Department Store accidently giving away $800 to $4000, and the unintended consequences including physical violence in the carpark.
When conducting usablity testing, it is super critical to have non leading goals and tasks, otherwise the participant will simply do what you ask and won’t reveal any usability issues. I found this example about usability testing of the IKEA site the most useful to explain why.
Years ago, we helped with a study of Ikea.com, looking at how people found products on the site. When we got there, they’d already started the testing process and were using tasks like “Find a bookcase.” Interestingly, every participant did exactly the same thing: they went to the search box and typed “bookcase”.
Upon our suggestion, the team made a subtle change to the instructions they were giving their participants: “You have 200+ books in your fiction collection, currently in boxes strewn around your living room. Find a way to organize them.”
We instantly saw a change in how the participants behaved with the design. Most clicked through the various categories, looking for some sort of storage solution. Few used Search, typing in phrases like “Shelves” and “Storage Systems”. And, nobody searched on “bookcase”.
The way you design tasks could have a dramatic outcome on the results, without you even realizing it.
Acceptance Test Driven Development
I’ve seen acceptance test driven development (ATDD) frequently implemented incorrectly, especially with developers writing acceptance tests in FitNesse. I think it’s because the developers I have seen have a natural tendency to make things technical. This article, about the misuse of Cucumber, represents my thoughts correctly, it’s a design vs implementation problem, and it’s the fault of the ATDD tools as they encourage it.
You’re Cuking It Wrong
Opinions on cucumber seem to be divided in the Ruby community. Here at Elabs we’ve been using cucumber to fantastic success on all of our projects for more than a year. At the same time Steak and projects like it seem to be gaining traction; some people are seemingly frustrated and fed up with cucumber.
So where does this gulf of experiences come from, why is cucumber loved by some and hated by others. At the risk of over-generalisation and mischaracterisation I recently came up with a theory: the cucumber detractors are not using cuke the way it was intended.