AMA: automation hooks bad practice?

Miguel Juteau asks…

Some people contend that deploying markup code in production that contains automation hooks (e.g. Html5 data tags) is bad practice (for the same reason that we don’t deploy unit tests to production: it doesn’t serve the customer).
However it seems very desirable to build a solid contract for automation selectors that does not get affected by UI changes.

Any thoughts on that ?

My response…

Continue reading “AMA: automation hooks bad practice?”

Three software testing links

Soap Opera Testing

A couple of years back I read about soap opera testing, a derivative of scenario testing that involves complicating a test scenario to the point of it resembling a soap opera plot. Most of the material on soap opera testing is now unavailable, but Eric Petersen wrote about a real life example of a Department Store accidently giving away $800 to $4000, and the unintended consequences including physical violence in the carpark.

Usability Testing

When conducting usablity testing, it is super critical to have non leading goals and tasks, otherwise the participant will simply do what you ask and won’t reveal any usability issues. I found this example about usability testing of the IKEA site the most useful to explain why.

Years ago, we helped with a study of, looking at how people found products on the site. When we got there, they’d already started the testing process and were using tasks like “Find a bookcase.” Interestingly, every participant did exactly the same thing: they went to the search box and typed “bookcase”.

Upon our suggestion, the team made a subtle change to the instructions they were giving their participants: “You have 200+ books in your fiction collection, currently in boxes strewn around your living room. Find a way to organize them.”

We instantly saw a change in how the participants behaved with the design. Most clicked through the various categories, looking for some sort of storage solution. Few used Search, typing in phrases like “Shelves” and “Storage Systems”. And, nobody searched on “bookcase”.

The way you design tasks could have a dramatic outcome on the results, without you even realizing it.

Acceptance Test Driven Development

I’ve seen acceptance test driven development (ATDD) frequently implemented incorrectly, especially with developers writing acceptance tests in FitNesse. I think it’s because the developers I have seen have a natural tendency to make things technical. This article, about the misuse of Cucumber, represents my thoughts correctly, it’s a design vs implementation problem, and it’s the fault of the ATDD tools as they encourage it.

You’re Cuking It Wrong

Opinions on cucumber seem to be divided in the Ruby community. Here at Elabs we’ve been using cucumber to fantastic success on all of our projects for more than a year. At the same time Steak and projects like it seem to be gaining traction; some people are seemingly frustrated and fed up with cucumber.

So where does this gulf of experiences come from, why is cucumber loved by some and hated by others. At the risk of over-generalisation and mischaracterisation I recently came up with a theory: the cucumber detractors are not using cuke the way it was intended.

Captchas on LinkedIn

I’ve noticed last week that LinkedIn started displaying a CAPTCHA every time I signed in. I thought it may have just been my account, so I asked around and realized they were appearing for everyone I work with. It’s the first time I have seen CAPTCHAs in use for each sign in, as opposed to each sign up. In my opinion a CAPTCHA on sign in is over the top, and the particular CAPTCHA implementation LinkedIn used was too cryptic, and in discussing this with colleagues I found it was not uncommon to try three or more attempts before being able to log in. Making it very difficult for users to log into your product or service means they’ll use it less. To quote Steve Krug who quotes his wife in Don’t Make Me Think:  “if something is hard to use, I just don’t use it as much”.

I was slightly relieved today to see that LinkedIn changed their CAPTCHA images to the popular reCAPTCHA. But after the first CAPTCHA quickly expired?!? (maybe to quickly timeout human solvers), I then realized what they were asking me to type in included a degree sign. Seriously? How do I type that in? There’s no key for that on my keyboard. Do you seriously want me to open up my character map and copy and paste it in? Will an average user even know how to do that?

I was beginning to think it was just me, that I am started to get too fussy and frustrated with technology, but a two second search on twitter confirms that’s not the case. Imagine having your customers saying these things about you:

  • @leightonhubbell: The @linkedin Captcha also doesn’t work too well on a mobile phone for logging in either. Clunky.
  • @NabilHarfoush: The Captcha of #LinkedIn log-in is driving me nuts. Now with Greek and accented letters!! WTF? #FAIL
  • @cabteix: @LinkedIn I’m considering leaving you because of that stupid captcha
  • @jgimer: RT @realex_tracy: Completing the CAPTCHA every time I log into LinkedIn is beginning to annoy me <- More motivation for evasion research. :)
  • @stevendiz: Hey #linkedin, I have no desire to fill in a damn captcha form EVERY time I use your site.
  • @highrockmedia: On @LinkedIn, why is there now a captcha after I authenticate with valid credentials? #uifail
  • @abishek: +1 RT @yaron_think_ver Recently I’ve been getting a captcha test each time I log in to #LinkedIn. What’s up with that? #Annoying
  • @MacBeer: @linkedin For what reason are you harassing clients with CAPTCHA “security” screens at each log in?
  • @guyindfw: Hey @LinkedIn – Captcha after signin is really, really, really, annoying. *sigh*
  • @DebbieHazelton: me too! RT @darrell: RT @TimNoonan: LinkedIn’s new useless verification CAPTCHA is discouraging me from using it so much :(

So, the big question is, are LinkedIn creating passionate users? I’ll let you decide.

Rocket Surgery Made Easy™ by Steve Krug

I am a really big fan of no-frills usability testing. Steve Krug is a pioneer in this space and I have a lot of respect for him.

I’ve had his first book “Don’t make me think” for a number of years, and have read it numerous times and I have enjoyed it immensely. I was very happy to find yesterday his second book titled: “Rocket Surgery Made Easy” has just been released. I ordered it straight away from The Book Depository (free worldwide shipping!) which is $30 cheaper than buying it locally. I can’t wait until it arrives so I can read it.

In the meantime I found a presentation Steve did in late 2008, which, whilst an hour long, is a great introduction into no-frills usability testing. I love how he doesn’t try to sell anything (actually the opposite, he gives away free PDFs), and  shares everything he knows. Amazing stuff.

[ ?posts_id=1566021&dest=-1]

Websites don’t need holidays

I am really dissapointed in the Australian Tax Office for taking down their entire web site for eleven days over Christmas/New Years 2008-2009. The website contains important reference information that should be accessible anytime. How can any organisation justify such a lengthy period of down time, especially for a web site with mostly static content for reference?


Twitter is over capacity

I like friendly error messages; I don’t think you can get much cuter than Twitter’s over capacity error message:

Twitter error message
Twitter error message

I just love how the whale is smiling. It makes it such a better error message. I don’t think many companies would be game to use a whale on their capacity error page (if they even had one) but in this case it just works.