AMA: automation hooks bad practice?

Miguel Juteau asks…

Some people contend that deploying markup code in production that contains automation hooks (e.g. Html5 data tags) is bad practice (for the same reason that we don’t deploy unit tests to production: it doesn’t serve the customer).
However it seems very desirable to build a solid contract for automation selectors that does not get affected by UI changes.

Any thoughts on that ?

My response…

I obviously don’t believe it’s bad practice to add testability features, such as HTML5 data attributes, to production code, that don’t directly serve the customer, otherwise I wouldn’t have shared an example of how I did this yesterday. I don’t like to encourage bad practices 🙂

However, I can see two potential reasons why people could believe this to be bad practice:

  1. Adding additional attributes to HTML5 elements adds to the size of the page, hence slows down the page for customers; and
  2. There is a potential opportunity cost of spending development resources on testability when they could be working on direct customer-facing functionality instead.

My views on these two potential arguments are:

  1. The example I gave yesterday of adding a data attribute would typically add ~50 Bytes (uncompressed) of data to the specific page. The current page size is ~313KB (~320,512 Bytes), which means the page size has increased by 0.015% by adding these attributes. I believe the testability benefit outweighs a 0.015% increase in page size.
  2. My primary role is that of testing, so when I add testability features to our application I am doing that as part of my role as a tester. Even if this were done by a development resource, I could argue that time spent adding testability features by a developer makes our application more testable, which means it’s more efficient and effective to test, which means it’s easier to find bugs, which means there will be less bugs. It is of my opinion that a customer who sees less bugs is being served well as a customer. I can’t see any point in deploying unit tests to production as they’re not a ‘deployable’ artifact (and we run them in a Continuous Integration environment before every release); however, we do run our automated end-to-end tests against production to ensure that we have a consistent end user experience.

I have also seen situations where adding testability features, such as progress indicators to make our automated tests more deterministic, also indirectly resulted in a large increase in usability for our customers by meeting this usability heuristic: “The system should always keep users informed about what is going on, through appropriate feedback within reasonable time”.

Author: Alister Scott

Alister is an Excellence Wrangler for Automattic.

2 thoughts on “AMA: automation hooks bad practice?”

  1. Alister,
    Totally and completely agree with your points. Both the cost and impact to the code and project itself is miniscule, but the payoff to the team/company and its end-users is huge. I had this happen on a project where the Development Lead whined, and I do mean whined, about building in testability and how it would take too much time and blah, blah, blah. One of the other developers on the project was working with his Unit Tests and had already built in what he needed. He asked how he could help me and the automation team, I explained to him what I was looking for (static values for Name or ID properties in the objects and page) and he told me it would take a 1 to 1 1/2 days to get it done. He came back to me the next morning and told me it was done, took him 1/2 a day once he got into the code base.

    A couple of days later after working with the new build and hooking things up we were in our Standup meeting, and I mentioned the need for testability (yes, I setup the Dev Lead) again. The Dev Lead again protested, on which the other developer (who had done the work) spoke up and said it was already done and that it only took him 1/2 day to complete it all. The Dev Lead stood there just smoking (as in mad and embarassed) and dumbfounded. The Scrum Team Lead was blown away, but happy. I told the group that myself and my team were cranking away on automating tests and that the “testability” improvements made things easier and more maintainable.

    That discussion went up the ladder to higher powers (Director of Development, CTO, CEO) and they supported what had been done for adding in Testability to the code. The mandate came down to other development teams to do the same.

    Liked by 1 person

  2. Yeah, absolutely. It makes more sense to allow test hooks in production than to spend the overhead cost to remove them from production and keep them in development or else do without them altogether. It’s a purist viewpoint, and not a very effective one, that nothing should go into production unless it directly benefits the customer.

    Liked by 1 person

Comments are closed.