The importance of testability, or how to avoid the nastiest xpath selector known to mankind

Our automated end-to-end tests for WordPress.com include searching for a domain and selecting a .com result from a screen that looks like this:

find a domain component.png

We want our tests to be consistent, so even though we search for a different address each time, we want to select the ‘.com’ address result each time by clicking ‘Select’ next to that result. But how do we click the correct ‘Select’ button when there’s so many of them?!?

If we have a look at the DOM, we can see how each of these elements are rendered:

<button class="button domain-suggestion__select-button add is-primary" data-reactid=".2.1.$=1$domains.1.1.0.$domain-search-results.1.0:$mycoolwebsiteisrad=1com.1.0">Select</button>

We can’t use the class such as domain-suggestion__select-button since that isn’t unique to our .com result Select button. We could click the second Select button but that’s not right since we don’t know the results are always going to be in the same order, or how many there are.

You might notice there’s a data-attribute provided by React (our front end UI library) which contains this: data-reactid=".2.1.$=1$domains.1.1.0.$domain-search-results.1.0:$mycoolwebsiteisrad=1com.1.0" which looks a bit messy and ‘generated’ but it contains our domain which we’re looking for, well kinda, like this: mycoolwebsiteisrad=1com

So we can use a css selector to select this button in JavaScript:

selectDotComAddress( dotComAddress ) {
  const reactDataAttr = dotComAddress.replace( '.', '=1' );
  const selector = By.css( `button[data-reactid*="${reactDataAttr}"]` );
  this.driver.findElement( selector ).click();
}

This does a fuzzy match on the .com address which we’ve made to look like the format it uses, which works a treat, it’s not great, but not too bad. It’ll do.

That is until we decide to upgrade to React 15 which includes this in its release notes:

“We are now using document.createElement instead of setting innerHTML when mounting components. This allows us to get rid of the data-reactid attribute on every node and make the DOM lighter.”

So the method we were using the locate our button will no longer work as that element won’t have our data attribute.

So taking this into consideration we can look at the DOM a little bit broader:

<div class="domain-suggestion card is-compact">
<div class="domain-suggestion__content">
<h3>
<!-- react-text: 463 -->mycoolwebsiteisrad.com<!-- /react-text --><!-- react-empty: 464 --></h3>
<div class="domain-product-price">
<span class="domain-product-price__price">
<!-- react-text: 467 -->A$24.00 <!-- /react-text --><small>/year</small></span>
</div>
</div>
<div class="domain-suggestion__action">
<button class="button domain-suggestion__select-button add is-primary">Select</button>
</div>
</div>

So we can see the text we are looking for is in a h3 element, but css selectors don’t support selecting by text value. So we can write an xpath to select by text, but the text isn’t on the button, it’s on a sibling to the button in the same card container.

So we end up with one of the nastiest xpath selectors known to mankind:

selectDotComAddress( dotComAddress ) {
  const selector = By.xpath( `//h3[text()='${ dotComAddress }']/../..//div[contains(concat(' ', @class, ' '), ' domain-suggestion__action ')]/button` );
  this.driver.findElement( selector ).click();
}

Translating this xpath into English becomes: ‘find any h3 element where the text is our .com address, then look at the parent of that element, and the parent of that parent element (which gives us the card), then look for any child div of that card element whose class contains domain-suggestion__action, and finally find the button in that div’ 🙄

At this point, I can’t see a better solution, except, we could make our application more testable.

Instead of relying on React’s data attributes, which are fuzzy and are about to be removed anyhow, let’s add our own data attribute. So I created a pull request to add a data-e2e-domain attribute to each button which will give us the domain it is selecting. I asked for some help adding a unit test for the attribute (to ensure it doesn’t inadvertently removed) and released this to Production myself yesterday.

So our DOM now looks like

<div class="domain-suggestion card is-compact">
<div class="domain-suggestion__content">
<h3>
<!-- react-text: 463 -->mycoolwebsiteisrad.com<!-- /react-text --><!-- react-empty: 464 --></h3>
<div class="domain-product-price">
<span class="domain-product-price__price">
<!-- react-text: 467 -->A$24.00 <!-- /react-text --><small>/year</small>
</span>
</div>
</div>
<div class="domain-suggestion__action">
<button class="button domain-suggestion__select-button add is-primary" data-e2e-domain="mycoolwebsiteisrad.com">Select</button>
</div>
</div>

Note the data-e2e-domain attribute on the button. This means we can use this selector:

selectDotComAddress( dotComAddress ) {
  const selector = By.css( `button[data-e2e-domain="${ dotComAddress }"]` );
  this.driver.findElement( selector ).click();
}

Simple, straight to the point, precise, easy to understand, and testable.

We have about nine other places in our e2e tests were we are using xpath selectors for similar reasons so I will be working on pull requests to add data-e2e- attributes to our user interface to continue to make it more testable. Testability FTW.

Author: Alister Scott

Alister is an Excellence Wrangler for Automattic.

10 thoughts on “The importance of testability, or how to avoid the nastiest xpath selector known to mankind”

  1. This is a never ending battle and a frustrating argument to have with designers and developers who insist on ‘clean markup’. You did a great job of showing how brittle the original selector would be vs. your data tag solution.

    Liked by 1 person

  2. I haven’t seen better approach than using data attributes if you really need to test your application directly through UI :) XPath approach was the most popular one before data attributes were introduced. Indeed it was very fragile way of finding element you wanted to use in your test. I also remember that QA teams tried to mitigate that using class names to simplify XPath expressions, but it had its own downsides. In many cases you ended up with HTML classes that were obsolete, but it was hard to remove them because you had to check if they are used in CSS files and also in end to end tests.

    Have you considered testing WordPress.com with bypassing HTML layer at all? :) We are moving very quickly to Redux. I anticipate that at some point we could expose API that would allow to call the same Redux action that is executed whenever you click on a button or another HTML element. Just an idea to think about. I’m a JS wrangler, so I don’t even know if that makes any sense in that context. I only recently started following your blog and I’m happy to discover more how our test automation at Automattic works in details :D

    Liked by 1 person

  3. It looks likely there is a related accessibility issue here. With every button having the same label, folks tabbing through controls are going to get ‘Select’, but select what? On high zoom levels or with a screen reader the surrounding context may not be easy to see. The same kind of issue that makes it difficult to pinpoint with a selector.

    There should be nice solutions to both issues. Rather than using a custom data attribute, use an aria label or title or other feature that describes the button and makes a nice selector hook for testing! Double win :)

    Liked by 2 people

    1. Actually I tested this in a screen reader and the domain and price elements are read by the screen reader just before each Select button, so aria is not needed in this case where it would duplicate this other content.

      Like

      1. That sounds fine Alister!

        It’s when people use tab to jump between buttons that I’d worry they may skip that context. Generally I try to avoid a bunch links, buttons or other controls that use the same text (e.g. “Read more” links which could be “More name_of_topic”). Something like this button I might put the extra text inside the button but hidden (which may help screenreaders but not zoom users—maybe a tooltip for them). Actually, thinking about the html I’d probably want the value in there (maybe react uses its own hidden magic), something like:

        <button type="button" value="mycoolwebsiteisrad.wordpress.com">Select<span class="sr-only"> mycoolwebsiteisrad.wordpress.com</span></button>

        Your solution sounds good though, just sharing thoughts ~:)

        Liked by 2 people

  4. interesting that this is still “new” most individuals, I gave a lightning talk @ the Selenium Conf15 over data-* attributes

    Like

Comments are closed.