Lance Willett asks…
Do you think QA specialists are more effective embedded on product teams directly (working alongside dev and design) or on support teams (working alongside user-facing help teams)? Or something in between, not tied to any other teams?
This is a great question Lance, and something I have also been thinking a lot about lately.
When I first started working, ‘QA’ was a fairly lengthy process done by a different team, often located somewhere else (whether that be a different floor, building, or country), as the final step before we shipped the next version of a product.
As you can now imagine, this had quite a few downsides. The feedback time between when an issue was introduced and when it was discovered was often quite long, so it was a lengthy process to fix and re-test issues. There often was a culture of developer’s throwing things over the wall to QA, and a tendency for the QA team to act as gatekeepers between what developers were creating and what was being released to our users.
But one huge benefit, that is often forgotten and underrated, was that the QA team would be really good at systems thinking: understanding how all the different components and products worked together as whole for our users, and how any change would impact that.
I remember working on such a team straight out of University, and after a particularly large and impactful software release the entire QA team was asked to help answer help-desk calls. This was recognising that our QA team had great end-to-end systems knowledge and that we could understand user’s problems.
Two of the biggest changes in moving towards using agile software delivery models (over using the waterfall model) have been team composition and team location. Typically agile product teams are cross-functional, consisting of embedded design, development and QA folk(s), and co-located in the same physical work area.
This has tended to reduce the throwing things over the wall mindset since there’s much less time between a change being developed and tested, and iterating is encouraged, and also since the team is typically physically co-located there isn’t a wall to throw things over.
But, in my opinion, the biggest downside to embedded QA specialists on these cross-functional product teams is the tendency for these roles to have a much narrower view of QA.
Since a product team is typically responsible for development of a particular component or product, changes to these products can be developed, tested and released rapidly by the product team, which is great.
But the bigger questions about how these changes impact or fit into larger end-to-end user flows are often left unanswered.
One potential solution to this would be to, as you mentioned in your question, embedding QA specialists instead on user support teams, instead of product teams, since this ensures a user-centric, end-to-end approach to testing.
The downside I can see to this approach is it tends to be a more reactive approach to testing, focusing on what are the current user facing issues, over being a proactive approach in testing upcoming functionality, and assessing potential user impact.
This may lead to testing being too late in identifying issues after users have already been impacted by these changes.
So I believe the answer to this, like the answer to most problems, lies somewhere in between these two approaches.
My ideal scenario would be to have a small team of QA specialists with a mindset of systems-thinking and empathy to our users.
These specialists ‘float’ around working on different things. One day this may be working alongside user-facing help teams to see what the current issues impacting users are, and working to clearly articulate these, and make sure they are prioritised appropriately in the product roadmap.
Another day this may mean working on new and upcoming changes in product teams highlighting how these changes will impact end-to-end user flows, and generally pushing everyone to strive towards an excellent user experience.
I call these folk Quality Advocates.
This approach may be trickier to achieve in a physical office environment where various product teams and user-facing help teams may be in different buildings, or cities, which would mean it’s a bit trickier to ‘float’ around.
But in a fully distributed environment, like Automattic, this is much easier to achieve since all teams exist only in a virtual sense anyway.
I believe a hybrid approach may also work. At a previous organisation, we had a QA specialist on each product team, but, to ensure we still had an excellent user experience across all our products, I took on the role of end-to-end testing and quality advocacy as part of being the QA lead for the organisation.