Should you raise bugs you don’t know how to reproduce?

At Automattic we’re always dogfooding; we’re always testing as we develop and therefore we’re always finding bugs.

When I come across one of these bugs I try to capture as much as I can about what I was doing at the time when it happened, and also capture some evidence (a screenshot, visual recording, error messages, browser console) of it happening. But even then on occasion I can’t work out how to reproduce a bug. 

This doesn’t mean the bug isn’t reproducible; it’s just that because there’s so many factors in the highly-distributed, multi-level systems that we develop and test, this means I don’t know how to reproduce these exact circumstances, therefore I don’t know how to reproduce this bug.

The exact set of circumstances that may cause a bug are never entirely reproducible; just like you can never live the exact same moment of your life ever again.

So, when this bug occurred, perhaps the server was under a particular level of load at that exact moment in time. Or one of the load-balancers was restarting, or maybe even my home broadband speed was affected by my children streaming The Octonauts in the next room.

So it can seem almost like a miracle when you do know how to reproduce a bug.

But what do you do with all of those bugs that you haven’t been able to reproduce?

Continue reading “Should you raise bugs you don’t know how to reproduce?”