Do you actively close bugs because they reach a certain age?
One of the (many) things I love about Automattic is the attention that is given to bug triage. Bug triage is the habit of continually grooming our bug lists to ensure they are constantly relevant, updated and reflective of the current state of our products. A benefit of this is that an up-to-date and prioritized bug list translates directly into a backlog of maintenance work items for a product development team.
The idea of maintaining a list of bugs as a credible source of truth was mentioned by Rands in Repose as a habit from his Apple days:
“My favorite internal application at Apple is a product called Radar. It’s a Cocoa application that served as our bug tracking system, and if you wanted to know what was going on regarding a specific application at Apple, you went to Radar.”
“Radar became a powerful tool because it became a credible source of truth regarding the product.”
One habit Automattic encourages but I personally don’t like doing in bug triage is actively closing older bugs because they reach a certain age.
The first reason that comes to mind on why I don’t like closing older bugs is because this means the bug log could no longer be considered a credible source of truth, as there are current production issues that are no longer represented as open bugs.
Over the course of my career I’ve seen many reasons why I personally don’t like closing old bugs:
Closed GitHub Issues are Second-Class Citizens
- Searching GitHub Issues automatically defaults to searching only open issues, so if there’s a known active issue, but it was closed due to age, there’s a good chance it won’t be easily found and it may be raised as a new bug which duplicates the old closed one.
- Even if you explicitly include closed bugs in search results, it’s not easy (without adding custom labels) to know at a glance which GitHub issues were closed due to being fixed versus being closed because they were too old. I’ve recently started working on janitorial pull requests to fix known production issues, and I’ve found I’m limited to fixing the production bugs that are still left open as the bugs closed due to age are too hard to find.
Closing Low Priority Bugs
- Often older low-priority bugs are closed early because when looking at them individually the impacts are low. But I’ve seen lots of smaller, low priority bugs snowball together to create a larger poor user experience.
- Often bugs closed for inactivity are ‘edge cases’. Another way to look at edge cases is as stress cases for our users. As per the WordPress.org Flow Glossary/Design for Real Life Book (emphasis added):
But making digital products friendly isn’t enough to make them feel human.
Real life is complicated. It’s full of joy and excitement, sure, but also stress, anxiety, fear, shame, and crisis. We might experience harassment or abuse, lose a loved one, become chronically ill, get into an accident, have a financial emergency, or simply be vulnerable for not fitting into society’s expectations.
None of these circumstances is ideal, but all of them are part of life—and, odds are, your site or product has plenty of users in these moments, whether you’ve ever thought about them or not.
Our industry tends to call these edge cases—things that affect an insignificant number of users. But the term itself is telling, as information designer and programmer Evan Hensleigh puts it: “Edge cases define the boundaries of who [and] what you care about”. They demarcate the border between the people you’re willing to help and the ones you’re comfortable marginalizing.
That’s why we’ve chosen to look at these not as edge cases, but as stress cases: the moments that put our design and content choices to the test of real life.
It’s a test we haven’t passed yet. When faced with users in distress or crisis, too many of the experiences we build fall apart in ways large and small.
Instead of treating stress situations as fringe concerns, it’s time we move them to the center of our conversations—to start with our most vulnerable, distracted, and stressed-out users, and then work our way outward. The reasoning is simple: when we
make things for people at their worst, they’ll work that much better when people are at their best.
Won’t Fix Now is Different to Won’t Fix Forever
It’s easy to close a bug saying “won’t fix”, but won’t fix now is different to won’t fix forever. Priorities change. In my career, I’ve seen “won’t fix” bugs suddenly become fix right now as our CEO personally experienced the bug.
Sending the Wrong Message to Bug Reporters
Raising bugs that are never fixed can be demotivating for bug reporters, but having bugs closed just because they’re old increases this emotional impact on bug reporters. I recently saw this comment on a GitHub Issue for a product I work on:
What I Propose Instead
Instead of closing old bugs, let’s use a label to identify them as old; this makes it easy to find old bugs and fix them. It would be great if our tools like GitHub Issues allowed us to automatically label older bugs, but in the meantime we could do something like this during bug triage.
Thanks to my team lead and bug triage extraordinaire Lance Willett for reviewing a draft of this post.