Intentionally Disposable Software

Update 11 Feb: Sorry, somehow comments were disabled for this post. This has been resolved now.

There’s a series of code retreats that take place each year where a group of programmers get together to work in groups to solve a problem (kata). They do this in iterations over and over again, and most importantly they delete their entire code at the end of every iteration (typically 45 minutes).

“It’s much easier to walk a new road when the old road is out of sight”

~ Drew Miller

Programmers don’t delete enough production code. Which is funny because I’ve met heaps of programmers, including myself, who love deleting code; it’s a strangely satisfying, cleansing ritual.

What if we could replicate what we do when doing these katas and delete the entire source code of a system every 6 months or a year and start again? Does that sound crazy? Can we do this?

We couldn’t do this with how we currently develop software. It’s because we build software systems that are way too complex, have features that no-one uses and are built to last way too long. We expect our software systems to last 10+ years, and we’re so anxious about long-lasting technology impacts we make overly cautious or conservative decisions that come back to bite us constantly in the years to come. We build technical debt into legacy systems that no one wants to work on. We obsess about re-usability, longevity and salvageability of every piece of code we write. We build massive overly complex regression test suites because we expect the system to work for so long and be maintained for so long we expect it to eventually deteriorate and we want a regression test suite as a ‘safety net’ when it inevitably does.

Maintaining legacy software is like painting the Sydney Harbour Bridge. You start painting it on one side and by the time you get to ‘finish’ it on the other side it’s taken you so long you now need to start repainting the original side again. Wouldn’t it be easier to just build a new bridge?

What we need is Intentionally Disposable Software. We need to build software only designed to last 6 months, a year max. As soon as we deploy it we start immediately on a replacement for it. We put bare minimum effort into maintenance as we’ll just replace what we have in Production as soon as we can: why wash up when you can use fresh paper plates for every meal? As soon as the replacement is ready, we deploy that and completely blow away the old software system. We rinse and repeat.

It’s somewhat similar to planned obsolescence but we don’t do it to annoy our customers and attempt to generate repeat purchases, we do it to refine our software system without any legacy.

We use analytics to tell us exactly what features of the system in production are used and most importantly, what features are little or never used. We don’t build those features into the replacement systems ever again so each system we build is leaner, more focused on the important things it is meant to do and does them better each time. We don’t waste any time on building or supporting unimportant things.

We don’t have time to build up technical debt. We aren’t anxious about choosing a wrong technology. Did we use AngularJS and now hate it? Never fear, we start work immediately on our new system replacement and use ReactJS (or whatever the latest/coolest framework is).

Developer happiness skyrockets! No legacy code! No technical debt! Everyone can work on the latest/best technology to get the job done and want to stick around in our organization to do just that. It’s like being a consultant without being a consultant. Because everyone has already implemented the same thing before, everyone is aware of the gotchas, so whilst people are constantly learning new technology, they’re efficient because they know what they’re actually trying to achieve. And because we’re building such a lean system we’re lean in our approach.

We do need to make sure we use open standards and have an easy way to export/import/migrate data in fast, clean ways – which is good.

The same applies to integration points, we need to be modular enough and use open standards and protocols exclusively to be able to drop out one system and replace it with another that integrates easily.

So what about testing?

If we expect a system to last six months to a year, we need just enough testing. We need just enough testing to make sure the system is built right (doesn’t break), but not too much requirements based testing around building the right system, because we know we won’t build the right system, we’ll be building a ‘more right’ replacement as soon as this one is built.

We need testing that focuses on intention of the system over implementation, because the implementation will constantly change each time we rewrite it. If we write our automated tests in a declarative style devoid of implementation detail we’ll be much better off.

Think


Scenario: Overseas customers must provide ID for expensive purchases
Given I am a non-Australian customer
When I order more than $100 of tea
Then I need to provide additional ID on checkout

Not

Scenario: Overseas customers must provide ID for expensive purchases
Given I go to the Beautiful Tea Home Page
When I click Sign InAnd I enter my username and password
And I click OK
...

Think of Disposable Software like building a house.

You want somewhere to live so you decide to build a house. You design and build a new ‘dream home’ with the best intentions but soon you realize there’s a big difference between what you actually need and what you thought you needed. There’s also some fundamental design flaws you didn’t even realize until it was built and you’re living in it, like mold in the bathroom as it has not enough air-flow and the bedrooms face the wrong direction and are constantly too hot to sleep at night. Plus life has since thrown a new baby into the mix, so 12 months later you find yourself with a house with a lot of design flaws that doesn’t meet your now much clearer, and since expanded, requirements.

So what do you do? Since you’ve invested a lot (financially and emotionally) into your house and you expected it to last ten years of more, you renovate. But the problem with renovating is that you’ve got to work around all the original design flaws, and since the house is already built it’s much more difficult and expensive to make changes to it, and also since you’re living in it, any renovation comes with risk of disruption/displacement to the occupants including an overly sensitive newborn. You find since it’s mainly custom work that the renovations you’re planning will cost nearly as much as the original house.

This is like legacy software. You can’t renovate it easily as it’s already in use, you’re constantly working around its design flaws so it’s much more difficult and costly to make changes to it. Plus it’s really hard to remove the unnecessary parts of your house by renovation.

But what’s the alternative? What if you built the house knowing that come 12 months time you could knock it down, recycle it, and build a new house knowing exactly what you want this time around. You’ll know not to face the bedrooms West. You’ll know the make the bathroom has plenty of air-flow. You’ll even include a bedroom suitable for the baby. But you don’t get too caught up in getting this house ‘perfect’ because come 12 months time you can do it all again. The house could be prefabricated so it’s much cheaper to build off-site in a construction friendly environment, and the migration involves temporarily moving some furniture from the old structure, placing the new house in place with the furniture and recycling the old. You own less objects (clutter) as you know this happens and are prepared for it. As you kids grow up their needs change so instead of doing ‘extensions’ and ‘renovations’ you simply redesign your new house which will be delivered in 12 months time.

This is like Intentionally Disposable Software.

As Alex Bundardzic wrote almost ten years ago in his fantastic Disposable Software post:

“In reality, a good software developer should be able to… deliver a disposable, short lived product that addresses only the pressing needs of the moment, and then simply disappears. No maintenance, no enhancements, no song-and-dance, nothing.”

What do you think?

Never compare your organization’s insides with another organization’s outsides

I once heard the brilliant suggestion that you should never compare your insides with another person’s outsides; because they’re not the same thing. For example, just because someone may seem happy and drive an expensive car, that’s only the outside view of that person and it doesn’t paint the full picture of that person’s insides, which you’re dangerously trying to compare with your own inner thoughts/status.

The same applies with comparing things at your organization to things you’ve heard about other organizations. Countless times, including just this week, have I heard managers and colleagues say things they’ve heard like: “Facebook don’t have testers”, “Google has 10,000+ engineers in 40 offices working on trunk” and “Flickr deploys to production 10 times a day so we can too”. These are examples of comparing our insides to other’s outsides, again.

Yes, Google may have 10,000+ engineers committing to one branch but having spoken to people who work at Google it’s not quite as amazing as it seems. For example, firstly the code-base is broken down into projects (imagine the checkout time without this), each and every change set must be code reviewed, have automated and manual tests performed against it (which can take hours/days) before it is even committed to the trunk, even before it can even be considered for a production release.

I didn’t realize it at the time but the keynote at GTAC last year captured this phenomenon perfectly:

Google from the outside (like a jet plane)Google from the inside (lots of people pushing a broken down car)

Summary

It can not only be really annoying/unhealthy for staff to constantly hear such comparisons, it can also be dangerous because doing something just because Google/Facebook/Twitter/Flickr does it without knowing the inner workings of their organizations will inevitably lead to failure when you try to do it without that context and experience.

So next time you are tempted to drop something you’ve heard from a conference or a blog post about how another company does something better than yours, or to justify that we can/should do it this way, remember, never compare your organization’s insides with another organization’s outsides.

Iterative vs Incremental Software Development

What’s the difference between ‘iterative’ and ‘incremental’ software development?

I know a lot of agile software development teams call their blocks of development time ‘iterations’ instead of ‘sprints’. Does that mean they’re doing iterative software development?

You’ve probably seen the Mona Lisa analogy by Jeff Paton that visually tries to show the difference between the two development approaches:

Incremental Development:

incrementing

Iterative Development:

iterating

But which is better?

Well, if for some (very likely) reason (lack of money, changed business conditions, change in management) we had to stop after iteration/increment one or two, which approach would yield a better outcome?

Mona Lisa

Incremental development gives us a painting of half a lady whereas iterative development gives us an outline of a lady, but both paintings really wouldn’t belong in The Louvre. Perhaps we could have just painted a smaller painting?

This is where I think the Mona Lisa art analogy falls apart. A work of art, like a book, but unlike a piece of software, has a pretty clear definition of done. An artist knows when their piece of art is done: not a single stroke more, not a single stroke less.

But I’ve never worked on a piece of software that was considered done: there’s always more functionality to add/remove/fix.

If we can recognize that software is never done, all we need to do it work out how to get it to where we want it to be (for now).

“We shall not cease from exploration, and the end of all our exploring will be to arrive where we started and know the place for the first time.”
~ T.S. Eliot

If we are driven by time to market we should internally iterate just enough so we can  release ‘increments’ fast and often, and iterate/release again and again.

If we are driven by user experience, we should internally iterate a lot to get things right, release increments only when necessary, and iterate again.

Both approaches are about iterating. Both are also about incrementing. The difference is how soon we release after how many times we iterate.

Compare the beginnings of the two dominant mobile operating systems. Google went for time to market with Android, they released an unpolished, yet feature rich, operating system quickly and made it better by iterating/incrementing again and again over time. Apple took the opposite approach: they released iOS with highly polished features relatively slowly (it took three major iOS releases to get MMS and copy & paste!) but focused on getting things right from the start.

Both approaches are different but neither are wrong: they highlight the differences between Apple and Google and their approach to developing software.

Summary

We can’t build anything without iterating to some degree: no code is written perfectly the second that it is typed or committed. Even if it looks like a company is incrementally building their software: they’re iteratively building it inside.

We can’t release anything without incrementing to some degree: no matter how small a release is, it’s still an incremental change over the last release. Some increments are bigger because they’ve already been internally iterated upon more, some are smaller as they’re less developed and will evolve over time.

So, we develop software iteratively and release incrementally in various sizes over time.

What is a good ratio of software developers to testers on an agile team?

The developer:tester ratio question comes up a lot and I find most, if not all, answers are “it depends”.

I won’t say “it depends” (it’s annoying). I will tell you what works for me given my extensive experience, but will provide some caveats.

I’ve worked on different agile software development teams as a tester for a number of years and I personally find a ratio of 8:1 developers to tester(s) (me) works well (that’s 4 dev-pairs if pair programming). Any less developers and I am bored; any more and I have too must to test and cycle time is in jeopardy.

Some caveats:

  • I’m an efficient tester and the 8:1 ratio works well when there’s 8 equally efficient programmers on the team – if the devs are too slow, or the user stories are too big, I get bored;
  • Everyone in the team is responsible for quality; I have to make sure that happens;
  • A story must be kicked off with the tester (me) present so I can question any assumptions/anomalies in the acceptance criteria before any code is written;
  • A story is only ready for test if the developer has demonstrated the functionality to me at their workstation (bonus points in an integrated environment) – we call this a ‘shoulder check’ – much the same way as monkeys check each others shoulders for lice;
  • A story is also only ready for test if the developer has created sufficient and passing automated test coverage including unit tests, integration tests (if appropriate) and some acceptance tests; and
  • Bug fixes take priority over new development to ensure flow.

What ratio do you find works for you?

Why to avoid t-shirt sizes for user story estimation

The more I work on agile software development teams who use t-shirt sizes (S,M,L,XL etc.) to estimate user stories the more I dislike this approach. Here’s why:

  • In my opinion, the most import thing about user story sizing is relativity, and t-shirt sizes are a subjective measure of relativity: someone in the team might think a large is two times as big as a small, whereas another person might think it’s three times as big. This isn’t helped by the t-shirt analogy where it’s actually hard to determine how much bigger is a large t-shirt than a small one?
  • You can’t create a single measure of team velocity unless you define a scale that converts t-shirt sizes into a numeric size so you can measure t-shirt size relativity and velocity.
  • As soon as create a scale to convert t-shirt sizes into a numeric size you’ve essentially started using story points (in a convoluted way).

TL;DR: Using t-shirt sizes for user story estimation is confusing and ultimately leads the team to using story points so just skip t-shirt sizes and use relative story points instead.

Waterfall, Agile Development & Hyperbole

Hyperbole. Love it or hate it, it’s been around for centuries and is here to stay. And, as someone pointed out this week, I’m guilty as charged of using (abusing?) it on this blog. You just need to quickly flick through my recent posts to find such melodramatic titles such as ‘Do you REALLY need to run your WebDriver tests in IE?‘, ‘UI automation of vendor delivered products always leads to trouble‘, and  ‘Five signs you’re not agile; you’re actually mini-waterfall‘. Hyperbole supports my motto for this blog and my life: strong opinions, weakly held.

But it’s not just me who likes hyperbole mixed into their blog posts. Only this morning did I read the catchy titled ‘Waterfall Is Never the Right Approach‘ followed quickly with a similarly catchy titled rebuttal: ‘Why waterfall kicks ass‘ (I personally would have capitalized ‘NEVER’ and ‘ASS’).

While I found both of articles interesting, I think they both missed the key difference between waterfall and agile software development (and why waterfall rarely works in these fickle times): waterfall is sequential whereas agile development is (at least meant to be) iterative.

I personally don’t care whether you do SCRUM or XP, whether you write your requirements in Word™ or on the back of an index card, or even if you stand around in a circle talking about what card you’re working on.

What I do care about is whether you’re delivering business value frequently and adjusting to the feedback you get.

Sequential ‘big bang’ development such as waterfall, by its nature, delivers business value less frequently, and chances are when that value is realized the original problem has changed (depending on how long ago that was), because as I stated and believe, we live in fickle times.

Iterative development addresses this by developing/releasing small fully functional pieces of business value iteratively and adjusting to feedback/circumstance.

Just because an organization practices what they call ‘agile’, doesn’t mean they’re delivering business value iteratively. I’ve seen plenty of ‘agile’ projects deliver business value very non-frequently, they’re putting a sequential process into agile ‘sprints’ followed by a large period of end to end, business and user acceptance testing, with a ‘big bang’ go live.

Whilst I believe iterative development is the best way to work; I’m not dogmatic (enough) to believe it’s the only way to work. Whilst I believe you could build and tests parts of say an aeroplane iteratively, I still hope there’s it’s a sequential process with a whole heap of testing at the end on a fully complete aeroplane before I take my next flight in it.