Software Rewrites – The Forever War
Recently I asked myself how much of my career has been spent rewriting “legacy” software. At first I thought the answer is probably a good four out of ten years, although if I’m really being honest with myself, it’s what I’ve been doing almost the whole time.
Allow me to describe my career history in broad strokes:
- Years 0–3, working at a payment processing company. Writing new SPAs front ends for .net WSDL web services that would replace software written in a little-known technology called “Visual Objects”.
- Years 4–5, working at a company selling train tickets from various European providers. Building websites and integrating with the ticket providers, e.g. Deutschebahn.
- Years 6–7, working at a brokerage turning excel macros into .net services and websites.
- Years 8–9, working at a company providing IT services to a large clothing manufacturer. Ostensibly turning parts of a huge winforms app into .net services and websites.
- Year 10, working at a delivery company rewriting the route planning software (from .net to .net).
In all that time the only place that didn’t involve rewriting software was at the firm retailing train tickets. It should be of note that the company acted as a start-up, incubated by the well-established parent company. They explicitly avoided taking on the parent company’s software and processes in order to remain agile.
Here’s a picture of the Ouroboros eating its own tail as a helpful visual metaphor:
Friendly little critter, ain’t he!
Why does software need to be completely rewritten?
Small talent pool for outdated technology
Sometimes you just can’t get the staff! This is one potential reason that I would find hard to disagree with. When the core product is written in a technology that few have even heard of, it’s difficult to hire staff to maintain it — it’s a major liability when the bus factor is that low. This was the case at the first company I worked at.
Now I look back on my younger self (five or six years ago) with the wisdom of age and realise my folly at trying to convert colleages to use a functional programming language with a much smaller market share than the object-oriented alternative.
Previous solution doesn’t scale
This was the case at the financial company I worked at. It really is awe-inspiring when you consider the software that people have built using Microsoft Excel and VBA scripting. I mean awe-inspiring in the sense of both incredible and terrifying.
You may find that the software doesn’t scale beyond a certain input size. Initially when the business is small, it’s acceptable. When the business grows though and more customers/orders/trades need to be reconciled, this is when the cracks start to show. Scheduled tasks take longer and longer, until eventually they still haven’t finished by their next scheduled trigger time.
It isn’t just a matter of run-time performance not scaling. It also needs to scale with regards to accountability and security. With multiple people working on the software it’s paramount to have changes versioned via source control. This also ensures that changes can be reviewed, and also that appropriate third-party libraries are used.
Poorly engineered, poorly unit tested solution
Sometimes people want to rewrite a solution for the simple fact that it’s so poorly tested, documented and understood, that feature development is perceived to be too slow or at too high a risk of introducing bugs.
This is something that I personally find especially irksome. It’s tough to excuse this kind of approach being taken today, when the waters of automated testing, version control, continous integration and automated deployment have been well charted by other pioneers.
What also frustrates me is the approach of writing a complete replacement for it, rather than refactoring towards a better solution. The current solution presumably works and has battle-tested under real-world conditions. By doing some kind of big-bang release, it greatly increases the risk of something going wrong.
Why are estimates for the rewrite so far off?
In the two companies I’ve worked where a complete stand-in replacement was developed, the projects managed to drag on interminably.
Not surprisingly, people are reluctant to accept new software that actually accomplishes less than what they currently have! This usually means that the rewrite needs to achieve feature-parity with that which it’s replacing.
If you also need to continue developing features for the existing solution, then the scope of the rewrite increases accordingly. Again, people are not enthusiastic if you ask for a feature-freeze when they need solutions to their problems at this moment in time.
Another confounding factor is that developers are infamously bad at any kind of estimation. Even knowing at the back of their minds that the legacy software is a behemoth, they still seem to lump large groups of functionality together, in the process diminishing its complexity.
Where do I go to find true greenfield development?
We’re rewriting this software because it’s there and it’s been proven to work, but we need something better to drive the business forward. I reckon to do any kind of work that’s completely new you’ll probably need to go and work at a start-up, where they don’t have anything existing. But please, please use modern disciplined engineering approaches like automated testing to ensure I don’t have to rewrite it!
