Scale, Sustain or Kill

The annual decision

Every live AI system should face three options once a year, deliberately rather than by default:

Scale. Extend to more volume, more cases, or an adjacent process.

Sustain. Keep it running as it is, with monitoring and maintenance funded.

Kill. Decommission it.

The default in most organisations is a fourth option nobody chooses: drift — the system keeps running because switching it off requires a decision nobody wants to make, while nobody funds its maintenance either. That is how organisations accumulate systems that half-work and that no one understands.

When to scale

Four conditions, and all four should hold:

📅 Timeline

The business outcome is realMeasured against the baseline, not asserted. If you cannot demonstrate it after a year, scaling multiplies something unproven.

Adoption is broadUsed across the eligible population, not concentrated in enthusiasts. Narrow adoption at scale becomes narrow adoption over a larger denominator.

The exception path holdsAdding volume adds exceptions proportionally. If handling is already strained, scaling breaks it.

Someone owns itWith time allocated. Scaling an unowned system multiplies an unmanaged risk.

Scaling to an adjacent process deserves particular attention because it is where the compounding return lives. The second document type on an extraction pipeline costs a fraction of the first — the integration, monitoring, exception routing and organisational trust already exist. This is the payoff for the capability project in module 3, and it is why the question "what does this leave behind?" mattered.

When to sustain

Most systems, most years. Sustaining is a legitimate decision and should be an explicit one, which means funding it: monitoring, the exception path, periodic retraining, and someone's time.

A system in sustain mode still needs its annual review. "Sustain" means stable, not forgotten.

When to kill

Be willing. The signals:

The business outcome never materialised, and you have run out of hypotheses about why.
Adoption never arrived, and the causes from module 6 have been addressed without effect.
The process it served has changed and it no longer fits.
Maintenance exceeds the benefit — common for bespoke builds whose original engineers have left.
A bought product now does this better, which happens frequently and is not a failure.

That last one deserves saying plainly: being overtaken by the market is a success condition for your organisation, not a defeat for your project. A team that built something in-house two years ago and can now replace it with a product at lower cost should do so without embarrassment.

Decommission properly

Killing a system badly leaves as much mess as running a bad one:

Tell the users, with time and a replacement route. People build workflows around tools.
Extract your data — including labels, corrections and configuration, which module 5 required contractually.
Preserve the audit trail for as long as your obligations require, which may be years after switch-off.
Remove access and delete data at the vendor, with confirmation.
Update the inventory, or the annual review will keep finding a system nobody can locate.
Write down why, so the same idea is not re-proposed in eighteen months without new information.

The post-implementation review

The single highest-return hour available to an organisation doing AI work, and almost nobody spends it.

Take the original business case and mark it. Not to allocate blame, but because this is the only mechanism by which an organisation gets better at predicting.

Four questions:

Which assumptions were wrong, and in which direction? Adoption is usually the one, and usually optimistic.
What did we not anticipate at all? These become checklist items for the next case.
What took longer than expected, and what was the actual constraint? Nearly always data access or the last-mile integration.
What would we do differently with the same information? Distinguishes bad luck from bad process.

Circulate it. An organisation whose third business case is informed by two honest reviews is meaningfully better at this than one on its tenth case with none — and the difference compounds, because better predictions produce better decisions about what to attempt next.

The same company's annual review, showing all three outcomes.

Delivery-note extraction — scale. Business outcome measured at 290 hours a month against 265 promised. Adoption at 88% across all four depots. Exception rate stable at 21% with the path staffed. Named owner with allocated time. Decision: extend to supplier invoices, which reuses the pipeline, the monitoring and the exception process. Estimated cost roughly a third of the original build, because everything except the extraction configuration already exists.

Email triage — sustain. Working. Adoption at 74%, which is lower than hoped but stable. Saving modest and real. No adjacent process obviously benefits. Decision: sustain, with monitoring funded and a review in twelve months. Explicitly not scaled, because the benefit does not justify the integration work into a second team's systems — a decision recorded so it is not revisited casually.

Renewal-likelihood scoring — kill. Technically fine, around 84% accuracy on a held-back set. But the account managers never used it: the score arrived in a weekly report rather than in the CRM, and there was never budget to do the integration properly. The causes from module 6 were identified and the fix was funded twice and deprioritised twice. Decision: decommission. The write-up said plainly that the model was adequate and the project failed on integration and adoption, and that the same idea should only return with CRM integration funded from the start.

That last write-up is the most valuable artefact of the three. It converts a failure into a precondition for a future success, and it means the next person who proposes renewal scoring starts from what was actually learned rather than from enthusiasm.

❓ Knowledge Check

A system has been live a year. It performs well technically, but the business outcome cannot be demonstrated and adoption stayed narrow despite the known causes being addressed. What is the appropriate decision?

📚 Flashcards1 / 6

Term

The annual three options

Click to flip

Definition

Scale, sustain or kill — chosen deliberately. The unchosen fourth option is drift, where a system keeps running because switching it off needs a decision nobody wants to make and nobody funds its maintenance either.

Click to flip back

💡Key Takeaway

Force an annual decision — scale, sustain or kill — because the default is drift, and drift is how organisations accumulate half-working systems nobody owns. Scale only when the outcome is measured, adoption is broad, the exception path will hold and someone owns it, and prefer adjacent processes where the plumbing already exists. Kill without embarrassment when the benefit did not arrive, decommission properly, and always mark the original business case against reality — it is the only way anyone gets better at this.