MODULE 2 · LESSON 2
Free — no login requiredSign in to track progress, save quiz attempts and enrol in the full course.
Sign in to track progress / enrolDesigning the Failure Paths
The failure modes
An AI product fails in more ways than conventional software, and each needs a designed response:
Only the fourth and fifth resemble conventional software failures. The others are specific to this technology and are usually undesigned, which means they are experienced by users as the product being broken.
Refusal is a feature
Products that cannot say no are worse than products that can.
A system that answers everything will answer things it should not: questions outside its knowledge, requests it has no basis for, questions where it will be confidently wrong. Every one of those answers damages trust more than a refusal would.
A good refusal has three parts:
- It is clear. "I can't answer that" rather than hedged text that reads like an answer.
- It says what the product does cover. Turning a dead end into navigation.
- It offers the next step. A search, a human, a document.
And a good refusal does not lecture. Users experience moralising refusals as insulting, particularly when the request was reasonable and the refusal was over-broad. Refuse, redirect, stop.
Refusal has to be tested like any other output. Over-refusal is a real product defect — a system that declines legitimate requests is failing, and it is invisible in accuracy metrics because refusals are not usually scored. Put out-of-scope and edge-case inputs in your evaluation set and check both that it refuses what it should and that it does not refuse what it should not.
Graceful degradation
Your product depends on infrastructure you do not control — covered properly in module 8. The design question here is: what is the product when that dependency is unavailable?
Three answers, in descending order of quality:
Reduced function. The generative feature is unavailable; everything else works. The user sees a clear message in the relevant place, not a broken page. This requires the AI feature to be genuinely separable, which is an architectural decision made early or not at all.
Fallback path. A simpler, cheaper, more available mechanism takes over — a rules-based classifier, a cached response, a template. Lower quality, still useful.
Queue and retry. Where the work is not immediate, accept it and complete it when the dependency returns. Requires telling the user honestly.
The failure to avoid is the one that happens by default: the feature hangs, the page spins, the user does not know whether to wait, and eventually concludes the product is broken.
Latency is a failure mode
Worth treating explicitly, because teams classify it as performance rather than as failure.
For a conversational surface, a slow correct answer frequently loses to a fast adequate one. Users abandon. Beyond a few seconds without feedback, the perception is not "thinking" but "broken".
The design responses:
- Stream the output so something appears immediately, which changes the perceived wait entirely.
- Show what is happening where you cannot stream — "searching your documents" — because named progress is tolerated far longer than a spinner.
- Set a timeout with a defined fallback, rather than waiting indefinitely.
- Route by difficulty. Easy cases to a fast path, hard ones to a slower path with an expectation set.
A useful exercise for any AI product team, and it takes an hour.
Write down every AI-dependent surface in the product. For each, answer three questions: what does the user see when the dependency fails, what can they still do, and how do they know when it is back?
A typical first pass on a customer-service product with reply drafting, ticket classification, sentiment flagging and a knowledge assistant produces answers like:
| Surface | Today, on failure | Should be | |---|---|---| | Reply drafting | Spinner, then a generic error after 30s | Draft area opens empty and editable with a one-line notice; agent works normally | | Ticket classification | Tickets stay unclassified in a queue nobody watches | Fall back to the pre-AI keyword rules; mark them as rule-classified | | Sentiment flagging | Silently absent | Silently absent is fine — flags are advisory and their absence changes nothing | | Knowledge assistant | Error page | Fall back to plain keyword search over the same documents |
Two things this exercise reliably reveals.
The classification fallback usually already exists, because there was a rules-based system before the model. Teams delete it on launch because the model is better, and thereby convert a degradation into an outage. Keeping the old path as a fallback is nearly free.
Not everything needs a fallback. Sentiment flagging is advisory; its absence is not a failure. Deciding that explicitly is as valuable as building the other three, because it stops the team over-engineering the parts that do not matter.
The output of the hour is a table you can hand to engineering, and it converts "what happens if the API goes down" from an anxiety into a specification.
Why should refusal behaviour be included in an AI product's evaluation set?
The six failure modes
Click to flipConfident wrong answer, out-of-scope input, genuine uncertainty, dependency unavailable, response too slow, and a request the product should decline. Only two resemble conventional software failures.
Click to flip backAn AI product fails in ways conventional software does not, and undesigned failure is experienced as brokenness. Treat refusal as a feature — clear, navigational, unmoralising — and test over-refusal, which accuracy metrics cannot see. Design what the product becomes when the dependency is unavailable, keeping the pre-AI path as a fallback rather than deleting it at launch. And treat latency as a failure mode, because past a few seconds without feedback users read waiting as broken.