Process and Judgement Risks

The final group of risks arises from how AI gets used inside a process, rather than from the model or from the law. These are the risks a Requirements Engineer has the most direct control over.

Context window overflow

When the amount of input exceeds a model's context capacity, earlier information may be dropped, leading to incomplete or inconsistent outputs.

Example: during a large requirements elicitation workshop, a model with a limited context capacity processes meeting notes in real time. Early stakeholder statements about security requirements are silently dropped because they exceed the model's capacity. The result is an incomplete specification with critical security features missing.

The word doing the work in that example is silently. There is no error, no warning, no gap marker. The output looks complete. This is why the mitigation cannot be "watch out for it" — the failure produces no signal to watch for. It has to be structural: chunk long inputs deliberately, process in passes, and check coverage against a known list of inputs rather than against the plausibility of the output.

The terminology chapter adds a nuance worth carrying: this is not simply a "bigger is better" problem. The more input you give a model, the better it performs at first, but too much input can also cause the model to get distracted. A larger window is not a licence to paste everything.

Scope misalignment

AI may generate requirements that are out of scope for the project, or that conflict with agreed objectives — particularly if prompts are vague or incomplete.

Example: while drafting requirements for a mobile app, the AI begins proposing features for a desktop version that was never planned. The development team spends several days analysing irrelevant features before realising they are out of scope, delaying the agreed design phase.

Two things are worth noticing. First, the cause is named: vague or incomplete prompts — which ties straight back to educational objective 3.1 on context. Second, the damage in the example is not a bad requirement in the specification; it is several days of a team's time. Scope misalignment is expensive even when it is caught.

Over-reliance on AI

Relying too heavily on AI output without adequate human validation can lead to flawed or biased requirements being accepted, reducing overall quality and increasing project risk.

Example: a project team in the automotive sector adopts AI-generated safety requirements for an autonomous driving feature without proper human review. Months later, during regulatory inspection, auditors find that several requirements contradict mandatory safety standards, forcing a costly redesign and delaying the product launch.

Over-reliance is the risk that makes the others worse. Every other item on this list is survivable if a competent human reviews the output. Over-reliance is precisely the decision not to.

Opacity

Many AI models function as black boxes, making it difficult to trace how specific outputs were derived. In RE, this lack of transparency can undermine stakeholder trust and complicate the validation of requirements.

Example: the AI proposes prioritising certain requirements but cannot explain the ranking logic, making review difficult.

Opacity is worth thinking about as a validation problem rather than only a trust problem. Validation asks whether a requirement genuinely reflects stakeholder needs. If nobody can say why the AI ranked a requirement highly, there is nothing to validate against — no reasoning to agree or disagree with, only an output to accept or reject.