MODULE 4 · LESSON 1
Free — no login requiredSign in to track progress, save quiz attempts and enrol in the full course.
Sign in to track progress / enrolRisks That Come From the Model
The use of AI in RE offers real benefits and introduces specific risks. Those risks arise from two places: the technical limitations of the models themselves, and the way they are applied in RE contexts. This lesson covers the first group — five risks that follow from what the model is.
Hallucination
LLMs can produce plausible but factually incorrect outputs, inventing details or references that simply do not exist. In RE this can result in fabricated standards, stakeholder statements or system behaviours being treated as valid requirements.
The reason it cannot be trained away was covered in module 2: the model has no internal concept of "knowing", so it cannot indicate when it is guessing or when it lacks sufficient data.
Example: an AI cites a security guideline by a plausible-looking standard number that no standards body has ever issued. The citation format is right, the context is right, the standard does not exist.
False precision
Some AI outputs appear formally correct but are semantically meaningless or unverifiable. Requirements phrased in technically precise language may still be untestable or ambiguous in practice.
Example: "The system shall perform optimally at all times." It sounds like a requirement. It has the right shape. But "optimally" is undefined and "at all times" is both unrealistic and untestable.
False precision is the risk most specific to Requirements Engineering, and the one a general AI-literacy course would miss entirely. It is dangerous precisely because it survives a superficial review — a reader checking that requirements follow the phrase template will pass it.
Bias amplification
AI models may reinforce or amplify biases present in their training data, leading to systematically skewed requirements or personas. Undetected, this can introduce unfairness or discrimination into the system design itself.
Example: on an HR system project, an AI is asked to generate personas for job applicants. Drawing on biased historical recruitment data, it consistently describes leadership candidates as male, aged 35 to 50, with a finance background. If uncorrected, that bias risks embedding discriminatory assumptions into automated hiring workflows and may violate equal opportunity law.
Note what makes this an RE risk rather than a general ethics topic: personas and requirements are inputs to design. A biased persona does not stay in the document. It becomes the user the system is built for.
Outdated knowledge
Models trained on historical data may already be operating with outdated information, lacking recent standards, technologies or regulations. The syllabus is careful with the wording here: this is a static limitation present from deployment. It is particularly critical in regulated domains, where an outdated requirement can mean non-compliance.
Example: the AI recommends a constraint involving a network protocol that was deprecated two years ago.
Model drift
Over time, the accuracy of an AI system's output can degrade as domain knowledge, data patterns or operational conditions evolve. The syllabus contrasts it explicitly with the previous risk: this is a dynamic risk that appears during use. Without monitoring, drift produces subtle but significant errors.
Example: a model fine-tuned for banking compliance in one year starts producing outdated recommendations after regulatory changes the following year.
These two risks are easy to blur and the syllabus draws the line sharply, which makes it prime exam material.
Outdated knowledge is static and present from day one. The model was trained up to a cut-off. Everything after that cut-off is missing, and it was missing the moment you deployed. Nothing changes over time; your awareness of it might.
Model drift is dynamic and emerges during use. The model has not changed. The world has changed around it, so output that was accurate at deployment becomes gradually less accurate.
A test that separates them: was this output ever correct?
- Never correct, because the information post-dates training → outdated knowledge.
- Correct at deployment, wrong now, because conditions moved → model drift.
The mitigations differ accordingly. Outdated knowledge is addressed by grounding — RAG over current sources, so the model reads today's regulation rather than recalling last year's. Drift is addressed by monitoring — someone has to be checking, on a schedule, whether output is still right, because nothing in the system will raise its hand.
An AI-assisted compliance checker was accurate when deployed, but after a regulatory change its recommendations no longer match current obligations. Which risk is this, and what mitigation does it call for?
Hallucination (as a risk in RE)
Click to flipPlausible but factually incorrect output, inventing details or references that do not exist. In RE it can result in fabricated standards, stakeholder statements or system behaviours being treated as valid requirements.
Click to flip backFive risks come from the model itself: hallucination invents what does not exist; false precision produces well-formed but untestable requirements; bias amplification turns skewed training data into skewed personas and designs; outdated knowledge is a static gap present from deployment; model drift is a dynamic decay that appears during use. The last two are examined as a pair — the question that separates them is whether the output was ever correct.