MODULE 5 · LESSON 2
Free — no login requiredSign in to track progress, save quiz attempts and enrol in the full course.
Sign in to track progress / enrolThe OWASP Top Ten in Plain Language
OWASP is a long standing non profit that publishes widely used lists of the most important security risks in a given area. Their Top 10 for Large Language Model Applications is the closest thing to an agreed vocabulary in this field.
You do not need to memorise it. You need to recognise the categories well enough to ask the right question when someone proposes deploying something.
| Code | Risk | What it means in practice |
|---|---|---|
| LLM01 | Prompt injection | Hidden instructions in input hijack the model's behaviour |
| LLM02 | Sensitive information disclosure | The model reveals private data from its training or its context |
| LLM03 | Supply chain | A downloaded model, plugin or dependency carries a backdoor |
| LLM04 | Data and model poisoning | Corrupted training or reference data creates hidden wrong behaviour |
| LLM05 | Improper output handling | Model output is passed unchecked into another system |
| LLM06 | Excessive agency | The model can take actions far beyond what its task requires |
| LLM07 | System prompt leakage | Internal instructions leak, revealing how to bypass them |
| LLM08 | Vector and embedding weaknesses | The knowledge base behind a chatbot is read or poisoned |
| LLM09 | Misinformation | Confident, fluent, wrong output is treated as fact |
| LLM10 | Unbounded consumption | Unlimited usage causes an outage or a very large bill |
Four of these deserve more than a table row.
LLM02: sensitive information disclosure
Two distinct failures share this heading, and conflating them causes confusion.
The first is context leakage. Whatever is in the model's context during a session can potentially come out. If your assistant retrieves a customer record to answer a question, a manipulated conversation may cause it to reveal that record to someone who should not see it.
The second is training leakage. If a model was fine tuned on internal data, fragments of that data can surface in its outputs. This is why fine tuning on unfiltered internal material is a decision that deserves care, rather than a purely technical step.
The practical question to ask: what is the most sensitive thing this system can reach, and who can talk to it?
LLM04 and LLM08: poisoning the sources
These two are related and frequently missed by people who focus on the model itself.
Most useful business assistants do not rely on the model's training. They retrieve from a knowledge base, typically an internal wiki, a document store, or a database of past tickets, and answer using what they find.
That makes the knowledge base part of your attack surface, and it is usually far less protected than anything else in the chain.
Ask the question directly: who can edit the content your assistant reads? In most organisations the honest answer for an internal wiki is "any employee", and often "any contractor". Which means any one of those accounts, or anyone who compromises one of them, can influence what the assistant tells the entire company.
The attack is elegant and requires no technical skill. You do not touch the model, the infrastructure or the permissions. You edit a page. Every subsequent answer that cites it is confidently, authoritatively wrong in the direction you chose, and it carries the institutional credibility of the assistant.
This is the integrity failure from Module 2, delivered through a new route.
LLM09: misinformation
The risk that will affect you most often in ordinary work, and the one least likely to be described as a security incident.
Language models produce fluent, confident, well structured output regardless of whether the content is correct. There is no internal signal distinguishing a fact the model represents reliably from a plausible construction. Fabricated citations, invented case law, incorrect figures and non existent product features all arrive in exactly the same register as correct information.
The security dimension appears when that output enters a decision. A fabricated statistic in a board paper, an invented clause in a contract summary, a wrong configuration in a runbook.
The mitigation is procedural rather than technical, and it belongs in Module 6: verify anything factual before you forward it, because your colleagues will trust it once it carries your name.
Questions to ask without being a specialist
You can conduct a useful review of any proposed AI deployment with five questions.
- What can it read? That defines the worst case disclosure.
- What can it do? That defines the worst case action. Reading is recoverable, acting frequently is not.
- Who can put text in front of it? If the answer includes the public, assume hostile input will arrive.
- Who can edit what it reads? That is the poisoning surface, and it is usually broader than anyone expects.
- What happens to its output? If it flows into another system unchecked, that is LLM05.
Those five questions cover the majority of the list and require no technical background. They are also, deliberately, the same shape as the Module 2 questions about confidentiality, integrity and availability.
LLM03 receives less attention than prompt injection and is arguably more dangerous, because it fails silently and at a lower level.
Modern AI development involves downloading a great deal from public repositories: base models, fine tuned variants, embedding models, tokenisers, plugins, framework libraries. A working prototype might pull in dozens of components, most chosen because they appeared at the top of a search or in a tutorial.
Several distinct risks live here.
Malicious model files. Some model formats are serialisation formats that can execute code when loaded. Loading an untrusted model file has historically been equivalent to running an untrusted program, which is why safer formats exist and are worth insisting on.
Backdoored weights. A model can be trained to behave normally except on a specific trigger phrase, at which point it produces attacker chosen output. This is extremely difficult to detect by evaluation, because the model behaves correctly on every input you think to test.
Typosquatted packages. The same attack that has plagued software package registries for years, now aimed at AI libraries with plausible names.
Compromised plugins and tool integrations. A plugin that gives an assistant a new capability also gives whoever controls the plugin that capability.
The mitigations are the ordinary supply chain disciplines rather than anything AI specific. Obtain models and libraries from known publishers, prefer formats that cannot execute code on load, pin versions rather than always pulling the latest, keep an inventory of what your systems actually depend on, and treat the addition of a plugin with the same seriousness as granting an employee system access, because it is the same decision.
The reason this deserves attention is timing. Prototypes assembled quickly with whatever worked have a way of becoming production systems without ever being reviewed, and by then the dependency list is long and nobody remembers where any of it came from.
A company assistant answers staff questions using the internal wiki, which any employee can edit. What is the principal risk?
OWASP Top 10 for LLM Applications
Click to flipThe most widely used catalogue of language model application risks, led by prompt injection for two editions running.
Click to flip backThe OWASP list is a vocabulary rather than a checklist to memorise. Four entries matter most in ordinary organisations: disclosure of whatever the model can reach, poisoning of the knowledge base it answers from, unchecked handling of its output, and confident misinformation entering real decisions. You can review any proposal with five non technical questions: what can it read, what can it do, who can put text in front of it, who can edit what it reads, and what happens to its output. The fourth question is the one most often forgotten, and in most organisations the honest answer is "any employee".