Embeddings, Vectors and Semantic Meaning

The syllabus presents this material as optional depth — "if you are interested in why this may work" — but it also lists embeddings and vectors in the examinable terminology chapter, and both underpin RAG. It is worth a lesson.

Tokens as numbers

The model represents tokens as vectors: long, ordered lists of numbers. Tokens that appear in similar contexts get similar vectors. This creates a kind of geometry that lines up with meaning in practice.

The classic illustration comes from earlier methods and makes the idea concrete: the difference between the vectors for king and queen tends to resemble the difference between man and woman. The system is not storing dictionary entries. It is learning numerical patterns that reflect how people actually use words.

These numerical representations are called embeddings, and they are the reason a model can work with semantic meaning rather than exact string matching.

The syllabus gives the RE-specific version of the example, and it is the one to remember: the vector for user requirement is closer to stakeholder need than it is to database schema. No rule was written down saying those two phrases are related. It fell out of how they are used in text.

What this buys a Requirements Engineer

Embeddings are what allow an AI tool to identify similar requirements or find dependencies even when different terminology is used. That single capability underlies most of the analysis functions you will meet in chapter 5 of the syllabus.

Semantic search. Finding the requirement about "session expiry" when you searched for "automatic logout", because the concepts sit near each other in vector space.
Duplicate detection. Two backlog items describing the same need in different words are close together numerically, even with no shared keywords.
Clustering. Grouping a large set of extracted requirements by topic or feature area without anyone pre-defining the categories.
Traceability. Linking requirements to related artefacts — design documents, test cases, stakeholder notes — where the wording differs.

Take two entries that a real backlog might contain:

"The system shall log the user out after a period of inactivity."

"Sessions must expire when unused, to satisfy the bank's security policy."

A keyword search finds no overlap worth acting on. There is no shared noun. "Log out" and "expire" are different verbs; "user" and "session" are different subjects. A conventional duplicate check reports two unrelated items.

In embedding space they sit close together, because across an enormous quantity of technical text, sentences about inactivity timeouts, session expiry and automatic logout appear in the same contexts. The geometry encodes that co-occurrence.

That is why the syllabus links embeddings to duplicate detection and traceability specifically. It is not that the tool is cleverer than a search box. It is that it is measuring a different thing: proximity of usage rather than identity of characters.

This is also, incidentally, exactly the mechanism a RAG system uses to decide which passage of a knowledge base to hand the model. Retrieval quality — the thing the previous module warned you about — is embedding quality.

The limitation to remember

The syllabus states it plainly, and it is a likely exam distinction: embeddings form the basis for many retrieval and analysis functions but do not generate text themselves.

Embeddings measure and locate. Generation is the language model's job. A system that finds the ten most relevant requirements has used embeddings; a system that then writes a summary of them has used an LLM. Conflating the two leads to the mistaken idea that "the embeddings hallucinated", which is not a thing that can happen.

❓ Knowledge Check

Which of the following are things embeddings make possible in Requirements Engineering? Select all that apply.

📚 Flashcards1 / 4

Term

Embeddings

Click to flip

Definition

Numerical vector representations of text that capture semantic meaning. Tokens appearing in similar contexts receive similar vectors. They enable semantic search, duplicate detection, clustering and traceability, but do not generate text.

Click to flip back

💡Key Takeaway

Embeddings turn text into vectors whose geometry reflects how words are used, so semantically related content sits close together even with no shared vocabulary. That is what makes semantic search, duplicate detection, clustering and traceability possible across differently-worded requirements — and it is the retrieval half of RAG. Embeddings locate and compare; they never generate.