Model and Architecture Terms

The technology families

Machine learning (ML). Algorithms learn patterns directly from data. Classic ML predicts numbers or categories; deep-learning networks such as transformers and convolutional neural networks handle language, images and reinforcement-learning policies. Output quality rises with training-set size and relevance. Example: a model taking past house sales data and predicting prices for properties yet to be sold.

Deep learning. A subset of machine learning that uses neural networks — transformers for language, convolutional networks for images — to handle complex data types and tasks, including reinforcement learning.

Natural language processing (NLP). Everything that turns speech or text into meaning and back again. It ranges from rule-based parsers to today's LLM chatbots, plus speech-to-text and text-to-speech modules.

Computer vision. Transforms raw pixels into structured facts. Modern vision networks detect objects, read text from an image and estimate spatial depth. The best-known example is probably the face-unlocking feature of a modern phone.

Robotics. Combines sensor fusion (integrating data from multiple sensors), mapping (building a representation of the environment), path planning (deciding how to move) and feedback control (adjusting actions based on sensor input), so machines can navigate and manipulate the physical world. Applications range from warehouse pickers and autonomous drones to vacuum cleaners that map a living room and avoid obstacles.

Expert systems. Capture human expertise as explicit rules and ontologies, producing auditable, deterministic answers. Still favoured for domains such as tax law or medical triage, where transparency trumps probability and no room for error is allowed. Modern tax software can, for example, use AI to analyse bills and income statements while the rules compute what is owed.

Architecture and behaviour

Transformer. The neural network architecture behind most modern LLMs. It uses an attention mechanism to assess the importance of words relative to each other regardless of position, enabling efficient handling of long sequences and complex language patterns. In RE, transformers allow tools to interpret and generate requirements text with high contextual awareness — for example identifying dependencies or maintaining consistent terminology across documents.

Token. A unit of text — often a word, part of a word, or punctuation — that the model processes. LLMs operate on tokens rather than entire sentences, and their performance and cost are partly determined by the number of tokens processed. In RE, token limits affect how much specification text, stakeholder input or glossary content can be handled in a single interaction.

Context window. Defines the maximum number of tokens an LLM can consider at one time, including both the prompt and the generated output. This limit constrains how much of a requirements specification or dialogue history can be taken into account. The syllabus adds a caution worth remembering: this is not always a case of bigger being better — the more input you give an LLM, the better it does at the start, but too much input can also cause the model to get distracted.

Temperature. A parameter controlling the randomness of an LLM's output. Low values make the model more deterministic, producing consistent phrasing; higher values create more varied and creative results. In RE, lower temperatures are often preferable for consistency in requirements wording, whereas higher values can help generate diverse solution ideas. In many LLMs, temperature cannot be set by users.

Meaning and knowledge

Embeddings. Numerical vector representations of text that capture semantic meaning. In RE, embeddings enable semantic search, duplicate detection, clustering, and traceability between requirements and related artefacts — even when the wording differs. They form the basis for many retrieval and analysis functions but do not generate text themselves.

Vector. A mathematical representation of data as an ordered list of numbers. In the context of embeddings, vectors capture semantic meaning by placing similar words, sentences or requirements close together in a multi-dimensional space. In RE, vectors enable similarity search, clustering of related requirements, and linking stakeholder needs with existing documentation even when different wording is used.

Fine-tuning. Adapting a pre-trained LLM to a specific domain or task by training it on additional, relevant data. For RE tasks, fine-tuning can align a model with domain terminology, regulatory language or organisational standards, improving the precision of generated requirements or compliance checks. It requires effort, domain expertise and quality-controlled data to be effective.

Retrieval-augmented generation (RAG). A process that uses embeddings to find relevant documents and then passes them into an LLM to generate an answer. In RE, RAG ensures outputs are based on actual specifications, stakeholder notes or regulatory documents — reducing hallucination risk and keeping results project-specific rather than drawn purely from the model's internal training data.