MODULE 1 · LESSON 2
Free — no login requiredSign in to track progress, save quiz attempts and enrol in the full course.
Sign in to track progress / enrolInside a Chatbot: From Prompt to Response
Educational objective 1.2 asks you to explain what happens behind the scenes when you interact with a chatbot. The syllabus gives a practical reason for learning this rather than a theoretical one: knowing the backstage flow helps you troubleshoot odd responses, write sharper prompts, and specify realistic system requirements when your project is the one building the chatbot.
A chatbot looks like a black box. A request actually passes through several stages, and each one contributes to the answer you finally get. The pipeline below varies slightly between providers, but the stages are stable enough to reason with.
The six stages
Two details in that list do more explanatory work than the rest.
The model has no memory. In a chat, every previous turn is added to the prompt as pre-text. The syllabus puts it vividly: imagine the chatbot has no memory and has to read the whole conversation again with each input. This single fact explains a family of behaviours that otherwise look arbitrary. It is why long conversations get slower and more expensive. It is why something you said thirty turns ago can suddenly stop influencing the answers — it fell out of the window. And it is why starting a fresh chat is often the fastest fix for a conversation that has gone off the rails.
Providers may loop. The pipeline is not strictly one pass. Providers may cycle through context assembly and inference repeatedly, call additional tools, or run follow-up chain-of-thought prompts until quality criteria are met. What you see as one reply may be several internal rounds.
Diagnosing real failures with the pipeline
The value of knowing these stages is that odd behaviour stops being mysterious and becomes locatable.
"I attached a 200-page specification and it ignored half of it." Input and context assembly. The document exceeded what could be carried in the context window, so part of it was never available at inference. The fix is at the input stage — chunk the document, retrieve only relevant sections — not at the prompt stage.
"It answered in English when I wrote in German." System context. Default language is set by the system prompt, the training data and the provider's built-in settings. You can complement it with user input, but you are pushing against a default, not setting a blank field.
"The same prompt gave me a different answer this morning." Inference. Generation is probabilistic by default. Nothing broke.
"It swore at me / it refused a harmless request." Pre-processing and post-processing. Unsafe-content handling sits at both ends of the pipeline, and it varies strongly between providers. This is also why the same prompt sent to two different vendors can produce one refusal and one answer.
"It gave me the answer, then corrected itself mid-reply." A provider looping through inference more than once, or a reasoning model exposing part of its internal drafting.
When your project is building the assistant rather than using one, each stage in this pipeline becomes a place where requirements live. Working through Elily, Srileo's WhatsApp assistant, the stages map onto real specification questions:
- Input — which media types must be accepted? A customer sending a voice note to a clinic is common, and deciding whether that is transcribed or rejected is a requirement, not an implementation detail.
- Pre-processing — what counts as unsafe or out-of-scope content, and what happens to it? Silently dropping a message and telling the sender it was dropped are different products.
- Context assembly — which business facts must be retrieved for this tenant, and what happens when retrieval returns nothing relevant? The fallback behaviour is a requirement with real consequences: an assistant that guesses opening hours is worse than one that says it does not know.
- Inference — what latency is acceptable, and is a slower, more careful answer better than a fast one? On a messaging channel where people expect a human-like delay, this is a genuine trade-off.
- Post-processing — what formatting, length limits and disclosure does the channel require?
- Output — what is the escalation path to a human, and what triggers it?
Notice that most of these are quality requirements and constraints rather than functional ones. That is typical for AI-supported systems, and it is one reason a Requirements Engineer who understands the pipeline writes better specifications for them.
A stakeholder complains that after a long chat session, the assistant "forgot" a constraint they stated at the very beginning. Which stage of the pipeline best explains this?
Pre-processing
Click to flipThe pipeline stage that tokenizes text, removes unsafe content and normalises language. Non-textual input such as speech is converted to text here.
Click to flip backA request passes through input, pre-processing, context assembly, inference, post-processing and output, and providers may loop through context assembly and inference several times before answering. The model has no memory: every turn re-reads the entire conversation. Knowing these stages turns strange behaviour into a diagnosable problem and tells you where the requirements live when your own project is building the assistant.