MODULE 2 · LESSON 1
Free — no login requiredSign in to track progress, save quiz attempts and enrol in the full course.
Sign in to track progress / enrolThe Workflow of a Machine Learning Project
The three stages, and the loop
At the highest level a machine learning project has three stages:
The stages are accurate. The arrow between them is a lie. Real projects loop:
You collect data. You train a model. It performs badly. You investigate and discover the data is missing something important, or the labels are inconsistent between two annotators, or an entire customer segment is absent. So you go back and collect different data. Then train again.
Nobody gets it right first time, and a team that appears to be going backwards in month two is usually doing the job properly.
This has a direct implication for how you sponsor a project: budget for iteration, and do not treat "we need to go back to the data" as a failure report. A team that never revisits its data is either working on an unusually clean problem or not looking hard enough.
Where the time really goes
A common shape for the effort, and it surprises most sponsors:
- Understanding the problem and finding the data — a large fraction, often the largest.
- Cleaning, joining and labelling — the other large fraction.
- Actually training models — often the smallest part, and the part everyone pictures.
- Deployment and integration — consistently underestimated.
The modelling step is the glamorous one and the short one. If a plan allocates most of the schedule to modelling, the plan was written by someone who has not shipped one.
Deployment is a beginning
Three things start when a model goes live:
Monitoring. Accuracy in the lab is not accuracy in the world. Real inputs are messier, and the world moves.
Maintenance and retraining. Because the world moves, a model that was accurate at launch degrades. This is often called model drift, and it is normal rather than a defect. Somebody has to own noticing it.
Feedback. A deployed model generates new data — including its own mistakes, once someone corrects them. That is the most valuable training data you will ever get, and it only accrues if you designed a way to capture it. Many teams do not, and throw away the compounding advantage.
Concretely, for a mid-sized company automating support ticket routing.
Month 1. The team defines the problem: ticket text in, destination team out. They pull 40,000 historical tickets. It emerges that the "assigned team" field was frequently overwritten when tickets were escalated, so the recorded team is often the final team, not the correct first routing. The label is not what everyone assumed.
Month 2. They reconstruct better labels from the ticket history, and have two support leads manually label 3,000 tickets to create a trustworthy evaluation set. The leads disagree on about one in eight, which forces a genuinely useful conversation about what the routing rules even are. That conversation improves the human process regardless of the model.
Month 3. First model: 71% accurate. Investigation shows most errors are on billing-versus-account questions, which genuinely overlap. They decide those two queues can be merged for routing purposes — a process decision, not a technical one. Accuracy jumps to 84%.
Month 4. Deployment. Auto-route above 90% confidence; below that, show a suggestion to the human triager. About 60% of tickets auto-route.
Month 5 onwards. The triager's corrections are captured as new labels. A new product launches and a category of ticket appears that the model has never seen; the confidence threshold sends them to humans, exactly as designed, and within weeks there are enough labelled examples to handle them.
Notice what this project actually was. Two months on data and definitions, one month of modelling with the biggest gain coming from a process decision, and a design where the model's uncertainty is a feature rather than a problem. That is what a good machine learning project looks like from the inside.
Three months into a machine learning project, the team reports they are "going back to collect different data because the labels were inconsistent." How should a sponsor read this?
The three stages
Click to flipCollect data, train the model, deploy the model — with loops back to earlier stages throughout, because training reveals problems in the data.
Click to flip backA machine learning project collects data, trains a model and deploys it — but loops constantly, because training is what reveals what is wrong with the data. Most of the effort goes into understanding the problem, cleaning and labelling, not into modelling. And deployment starts the work rather than finishing it: monitoring, retraining as the world drifts, and capturing corrections as the next round of training data.