MODULE 2 · LESSON 4
Free — no login requiredSign in to track progress, save quiz attempts and enrol in the full course.
Sign in to track progress / enrolWorking With an AI Team
Who does what
Titles vary between companies, but the functions are consistent.
Two observations that matter more than the list.
Data engineering is usually the bottleneck, and is usually under-resourced. Organisations hire data scientists first because that is the glamorous title, then discover the scientists spend most of their time doing data engineering badly. If you can fund only one role, and your data is scattered, fund the engineer.
Most companies do not need research scientists. The overwhelming majority of business AI is applying well-understood techniques to a specific problem. Hiring researchers to do applied work is expensive and frustrates everyone.
Specify the outcome, not the method
The single most common way non-technical stakeholders damage AI projects is by specifying the solution.
Unhelpful: "Build us a neural network that classifies incoming emails."
Helpful: "We receive about 400 emails a day. Three people spend two hours each morning sorting them into six categories. We want to cut that time substantially without misrouting anything urgent. Here is a year of past emails and how they were sorted."
The second version gives the team the problem, the constraint, the success measure and the data — and leaves the method to the people qualified to choose it. It may turn out that a simple keyword system solves 80% of it, which is a better outcome than a neural network and one you would have forbidden.
Agree the acceptance criterion in advance
Ask for a number, before work starts: what level of performance would make this worth deploying?
This is uncomfortable because nobody knows the answer at the start, and it is precisely why it must be discussed. Two failure modes it prevents:
- The moving goalpost. Without an agreed target, a model at 88% accuracy triggers "can you get it to 92?" indefinitely. Every increment costs more than the last, and there is no defined finish.
- The wrong metric. Accuracy is often the wrong measure. For fraud detection, where perhaps one transaction in a thousand is fraudulent, a model that flags nothing at all is 99.9% accurate and completely useless. What matters is how many frauds it catches and how many legitimate customers it annoys — and the balance between those two is a business decision, not a technical one.
That last point is where a non-technical stakeholder adds the most value in the whole project. Only the business can say whether missing one fraud is worse than wrongly blocking fifty good customers. The team cannot decide it, and if you do not decide it, someone will decide it by default.
Questions worth asking
You do not need to evaluate the mathematics. These questions are answerable in plain language and reveal most problems:
- "What would make this project fail?" A good team has a ready answer. A vague one is a warning.
- "What is the model getting wrong, and is there a pattern?" Errors usually cluster — a customer segment, a document type, a time of day. Patterns are actionable in a way that an overall accuracy figure is not.
- "How does this compare with just doing the obvious simple thing?" Every project should be measured against a baseline: current human performance, or a simple rule. Sometimes a model beats the baseline by very little, and it is much better to learn that in month two.
- "What happens to the people whose work this changes?" Ask early. It is both a humane question and a practical one, because a team that fears a system will find reasons not to use it.
The first conversation. A head of operations tells a data science team: "We want AI to predict machine failures. Use deep learning — I've read it's the most accurate."
Four months later there is a deep learning model with 76% accuracy on a small sensor dataset. It is not accurate enough to schedule maintenance around, and nobody can explain its reasoning to the plant engineers, who do not trust it. The project is shelved. The team is blamed for underperforming.
The second conversation. The same head of operations says: "Unplanned failures on line 3 cost us roughly two days of production each time, and we have about eleven a year. If we could catch even half of them a week ahead, we would schedule maintenance into planned downtime. We have three years of sensor logs and a maintenance record of every failure. What is possible?"
The team investigates and comes back with something unexpected: two failure modes account for most incidents, and both are preceded by a temperature pattern that a simple threshold rule detects with high reliability. No deep learning. Deployable in weeks. The plant engineers understand and trust it immediately, because they can see the rule.
They also report that the remaining failure modes look genuinely unpredictable from the available sensors, and recommend not spending money there until new sensors are fitted.
Same organisation, same data, opposite outcomes. The difference is entirely in the framing: the second conversation supplied the business context, the cost, the constraint and the data — and asked what was possible, rather than dictating the method.
For a fraud detection model where about one transaction in a thousand is fraudulent, why is overall accuracy a poor acceptance criterion?
Data engineer
Click to flipBuilds and maintains the pipelines that make data available, reliable and joined. Usually the bottleneck and usually under-resourced.
Click to flip backKnow who does what, and remember that data engineering is usually the real bottleneck. Specify projects by outcome, constraint, success measure and data — never by method — because prescribing the technique can forbid the simpler solution that would have worked better. Agree an acceptance criterion in advance, and take responsibility for the metric trade-off, because deciding whether a missed fraud costs more than a blocked customer is a business judgement no technical team can make for you.