How to Choose an AI Project Worth Doing

Choosing badly is the most expensive mistake available in AI, because a failed first project does not just waste its budget — it poisons the organisation's appetite for the next three years.

Two lenses, both required

Every candidate project must pass two independent tests.

Business value. Is this worth money or materially better outcomes? Would anybody notice if it worked?

Technical feasibility. Can it actually be built with the data that exists? This is where module 1's rule of thumb earns its keep: could a person do this in about a second, and are there thousands of labelled examples?

Projects fail when one lens is applied without the other. Business people propose valuable impossibilities. Technical people build feasible irrelevancies — an accurate model of something nobody needed predicted. You need people from both sides in the room, early, and the most useful thing a non-technical sponsor does is insist on that conversation before the budget is committed.

🔗 Match the Pairs

Valuable and feasibleDrop here

Valuable but not feasible with current dataDrop here

Feasible but low valueDrop here

Neither valuable nor feasibleDrop here

Choose your first projects to succeed

For an organisation's first AI work, the primary objective is not the direct return. It is to prove to your own organisation that this works, and to build the muscle — the data plumbing, the deployment path, the working relationship between the technical team and the business.

That argues for a first project that is:

Quick. Something showing results in six to twelve months, not three years. Sponsors and priorities change.
Meaningful but not existential. Big enough that success is noticed, small enough that failure does not damage the company. Never make the first project the one the year's numbers depend on.
Well-instrumented. Choose an area where you can measure whether it worked. A project whose success is a matter of opinion will be argued about instead of learned from.
Supported by data you already have. A first project that begins with an eighteen-month data collection programme will not survive.

This is deliberately unambitious, and the reason is organisational rather than technical. The second project is much easier to fund and staff if the first one worked.

A due-diligence checklist

Before committing, get answers to these. None require technical knowledge to ask, and vague answers are themselves the finding.

What is the input and what is the output? If this cannot be stated in one sentence, stop here.
Where do the labelled examples come from, and how many are there? "We'll get the data" is not an answer.
How will we know it worked? Name the metric and the number that counts as success, in advance.
What does the model's output actually change? Who sees it, in which tool, and what do they do differently? If nobody changes any behaviour, the project has no value regardless of accuracy.
What happens when it is wrong? Every model is wrong sometimes. Who absorbs that, and how bad is it?
Who owns it in eighteen months? Models need monitoring and retraining. An unowned model quietly rots.
Should we buy this instead of building it? If your problem is genuinely generic — transcription, translation, document extraction — buying is usually correct. Build where your data or your problem is distinctive.

Question 4 is the one most often skipped and most often fatal. A team can deliver an accurate model that changes nothing because its output arrives in a report nobody opens.

In 2006 Netflix offered a million dollars to anyone who could improve its recommendation algorithm by 10%. It became one of the most famous competitions in the field's history, ran for three years, and drew thousands of teams.

In 2009 a team called BellKor's Pragmatic Chaos won with a 10.05% improvement. Netflix paid the million dollars.

And then never deployed the winning solution.

Netflix explained the reasoning publicly. The winning entry was an enormous ensemble of many models blended together. The accuracy gain was real, but the engineering effort needed to bring it into production could not be justified by the additional benefit. The competition models had been built to handle around 100 million ratings, while Netflix's live system was dealing with billions. And the business had moved: by the time the prize concluded, customers were streaming rather than queuing DVDs, which changed what the recommendation problem even was.

Netflix did take the two best-performing individual models from the ensemble and put a simplified version into production — capturing most of the value at a fraction of the complexity.

Three lessons a non-technical sponsor should take from this:

Accuracy is not the objective; value delivered in production is. A 10% gain that cannot be run is worth nothing, and a 6% gain that ships beats it entirely.
Complexity has a permanent cost. Someone maintains every model, forever. A simple model that a small team can operate often beats a sophisticated one that only its author understands.
Long projects get overtaken by the business. Three years is long enough for the problem itself to change. This is the strongest practical argument for choosing first projects you can finish quickly.

❓ Knowledge Check

Which of these are good reasons to prefer a modest first AI project over an ambitious one? Select all that apply.

📚 Flashcards1 / 5

Term

The two lenses

Click to flip

Definition

Every candidate project must pass both business value and technical feasibility. Applying one without the other produces valuable impossibilities or feasible irrelevancies.

Click to flip back

💡Key Takeaway

Judge every project through two lenses at once — is it valuable, and is it feasible with the data that exists — and get both kinds of people in the room before the budget is set. Pick early projects to succeed rather than to impress: quick, measurable, survivable, and supported by data you already have. Then run the checklist, paying particular attention to what the output actually changes and who owns it in eighteen months.