Unit Economics and the Margin Problem

The shape has changed

Conventional software: build it once, serve the millionth customer for approximately nothing. Gross margins of 75–85% are ordinary, and the business improves as it grows.

An AI product: every request costs money. More engaged customers cost more. Success raises cost of goods sold in direct proportion to the thing you are trying to maximise.

This is the central economic fact of the category, and its consequences run through everything:

• Gross margin is a design outcome, not a given.
• Your most enthusiastic users may be unprofitable.
• Unlimited-use pricing is dangerous in a way it never was for software.
• A quality improvement that doubles cost is a business decision, not an engineering one.

Calculating cost per unit of work

Prices for model inference change frequently and in both directions, so learn the calculation rather than any particular number:

Cost per request ≈ (input units + output units) × price per unit

Then aggregate up:

Cost per customer per month = requests per customer per month × cost per request

And the number that matters:

Gross margin = (price − cost to serve) ÷ price

Two things routinely omitted from "cost to serve" and both belong in it:

Retrieval and supporting calls. A single user-visible answer often involves several model calls — a query rewrite, a retrieval step, the generation, sometimes a check. Cost the whole chain, not the final call.

The retry and failure tail. Failed requests, retries and abandoned interactions all cost money and produce nothing.

Get a measured figure from a real sample rather than an estimate. If nobody can produce one, that is itself the finding — it means the product's margin is unknown.

The levers that move unit cost

Unit cost is not fixed by the provider. It is substantially a design outcome, and these are the levers:

Routing by difficulty is usually the largest single saving available. Most products send every request to their most capable model, and most requests do not need it. Classifying difficulty and routing accordingly frequently cuts cost substantially with no perceptible quality change on the easy majority.

Customers who destroy margin

A dynamic with no real SaaS equivalent: individual customers can be unprofitable, and the ones who are tend to be the ones who love the product most.

Where pricing is per seat or flat, a customer using the product ten times more than average costs ten times more to serve while paying the same. In software that is a success story. Here it is a loss.

Watch for:

• Power users on flat pricing. The distribution of usage is usually far more skewed than the distribution of price.
• Customers whose workflow encourages retries. Products where users regenerate repeatedly until satisfied.
• Large document processing on unlimited plans. Cost scales with input size, and document sizes vary enormously.
• Automated or programmatic use where a human is not naturally rate-limiting consumption.

The response is not to punish engagement. It is to know your cost per customer, price so that heavy use is either paid for or bounded, and design so that heavy use is not disproportionately expensive to serve.