Capacity, Scarcity and the Roadmap

Compute is not simply available

Ordinary cloud computing conditioned a generation of product teams to treat capacity as infinite and instantaneous. Need more servers, request more servers.

AI accelerator capacity has behaved differently. It is concentrated in a small number of suppliers and providers, it requires data centres with power and cooling that take years to build, and demand has repeatedly outpaced supply. The practical result for a product team is that capacity is allocated rather than simply purchased, and the allocation may not match your growth curve.

This is why the terms most worth negotiating with a provider are frequently not about price:

• Committed capacity at a defined rate, especially at peak.
• Rate limits high enough for your projected growth, with a defined path to raise them.
• Regional availability for the models you depend on, in the regions you need.
• Notice of deprecation, so a model you have built on is not withdrawn on a timeline you cannot absorb.

A cheaper per-unit price with a rate limit you will hit in six months is a worse deal than a higher price with headroom, and the comparison is easy to get wrong because only one of those numbers appears in the quote.

Sovereign and national compute

A development shaping availability: governments have moved to treat AI compute as strategic infrastructure, funding national or regional capacity rather than relying entirely on foreign providers. The pattern recurs across many countries — a national fund, a cloud or telecommunications partner, an allocation of accelerators, and a sovereignty condition attached.

Specific capacity figures in this area move very fast and are reported inconsistently, so treat any particular number you read as provisional and check it against a primary source before relying on it. The structural point is stable and is what matters for planning: where compute is located is increasingly a policy question as well as a commercial one, and in some markets the available option may be a national provider rather than a global one. Module 6 covers the residency consequences.

Planning a roadmap under capacity uncertainty

Four practices:

Know your headroom. What is your current rate limit, what is your current peak, and what growth does the gap allow? Teams frequently cannot answer, and discover the limit during their best week — a launch, a campaign, a seasonal peak.

Load-test to the limit deliberately. Find out what happens at your ceiling before your customers do. The behaviour at the limit — hard rejection, queueing, degraded latency — determines what your fallback must do.

Design for graceful degradation at capacity, exactly as module 2 required for outright failure. Capacity exhaustion is more common than provider outage and is usually less well handled.

Keep a second path viable. Module 8 covers this properly. The short version: an alternative provider you have actually tested, with evaluation confirming acceptable quality, is the difference between a capacity problem and an outage.

Signals you are approaching a limit

• Rising rates of throttling responses, even if the product still works.
• Latency rising at peak hours specifically, with off-peak unchanged.
• A provider declining to raise a limit, or offering a longer timeline than your growth allows.
• Growth in requests outpacing growth in users — often caused by a feature that retries, or by a new surface that calls the model more per interaction.

That last one deserves attention because it is self-inflicted and easy to miss. A product change that adds a second model call per interaction doubles consumption without any change in customers, and it surfaces as a capacity problem months later with no obvious cause unless you track requests per user.