The GPU is the new oil

NVIDIA’s H100 GPU costs about $25,000. The waiting list is 6-12 months. Every major tech company, every AI startup, every government with AI ambitions wants them. There aren’t enough.

I spent a week trying to map who has how many GPUs. The numbers are approximate because nobody publishes exact counts. But the concentration pattern is clear, and it’s disturbing.

The map

The United States has the most. By far. Microsoft, Google, Meta, Amazon, and the hyperscalers collectively hold hundreds of thousands of H100s. The US government has its own stockpile for defense and intelligence AI programs.

China had significant GPU deployments before the October 2022 export controls. Since then, the flow of new NVIDIA GPUs to China has been restricted. China is now building domestic alternatives (Huawei’s Ascend, Biren, Moore Threads), but these are generations behind in performance.

The UK, through DeepMind and various cloud providers, has a meaningful cluster. The EU is fragmented, with individual countries (France, Germany, Finland) hosting smaller deployments.

The Gulf states (Saudi Arabia, UAE) are investing aggressively. Saudi’s NEOM project and UAE’s G42 are building GPU clusters with newly acquired hardware.

Everyone else has relatively little.

Why it matters

Here’s the thing about GPUs: they’re the bottleneck for AI development. You can have brilliant researchers, great datasets, and clever algorithms. Without GPUs (or equivalent accelerators), you can’t train large models. Period. The compute is the constraint.

Epoch AI tracks the compute used to train frontier models. The trend is clear: each generation of model uses 10x more compute than the last. GPT-4 reportedly used roughly 10x the compute of GPT-3, which used roughly 10x the compute of GPT-2.

If this trend continues, the organizations that can train frontier models will be the ones with access to tens of thousands of top-of-the-line GPUs. That’s a very short list.

The oil analogy

SemiAnalysis and Reuters have both drawn the oil comparison, and I think it’s more apt than it first appears.

Oil is geographically concentrated. So are GPUs (manufactured by TSMC in Taiwan, designed by NVIDIA in the US).

Oil access determines military and economic power. GPU access increasingly determines AI capability, which is becoming a determinant of military and economic power.

Oil created geopolitical alliances and conflicts. GPUs are already doing the same. The US export controls on China are explicitly about denying a strategic competitor access to the resource that powers the most important technology of the decade.

Oil required massive infrastructure (drilling, refining, pipelines). GPUs require massive infrastructure (fabs, data centers, power grids, cooling systems).

The analogy breaks down in some places. Oil is consumed when used. GPUs can be used indefinitely. Oil exists in nature. GPUs are manufactured. But the structural pattern of a critical resource with concentrated supply and universal demand, that pattern is the same.

The uncomfortable part

The countries that control GPU supply chains will have disproportionate influence over AI development. Right now, that’s the US (design, software), Taiwan (manufacturing), and the Netherlands (lithography equipment). A very small number of chokepoints controlling a technology that will shape everything.

The countries that don’t have GPUs will fall behind in AI capability. Not linearly. Exponentially. Because compute advantage compounds. More GPUs means better models, which means better tools for building even better models, which means an even larger advantage.

The concentration of compute power mirrors the concentration of other forms of power in ways that should make us uncomfortable. But it’s happening anyway. The GPU is becoming what oil was in the 20th century: the resource that determines who gets to shape the future.

I don’t know what to do with this observation except make it. And keep watching the allocation of silicon as closely as previous generations watched the allocation of petroleum.

Related thinking: