GPU Poor: the Shortage and the Surplus Are Both Real

Illustration: chili pepper mascot illustrating the shortage and the surplus

There are two GPU crises in 2026, and they are happening in the same building.

In one room, an indie developer is refreshing a distributor page for the fourth week, waiting on a card that sells for €4,500 if it ever shows up. In another room — sometimes literally the same data center campus — a rack of H200s burns idle wattage at a rate that would make a CFO weep, except the CFO has already stopped looking.

The shortage and the surplus are both real. That's not a paradox. That's a business model.

The discourse

Matt Busel put the stakes bluntly: "Compute access is the great filter of modern intelligence." Talent is abundant. Ideas are abundant. Models are abundant, mostly because open weights keep falling out of labs faster than anyone can fine-tune them. What's scarce is the substrate underneath — the actual silicon that turns a clever architecture into something that runs. Busel's point isn't that innovation is dying. It's that innovation is being rationed by credit limit.

Sameer Koli went further and named the incentive structure. "We can't get enough GPUs" flatters everyone who says it. The infra team gets an external villain. Finance gets a documented reason to keep signing checks. Leadership gets to describe a spending spree as ambition rather than an admission that nobody is watching what's actually running. Koli's argument, drawn from Cast AI's data, is that your GPU bill isn't a capacity problem. It's a utilization problem. Laurent Gil, Cast AI's president and co-founder, summed the asymmetry up in one line: "A GPU sitting idle costs dollars per hour. A CPU sitting idle costs cents."

TheSageKhan ran the numbers on what that means for someone who just wants to run a model. Llama 3 70B at full precision wants ~140GB of VRAM. That's not one GPU. That's a small fleet, and the cards alone clear $10,000 before you've paid for the machine around them. Enterprise LLM API spending doubled to $8.4 billion in 2025. Meanwhile: "The indie developer of 2010 could build a competitive web application on a $500 laptop. The indie developer of 2026 must pay tribute to Amazon, Microsoft, or Google just to participate."

That word is doing a lot of work. Tribute.

Then on October 7, 2026, a consortium called the National Compute Network launched and said the quiet part in a press release: the industry has spent two years treating compute as scarce while a meaningful share of already-bought capacity sits idle. Their number, for independent single-tenant data centers, is under 15% net utilization. Their proposal is almost boringly sensible — connect existing data centers through a shared scheduling layer, match capacity to chip type, location, and price. Roughly 760 megawatts connected or scheduled today. A stated goal of 2 gigawatts by 2030. Organizers include Anjney Midha, a former Andreessen Horowitz general partner, and Sam Singh, who leads AI at 1X and is quoted in support.

If that sounds like a boring infrastructure story, it's because the boring version is the scandal. Fifteen percent. Imagine renting a warehouse, filling it with the most expensive equipment on earth, and using one-sixth of it. Then imagine telling everyone else there isn't enough warehouse.

Nobody's building for you

Here's the part the shortage narrative conveniently skips.

Every dollar of "compute scarcity" panic gets converted into construction. Hyperscalers pour concrete. Nvidia redirects GDDR7 modules away from consumer cards and into high-bandwidth memory for AI servers — which is why WebProNews reports there's no new gaming GPU launching in 2026, ending a roughly thirty-year streak of annual releases. Nvidia cut RTX 50-series production 30–40% in the first half of 2026. An RTX 5090 runs €3,800–4,500 in some markets; one premium variant is listed at €6,499. Availability is spotty, waits run weeks, and the forecast for anything resembling normal is late 2027 or mid-2028. Gamers, startups, and academic researchers are all in the same line, holding the same number, watching the same screen.

So who is all this new capacity for?

Not you. Empires. The hyperscaler buildout isn't aimed at the developer who wants to fine-tune a 7B model on a single workstation. It's aimed at the enterprise customer who will sign a three-year reserved-instance contract and never ask why utilization is a rounding error on a slide nobody presents.

And notice what happens to price when scarcity is the story. Cloud vendors raised H200 pricing roughly 15% this year, per Cast AI's report — breaking about two decades of falling compute costs. That's not supply and demand. That's supply, demand, and a marketing department.

The waste nobody photographs

Here's the sentence an AI press release would never print: the compute that could have trained your model was already bought, already installed, and already sitting there.

Go look at what the National Compute Network is actually proposing. Not new fabs. Not new megawatts. A scheduling system. Match a lab that needs compute with a cloud provider that has idle chips — by capacity, chip type, location, and price. That's it. That's the whole pitch.

If that works at even partial scale, it proves the shortage was never physical. It was organizational. It was contractual. It was a choice made by people whose compensation depends on the shortage continuing.

Seventy-six hundred megawatts of compute connected or scheduled today is genuinely good news. Two gigawatts by 2030 is a real target. But notice how modest it is relative to the headline numbers being thrown around. The industry has been screaming about scarcity for two years, and the fix being proposed is essentially "let the idle stuff talk to the needy stuff." That's not a moonshot. That's a scheduling product.

The scarcity was always the point. Scarcity is what lets you raise prices 15% in a year after two decades of declines. Scarcity is what makes "we're building the future" sound heroic instead of "we're building a landlord." Scarcity is what turns an indie developer with a genuinely good idea into someone who has to beg for API credits.

What "GPU poor" actually means

There's a meme version of GPU poor — the person running Stable Diffusion on a 3060 and complaining. Fine.

Then there's the real version. The solo researcher who can't replicate a paper because the reference implementation assumes eight A100s. The two-person startup that shipped a demo on borrowed credits and now has to choose between paying rent and paying for inference. The open-source maintainer who spent three months optimizing a model that half the internet now depends on and still can't afford to run it.

These people aren't poor because they lack talent. They're poor because the distribution of compute has been deliberately shaped to make them renters.

The self-built future got squeezed out of the room while everyone was arguing about whether AI would take their job. The infrastructure squeeze is the more interesting story: it's not about AI replacing developers, it's about AI infrastructure making sure developers can never quite own their own stack again.

The honest ledger

So here's the whole thing, laid out plainly.

Utilization under 15% at independent data centers. A consortium launched October 7, 2026, to connect what's already built. A cloud pricing environment that reversed twenty years of declining costs in a single year. A consumer GPU market that skipped its first annual release cycle in three decades. An indie developer who needs $10,000 in silicon to run a model that was given away for free.

The compute exists. The compute is running at a sixth of its capacity. The price is going up anyway.

Nobody is building more compute for you. They're building empires, and then renting you back the parts they're already not using.

That's not a shortage. That's a toll booth, and the line you're standing in was designed by the person collecting.

Sources & further reading

• The National Compute Network's case: independent data centers averaging under 15% utilization, and a plan to connect them.
• Matt Busel: "Compute access is the great filter of modern intelligence."
• Sameer Koli: your GPU bill is a utilization problem, not a capacity problem — including the $15%-higher H200 pricing that broke twenty years of falling compute costs.
• TheSageKhan: how Big Tech's infrastructure squeeze is killing the self-built future