Signals Inbox·August 30, 2026·AI Infrastructure

AI compute shortage: how bad is it now?

The AI compute shortage is still severe where it matters most: not for a developer renting a few GPUs, but for companies trying to assemble thousands of accelerators, HBM, data-center space and power at once.

Q1Is AI compute still in short supply today?

Yes, the AI compute shortage is still severe today once we look at the capacity needed by major AI companies rather than the ability to rent a few GPUs online.

The freshest numbers make the shortage difficult to dismiss. In Microsoft's latest fiscal-year results, Azure grew 43% and the company again said customer demand was higher than its available capacity. Microsoft had added another gigawatt of infrastructure during the quarter and says it is on track to roughly double its overall capacity in two years.

Nvidia is seeing the same pressure from another part of the stack. Its latest quarter produced $89 billion of Data Center revenue, up 117% from a year earlier. More revealingly, Nvidia's latest SEC filing shows that its commitments to secure supply and manufacturing capacity jumped from $119 billion to $279 billion in a single quarter. Nvidia says these commitments are mainly for memory and manufacturing capacity needed to meet demand over the next several years.

The physical data-center market is just as tight. CBRE's latest H1 2026 study found that North America's eight main data-center markets increased total supply by 33.7% year over year. Vacancy still sat at only 1.4%.

So this is a strange market: chip production, cloud capacity and data-center construction are all growing extremely fast, while spare capacity remains scarce. The AI compute shortage is still a real problem now.

Q2What is actually causing the AI compute shortage now?

Today's AI compute shortage comes from several bottlenecks piling up at once: advanced accelerators, HBM memory, complete server systems, data-center space and especially electricity.

The easiest mistake is to reduce “compute” to GPUs. Nvidia can manufacture another accelerator, but that accelerator still needs high-bandwidth memory, networking equipment, racks, cooling, a suitable data center and enough electricity to run continuously. A missing component anywhere in that chain can delay usable compute.

The bottleneck has also been moving. GPU availability has improved for smaller buyers. Memory has become much tighter lately. Large data-center projects are running into power constraints. Frontier labs increasingly want hundreds of megawatts or several gigawatts at once, which makes the electricity connection almost as important as the hardware.

That explains why we can simultaneously see H100s available for hourly rental and multi-billion-dollar companies saying they cannot get enough compute. Those customers are buying very different things.

Where the AI compute market is tightest now

Q3Are Nvidia's newest AI GPUs still actually hard to get?

For large deployments, Nvidia's newest AI systems are still hard to secure in the quantities customers want.

Nvidia's latest quarter is a useful reality check. Data Center revenue reached $89 billion, more than double the level a year earlier, while Blackwell remained the majority of system shipments. Nvidia has already started production shipments of Vera Rubin and says it is experiencing supply constraints across current and upcoming systems.

The company's purchasing behavior is even more striking. Nvidia increased supply and capacity commitments from $119 billion to $279 billion in one quarter. According to its SEC filing, those commitments mainly cover memory and manufacturing facilities needed for current and future data-center products.

That $160 billion increase is larger than the annual revenue of most technology companies. Nvidia is effectively reserving years of upstream capacity because it expects customers to absorb enormous volumes of hardware.

Availability also depends heavily on scale. A company looking for eight B200s has many more options today than one trying to deploy 20,000 of them in a single high-speed cluster. Once thousands of accelerators have to arrive together with networking, memory, cooling and power, the market gets tight very quickly.

So the current Nvidia shortage is mainly a scale problem. Individual GPUs have become easier to find, while very large blocks of the newest Nvidia infrastructure remain heavily competed for.

Q4Has HBM memory become one of the worst AI compute bottlenecks?

Yes, HBM memory has become one of the nastiest AI compute bottlenecks currently, and the latest forecasts suggest the pressure will continue.

TrendForce now expects HBM bit shipments to grow another 50% to 60% in 2027 and still believes supply will fail to match demand. It also estimates that HBM will consume roughly 30% of DRAM wafer input by the end of 2027, up from about 18% at the end of 2025.

That has consequences well beyond HBM itself. Producing more HBM consumes wafer capacity that could otherwise produce conventional server memory. TrendForce expects the broader DRAM market to remain undersupplied through 2027, with meaningful new factory output arriving too slowly to solve the problem in the near term.

The shortage is already changing hardware decisions. TrendForce says Nvidia has been evaluating lower-memory alternatives for Rubin Ultra because of HBM4E supply and production-yield uncertainty. Cloud providers and server manufacturers have also reduced memory configurations on some systems.

Micron's financial results show how unusual the market has become. Its Cloud Memory business generated $13.8 billion in its latest reported quarter, up from $3.4 billion a year earlier. Its Core Data Center business went from roughly $1.5 billion to $11.5 billion over the same period. Gross margins in those two businesses reached 83% and 87%.

Those margins tell us that buyers are competing aggressively for memory. Nvidia's $279 billion of supply commitments, which it says are primarily related to memory and manufacturing, reinforce the same conclusion from the customer side.

HBM has gone from a component people rarely discussed to one of the main limits on how quickly new AI compute can come online.

Q5Is electricity now slowing AI data centers more than chips?

At the largest AI campuses, electricity is currently one of the hardest constraints because data centers can be built much faster than the power grid around them.

The International Energy Agency estimates that a new data center can often be delivered in one to three years. Planning, permitting and building major grid infrastructure can take five to fifteen years. Those timelines simply do not match.

The industry is adjusting around that problem. CBRE says the ability to secure more than 300 MW of electricity within roughly three years has become one of the main factors determining where developers build new U.S. data centers. Fiber connectivity used to dominate site selection. These days, a site with power can beat a more obvious technology hub without it.

Large AI campuses make the mismatch worse. A 50 MW data center was already a major project a few years ago. Developers are now discussing 500 MW sites and multi-gigawatt campuses. Those projects can require new substations, transmission lines and sometimes new electricity generation.

The IEA estimates that AI-focused data-center electricity consumption grew about 50% in 2025. It also expects data centers to drive roughly half of the increase in U.S. electricity demand through 2030.

This is why technology companies are getting involved directly in electricity supply. Long-term nuclear agreements, natural-gas projects, renewable contracts, batteries and behind-the-meter generation have all become part of AI infrastructure planning.

At frontier scale, getting enough electricity in the right place can now take longer than getting the chips.

Q6If so many data centers are being built, why is there still almost no spare capacity?

Because companies are reserving new AI data-center capacity almost as quickly as developers can build it.

CBRE's newest North American numbers are unusually clear. Capacity across the eight main markets reached about 10.9 GW, up 33.7% in one year. Another 7.48 GW was under construction during H1 2026, a record amount.

Yet more than 80% of that construction had already been preleased. A year earlier, the figure was 74.3%. The amount of uncommitted capacity under construction was below 1.5 GW, which CBRE estimates represents roughly six months of supply at the current pace of demand.

Vacancy remained at 1.4%. For context, CBRE says preleasing historically tended to run around 40% to 50%. We are now above 80% while the construction pipeline itself is at a record.

JLL reaches a similar conclusion from a broader dataset. It says 66 GW of data-center capacity is currently under construction and still expects vacancy to remain near zero through 2028.

The trajectory is the part to watch. Supply is already growing by roughly one-third a year in the largest North American markets. Demand is absorbing enough of that growth to prevent vacancy from recovering.

North American primary data-center markets, H1 2026

Q7Are Microsoft, Google, Amazon and Oracle actually running short of AI capacity?

Yes, the biggest cloud companies are still spending extraordinary amounts because AI capacity remains tight enough to limit how much business they can take.

Microsoft gives us the clearest direct admission. Azure grew 43% in its latest quarter, but customer demand still exceeded available capacity even after Microsoft added another gigawatt. The company says it is trying to roughly double overall capacity within two years.

Google's numbers show the same race from a different angle. Google Cloud revenue recently jumped 82% to $24.8 billion in one quarter. Alphabet increased its planned capital spending again, to roughly $195 billion to $205 billion for the year, after entering 2026 with a lower forecast. Earlier Alphabet earnings calls had already described the cloud business as operating in a tight supply environment.

AWS is growing faster too. Its latest quarterly revenue rose 37%, the fastest growth rate in 18 quarters, taking AWS to a $169 billion annualized revenue run rate. Amazon expects around $200 billion of capital spending this year, much of it tied to AI and cloud infrastructure.

Oracle provides perhaps the strangest evidence. Its remaining performance obligations reached $638 billion, up 363% year over year. Oracle said most of the huge increase in its previous two quarters came from large AI contracts where customers either prepaid Oracle to buy GPUs or supplied the GPUs themselves. Those prepaid and customer-supplied hardware commitments totaled $75 billion.

Customers paying in advance for hardware so their cloud provider can build capacity faster is far from normal cloud procurement. It tells us how valuable guaranteed access to future AI infrastructure has become.

Q8Can a normal AI startup still rent enough GPUs today?

Usually yes, because the AI compute shortage is much less painful for a startup renting a handful of GPUs than for a company trying to secure an entire AI campus.

Current GPU rental markets are fairly liquid at small scale. Silicon Data's daily market benchmark currently puts an H100 around $2.65 per GPU-hour and a B200 around $5.58. Individual neocloud providers advertise H100, H200 and B200 capacity at similar orders of magnitude.

This changes the practical meaning of “shortage.” A founder who needs four H100s for an experiment can often start quickly. Availability across RunPod, GMI, Lambda, Vast.ai, CoreWeave and other providers has made that part of the market far healthier than during the first H100 rush.

Large clusters behave differently. Prices vary sharply between marketplaces, neoclouds and hyperscalers, and the cheapest capacity may lack the networking, reliability, location or long-term guarantee required for a serious production workload.

Geography adds another complication. Capacity available in one country may be useless to a company that needs European data residency, a particular cloud region or locally hosted sovereign AI infrastructure.

So small-scale GPU access is currently fairly good. The shortage becomes much more obvious once the buyer needs thousands of identical accelerators working together for months.

Current GPU rental market references

Q9How much worse is the compute shortage for frontier AI labs?

The AI compute shortage becomes extreme at frontier-lab scale because companies such as Anthropic and OpenAI are securing capacity in hundreds of megawatts and gigawatts rather than in individual GPUs.

Anthropic gives us a useful sense of scale. Earlier this year it committed more than $100 billion over ten years to AWS technologies and secured up to 5 GW of new compute capacity. Anthropic says it already uses more than one million Trainium2 chips.

The company then signed another agreement with Google and Broadcom for multiple gigawatts of next-generation TPU capacity. More recently, Anthropic agreed to use all of the compute at SpaceX's Colossus 1 data center, giving it access to more than 300 MW and over 220,000 Nvidia GPUs.

OpenAI is moving in the same direction. Its partnership with Amazon includes plans to consume 2 GW of Trainium capacity through AWS. That sits alongside its other large infrastructure projects and cloud relationships.

These numbers make the scale problem easier to understand. One gigawatt of continuous load is roughly the power consumption of a substantial city. Frontier companies are securing several gigawatts each.

At that level, choosing a GPU model becomes only one part of the job. The buyer needs chips, memory, high-speed networking, buildings, cooling, electricity and financing to arrive together.

That is why frontier labs sign infrastructure agreements years before all the hardware exists. Waiting for an open cloud instance would be completely unrealistic at their scale.

Q10Is inference now eating as much AI compute as model training?

Inference is increasingly becoming the bigger long-term source of AI compute demand because every new user, agent, video and reasoning task creates recurring consumption.

Training still produces some of the largest single clusters in the world. A frontier model may occupy tens or hundreds of thousands of accelerators during training. But once that model is released, inference can run millions or billions of times.

The workload itself has also become heavier. According to the IEA's latest analysis, energy use per basic AI task has been falling by at least an order of magnitude annually thanks to better hardware and software. At the same time, reasoning, video generation and agentic tasks can consume hundreds or even thousands of times more energy than a simple text query.

Usage is expanding fast enough to overwhelm much of the efficiency gain. The IEA found that electricity consumption at AI-focused data centers still grew about 50% in 2025.

Amazon offers another clue. Trainium2 already powers the majority of inference on Bedrock, and Amazon said the 1.4 million Trainium2 chips it had deployed were fully subscribed. The company designed later Trainium generations partly around the economics of serving enormous inference workloads.

TrendForce also expects agentic AI to increase memory consumption because agents repeatedly call models, maintain longer contexts and create larger KV caches rather than producing one short answer and stopping.

Training still gets the headlines, but inference is what turns AI compute demand into a recurring infrastructure load. As AI products get used more often and perform longer tasks, that load keeps expanding.

Q11Are Google's TPUs, Amazon's Trainium and Microsoft's Maia fixing the shortage?

Custom AI chips are easing the pressure, but demand is growing quickly enough that Google, Amazon and Microsoft are filling much of the extra capacity themselves or selling it almost immediately.

Amazon is already a good example. Earlier this year, the company said all 1.4 million deployed Trainium2 chips were subscribed. It also expected almost all Trainium3 supply to be committed by mid-2026. Anthropic uses more than one million Trainium2 chips, while OpenAI has agreed to consume 2 GW of Trainium capacity in the future.

Google has been building TPUs for more than a decade, giving it the most mature Nvidia alternative among the hyperscalers. Anthropic's agreement for multiple gigawatts of next-generation Google and Broadcom TPU capacity shows that these chips are now becoming infrastructure for companies outside Google itself.

Microsoft is pushing Maia in the same direction. Maia 200 is already supporting OpenAI and Microsoft's own models, and Microsoft says the system delivers about 30% better performance per dollar than the latest-generation hardware in its existing fleet.

These chips clearly increase the world's effective AI compute supply. They also let hyperscalers match different workloads with different hardware rather than using an expensive Nvidia GPU for everything.

The interesting part is how quickly the alternatives are being consumed. Amazon's proprietary chips can be fully subscribed. Anthropic is reserving TPU capacity years ahead. Microsoft continues adding its own accelerators while Azure remains capacity-constrained.

Custom chips are widening the supply base, and for now the extra capacity is finding buyers very quickly.

Q12Won't faster AI chips eventually make the compute shortage disappear?

Probably not anytime soon, because every big efficiency improvement is currently encouraging companies to run more AI rather than leaving accelerators idle.

The efficiency gains themselves are enormous. The IEA estimates that energy consumption per basic AI task has fallen by at least tenfold each year in recent years. New accelerators generate far more tokens per watt and per dollar than previous generations.

Usage is moving just as quickly. AI companies are adding reasoning, video, persistent agents, voice, longer contexts and more frequent model calls. Many of those tasks consume vastly more compute than basic text generation.

We can see the rebound effect in aggregate electricity demand. AI-focused data centers used roughly 50% more electricity in 2025 even though the cost and energy required for basic inference were collapsing.

Microsoft has seen the same phenomenon inside its own infrastructure. It increased Copilot workload throughput fourfold through hardware and software optimization during the year. Azure still ended the period with customer demand above available capacity.

Efficiency therefore changes how much useful work one accelerator can do, but users keep finding new work for it. Cheaper tokens make more applications economical, which creates another layer of demand.

For the compute shortage to disappear through efficiency alone, improvements would have to outrun both AI adoption and the rapid increase in compute used per advanced task. These days, that is clearly not happening.

Q13Could the industry be building too much AI compute anyway?

Yes, an AI compute glut is a real future risk, although current utilization and capacity data still look much more like shortage than oversupply.

The amount of money being committed is extraordinary. The IEA estimates that capital expenditure by five major technology companies exceeded $400 billion in 2025 and could rise another 75% in 2026. That would put their combined annual spending around $700 billion.

Oracle alone raised $43 billion of debt and $5 billion of equity during its latest fiscal year to support expansion, and it expects to raise roughly another $40 billion during the following fiscal year. Nvidia now has $279 billion of supply and capacity commitments, plus large commitments tied to cloud agreements, data-center leases and AI infrastructure partners.

This creates obvious risk. Hardware improves quickly. Some projects will be cancelled. Some customers may reserve more capacity than they eventually need. AI revenue could grow more slowly than infrastructure spending. Financing costs could also force weaker developers to abandon projects.

Recent investor scrutiny of giant AI financing arrangements shows that the market has started asking harder questions about who ultimately pays for all this infrastructure.

Current operating data still points in the other direction. Nvidia's Data Center business is growing more than 100% year over year. Major clouds are expanding quickly. HBM remains constrained. North American data-center vacancy is 1.4%, and more than 80% of capacity being built in the main markets has already been reserved.

A future glut is plausible, perhaps even likely somewhere in the cycle. We just do not see broad spare capacity yet.

Q14What would tell us the AI compute shortage is finally ending?

We will know the AI compute shortage is genuinely easing when spare capacity appears across several parts of the market at the same time.

The first place to watch is data centers. A sustained rise in vacancy from today's extremely low levels would show that construction is finally outrunning new leasing. Preleasing would also need to fall from the current 80%-plus level toward something closer to historical norms.

Memory would give us another clue. HBM and server DRAM prices should weaken, suppliers should lose some bargaining power, and hardware companies should stop redesigning products around memory availability.

The cloud providers would change their language too. Microsoft saying that capacity is comfortably ahead of Azure demand would carry far more weight than a lower H100 rental price. The same applies to hyperscalers reducing infrastructure investment because they have enough compute rather than because customer demand has weakened.

At the frontier, Anthropic, OpenAI and other labs would have less reason to reserve gigawatts several years ahead. Shorter contracts and more flexible procurement would suggest that large blocks of capacity had become easier to find.

GPU rental markets by themselves can give a false impression. H100 prices can fall because newer hardware is replacing them while the frontier remains constrained.

For now, the indicators that matter most still show a tight market: very low data-center vacancy, heavily preleased construction, constrained memory supply, enormous forward hardware commitments and multi-gigawatt reservations from the biggest AI labs.

Q15So how bad is the AI compute shortage now?

The AI compute shortage is still very bad at the top end today: we would rate it around 8/10 for frontier AI companies, while a small developer who needs a few GPUs faces something closer to 3/10.

This difference explains much of the confusion around the topic. Anyone can open a GPU marketplace and find an H100. At the same time, Anthropic is securing several gigawatts of future capacity, Oracle customers are prepaying billions for hardware, and cloud providers are spending hundreds of billions of dollars to expand infrastructure.

The freshest numbers reinforce that gap. Nvidia's Data Center revenue reached $89 billion in its latest quarter, up 117%, while the company raised supply commitments from $119 billion to $279 billion. CBRE found that North American primary-market data-center supply grew 33.7% year over year while vacancy remained stuck at 1.4%. HBM suppliers are still expected to struggle to keep up with demand next year.

As seen above, Microsoft has even added another gigawatt of capacity in a single quarter while Azure demand continues to run ahead of what the company can provide. That's a particularly strong test because Microsoft has the money, purchasing power and technical expertise to build almost anywhere.

The real constraint today is complete, power-ready AI infrastructure at scale. GPUs are part of it, but the hardest problem is getting accelerators, HBM, networking, cooling, buildings and electricity to arrive together in the quantities frontier customers now require.

The AI compute shortage remains severe now, especially for the companies pushing the frontier. Small-scale GPU access has improved a lot, but the industrial-scale shortage has moved deeper into memory, data centers and electricity, where adding supply takes much longer.

AI compute shortage severity by buyer type