Is AI a Bubble?

No matter how big it gets, it can't suspends the laws of economics..

We are in the first innings of a massive infrastructure build-out in which demand for AI chips still far outweighs the supply, and we see announcements of new AI-data centers costing tens of billions of dollars.

Some, like famous short-seller Jim Chanos, are starting to wonder whether this is a bubble. In fact, Chanos does more than wonder; he argues it alread is, comparing the AI build out to the fracking revolution, which did create a huge boom in energy production, but many frackers struggled to generate an ROI because of the steep decline rate of wells. Many went bankrupt.

At first sight, this analogy seems persuasive:

• The CapEx costs are enormous; Chanos talks about $400-$500B by just a handful of hyperscalers.

• For steep well decline rates, one can substitute steep depreciation of the gear that CapEx has bought (mostly servers and GPUs). The economic lifetime of these doesn’t exceed 3-4 years.

• The running cost of the biggest AI server farms is also steep, they can need as much energy as a decently-sized US city.

We wondered ourselves about ROIs, especially after DeepSeek appeared, as the latter:

• Using partly innovative approaches, the model performs on par with peers but was much cheaper to train.

• Deepseek used considerably less data to train, and others (Hong Kong University) have emulated this with even much less data, especially relevant as scaling seems to have hit a limit.

• Its cost advantage triggered a price war in China, which has spread to the rest of the world with the model almost an order of magnitude cheaper than most other leading models.

• It’s a victory for open-source models and is democratizing AI with Hugging Face, noting 500+ derivative models appearing within the first week of R1’s appearance.

Basically, it commoditized these large LLMs, which were running out of data to train on and didn’t produce much improvement over previous generations. Scaling was hitting a wall even before DeepSeek arrived:

OpenAI struggled to get Orion to deliver significant performance gains over GPT-4 however, according to a November report in The Information, and a Wall Street Journal report in December... OpenAI conducted at least two large training runs for Orion, each lasting several months, but the results fell short of what company researchers were hoping for.

Indeed, ChatGPT 4.5 just arrived:

GPT-4.5 offers marginal gains in capability and poor coding performance despite 30x the cost.

30x the cost, how about that for a dead end..

So, with large LLMs largely commoditized, these aren’t producing the ROIs on the tens of billions poured into new data centers. But what is? Well, to answer that, we have to look into the future, but fortunately, given the steep depreciation rates, we don’t have to look very far.

AI’s near future

We see several shifts ongoing:

• From training to inference

• The emergence of reasoning and agentic models

• From LLMs to specialist SLMs running at the edge.

Keep in mind this is by no means exhaustive, and something new and unexpected may arrive in the 3-year timeframe we’re taking into consideration.

Inference (using the models to do stuff) will be a much bigger market, orders of magnitude (a billion times bigger, according to Nvidia CEO Jensen Huang has argued that inference will be 1Bx larger than training).

That should greatly increase the demand for AI servers, although it’s somewhat countered by the fact that inference is much simpler and requires fewer resources (and, not unimportant for Nvidia, can be more easily done on a variety of hardware).

However, the advent of reasoning models (like DeepSeek’s R1 and OpenAI’s o1 and o3 models) tax resources a great deal more (and are in part a substitute for training, but never mind that at the moment).

While inference and reasoning models are likely to greatly increase the demand for AI server capacity, we think reasoning models could very well be commoditized as LLMs, in which case these AI data centers, while a great growth story, will be like solar panels, a great growth story without a moat.

Jury still out.

SLMs at the edge

We think most of the economic impact of AI in the near future will be in small language models (SLM) that are trained on high-quality data, solve specific problems, and run in enterprise (to keep data proprietary) on devices. Here is Jahan Ali, founder and CEO of MobileLive:

Small language models allow us to train AI on domain-specific knowledge, making them far more effective for real-world business needs... Why pay millions to train and run a massive LLM when you can achieve better business outcomes with a smaller, cheaper model tailored to your exact needs?

Many of these will be agentic models, optimized for specific tasks they will perform autonomously, powered by domain-specific knowledge could not just produce financial market insights, but actively executing trades on real-time data, or track supply-chains and autonomously optimizing delivery routes and inventory levels, or proactively detect equipment failures in factories, adjust machine settings or schedule maintenance.

Most of these might not even need to run on these big server farms from hyperscalers (although some probably will, with enterprises using them through APIs). However, it’s of course possible that they could enter this market by not just hosting these but also building some of these models themselves (perhaps with the help of LLMs).

Conclusion

At some point, the tens, if not hundreds of billions of dollars that are poured into AI data centers need to produce an ROI. We were told that we don’t see the bigger picture, but the AI market can’t escape the laws of economics.

Demand for AI may be so huge that these data centers can sustain themselves on a simple markup pricing to recoup the investments, but given the sums involved and the steep depreciation rates, it could also devolve into a situation like many of the crackers found themselves in.

Moat would help, as that builds margins, but we see little moat in these LLMs, especially after DeepSeek, and there isn’t any guarantee that reasoning models will find themselves in the same hole.

Even worse, it’s possible much of the economic contribution from AI will occur outside of the data centers from hyperscalers, on the edge with specialist SLMs run by enterprise.

That we can’t see moat, or ROI doesn’t mean it doesn’t exist. But we think it’s useful to think about these things with an economist's view, using concepts like barriers to entry, defensible moat, ROIs, and the like.

It could very well be that the demand for AI infrastructure will increase manifold over time, but we remember the fate of these optical data cable connection build-outs during the dot.com bubble, 97% of which was kept dark as a simple innovation increased the capacity of existing strands by an order of magnitude.

So, to us, these massive investments seem a bit like a leap of faith. It could all work out, but this is far from guaranteed.