The Frontier Model Era Is Cooling Off

Here's the thing about AI infrastructure: everyone got obsessed with training the biggest, most powerful models, but the real money is starting to move somewhere else. According to Sri Kanajan, a data scientist at Scale AI and former senior data scientist at Meta Platforms Inc. (META), the compute spending that fueled the training boom is now pivoting toward inference — and it's happening faster than most people expected.

Kanajan broke down the evolution during a JPMorgan-hosted call, and his takeaway was clear: the era of throwing unlimited resources at frontier models is giving way to something more practical. Techniques like distillation, quantization, chain-of-thought reasoning, and multi-step optimization are making inference cheaper and more efficient. Meanwhile, training cycles are hitting diminishing returns. Companies are figuring out they don't need the biggest model anymore — they need the cheapest one that actually works.

Kanajan expects inference to capture the majority share of incremental compute spending by 2027, with the tilt already beginning in 2025 and 2026. That's not a distant future problem. It's happening now.

Who Wins When Inference Takes Over?

This shift isn't just about workloads. It's about which companies end up with the business. And according to Kanajan, Broadcom Inc. (AVGO) is sitting in an enviable spot. The company's custom ASICs are powering inference for some of the biggest players in tech: Alphabet Inc. (GOOGL) (GOOG) with its TPUs, Amazon.com Inc. (AMZN) with Inferentia chips, and Meta with MTIA. In a world where smaller, cheaper, more efficient models are the goal, Broadcom is perfectly positioned.

Then there's Marvell Technology Inc. (MRVL), which benefits from inference workloads leaning heavily on Ethernet and PCIe instead of the expensive, training-focused NVLink and InfiniBand fabrics. As AI networks standardize and companies start multi-sourcing their hardware, Marvell's solutions become increasingly relevant.

But it's not just about chips. Celestica Inc. (CLS) is well-positioned as the industry moves toward white-box, OCP-aligned hardware. Operators want cheaper, standardized inference racks they can source from multiple vendors, and Celestica is ready to supply them.

Arista Networks Inc. (ANET) still dominates the highest-performance training networks, but Kanajan noted that the broader shift toward Ethernet in inference environments opens the door for more networking players to benefit in the coming years.

Power Is The Hidden Driver

There's another reason this shift makes sense: power. Training remains absurdly power-hungry, often requiring 5 to 10 times more energy than inference. Many datacenters simply don't have the grid capacity to run large training clusters at full utilization. Inference, on the other hand, scales much better across distributed servers and edge clusters.

That dynamic makes inference not just cheaper to run — it's also easier to deploy. When you're constrained by power availability, inference becomes the practical choice.

The Bottom Line

AI's next chapter isn't about who can build the most impressive model. It's about making AI cheaper, faster, and easier to operate at scale. The companies that enable that transition — Broadcom with its custom silicon, Marvell with its networking infrastructure, Celestica with its standardized hardware — are positioned to capture the spending that follows. The AI money is moving, and it's moving toward inference.