World Models and the AI Hype Cycle

The Next Big Thing Doesn’t Work

World models are the latest big thing in Artificial Intelligence. (Image from Canva.)

A world model is a buffer placed in front of a Large Language Model designed to keep it from telling lies. You code facts into the model, like people don’t have six fingers, or gravity is real (and here’s a formula). The LLM must consult the world model before spitting out its latest theory. It’s like an editor for a writer, or a teacher for a student.

Gary Marcus, who is often portrayed as an enemy of AI, is in fact an advocate for these world models. He says LLMs won’t work without them, because they’ll make stuff up, not being grounded in the real world. He’s right.

But world models are tricky. They must be as complete as the data the LLM is consuming. They must be robust enough, and fast enough, so that the LLM consults them, and adjusts, with minimal interruption in workflow. In other words, they must be instinctive.

They’re not. We’re still trying to model the instincts of ants. Google is pushing its Genie 3 as a way to make pretty pictures, although that’s not the point of a world model at all. Beautiful plumage!

Nvidia, which has its feet on the ground on AI limitations (even as it pushes AI hype to insane levels), emphasizes that World Foundation Models are meant to simulate real world environments, then predict accurate outcomes based on text or visual inputs. They talk about using these in cars and robots. Developers can in theory use them to keep LLMs from making stuff up, creating predictive intelligence and reinforcement learning, where mistakes become inputs that steadily ground the LLM’s performance.

Problems With World Models

There are big problems here. Start with the cat that jumps on a hot stove. It won’t jump on a cold stove, either. Then understand that world models, like LLMs themselves, take an enormous amount of computing power to create and to run. Even before these models become real, in other words, they’re making the AI bubble bigger.

The result is that Marcus himself is being used in the hype surrounding the bubble. It’s assumed that, as his techniques are perfected and adapted, that LLMs will stop with the lying, that real “superintelligence” will result.

There’s a Moore’s Second Law effect in action. (Moore’s Second Law holds that the costs of getting a chip into production grow as the chip becomes more complex.) But what if the costs of human intelligence are higher than the cost of just hiring humans?

That’s why, just as AI boosters are claiming “world models will fix it” when it comes to LLMs, they’re also hyping the idea of “super intelligence,” an AI that’s smarter than any human.

Question. How much will it cost to get answers out of this super intelligence? And wouldn’t a team do just as well?

The Only Possible Outcome

The hope lies, again, with Nvidia. While the Blackstone chip takes more power and uses more water to run than the Hopper chip it replaces, it’s also more efficient in making its calculations. The same will be true with the Rubin chip. This is Huang’s Law, the idea that calculation efficiency can double faster than Moore’s Law saw computing efficiency doubling in the Intel age.

But the hunger for calculation is growing faster than even Huang’s Law can account for. And the productivity from using LLMs isn’t growing.

Either customers will turn off the demand spigot, or Nvidia will overcome the calculation shortage. Either way, we’re heading for a market crash of Biblical proportions.