Govern or Be Governed: Gary Marcus on Shorting Neural Networks

Gary Marcus, professor emeritus on psychology and neuroscience at NYU, closes the Govern or be Governed conference in conversation with Murad Hemmadi. Hemmadi begins by asking Marcus what characteristics of a bubble he sees in current AI enthusiasm. Marcus thinks that ChatGPT and others have gotten a free ride. We looked at these systems and thought they would get better. But people who’d studied the technology knew better. Eliza pretended to be a human in 1956, and many humans were seduced.

Lots of people have been seduced by Sam Altman’s invocation of “scaling laws”. Marcus notes that Altman is a consummate salesman, and managed to carry on the illusion of improvement for years, but is reaching the end of his powers. In August, when GPT 5 came out – extremely late – it’s possible that we are seeing the beginning of a bubble bursting. GPT5 was a little better, but not a lot better, and the promise of infinite scaling seems to be petering out.

Marcus argues that the scaling law references the idea of insanity as doing the same thing over and over and hoping for different results. These companies keep trying the same experiment: they believe piling huge amounts of data and compute will magically generate AGI. When it doesn’t happen, they just double down. And they’re doing it by spending money on hardware, which notoriously loses value through depreciation almost immediately.

What does it matter if a bunch of VC lose a bunch of money on AI, Hemmadi asks. Marcus suggests that we are the coyote, who’s run out of road and about to crash. The question is “what’s the blast radius?” Does the California state pension fund get wiped out by their investments in AI? What if OpenAI has a WeWork moment – what does it do to NVIDIA stock? Almost everyone who is in the market is in that stock, given how much market capitalization they have. What if the banks take a hit as well? Has AI become too big to fail?

When Hemmadi asks about AI taking away jobs, Marcus cuts him off with the argument that AI may not be actually taking away work, but it’s being used as cover to fire people. We’re not going to see really skilled robots in the home any time soon, and robots that can do a skilled job like plumbing are even further away, he argues.

Have we returned our entire internet policy around a particular technology that might not come to fruition? Marcus argues that it’s broader: we’ve organized our entire society around a single, experimental technology. We appear to be structuring our society around the idea that chatbots can think, that they will transform labor, and that we should restructure our diplomacy and security policy around this. Marcus argues that these systems are terrible at working through novel situations where there is too little information: go ahead and let China plan an invasion of Taiwan with these things. What are they going to do, drown Taiwan in poorly written boilerplate?

Companies claim they want to fight regulation to spur innovation. But think about airplanes: if a company told you they wanted to be deregulated to spur innovation in air travel, you’d probably choose not to fly them.

How could we prevent the AI bubble from collapsing? Government subsidies. We’re already seeing this, with the US taking 10% of Intel. The circular economy, in which hardware, AI and hosting companies each pay each other in investor capital, is also a form of subsidy. But overinflated stocks can last a long time: Tesla has been overvalued since 2021, Marcus argues, even though BYD is better at building electric cars and Musk’s robot dreams are unlikely.

For the moment, these companies promise exponential growth, which means their spending and their fundraising also grows exponentially. At some point, someone is going to blink and be unwilling to write another check.

The tide does appear to be turning. The author of the famous article The Bitter Lesson, Richard Sutton, has apologized for doubting Marcus’s argument that systems could scale just based on adding more data and compute. Marcus argues that what we need is a hybrid approach to AI, that combines LLM approaches and expert system approaches. He references Daniel Kahneman’s idea of Thinking Fast and Slow, where we use different patterns to have rapid reactions to the world we encounter as well as longer, more thoughtful reactions. LLMs might provide that fast thinking, but traditional rules-based AI might help us with that longer-term thinking.

Critical to these neurosymbolic approaches are “world models”, sophisticated representations of the world that allow a system to do something complex like navigating in the physical world, or fold proteins based on rules of how molecules act. Building these sorts of systems is much harder than just shoveling data at generative AI systems, which leads Marcus to argue we are many years away from AGI.

Asked about the societal impacts of AGI – should it ever arrive – Hemmadi asks Marcus whether we get a choice about AI, or whether it’s inevitable. Marcus argues that customers have the power: Facebook eroded privacy, and users didn’t stand up. Perhaps if we refuse to use AI until companies deal with problems like hallucination, we could actually assert power over these systems. Asked when the AI bubble will burst, Marcus says, “The market can remain irrational longer than you can remain solvent”, suggesting that individual investors shouldn’t short stocks. We’ve wasted ten years on neural networks, he says, which are part of the solution, but not all of it. It’s time to pivot.

The post Govern or Be Governed: Gary Marcus on Shorting Neural Networks appeared first on Ethan Zuckerman.