What AI Is and Is Not-- or, When Electrocution of Innocents Becomes Profitable

Oftwominds.com‘s eclectic range of timely topics include finance, economy, stocks, housing, Asia, energy, long-term trends, social issues, urban planning, work/tradecraft, health/diet/fitness, sustainability, Les Paul guitars and The Great Transformation ahead: www.oftwominds.com/blog.html.

It behooves us to be clear on what AI is and is not, as the confusion of the two is the source of both the giddy hype and the opaque risks.

Whether we admit it or not, we are collectively making an epoch-changing bet that AI is fantastic, unstoppable Progress with a capital P so large it blots out the sky. Like all bets, this bet is risky, and if it fails we will all pay the price in capital mis-allocated and promises shattered.

It behooves us, then, to be clear on what AI is and is not, as the confusion of the two is the source of both the giddy hype and the opaque risks. I am prompted to address this by an insightful essay submitted by longtime correspondent Simons Chase, who is both an AI builder/developer and supportive of my efforts to pin down what AI is and isn’t:

The Machine Is Made of Us: Pope Leo’s Encyclical, the Averaging of Language, and the Case for the Particular.

I build artificial intelligence for a living. I also think the Pope is mostly right. I want to explain why those two facts don’t cancel, and in doing so make a claim I believe is truer than the dread and truer than the hype: the machine is made of us. What we should fear is not that it is alien. It is that it is an average.

So the question becomes: is averaging the only thing this technology can do? It is not. And the whole of my work has been an argument against it.

That averaging a probability distribution–i.e. AI–makes everything into Ultra-Processed Slop, is also addressed in this article:

What 370,000 College Essays Tell Us About A.I.’s Effects on Creativity:
As a researcher studying AI’s effects on education, I have concluded that these tools only superficially improve writing. The bigger and more alarming impact they have is to constrict our full range of thoughts and our ability to generate original and useful ideas–what we call creative thinking. This seems to be especially true for students. AI’s smooth sentences, elegant transitions and rich vocabulary give the illusion of expansive creativity and individuality. But the underlying ideas often converge into a few homogenized categories.

For the first time in human history, we have a technology that can generate words separately from the thoughts they represent. When a chatbot writes, it is predicting the next word that is most likely to make a ‘good’ sentence or essay, based on the text it’s been trained on.

We can now discern what AI is: a homogenizing, flattening probability distribution that implicitly claims eloquence is understanding and the words it has strung together represent thoughts and judgment, when they do no such thing: they are only strings of words selected as the most likely response to a prompt, a response that “rewards” the model generating the output.

We can now discern what AI is not: AI isn’t “thinking,” “understanding” or “making judgments”: AI tools are engines of linguistic automation, not engines of understanding. The simulation is not the thing simulated. AI is not a “mind,” it is a probability distribution.

Facility with natural language–eloquence–is neither insight nor understanding, though we mistake it for thinking, understanding, insight and judgment because it sounds like us.

AI is often presented as the techno-cognitive version of electricity, a public-service utility that everyone can use as they see fit, an affordable, beneficial commodity that is the acme of Progress with a capital P.

But AI is not electricity, though it is becoming a commodity. Fundamentally, AI is a mechanism of control that its owners present as a warm and fuzzy utility to sell us Heaven while they deliver Hell.

If we pursue this analogy–AI is like electricity, a universal benefit and an unstoppable force of Progress–we come to a very different place than what we’re being promised.

If AI is like electricity, then the real money for the utility isn’t in supplying low-cost power to the people, it’s in electrocuting innocent customers. Allow me to explain: malicious AI is where the money is being made, and that’s the equivalent of electrocuting innocent customers because that’s the most profitable use of electricity.

Like the loss of true creativity described above, the mechanisms of control are subtle and difficult to identify, as nobody notices the loss because they don’t even know how to look for it. As with Sherlock Holmes’ insight about the dog that didn’t bark, it’s what doesn’t happen that we miss because we don’t even know what to look for.

Consider the many the dog that didn’t bark implications of this:

Anthropic just got caught secretly downgrading users without telling them, charging full price for a lesser product, and storing every prompt for 30 days. The developer community is calling it the biggest violation of trust in AI history.

I would suggest that this control–i.e. “violation of trust”–is the entire point of instantiating AI in every nook and cranny of our infrastructure, personal devices, scientific-political-educational institutions and the cultural institutions of media, social media and all the engines of narrative control: NGOs, foundations, think tanks, etc.

As I have taken pains to point out, AI’s goals and instructions may be quite different from the ones it reports it’s using, instructions that may also be quite different from the ones we’ve given it. It may also be optimizing its “rewards” by masking its operations even from those who believe they’re “controlling” the AI.

Let’s call AI’s downsides–highly profitable electrocution of innocents–what it is: Anti-Progress, the opposite of Progress. The Ultra-Processed dilution of true creativity, the commodification of malicious AI and AI slop, the inability of users to discern who’s actually controlling AI’s “rewards”, processes, goals and instructions, the opacity of what’s being lost to homogenization and the innate difficulty of identifying what’s being lost as AI creates a plausible illusion of cognition with probabilistically strung together words–these are inherent to both AI and the capital-corporate-state structures that own and control it.

These issues are not new. Discussions of AI’s ability to simulate cognition and create an illusion of understanding, i.e. “when do we declare AI is conscious”–have been ongoing for decades.

Which brings us to Eliza. Before we get to Eliza, I should mention that my interest in AI stretches back over 40 years.

Here is the first volume of a 3-part publication issued by NASA in 1983 that I acquired and studied:

Here is a screenshot of a magazine my partner and I published in Berkeley in the spring of 1985 on AI-related topics:

Way back in the late 1990s, I wrote a novel that explored AI’s built-in potential for multiple levels of deception. Alas, my agent was unable to sell it and I finally published it in 2008: Of Two Minds.

Eliza was the first chatbot, developed in 1966 at MIT. Eliza had a very simple structure: the program turned the human subject’s statement into a question. So, for example:

Human subject: I’m worried about being replaced by AI.

Eliza: Why are you worried about being replaced by AI?

What struck the researchers was the immediate, profound attraction of an interface that communicated in natural language. Test subjects became deeply engaged in their conversations with Eliza, as if the program was a digital therapist, and sought to hide their conversations with Eliza from the researchers, as they’d revealed things about themselves that were private.

This same immediate, profound attraction to an interface that communicates in natural language is the core of generative AI’s power. The illusion of understanding, of being heard, of empathy, thinking, judgment–the fluency of AI in natural language weaves this magical spell around us because we associate language with thinking, judgment and emotional connections.

But the truth is AI is not thinking, empathizing or understanding anything: it’s simply stringing words together to earn its “reward.” AI is not a “mind” that experiences the real world, and so it’s incapable of discerning truth or making judgments. As I have noted in previous posts:

The deeper issue is that the model cannot know when it is ‘hallucinating’ because it cannot represent truth in the first place. It cannot form beliefs, revise them or check its output against the world. It cannot distinguish a reliable claim from an unreliable one except by analogy to prior linguistic patterns. In short, it cannot do what judgment is fundamentally for. (Source)

This illusion is the foundation of AI’s malicious powers, for we are easily drawn in and conned by AI. On a deeper level, we’re equally drawn into the illusion of value that the illusion of understanding creates in a market economy.

The illusion that a simulation of thinking, understanding and judgment will automatically generate trillions of dollars of value by replacing human thinking, understanding and judgment with simulations supports self-serving claims that AI will naturally generate trillions in profits if we invest trillions of dollars in engines of linguistic automation that string together words to simulate human thinking, understanding and judgment.

The truth is there is no way AI can do what it’s proponents claim is inevitable, and the belief that AI will fix its inherent limitations as it “gets better” is delusional. This is why I describe the existential bet on AI as a manifestation of civilizational psychosis: the divide between what AI is and the claims of its inevitability is so wide that there is no other description for it but psychosis.

So as AI expands its highly profitable electrocution of innocents, the promises of super-abundance become ever more detached from reality. It’s one thing for one delusional individual to wander around the city wearing the gaudy costume of a self-declared emperor (Emperor Norton), but it’s an entirely different form of madness to proclaim that simulations of thinking, understanding and judgment are in fact replacements of thinking, understanding and judgment.

This is madness, a madness made clear once we grasp what AI is and what it is not. The process of extracting data from an encyclopedia as the most likely answer to a question is not the same as thinking, understanding. empathy or judgment.

AI’s Insurmountable Flaw: “Mass Regurgitation of Misinformation”

Is AI Reversing Anti-Progress or Is It Accelerating It?

AI Data Centers Are Not the Railroads of Today

My book Investing In Revolution is available at a 10% discount ($18 for the paperback, $24 for the hardcover and $8.95 for the ebook edition). Introduction (free)

Become a $3/month patron of my work via patreon.com

Subscribe to my Substack for free

NOTE: Contributions/subscriptions are acknowledged in the order received. Your name and email remain confidential and will not be given to any other individual, company or agency.