The Great Hallucination

What happens when you can no longer trust anything?

I am offering a new seminar this July, Breaking the AI Barrier. During the month-long course, we will seek to understand, integrate, and anticipate where this rapid synthetic intelligence evolution is driving us, both as individuals and, collectively, as a society. Of course, I hope you will sign up for the course, if you haven’t already.

I don’t pretend to be some technical super-expert on AI. I’m trying to find a multidimensional, philosophical and practical approach that can help us deal with the intensifying impacts of this fascinating yet wildly out-of-control technology. I feel we need opportunities to learn together to understand what’s happening, what’s at stake, and what can be done. Tens of millions of jobs may vanish in the next few years because of AI, yet the systems have severe flaws.

In the last few days I’ve learned that AI’s hallucination problem – when AI makes up information or cites nonexistent books and articles – is getting much worse as AI “advances.” I realize I’ve found this in my recent research and explorations with AI. I had convinced myself that AI was hallucinating less, but it turns out that is not the case. One reason for this may be that AI companies have run out of human-produced training data (like all of our books and art works through history). It is starting to churn on AI–produced content, leading to rapid loops of deepening “enshittification”, and increasing volumes of “AI slop.” 

I like to ask ChatGPT for daily or weekly digests, getting articles and information on topics that interest me. I created one daily digest so I could follow how the Trump administration and its allies seek to compromise and corrupt the electoral process to steal future elections. Every day, Chat GPT delivers headlines, plus sensible sounding (if scary) descriptions of articles. The digest includes links to The Guardian, Propublica, The NY Times, and so on. I forgot about the digest and didn’t get around to checking the links for a few weeks. When I went back to read up on the sources, I found that basically all of the articles it highlighted were nonexistent, with links leading nowhere. 

While the hallucination problem is getting worse for unknown reasons, the world‘s most powerful technology companies such as Google and Microsoft rush to insert AI agents and Chatbots in almost everything. The whole Internet is being rapidly reconfigured – enshittified – around AI. We now know that AI is incredibly unreliable. Instead of this problem getting better or going away, it seems to be getting much worse, very quickly. As The New York Times reported:

For more than two years, companies like OpenAI and Google steadily improved their A.I. systems and reduced the frequency of these errors. But with the use of new reasoning systems, errors are rising. The latest OpenAI systems hallucinate at a higher rate than the company’s previous system, according to the company’s own tests.

The company found that o3 — its most powerful system — hallucinated 33 percent of the time when running its PersonQA benchmark test, which involves answering questions about public figures. That is more than twice the hallucination rate of OpenAI’s previous reasoning system, called o1. The new o4-mini hallucinated at an even higher rate: 48 percent.

When running another test called SimpleQA, which asks more general questions, the hallucination rates for o3 and o4-mini were 51 percent and 79 percent. The previous system, o1, hallucinated 44 percent of the time.

Many scientists and engineers are saying this problem cannot be solved. 

I’m not sure we can deal with this as a society at this point. I’m increasingly afraid that what’s immediately ahead for us in the U.S. is a chaotic melt-down into collective psychosis and catastrophic collapse, exacerbated by AI. Personally, I feel like the entire Republican establishment, the Right, the MAGA world, as well as the NxReactionary tech elite have lost their minds. Trump, the avatar of this movement, makes less sense when he speaks, with each passing month. Trump’s lies and AI’s hallucinations seem to mesh well together, expressing the archetypal process of psychic disintegration or fragmentation we are undergoing. This is made clear when Trump presents himself as the Pope or a rock guitarist on Truth Social, putting AI to heinous uses such as the infamous “Trump Gaza” video where he fantasizes about building a resort over a site of genocide.

If the U.S. is entering a collective state of psychosis, AI is intensifying the condition. As Tristan Harris and Asa Raskin from the Center for Humane Technology noted a few years ago, one danger of AI is the “total decoding of reality.” Once reality has been “decoded,” broken apart, it can be “recoded” to fit any ideologically fixed position, no matter how extreme or biased. This can be maintained until reality finally intrudes, via famine, energy depletion, bankruptcy, systemic collapse or mass violence.

The Republican Congress is about to pass a budget bill that takes healthcare away from 14 million poor people, slashes social services, destroys education, eliminates scientific research, stops providing emergency aid to states during natural disasters (Trump has already stopped sending FEMA in to disaster zones), and beefs up the military (75% of the “Trump Suicide” budget goes to military and police). The bill increases the deficit by $5 trillion to give further tax breaks to the wealthy and super wealthy.

Historically, we know that human societies are prey to episodes of mass psychosis that usually lead to horrifying wars and mass death. It feels like we are hovering on the brink of one of those historical episodes that we read about in the past. I’m amazed and even flabbergasted at how incredibly uninformed people are these days. An artist I met last weekend just sent me a Tweet supporting Trump’s fantasy that there is a “white genocide” currently happening in South Africa. But this is a total fabrication. This person was clearly intelligent enough to do their own research, but they outsourced their thinking capacity to MAGA.

Trump just orchestrated another mafia-style ambush of a political leader when he and his crew met with President Cyril Ramaphosa of South Africa, with Elon Musk, Pete Hegseth, Stephen Miller, JD Vance, and his other pasty-white “good ol’ boys” hovering around. During the meeting, Trump presented bogus “evidence“ that he claims support the white genocide thesis, but actually does not. Trump wouldn’t respond to President Ramaphosa, who is black. But he was able to listen to the white South African golfers Ramaphosa brought with him. They agreed with President Ramaphosa’s statements. This video from The Bulwark is helpful, if you need background, as is this video from MeidasTouch. People do need to get informed.

I can maybe envision how we could defeat this new American fascism or “competitive authoritarianism” or whatever we have via a massive surge of public opposition, both through electoral means and popular protest. It is harder for me to know what we can do when the space of reflective thought has shrunk to such a degree. People seem increasingly reactive and instinctive, tribal and almost animalistic. They make rapid-fire judgments of complex events on the historical stage, with no study or investigation: They barely think. They want someone else or AI to do their thinking for them.

We have every reason to be very worried that artificial intelligence — which turns out to be inherently deceptive, unstable, and untrustworthy — could exacerbate these already dangerous tendencies in the collective psyche to the point where any form of democratic agreement becomes fundamentally impossible. At that point, some kind of mass violence becomes inevitable.

I’m not saying that I really know what we can do here. I don’t. I’m not sure that anybody does. Of course, a massive platform like Google or Facebook could be used to spread only good information with a solid factual basis to its billions of customers. So far, the billionaire owners of the major tech companies have chosen to accommodate Trump’s authoritarian political movement and make profit from hate, outrage, and false conspiracy theories.

One disturbing trend is the apolitical, vacuous content shared by most public artists and social media influencers. Unless somebody’s “brand” is specifically focused on raising ecological or political awareness, most artists and influencers no longer use their platform to discuss anything important or meaningful. Their media feeds are full of solipsistic drivel. Artists should be truth-tellers as well as the “antennae of the race,” but at this point, they mainly provide another luxury service for the rich.

Sadly, younger people have been indoctrinated to believe it’s supposed to be like this. There’s almost no moral authority, gravity, or courage left in the U.S. AI exacerbates the general sense of nihilistic cognitive collapse and ethical capitulation. And I write this as somebody who’s intrigued — even fascinated — by the creative possibilities of the new technology, as we will also explore in the seminar. (I’ve included some of my recent AI video experiments, for what it is worth).

It seems obvious that Trump intends to crash the U.S. economy.