And a third example: with its Mythos model, Anthropic said it had an AI that was too dangerous to release to the public, as it finds vulnerabilities in widely used software before they can be fixed. Dan O’Dowd, the veteran entrepreneur whose software is used in fighter jets and nuclear power plants, and many cybersecurity researchers, have all thoroughly debunked this. Some 48,000 bugs are found a year and reported to the CVE database. Mythos found just 20. But even the Governor of the Bank of England bought into that one.
And a fourth.
Exaggeration and anthropomorphism are tropes I first reported in 2019 after talking to robotics and AI experts. They’d noticed it too, and sarcastically called it an “Igor, it’s alive!” trick.
But the AI doomer hype isn’t just a marketing exercise.
The Killer AI or “out of control AI” narrative can be traced to the emergence of the rationalist networks, particularly Effective Altruism (EA), and associated groups. Anthropic’s sibling founders, Dario and Daniella Amodei, and close associates emerged from EA circles, are prominent EAs and wrote the pieties into the company’s mission statements.
It’s a big money business. Nirit Weiss Blatt, the researcher who runs the AI Panic website that documents the money flows from big EA philanthropy, tells me that $1.5bn has been donated to create the pseudo-academic field of “AI safety” to buttress the myths. She calls it the “AI existential risk industrial complex”.
These networks captured Rishi Sunak’s AI summit in 2023, Politico reported at the time, attracting money to EA organisations. They even got the taxpayer to cough up £100m to fund a new one, the AI Safety Institute, now called the “AI Security Institute”. Guess how that reports on Anthropic’s stunts like Mythos?
Years ago, I was watching footage of an audience watching a playback of a magic show they had attended. They were chuckling and kicking themselves: the sleight of hand had been so obvious, but they couldn’t see it. In the room, at the performance, they wanted to believe in magic.
Anthropic knows that the audience completes the illusion, and until we get more sceptical, the AI doomers will keep writing the same movie scripts, over and over again.
No wonder Hollywood is in trouble. The dream factory’s role as our great storyteller and myth-maker has been usurped by artificial intelligence companies, who now create more compelling fiction than the screenwriters can.
The problem is, it’s just fiction. AI just isn’t as important as the stories make it out to be – and if people start to realise this, there will be implications for the AI companies’ share prices.
Take the franchise called “AI is going to kill us all”. AI lab Anthropic released its latest instalment this week. It’s purportedly a self-improving AI: software that designs itself, getting better all the time. The studio says this will lead to an intelligence explosion, and maybe a global catastrophe. This would be a remarkable moment in human history if it were likely, but even Anthropic acknowledges it isn’t.
“We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for,” it warns. But the critics are sceptical.
All it’s doing, they pointed out, is applying some fancy steps to the output of an AI model, much like double checking your maths assignment before handing it in. The technical term is “meta-harnessing”. This isn’t going to get us to an intelligence explosion any more than a child drawing a rocket lands them on the moon.
And besides, say critics, why should adding a process of introspection turn out better results? AI that has been trained on its own slop degrades rapidly, rather as cows fed their own spines and brains go mad. It’s even called Mad AI disease. Large language models can be very useful, but they lack any reasoning or intelligence. In reality the model can’t improve the code, because it doesn’t know what good looks like.
Another narrative that Anthropic peddles is that the models are acquiring consciousness. Claude exhibits feelings, it insists. Anthropic has even engaged a philosopher, Amanda Askell, to write “a constitution” – human rights for robots – in case we hurt its feelings. This is another exercise in anthropomorphism: making something seem alive that isn’t. Christopher Olah, the baby-faced Anthropic co-founder, gave us yet another version at the Vatican last week, as Pope Leo XIV published his first encyclical. Olah insisted that there are mysterious and troubling signs of life in the models. He was firmly slapped down by the Pope himself.
And a third example: with its Mythos model, Anthropic said it had an AI that was too dangerous to release to the public, as it finds vulnerabilities in widely used software before they can be fixed. Dan O’Dowd, the veteran entrepreneur whose software is used in fighter jets and nuclear power plants, and many cybersecurity researchers, have all thoroughly debunked this. Some 48,000 bugs are found a year and reported to the CVE database. Mythos found just 20. But even the Governor of the Bank of England bought into that one.
And a fourth.
Exaggeration and anthropomorphism are tropes I first reported in 2019 after talking to robotics and AI experts. They’d noticed it too, and sarcastically called it an “Igor, it’s alive!” trick.
But the AI doomer hype isn’t just a marketing exercise.
The Killer AI or “out of control AI” narrative can be traced to the emergence of the rationalist networks, particularly Effective Altruism (EA), and associated groups. Anthropic’s sibling founders, Dario and Daniella Amodei, and close associates emerged from EA circles, are prominent EAs and wrote the pieties into the company’s mission statements.
It’s a big money business. Nirit Weiss Blatt, the researcher who runs the AI Panic website that documents the money flows from big EA philanthropy, tells me that $1.5bn has been donated to create the pseudo-academic field of “AI safety” to buttress the myths. She calls it the “AI existential risk industrial complex”.
These networks captured Rishi Sunak’s AI summit in 2023, Politico reported at the time, attracting money to EA organisations. They even got the taxpayer to cough up £100m to fund a new one, the AI Safety Institute, now called the “AI Security Institute”. Guess how that reports on Anthropic’s stunts like Mythos?
Years ago, I was watching footage of an audience watching a playback of a magic show they had attended. They were chuckling and kicking themselves: the sleight of hand had been so obvious, but they couldn’t see it. In the room, at the performance, they wanted to believe in magic.
Anthropic knows that the audience completes the illusion, and until we get more sceptical, the AI doomers will keep writing the same movie scripts, over and over again.