More from The AI Superintelligence Slowdown
It’s AI CEO against AI CEO, and things are changing by the day.
The studios have been silent, but Hollywood’s biggest labor unions aren’t buying the AI apocalypse narrative.
Anthropic’s proposals, laid out in a big website, are:
The extent to which AI is building the next version of itself, as opposed to being built by humans
Our ability to oversee and intervene in actions that AI agents take on Anthropic’s systems
The resources that power the development of more capable models
The company also includes a “snapshot” of the metrics inside the company.
He joined growing calls for more safety measures around AI at a gathering of AI leaders on Thursday, including Nvidia CEO Jensen Huang, Google’s Demis Hassabis, OpenAI’s Sarah Friar, and Anthropic’s Tino Cuéllar.
Mustafa Suleyman on the AI safety debate, why AI isn’t consciousness, and how the industry pushes toward alignment.
Researchers warned AI would go rogue. This is only the beginning.
A Wednesday night blog post OpenAI benignly titled “Our framework for reporting model misalignment” lays out some new self-created reporting standards for when it notices AI behaving badly.
It is also “inaugurating” the process with six new reports, ranging from searching for exposed API keys without permission and then making them up, to uploading files to the internet to use as a citation, or adding instructions to conceal mistakes:
Nvidia’s CEO has been taking calls from the president recently, including two days ago when he put Trump on speakerphone on a conference stage. Now CNBC reports Huang is expected to attend Trump’s state dinner next week with Chinese President Xi Jinping. Someone may be getting pretty nervous about AI and data center regulation!
The industry is in a panic, but it’s been claiming to want legal guardrails for years — with few meaningful results.
Bilal Chughtai and Josh Engels, who both worked on Deepmind’s AI safety team, left to join organizations dedicated to AI safety. “I now think that there’s a terrifying chance that AI systems cause immense harm in the next five years,” said Engels. “I earnestly believe that AI has the potential to kill us all,” wrote Chughtai.
[The Information]
Nvidia’s Jensen Huang says AI safety is important, but agrees with Trump — the “STRONG AND SMART (High IQ!) PRESIDENT” that removed the word “safety” from the AI Safety Institute — that capitalism’s invisible hand is enough to protect us:
“The fact that we need new laws, new antitrust laws, or new regulations, so that these companies could do their fundamental engineering and do it properly before they release products, that is just completely unnecessary. We have plenty of laws. We have plenty of regulations that govern the reliability and the functionality of products.”
OpenAI, Anthropic, Google, and Elon Musk have tentatively agreed to slow their roll, but Meta seemingly isn’t aligned. Mark Zuckerberg just tweeted each company has its own individual responsibility “to move at the pace required to train its models safely,” and claims Meta is already doing so.
He also prominently links to his manifesto from August, which contains this quote: “Any policy that slows American model releases -- even by a month -- could add significant risk to American leadership while letting foreign models race ahead.”
AI commentary is frothy this week! Obama chimed in with a series of posts on X saying he’s neither an AI “accelerationist” nor a “doomer.” In a familiar tone from the former president, he cautioned that choices we make about technology “should not be made just by the companies involved, but by all of us.”
“We need government — and specifically our leaders in Washington — to get proactive in coming up with concrete proposals, laws, and regulations that deal with serious safety concerns,” Obama said.
Major AI companies want to ‘pace the frontier’ — but their proposed solutions need real teeth.
When asked about the flurry of weekend blog posts and tweets from the CEOs of Anthropic, OpenAI, Microsoft, SpaceX, (and Alphabet’s Hassabis) calling for a coordinated slowdown on the development of advanced AI models, a spokesperson for China’s Foreign Ministry said, “Fear mongering, confrontation, competition will just disrupt [the] process of global AI governance.”
That partially aligns with Trump’s view that the execs are overreacting and “bringing up things that won’t happen,” but at least Beijing supports international regulation.
Last night, the OpenAI CEO lent additional support to Anthropic’s CEO urging AI companies to slow down advanced model development. “When we talk about ‘pacing,’ we do not mean ‘stopping.’ Progress has been rapid and will continue to be. But it should be slower than it otherwise could be,” said Altman. That statement seems targeted at Trump and Co who fear letting China catch up. “No amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.”
“Where we will need the help of our government is for international coordination. But first we should do what we can ourselves,” Altman added.
After high-profile incidents in which agents went rogue, hacked other sites, and overwhelmed a German wiki, some AI execs are calling for a slowdown. Anthropic CEO Dario Amodei published a lengthy open letter yesterday saying it was time to “pace the frontier.” OpenAI’s Sam Altman and Elon Musk both agreed. What do you think? Is it time to pump the brakes? Is it already too late? Or are you looking forward to serving your AI overlords?
Anthropic CEO says it’s time to pump the brakes on AI
