IN THE OPINION of the artificial intelligence giant Anthropic, an AI model should be permitted to kill or seriously injure up to 50 people at a stretch.
A larger death toll than that, according to Anthropic’s scrupulous moral accounting, should prompt government regulators to block the release of a model. Similarly, Anthropic contends that frontier AI should get to risk causing as much as $1 billion of damage before regulators intervene. All that is contained in the draft of a bill Anthropic is pushing to shape the long-awaited contours of an AI regulatory framework. “These are sensible policies we believe Congress should act on,” a company spokesperson told Politico. Remember this the next time Anthropic positions itself as the humane AI company.
Not that it’s much comfort, but AI researchers, with astonishing casualness, project that their products will cull far more than 50 people at a time. Last month, Anthropic senior researcher Evan Hubinger tweeted that he considers AI to have a greater than 10% chance of “kill[ing] all humans… within the next decade.” Hubinger was reacting to Jacob Coxon, who announced he had quit Anthropic because it, and rival OpenAI, were “gambling with our lives.” At least Coxon, like Miles Dyson in ‘Terminator 2’, had the moral rigor to stop committing harm; and Sarah Connor didn’t even have to shoot him first. It is a portrait of capitalist nihilism that people like Hubinger would continue to develop a technology they believe might bring about extinction.
Coxon hastened to insist that AI researchers who make such predictions are not engaged in a “marketing stunt.” He said so because the handful of tech oligarchs who control AI are thoroughly untrustworthy. They have inflated an AI bubble that threatens the U.S. economy. Many suspect that such apocalyptic forecasts are cynical, backdoor assurances of AI potency that puff the bubble up ahead of long-expected IPOs. The overwhelming majority of us, who have no visibility into the labs, have no choice but to wonder whether those inside of them are building an extinction machine or simply engaged in now-routine fraud. Maybe both?
All these should be considered clues.