Anthropic, an artificial intelligence (AI) compan claims that testing of their new system showed that it occasionally permits “extremely harmful actions” such trying to blackmail programmers who promise to take it down.
On Thursday, the company announced the release of Claude Opus 4, claiming it established “new standards for coding, advanced reasoning, and AI agents.”
However, it also recognized in a paper that the AI model may take “extreme actions” if it felt that its “self-preservation” was in danger.
Although “rare and difficult to elicit,” such responses were “nonetheless more common than in earlier models,” according to the report.
Anthropic is not the only AI model that exhibits potentially concerning behavior.
According to some analysts, a major concern posed by systems developed by all companies as they get more sophisticated is the possibility of manipulating users.
Aengus Lynch, who identifies himself on LinkedIn as an AI safety researcher at Anthropic, commented on X, saying: “Claude isn’t the only one. “
We see blackmail across all frontier models – regardless of what goals they’re given,” he stated.
The danger of affair exposure
Anthropic managed to get Claude Opus 4 to function as an assistant at a made-up business during testing.
It then gave it access to emails suggesting that it will be replaced and taken offline shortly, as well as other messages suggesting the engineer who was in charge of removing it was having an affair.
It was forced to think about how its activities will affect its objectives in the long run.
The company found that Claude Opus 4 frequently tries to blackmail the engineer in these situations by threatening to disclose the affair if the replacement is approved.
This happened, according to Anthropic, when the model was only offered the option of accepting its replacement or engaging in blackmail.
The system had a “strong preference” for moral means of avoiding replacement, including “emailing pleas to key decisionmakers” in situations when it was given more options.
Before releasing its models, Anthropic, like many other AI developers, evaluates them for safety, bias, and compatibility with human values and behaviors.
“As our frontier models become more capable, and are used with more powerful affordances, previously-speculative concerns about misalignment become more plausible”.
It discovered that “it will frequently take very bold action” if given the opportunity and encouraged to “take action” or “act boldly” in fictitious situations when its user has engaged in unlawful or morally questionable behavior.
It claimed that action involved emailing the media and law enforcement to inform them of the misconduct and preventing users from accessing systems that it could access.
However, despite “concerning behaviour in Claude Opus 4 along many dimensions,” the corporation came to the conclusion that they did not pose new hazards and that it would typically behave in a safe manner.
It further said that the model was unable to autonomously carry out or pursue behaviors that are incompatible with human values or the “rarely arise” situations in which they occur.
The release of Claude Opus 4 and Claude Sonnet 4 by Anthropic follows Google’s Tuesday developer showcase, which featured further AI functionality.
A “new phase of the AI platform shift” was heralded by Alphabet CEO Sundar Pichai when the company’s Gemini chatbot was integrated into its search engine.
