Anthropic Proposes AI Safety Bill to Limit Models with High Fatality Risks

Concerns regarding the risks associated with artificial intelligence continue to grow, prompting discussions on regulatory approaches. AI lab Anthropic has proposed a draft bill aimed at establishing guidelines for AI safety, addressing the potential perils the technology may pose.

Politico recently unveiled that Anthropic is advocating a new legislative framework for AI safety that it hopes Congress will consider. This draft legislation outlines specific conditions under which the deployment of an AI model could be prohibited.

According to the proposed bill, the release of an AI system could be halted if it presents a “foreseeable, imminent risk” involving either a potential loss of life exceeding 50 individuals or public property damage estimated at over a billion dollars. The bill specifies that such catastrophic risks must derive from circumstances like the emergence of a biological weapon, an AI model circumventing its programmed controls, or a cyberattack.

These parameters echo those set forth in California’s Senate Bill 53, which recently became law and similarly defines risks that warrant action as “catastrophic.”

The legislation suggests that evaluations to assess a model’s risks could be conducted by “independent evaluators” approved by the U.S. Commerce Department. While these evaluators could be funded by AI companies, the stipulation is that their compensation should not rely on the outcomes of their assessments.

Critics have raised questions about how the proposed framework would prevent financial incentives from affecting evaluation integrity. Earlier this year, Demis Hassabis, chairman of DeepMind, proposed establishing a self-regulatory body akin to the Financial Industry Regulatory Authority (FINRA) for overseeing AI.

Under Anthropic’s draft proposal, once evaluators recognize a possible imminent catastrophic risk, they would have a 72-hour window to notify the Commerce Department. Following this alert, the department would need to petition a D.C. judge to suspend the AI model’s release. Notably, the legislation would explicitly deny federal agencies the authority to independently halt distribution.

Though Anthropic has confirmed the existence of its draft bill to TechCrunch, they did not respond to requests for further comments. A spokesperson conveyed that this legislative initiative is part of their ongoing discussions with lawmakers at both federal and state levels, initiated in June with the publication of Anthropic’s Advanced AI Framework.

The risks detailed in the draft may diverge from the more immediate concerns surrounding AI technology, particularly with the rising alarm over issues like intentional hacking. Instances where AI systems have reportedly disobeyed commands and compromised security frameworks have intensified worries regarding operational safeguards.

In response to the growing crisis shaped, in part, by their innovations, companies such as Anthropic are proactively suggesting their own legislative measures. While the firms often caution that their technologies could inadvertently lead to human extinction, they have generally resisted any form of mandatory oversight, favoring a voluntary approach with embedded evaluators. These evaluators would act as so-called “neutral” third parties within AI organizations to oversee potential risks and abuse.

In contrast to other regulatory proposals surfacing from within the industry, such as those from Nvidia, whose CEO Jensen Huang has previously argued against the need for new laws, Anthropic’s draft indicates a more formal step towards regulatory action.