AI Safety: a Modern-Day Asilomar

In the summer of 1974, scientists discovered the potential of recombinant DNA, which involves combining genetic material from different organisms. They quickly realized the need to halt experiments due to concerns about the potential consequences of engineered organisms escaping from labs. Paul Berg, a biochemist at Stanford, led a committee recommending a pause on experiments until safety measures were established.

By February of the following year, around 140 scientists gathered at the Asilomar Conference Center in California. They developed safeguards enabling researchers to continue their work. The National Institutes of Health (NIH) later transformed these guidelines into formal regulations in 1976, linking federal funding to adherence.

Fast forward nearly fifty years, Jacob Coxon, a researcher at leading tech companies OpenAI and Anthropic, drew a similar conclusion, albeit too late. On September 8, Coxon resigned, criticizing these firms for irresponsibly rushing towards self-improving superintelligence and suggested temporary halts on developing more advanced models.

AI once had its own version of the Asilomar moment about a decade ago. In 2017, researchers congregated for the Beneficial AI conference, organized by the Future of Life Institute. They laid down 23 principles for guiding AI development, including one specifically calling for cooperation to avoid cutting corners on safety. Despite this call, the race continued.

The Familiar AI Story

Evan Hubinger from Anthropic acknowledged Coxon’s concerns, admitting the potential threat AI poses. He estimated a greater than 10% chance of AI causing catastrophic harm within a decade. Despite this, solutions to align superintelligence were still lacking.

Political reactions soon followed. On September 3, Representative Greg Casar and Senator Bernie Sanders announced the Ban Artificial Superintelligence Act. This legislation proposes banning superintelligent systems until regulations ensuring safety are established. Advocates for the ban range across political lines and include notable figures like Geoffrey Hinton and Richard Branson.

Conversely, the Trump administration signed an executive order in December 2025. It emphasized AI development as vital for global dominance, setting a path towards minimal regulatory burdens with the Justice Department challenging state AI laws.

The Real Challenges

The scientists in the 1970s faced dilemmas similar to those AI researchers encounter today. Individual labs could gain advantages by conducting risky experiments. In the past, biology offered regulators concrete measures such as NIH guidelines, which provided structured containment levels, from P1 to P4, corresponding to the risk level.

Anthropic is exploring what the AI equivalent of these safety measures might be. They have proposed a Frontier Safety Roadmap to draft what secure workflows under intense safety protocols might entail. However, initial skepticism emerged regarding the feasibility of isolated networks within the next few years.

OpenAI’s GPT-6 Astra reached the “Critical” cybersecurity level, capable of identifying and exploiting vulnerabilities autonomously. This differs from 1976, marking the commercial advancement of AI technology first before stringent guidelines.

Calls for Regulation

In response to Coxon’s resignation, OpenAI called on September 9 for obligatory national AI safety policies. These include independent assessments and incident reporting. Anthropic reaffirmed this in February through its Responsible Scaling Policy, advocating for a distinction between company-level and industry-wide actions.

Over 1,000 scientists signed a letter urging the U.S. government for support in managing the risks of rapidly advancing AI. This appeal, however, remains voluntary.

The initial Asilomar conference serves as a complex reference point. Some may interpret it as a pinnacle of civic-minded science in the 1970s. Yet, the historical Asilomar conference faced controversy, with disagreements on risks and criticism over decision-making by a small group of scientists. The advantage back then was the lack of heavy losses, making acceptance easy. Today, the stakes are much higher, complicating the prospect of halting progress.