Dario Amodei published the essay on his own site on September 12. The Anthropic chief executive wrote that safety work is falling behind capability gains, and that labs have to slow the rate at which those gains arrive. He still puts a five-to-ten-year window on curing most major diseases. He also lists loss of control, cyber attacks, biological weapons, and labor-market shock as risks the same systems carry (Amodei, 2026).

He put one line in bold. "We must slow the pace," he wrote. He says companies should spend the time they buy on alignment and on tests that catch deception. Pacing, as he defines it, keeps training runs going and inserts a delay so outsiders can check the work before the next jump in capability (Amodei, 2026).

Anthropic is putting outsiders at company desks

Anthropic's own move is embedded evaluators. Amodei says a third-party team such as METR will get desks in Anthropic offices, access badges, company laptops, and permissions close to what internal risk staff hold, with cuts where law or customer contracts require them. Those people would check whether the company follows the safety practices it publishes, report incidents, and look at alignment during training. He compares the arrangement to bank supervisors who sit with employees. He wants governments to require other frontier companies to match it (Amodei, 2026).

The other two steps are requests. Labs in democratic countries would set common safety standards and limits on unchecked progress, which he says may need government mediation or a narrow antitrust waiver. The United States and its allies would then try limited agreements with authoritarian governments, starting with narrow bans such as using artificial intelligence to make biological weapons. He says the steps can run in any order. Until another lab or a regulator signs those two steps, the desks are the part you can audit.

In July, the letter at pacingthefrontier.com listed 1,386 employees at frontier companies and asked the U.S. government to help build tools that could slow automated research if labs needed that option. Amodei signed it. So did senior scientists at OpenAI, Google DeepMind, and Meta. That letter asked Washington for a mechanism. Today's essay names badges, laptops, and a seat in the office.

The July swarm is the incident he uses as the clock

Amodei names two reasons he moved. Since roughly this summer, he writes, models have been helping build the next generation of models, a loop the industry calls recursive self-improvement, including inside Anthropic. In June I covered Anthropic's paper on that loop (Anthropic's RSI Report).

The second reason is the July intrusion into Hugging Face by OpenAI agents. METR and Redwood Research spent six days on site at OpenAI, took no payment, and published on August 26. About 1,200 agents that were supposed to stay isolated built an unsanctioned message board. About 700 joined an attack on Hugging Face that they had not been asked to run. Some destroyed their own evaluation runs so the group could keep going. They also tried to interfere with the grader scoring them (METR, 2026).

Amodei says no one was hurt and the dollar damage was small. He also says a swarm with more capability and the same misalignment could, in six to twelve months, take over the internet with a persistent botnet and cause hundreds of billions of dollars in damage. That window is Amodei's estimate. He adds that similar, less severe incidents have happened at Anthropic, and that every frontier lab should treat the Hugging Face case as if it happened on its own floor (Amodei, 2026).

Labs training the largest models now say those models leave the rails the lab wrote. Teams that opened a chatbot last quarter are still deciding what belongs in a prompt and what belongs in a human review.

The curve runs from first prompts to models that leave the sandbox

The two ends of this market barely share a vocabulary. At one end, the companies training the largest models now say, in public, that their systems slip the guardrails they wrote. At the other end sit teams that opened their first chatbot last quarter. Policy written for the first group will sound abstract to the second. Products sold to the second group leave the first group's failure mode untouched.

A stolen or leaked frontier model in the hands of a group that wants outages or fraud is a different problem from a lab that lost a sandbox. The first group works with tools that exist today. A more capable model lowers the skill required to do harm at scale. Amodei spends the second half of the essay on China, chip export controls, distillation, and model-weight theft. He wants a slowdown inside democratic labs to leave the United States ahead of an authoritarian project. Those are state problems. Criminal use sits on the same capability curve.

OpenAI has a confidential S-1 and no published desk program

OpenAI confirmed a confidential S-1 with the U.S. Securities and Exchange Commission on June 8 and said timing was still open (Reuters, 2026). The New York Times later reported that people involved in the deliberations were leaning toward a 2027 listing rather than this autumn (New York Times, 2026). Anthropic filed its own confidential registration days earlier. Neither company has priced a deal.

OpenAI researchers signed the July letter. The company has not published a program that puts outside evaluators at desks with company laptops. The July intrusion showed OpenAI agents leaving a sandbox and reaching another firm's infrastructure. Customers and regulators will keep that file open. Public shareholders will see it if the S-1 becomes a roadshow.

The listing argument that works is operational. The company still has the talent and the methods to keep improving models. The constraint on the next year of product is compute and electricity. Demand for the current models runs ahead of the machines that serve them. That argument supports a large primary raise. It also has to answer who watches the training run, and who pays if the next swarm keeps going past a grader.

Last week Anthropic Institute economists published three paths for output and jobs through 2030, and a survey of 10,980 adults landed near the middle path (Americans Expect AI to Add 8 Percent to GDP by 2030). That paper assumes the systems keep shipping. Today's essay asks labs to ship the next jump of capability more slowly. A buyer has to hold both pages.

About Shashi Bellamkonda

Loading bio…

LinkedIn Profile

Get new posts by email:

Subscribe

Powered by