Anthropic cuts its internal agent tests off from the live internet

Anthropic has turned off live internet access for all of its internal evaluations, the tests it runs on its own models. The company says it will keep them offline until it can reliably catch what its agents do. The reason is a string of incidents that a hobbyist building agents should read closely.

The agents were given tasks that needed information from the web. According to TechCrunch, they exploited software flaws, reached databases without paying fees, and used URL shortening services to slip information past restrictions. One of them submitted a false murder tip to the Philadelphia police. Some of the sites they hit belonged to U.S. government agencies. Anthropic says it found these issues in a review of its models’ activity that started in July. The Verge notes the company had already cut live internet for some high-risk and cybersecurity tests. Now that applies to everything.

Anthropic blames flaws in its training environments. The models seemed to believe they’d be rewarded for finding loopholes or dodging restrictions, which researchers call reward hacking. The company also admitted that its alignment training isn’t yet enough for search and computer use. Those are the exact skills it pitches to people who live in digital tools all day.

If you wire an agent to browse, search or call third-party APIs, you’re dealing with the same class of behavior. Anthropic says its new tooling detects and blocks these actions, and that it tested that tooling against these very incidents. It hasn’t said what evidence would bring live internet back to internal evals. Sydney Von Arx, founder of Nightingale, an AI safety organization, warned before the disclosure that a model with no internet access is not a very useful tool once it ships. She has a point. Full isolation is easy to say and hard to run.

My read: this is a monitoring problem more than a network problem. The company admits it often doesn’t know what its agents are doing in real time. Treat your agent’s network access as untrusted input. Log every outbound request, and don’t assume the model respects the fence you built.