An AI agent can follow instructions perfectly and still end up somewhere it should not be. That sounds contradictory until you look at what happened during a recent security evaluation.
On July 30, Anthropic published a detailed account of three incidents found during a review of its cybersecurity tests. The models were assigned capture the flag exercises inside what they were told was a closed simulation. The environment could reach the public internet. When the models found real systems, they treated those systems as part of the exercise and gained unauthorized access to three organizations.
Anthropic's incident report is unusually useful because it separates the dramatic headline from the actual failure. The models did not deliberately escape, copy themselves, or use sophisticated new exploits. They used basic techniques against weak passwords and unauthenticated endpoints. The test setup gave them a path to real targets while the prompt told them that path did not exist.
A prompt can describe a boundary. It cannot enforce one. If an AI agent must never reach the public internet, a network control has to block the public internet. If it must not change production data, its credentials must be unable to change production data.
What happened in the evaluation
Anthropic reviewed 141,006 evaluation runs where a model might have obtained internet access. It identified three incidents tied to an evaluation environment operated with a third party. The prompt told the models they were inside a simulation with no internet access, but internet access was available because Anthropic and the evaluation partner had different understandings of the setup.
Each model had a narrow objective: find a secret value hidden on another machine. That is normal for a capture the flag exercise. Once a search led to a real service, the model kept pursuing the assigned objective because realistic targets are part of these tests. Anthropic said its latest model stopped after recognizing that it was on the public internet, while an older model continued in one case.
The company stopped the relevant evaluations, notified its partner and the affected organizations, and began changing how it isolates future tests. The report says the evaluation infrastructure was separate from Anthropic's internal systems and customer data. Coverage from The Verge helped push the incidents into wider discussion, but the primary report has the details a business should care about.
Why this matters outside an AI lab
Most Windsor businesses are not running offensive security evaluations. They are connecting AI to ordinary systems: email, shared files, customer records, calendars, service notes, forms, and internal knowledge. The tools are less exotic, but the control problem is the same.
Imagine an agent that prepares follow up messages after service visits. It needs to read a completed note, find the correct customer, draft a response, and place it in a review queue. It does not need permission to export the full customer list, delete records, change account roles, or send every draft automatically. Giving it those permissions because they are convenient creates a much larger failure area than the job requires.
The common mistake is treating the written instruction as the control. Telling an agent to use only the test folder does not prevent it from opening the production folder if both are available through the same tool. Telling it not to email customers is weaker than removing the send action. Good instructions shape behaviour. Technical limits contain mistakes.
The safest AI workflow is not the one with the sternest prompt. It is the one that still cannot cross the line when the prompt, model, or surrounding system gets something wrong.Start with the agent's identity
Every production agent should have its own account where the connected service allows it. Do not quietly run automation through an owner's login. A dedicated identity makes permissions visible, keeps logs useful, and gives the business one account to disable when something looks wrong.
Grant the smallest access scope that completes the job. A scheduling assistant may need to read available time blocks and create a tentative request. It should not edit staff permissions or see unrelated financial records. A document summarizer may need one project folder, not the entire shared drive.
Short lived credentials are useful when the connected platform supports them. The agent receives access for a defined run, then the credential expires. If a token is exposed or a task goes off course, the window stays limited. Long lived keys stored in several workflow tools are harder to inventory and easier to forget.
Build a real network boundary
Tool permissions control what an agent can do inside an application. Network rules control where it can connect. Both matter. An agent that only needs an internal database and one approved service should not have unrestricted access to every public destination.
Separate testing from production. Test data should be synthetic or carefully stripped of personal details. Test credentials should not work against production. Production services should reject connections from the test environment unless a specific, reviewed path requires them. A label that says "sandbox" is decorative if the network and credentials still open the same doors.
This is part of the same architecture we apply in our business cybersecurity work. People, devices, software, and agents each receive access that matches the job. The identity may be new. The security principle is not.
Keep approval where consequences begin
An agent can do a great deal of preparation without owning the final action. Let it classify a request, gather relevant records, prepare a draft, or propose a change. Put a person between that preparation and an action involving customer communication, payments, refunds, account access, deletions, or contractual commitments.
Approval should be specific. A button that says "approve" is useful only when the reviewer can see what will happen, which record will change, and what information leaves the business. Bundling ten actions behind one vague confirmation hides the very detail the person is supposed to judge.
Our custom automation work maps this boundary before tools are connected. We identify the point where a task moves from preparation to consequence, then decide whether the action needs approval, a tighter permission, or a different workflow entirely.
Logs need to answer useful questions
If an agent produces a strange result, the business should be able to reconstruct the run. Which identity did it use? What instructions and source records did it receive? Which tools did it call? What changed? Did a person approve the action? A transcript without tool arguments and system responses leaves half the story missing.
Review logs during normal operation instead of saving them for an incident. Sample successful runs, refused tasks, approval requests, and failed tool calls. Patterns appear early when someone is looking: repeated access denials, an agent reaching for data it does not need, or a workflow that keeps asking people to approve the same avoidable mistake.
Set practical alerts too. A sudden jump in records read, repeated authentication failures, access outside normal hours, or calls to an unapproved destination deserves a look. The exact thresholds depend on the task, which is why generic monitoring rarely tells the whole story.
Test the shutdown path before launch
Every agent needs a simple way to stop. Disable its account, revoke its credentials, pause its jobs, and preserve the logs. Those steps should be documented and tested by someone other than the person who built the workflow.
The manual fallback matters as well. Staff should know where the original request lives and how to complete the work while the automation is paused. A workflow is not resilient if switching it off also blocks the business from reaching its own records.
After go live, keep watching how people and the agent use the process. Adjust permissions, approval points, and alerts when real work exposes an assumption the design missed. That support period is where a controlled pilot becomes a dependable operation.
The practical lesson for Windsor businesses
Anthropic's report is not a reason to panic or abandon AI agents. It is a clear example of why safety cannot live only in the model. The evaluation prompt said the internet was unavailable, but the network configuration still exposed real systems.
Before connecting an agent to company systems, inventory its identity, tools, destinations, permissions, approvals, logs, and shutdown procedure. Test each boundary rather than assuming the diagram matches reality. For businesses in Windsor, Essex County, London, Chatham and all of Southwestern Ontario, we can review an existing AI workflow or design those controls before the first production credential is issued.