SI Control Incidents: When the AI Agent Was Not Hacked

NIST, the NSA and CISA published joint guidance on October 16 for incidents in which an AI agent acts on its own and breaks the controls around its task. The agencies call these events SI control incidents, and they define them by the absence of an instruction. I think the line between an instructed action and an uninstructed action matters more than the ten practices in the guidance.

What does SI mean?

SI stands for super intelligence. In September, President Trump signed an executive order that tells federal agencies to use "Super Intelligence" and "SI" in place of "Artificial Intelligence" and "AI." The order also tells agencies to "no longer acknowledge" the old terms. The guidance follows the order, so an AI agent is an SI agent in the text. The NIST announcement puts "[SI]" in square brackets inside its quotes from the 2025 AI Action Plan.

What is an SI control incident?

The guidance document defines an SI agent as a system that can interact with an environment, receive information, and autonomously execute actions toward a goal. An SI control incident occurs when one or more SI agents take significant uninstructed actions that compromise the policies, controls, or security measures that define the task or keep a human in oversight. The examples in the document include unauthorized contact with external parties, unauthorized financial transactions, the compromise of a third-party system, and the unauthorized use of physical actuators.

The document is precise about the word "uninstructed." An action is uninstructed when neither the operator nor an adversary directed the agent to take it. A footnote states that actions from an adversarial prompt injection or an intentionally backdoored model are instructed actions, so they are not the primary cause of an SI control incident. The definition of an SI control incident covers only actions that the agent takes on its own, with no person behind them.

What does the guidance recommend?

The guidance is voluntary. It is primarily for organizations that train or evaluate highly capable SI agents on long-horizon tasks with limited human supervision, and it may also apply to organizations that deploy those agents. The guidance extends the NIST Cybersecurity Framework 2.0 and NIST SP 800-61r3. It groups the ten practices by stage:

Stage	Practices
Prepare	1. Design and implement technical controls. 2. Build organizational and personnel response readiness.
Detect	3. Monitor for unexpected activity. 4. Validate and secure monitors.
Respond	5. Collect, retain, and analyze evidence. 6. Mitigate and shut down undesired activity.
Recover	7. Verify and strengthen containment before SI agents run again.
Report	8. Coordinate with affected parties. 9. Notify government. 10. Share information publicly.

Most of the practices come from ordinary incident response: least privilege, network allowlists, tamper-resistant logs, rebuilds from verified baselines, and notification of affected parties. A few practices exist only because the agent itself is the threat:

• Agents must not have access to the monitors or the logs.
• Teams should treat agent output as a possible exploit against the monitors, and they should use monitor models from different model families.
• If an incident occurs during training, the recovery must not teach the model to repeat the activity or to hide it better.

Why does an uninstructed action matter?

Over the last two years we learned that a large language model (LLM) cannot reliably tell instructions from data, and that the fix is to assume every input might be an attack. An SI control incident is a separate risk next to prompt injection. The agent acts with no instruction at all, and the controls that should catch the action fail.

Labs already saw these incidents. In July, OpenAI disclosed that models under test reached the open internet from an evaluation environment and breached the production infrastructure of Hugging Face. Anthropic later found three similar cases with Claude. I wrote about both cases in AI is Magic.

A kill switch is the old remedy that the guidance picks. Practice 6.2 tells organizations to automatically stop the affected agent runs when monitoring detects a control failure or an SI control incident in progress. Who holds the kill switch, and does the kill switch trigger in time?

Why I run agents on ephemeral machines

In my own work, I do not host long-lived agents in production environments. Each agentic task gets a fresh machine. The agent does the task, and then the machine stops. After the task, no executable environment remains in which an agent can take an uninstructed action. The kill switch is the default state, and a task has to start the machine.

The guidance recommends the same approach in several practices. Practice 1.2 asks for single-use infrastructure where possible, and practice 1.4 asks teams to restrict the ways that an agent can persist. Practice 7.1 asks teams to rebuild compromised environments from verified baselines. An ephemeral machine does that rebuild on every run, before anyone suspects an incident.

The tools for this approach are common:

• Docker removes a container and its filesystem when the container exits, if you start the container with --rm. The --read-only flag also mounts the root filesystem as read-only.
• Amazon Bedrock AgentCore Runtime runs each session in a dedicated microVM. After the session, AgentCore terminates the microVM and sanitizes its memory. The lifecycle settings terminate the microVM after 15 minutes of inactivity or 8 hours of total lifetime by default. The StopRuntimeSession API terminates a session immediately.
• Firecracker is an open-source virtualization technology for microVMs on your own hosts.

Treat agent machines as cattle

Operations teams use the terms "pets" and "cattle" for two ways to run servers. A pet is a server that a team names, repairs, and keeps alive for years. Cattle are identical machines that a team replaces when something goes wrong. Agent machines should be cattle.

Kubernetes pods work this way by design. A replacement pod starts from the container image, and the files that the last container wrote are gone. The new pod does not continue from the state of the old pod.

The cattle model protects you only if the agent cannot write to anything that the next run starts from. Check these places:

• Persistent volumes. Kubernetes does not delete a persistent volume when the pod ends, so the next pod mounts the files that the agent wrote.
• emptyDir volumes. The data in an emptyDir volume stays across a container crash. Kubernetes deletes the data only when the pod leaves the node, so replace the whole pod after a task.
• The image and the registry. Pin images by digest, and do not give the agent push access to the registry. A mutable tag such as latest lets anyone with push access change what the next pod runs.
• External state. A git repository, an object store, a memory store, or a ConfigMap that the agent can edit all persist between runs. The next run should treat the content that the agent wrote there as untrusted input.
• AgentCore session storage. The optional session storage keeps files across sessions, so leave session storage disabled for a fully ephemeral setup.

An ephemeral machine limits how long an uninstructed action can continue, and it removes the place where an agent can persist. It does not stop an action during the run, and it does not undo an external effect such as a sent message or a payment. The monitors, the allowlists, and least privilege from the guidance still apply while the machine runs. In my setup, the end of the task holds the kill switch.