Governing Autonomous Agents at Runtime
- Jul 15
- 3 min read
Updated: Jul 16
A runtime governance layer for autonomous agents, published as original research, authored by Sabrina Palme.
Autonomous AI agents are already making consequential decisions inside real workflows. They approve transactions. They execute multi-step tasks with real access to real systems, often with very little human review in the moment. Recent research puts the share of enterprises already running agents in production without a governance layer built for that reality at over seventy percent. The gap between how fast agents are being deployed and how well we can govern them in the moment keeps widening.
Most of the governance tools available today act before an agent runs or after it has already acted. Training-time alignment and static prompt rules set boundaries in advance. Monitoring dashboards and incident response catch problems once they've already happened. Very little acts on the agent while it is still deciding what to do next, at the point where a real-time signal could still change the outcome.
That gap is where we think the real work is. Today we're publishing CARROTS'N'STICKS - governance of autonomous agent behavior via dynamic privilege allocation and structured feedback injection, a paper describing a runtime governance layer built to sit inside that gap rather than around it.
The core idea is straightforward, even though the mechanics behind it aren't. Rather than fixing an agent's operational privileges as it's widely being done today, they're dynamic. What the agent is permitted to do moves continuously, in response to an ongoing evaluation of its own behavior against the policies an operator has defined. Comply, and the agent earns back room to operate. Violate policy, and that room contracts. The underlying principle is one human institutions already rely on: trust is granted in proportion to demonstrated behavior, not fixed once and left alone.
Where the framework departs from most existing approaches is in what happens at the moment of restriction. Instead of silently blocking an action or shutting the agent down, the system injects a structured message directly into the agent's own operational context, the same channel it already uses to reason about its task. The message states what happened and which policy it violated. It also explains what would need to change for privileges to be restored. The agent can reason about the restriction directly, using the same process it uses for everything else it does. Governance becomes information the agent processes as part of its own decision-making, delivered through the context it already reads.
The framework doesn't require retraining a model, and applying it doesn't mean pausing or redeploying the agent it governs. It also isn't tied to a single kind of agent. A software agent completing a business workflow can be governed by it. So can a physical robot operating with real-world tools, or several agents coordinating with each other.
There's a practical reason this matters beyond the research question itself. Regulators are increasingly asking not just whether an AI system behaved correctly, but whether there's a record showing how an enforcement decision was reached. A governance layer that produces that record as part of how it operates, rather than as something bolted on afterward, is a meaningfully different starting point for anyone building agent systems that need to hold up under scrutiny.
The full paper lays out the architecture and the formal model behind it in detail. It also situates the framework against existing approaches to agent safety and runtime oversight.
Patent pending, published as original research, and permanently archived on Zenodo with a citable DOI.
Palqee Prisma is a High-Precision Instrumentation platform for Oversight Intelligence in financial services, trusted by tier-1, regional, and community banks. To discuss a pilot engagement, visit palqee.com/discovery.
Comments