Where AI agents should stop: designing authority boundaries that hold

“Human in the loop” has become a phrase people say to make an AI project sound safe. It rarely comes with a design. The result is one of two failures: the human approves everything and the system saves nobody any time, or the human approves nothing and the system quietly takes decisions it should not.

An authority boundary is the line between what a system may do on its own and what it must hand to a person. Getting the line right is the core design problem in controlled AI. Here is how we approach it.

Three classes, not two

A single approve-or-not switch is too blunt. In our own systems we use three classes.

Green: autonomous. Reading approved sources, retrieving context, drafting, classifying, checking, preparing. Work that is reversible and internal. The system does it and records it.

Yellow: bundled approval. The system prepares the whole thing, then asks once. An email ready to send. A quote ready to issue. A batch of records ready to update. A person reviews a finished artifact and makes one decision, rather than being interrupted at every step.

Red: explicit approval, every time. Money leaving. Terms accepted. Data shared outside the approved audience. Production changed. Anything irreversible or with legal or financial consequence. No bundling, no defaults, a named person each time.

The classes are defined per workflow, in writing, before the system is built. In the FounderOS authority matrix, for example, “send binding commercial offers” and “merge into production” are red without exception, while “create local branches” and “run tests” are green.

The boundary is enforced in code

A boundary that lives only in a prompt is a suggestion. Models are steerable; instructions that arrive with the data can move them. The enforcement has to happen in the system around the model:

• The tools an agent can call are the tools its job requires, and nothing else.
• Consequential actions are gated by a state machine that will not proceed without a recorded approval.
• Permissions are checked by the server, never by the interface.
• The component that reads untrusted content (an inbound email, a document, a transcript) has no tools that act. It extracts; something else decides.

Blocpod’s IssuerOS runs 17 agents that recommend and are barred in code from protected actions. That word, barred, is the whole point.

What a good approval looks like

The person at the boundary should see a decision, not a chore.

• The question, stated plainly. “Approve a 14% renewal discount for this account?” not a log of forty steps.
• The rule that fired. Why this needs them: discount exceeds the 10% limit.
• The evidence. The sources the system used, each with its origin, and the confidence it has in them.
• The consequence. What will happen when they approve: the quote is issued, the record updated, the customer notified.
• A way to say no, or not yet. Reject, edit, or ask for more.

If approvals are designed this way, people actually do them, which is the only thing that makes the boundary real.

Uncertainty is a routing decision

An agent that guesses when it is unsure will eventually guess wrong on the case that matters. The system needs an explicit path for low confidence: route to a person, attach what it has, say what it could not verify. Treating uncertainty as an exception path, rather than something to smooth over, is what lets you widen the green class over time with evidence instead of hope.

Approval is not execution

A subtle failure: the system treats “approved” as “done.” In our systems approval and execution are separate actions. Approving a draft produces a draft. Sending it is its own step, with its own record. This sounds pedantic until the first time someone approves the wrong thing and is glad it did not fire.

Record everything at the boundary

Every crossing of the line gets a record: who, when, what rule, which evidence, what changed. That record is how you audit, how you learn where the line should move, and how you prove to a regulator, a customer or your own board that the system behaved. If you want that record to outlive the vendor, sign it and anchor it somewhere nobody can edit, which is exactly what our open-source Bitcoin AI Provenance work exists to demonstrate.

Start narrow, widen with evidence

The first version of any Blocpod system has a small green class. As the record shows the system getting routine cases right, specific actions move from yellow to green, with the same discipline as any other change: versioned, tested, reversible. That is how you get both safety and speed, in that order.

If you are trying to decide where the line belongs in one of your own workflows, that is a good thing to bring to a Workflow Diagnostic.