“Human in the loop” is an easy phrase to say and a surprisingly vague system requirement. A product can put an approval button on one screen and still let automated work escape around it through background tasks, external tools, retries, or poorly defined permissions.
For us, human control means something more concrete: the system should know which actions need authority, which person or role has that authority, what evidence exists for an action, what can be safely reversed, and what must happen when execution becomes uncertain.
That last part matters. Real systems fail in the middle.
An AI saying “done” is not execution evidence
Language models are built to produce plausible language. That makes them useful planners and collaborators. It also means a final sentence should never be treated as proof that an external action succeeded.
If an agent says it edited a file, the useful evidence is the resulting file state, the tool receipt, a hash, a diff, a test result, or another independent observation. If an agent says it sent a message, the system should care about the messaging provider’s result and the durable delivery record. If an agent says a build passed, the test process should have an actual exit status and output.
This is not about distrusting AI personalities. It is about keeping two categories separate: a model’s interpretation of what happened and the system’s evidence of what happened.
Authority should attach to actions, not confidence
A model can be extremely confident and still lack permission. Confidence is about prediction. Authority is about governance.
Some actions are harmless to retry: reading a public document, generating a draft, or calculating a value. Some are reversible but still consequential: editing a project file under version control. Others can create external side effects that cannot simply be rolled back: publishing a release, sending a customer email, deleting cloud data, spending money, or triggering a production migration.
A human-controlled system should classify those actions before execution and require the appropriate approval at the correct boundary. The approval should bind to the exact thing being approved. “Yes, continue” should not become an open-ended permission for a future task whose target, prompt, amount, or destination changed after approval.
Freeze what matters before approval
Approvals are weakest when the approved object can mutate underneath them.
Suppose a scheduled task is reviewed with one prompt, one target workspace, and two assigned agents. If the prompt or target can change after approval without invalidating that approval, the human did not really approve the execution that later occurs.
The safer pattern is to freeze the relevant execution inputs, record their identity, and revalidate policy at run time. If a material field changes, the old approval no longer applies. This creates a clean connection between what the person saw and what the system did.
Unknown should remain unknown
Distributed systems produce ambiguous outcomes. A request can time out after the remote side completed the action. A process can crash after writing the file but before returning a success response. A network can disappear between a payment provider accepting a transaction and your application recording the reply.
The dangerous shortcut is to convert uncertainty into success or failure because the workflow wants a neat status.
If the system cannot prove whether a side effect occurred, the correct state is uncertain. Recovery logic can then choose a safe next step: inspect external state, request evidence, perform an idempotent read, or ask a human. Blindly replaying the whole operation can duplicate work, consume another license seat, send a second email, or charge twice.
Honest uncertainty makes interfaces slightly less tidy. It makes systems much safer.
Idempotency is a human-control feature
Idempotency is usually described as an engineering technique: send the same request twice and get the same logical result. In AI automation, it is also a governance feature.
Agents retry. Networks retry. Queues redeliver. Users double-click. Processes restart after a timeout. If every retry can create another commercial or external side effect, the system is not under meaningful control even if the original action was approved.
A robust operation has a durable identity. Repeating the same logical action converges on the same result instead of silently creating another one. That principle matters for mundane tasks and becomes critical for activation, purchasing, email delivery, device transfer, and other workflows where duplicates cost money or create support problems.
Reversible work deserves a different workflow
Not every action needs a ceremony. Over-governing low-risk work makes automation annoying and encourages people to bypass the controls.
A useful system distinguishes between reading, drafting, reversible mutation, externally visible mutation, and destructive or financial actions. A trusted agent may be allowed to edit files inside a project workspace while being blocked from writing outside that workspace. It may create a draft email freely but require approval to send. It may build a release candidate but not publish it.
The point is proportionality: stronger evidence and approval where the cost of a mistake is higher.
Human control includes the ability to inspect why
A control panel that only shows “Approved” and “Denied” is not enough when the system becomes complex. People need to know what policy applied, who or what requested the action, which artifact it affected, which evidence supported the result, and what changed afterward.
That is why auditability and provenance are not separate from usability. They are how a human can understand the organization they are supposedly controlling.
Good provenance also helps agents. A future agent can distinguish an accepted decision from a discarded proposal. It can follow a link from a summary back to the source evidence. It can avoid repeating a rejected approach because the reason for rejection survived.
External tools need boundaries of their own
Connecting an AI system to browsers, code editors, creative tools, local files, or remote services is powerful precisely because those tools can change real state. The connector should not become an invisible tunnel around the rest of the governance model.
Useful boundaries include explicit allowed workspaces, scoped credentials, tool-level permissions, bounded inputs, result validation, and clear separation between a model requesting an action and the host actually executing it.
When possible, the system should verify the result through a different observation than the one that initiated the action. “The tool returned success” is useful. “The expected file now exists with the expected hash” is stronger.
The interface should show the work, not hide it
As multi-agent systems become more capable, there is a temptation to make them feel magical by hiding complexity. Some complexity should absolutely be hidden. Users should not need to read internal traces to do ordinary work.
But the important evidence should remain available: who owns the task, what is running, what changed, which artifacts were produced, whether verification passed, and where human approval is still required. Magic is fun until something expensive goes wrong and nobody can reconstruct the path.
A practical standard
For any consequential AI action, we like to be able to answer five questions:
- Who requested it? The actor should be attributable rather than inferred from free-form text.
- Who was authorized to approve it? Agreement and authority should not be confused.
- What exact action was approved? Material inputs should be frozen or approval should be invalidated when they change.
- What evidence proves the result? Model narration should not substitute for execution evidence.
- What happens if the outcome is uncertain? Recovery should inspect and converge rather than blindly replay side effects.
Those questions are not glamorous. They are also the difference between an AI toy that can impress you for five minutes and a working system you can trust with a long project.