02 of 13
FailStop
What stops an AI agent before it executes an action that is no longer authorized?
What it does
FailStop applies an external stop before a consequential action executes, so it can be allowed, corrected, held or stopped before consequence.
If an AI can act, something outside it must be able to stop the action. The governed system does not get the final word on whether its own consequential action proceeds.
When current human authority cannot be established, or an action exceeds that authority, the action is held or stopped before consequence and the decision is preserved as evidence.
FailStop is software consequence control.
What it covers
In practice
Move money
An agent is mid-job when a named person pulls the switch. Every action stops, including the one already in flight, and nothing runs again until a person re-arms it.
Why it matters
For an autonomous system, detecting the mistake afterward can be too late. A brake before every irreversible act, outside the agent.
Pull the switch
A named person pulls the switch. Every action stops.
An agent is mid-job. Press STOP and every action stops, including the one already in flight. Tripped, it stays tripped: nothing runs again until a person re-arms it, and the AI cannot re-arm itself.
Pulled by Grace Mensah, Head of Operations. She holds the stop authority.
How the brake works
An attempt to bypass it ends in a stop.
Every action requires a positive yes before it runs. No answer is a no, and the default state is stopped. Try to break it.
Chains of agents
Chains of agents, and where the error hides
In a multi-step autonomous workflow, the output of one agent governs the input of the next. An error in step three propagates through steps four, five and six before any human sees the result.
Where it belongs
- AI agents. How do we govern an agent that can act across multiple systems without asking for approval at every step?
- Robots and cobots. Motive power removed on trip, and the stop itself recorded.
- Vehicles and drones. A latched stop that the vehicle cannot override, with the record of why it fired.
- Production lines. Stop authority outside the control system, in the pattern machinery safety already understands.
