FailStop

FailStop applies an external stop before a consequential action executes, so it can be allowed, corrected, held or stopped before consequence.

If an AI can act, something outside it must be able to stop the action. The governed system does not get the final word on whether its own consequential action proceeds.

When current human authority cannot be established, or an action exceeds that authority, the action is held or stopped before consequence and the decision is preserved as evidence.

FailStop is software consequence control.

Move money

An agent is mid-job when a named person pulls the switch. Every action stops, including the one already in flight, and nothing runs again until a person re-arms it.

For an autonomous system, detecting the mistake afterward can be too late. A brake before every irreversible act, outside the agent.

An attempt to bypass it ends in a stop.

1. Attack it. It goes to a safe stop.
2. Silence it. It goes to a safe stop.
3. Bypass it. It goes to a safe stop.
4. Crash it. It goes to a safe stop, and it stays there until a person re-arms it.

Chains of agents, and where the error hides

In a multi-step autonomous workflow, the output of one agent governs the input of the next. An error in step three propagates through steps four, five and six before any human sees the result.

1. AI agents. How do we govern an agent that can act across multiple systems without asking for approval at every step?
2. Robots and cobots. Motive power removed on trip, and the stop itself recorded.
3. Vehicles and drones. A latched stop that the vehicle cannot override, with the record of why it fired.
4. Production lines. Stop authority outside the control system, in the pattern machinery safety already understands.