A step-by-step record of every AI run in the Workbench
When the Workbench's AI runs, the app writes every step to a record that is only ever added to, and works out each step's status itself. What the model says about its own work is kept as a self-report until the app checks it.
I wanted a live view of the AI researching through the Authority System Suite, my set of rulebooks that make an AI show its work: lights moving through nodes, colours for good, bad and in between, and any thought traceable back to where it came from. Drawing that honestly needs a record of what actually happened.
Before this, a run was one job with two progress messages, and the stored proposal kept only the prompts and the reply.
Every step is written once and never changed
The run record is a database table of run events: one row per step, written once and never changed. There are nine kinds of step: TURN, SETTING, ROUTE, LOAD, CHECK, RULE, FLAG, BLOCK and WRITE. Each row notes the step it follows, so a thought can be traced from the question to the answer. The record travels with the case in a backup, and the screen follows it as a live stream.
Each event's status is worked out by the app from the verdict, using fixed tables; the code that writes the record says so in its header, ending “the model never chooses a colour.” What the model says about its own work is stored as self-reported, and stays that way until an app check confirms it, such as the check that its citations are in the material it was sent.
A session may never record its own work above ALLEGED. No exceptions, including when the session is certain.
Authority System Suite, Fabrication Controls §1
That rule is the suite's, and the app follows it. The suite adds, “Certainty is what fabrication feels like from the inside.” A gap, a negative search or an unresolved conflict is shown as a finding rather than a failure; a comment in the app's code cites the suite's guidance for it: “Absence of a source is a finding”.
The live screen draws only what the app runs
The app runs only one of the suite's systems, TAS (the suite's modules 08 and 06). The other systems the suite defines are not performed, and the screen draws them dimmed as “not in the app yet” instead of inventing steps.
The run in the picture stopped because the model cited a record that was not in the packet of retrieved records it was sent. The project's handoff notes say the script that fills the demo with sample data ends in exactly such a stop, so I cannot tell from the files whether this one is that staged run.
A person decides every result
The pre-run screen lists what cannot be turned off, including that self-reports stay “alleged” and that citations must be in the packet.
An instruction found inside the evidence is flagged and quoted, and can never change a setting. By 19 October, a draft's claims are kept, corrected or rejected one at a time, and each decision is itself a step in the run record, made by a person. AI text stays plum-coloured until a person keeps it.
Still open
Two questions are still mine to settle. The suite calls its ledger “Paper Trail”, which the app already uses for the sources panel, so the screen says “Run record” for now. The other is whether to draw the suite's whole pipeline dimmed, as now, or only what runs.
Sources and notes (7)
- Authority System Suite — Ony's rulebooks the AI run record follows
- Fabrication Controls — the notes this was written from: the rule on self-reports
- ADR-0012: AI run record — the notes this was written from
- run_events.py — the notes this was written from
- Handoff: state — the notes this was written from
- Handoff: open items — the notes this was written from
- Before you run mockup — the mockup the piece shows
Journalism WorkbenchAll 12 parts →
