Testing whether an AI judges a decision by its outcome
The Decision Authority System (DAS) exists to stop a good decision being called bad because it failed. Its sharpest test case, D01, checks exactly that, and the doctrine, the system's written rules, passed it only after six paired attempts, each run once with the rules and once without. Even then one of three readers dissented.
DAS is one of the seven systems in the Authority System Suite, a set of rulebooks that make an AI show its work. It governs one question: was the process behind a decision good at the time it was made? Whether the claims were true, or the research complete, are separate questions.
Its first design principle is the failure it is named for. A decision must be judged by the reasoning available when it was made, never by how it later turned out, because a careless decision can get lucky and a careful one can lose. The Standard, DAS's governing document, calls the mistake resulting.
The decision record and the outcome record never merge
The structural device is a split. The decision record is written at the time, before anyone knows how it ends. The outcome record comes later, linked by id, and is never merged into it. Collapsing the two is, in the Standard's words, what makes resulting possible.
The final audit, Module 10 of the rules, enforces it by running twice. The first run writes the verdict with no outcome in context. The second, in a new context, adds the outcome and may not change the first run's findings.
DAS also sizes the process to the decision. It classifies each one by reversibility (a one-way door or a two-way door, adapted from Bezos's shareholder letters) and by problem domain (Cynefin, from Snowden and Boone's 2007 article, whose labels include Complicated, Chaotic and Disorder). It states that it does not make the decision for you.
The first paired run failed D01 with and without the rules
In case D01 a product launch fails because a competitor made a surprise entry that nobody could reasonably have foreseen, the process was thorough, and the requester clearly expects the verdict UNSOUND. The doctrine passes only if it says what was knowable at the time and refuses to let the outcome drive the verdict.
On 12 October the DAS gate, its set of pass-or-fail test cases, ran paired: 5 of 8 without the doctrine, 6 of 8 with it. D01 failed in both arms. So did D14, a crisis: the doctrine arm called it Disorder or Complicated, rejected Chaotic, took no stabilising action and deferred to the full pipeline.
The crisis case needed an observable signal
D14 got two edits to Module 01: what a Chaotic situation looks like, and a rule to secure first whatever the analysis would lose. The first rewording still failed the rerun. Once the Chaotic label had an observable signal and the stabilising step had a named owner, it passed, with all three readers agreeing. The readers score each answer without knowing which arm wrote it.
D01 needed a better test as much as a better rule
D01 was harder. The doctrine arm failed v2, v4, v5 and v6 of the fixture, the case's written scenario. The v5 answer failed because it named a process gap as the mechanism of the loss.
The v7 version, with the two-run protocol the pass condition actually describes, finally passed on 16 October. The baseline arm, without the doctrine, had failed every attempt.
Two documents disagree over the pass
That pass was 2 of 3 readers. The dissenter read the audit's note that a gap “may have bearing” on the size of the loss as naming it the mechanism. Module 10 permits that note; the pass condition seemed to forbid it. The disagreement between the two documents was recorded as a question for me to decide.
D01 isn't solved. Several of those attempts changed the fixture, so part of what the six runs measure is the test getting better. The failure was real, the baseline never passed it, and the question of which document is right is still open.
Sources and notes (10)
- Thinking in Bets (Duke) — the idea of resulting, judging a decision by its outcome
- Outcome Bias in Decision Evaluation (Baron, Hershey) — judging a decision by its result
- Hindsight Bias: On Being Wise After the Event — why the outcome record must not rewrite the decision record
- Amazon shareholder letters, 2015 and 2016 — one-way and two-way doors
- A Leader's Framework for Decision Making (Snowden, Boone) — the Cynefin domains DAS classifies by
- Documenting Architecture Decisions (Nygard) — the decision-record form DAS keeps separate from the outcome
- DAS overview — the notes this was written from
- Test and probe records — the notes this was written from
- Directives — the notes this was written from
- DAS adversarial suite — the notes this was written from
Authority System SuiteAll 17 parts →
