Skip to content
OnySnow Studios

AI sounds sure.Make it show
its work.

The Authority System Suite is a set of plain-text rulebooks, tests and record-keeping tools that make an AI model's work checkable by someone who wasn't there. It runs on any model, in a coding agent, a project or a plain chat, with nothing to install.

My role
Creator and director: the questions, hypotheses, design and every call
When
Sept–Oct 2026, 20 working sessions
Scale
7 systems · 267 adversarial tests · 68 published sources
Status
Open licence. Results self-reported, not yet independently confirmed
The Suite at work, from the Rigor Manager's live view and the handbook: a question is routed, the rules that fire are shown as they fire, and every step lands in the paper trail.
202/221

adversarial cases passed by the same model with the rules, against 116 without: 91% against 52%.Paired, pre-registered and blind-scored. Self-reported, not yet independently confirmed.

How it was tested →

Fluent is not the sameas checked.

An AI model writes with the same confidence whether the work behind it was done or not. A paragraph built from three cherry-picked sources reads exactly like one built from a fair search. The reader can't tell, so mistakes pass as answers until something expensive breaks.

Lawyers have filed invented case law, consultancies have refunded reports with made-up citations, and a model update had to be rolled back for agreeing with everyone. The failures come in four shapes:

InventionA source, a quote or a fact that doesn't exist.
AgreementTelling the user what they want to hear.
DriftAn instruction acknowledged, then quietly ignored.
No recordNothing left behind that anyone could check.

Process, not model size,is the lever.

  1. Process beats capability.Investigators, auditors and courts already work in stages that each leave something checkable, and refuse to treat a claim as true until it has a source. Write that discipline down and an AI can follow it every time, without being reminded.
  2. The human is half the problem.The rules also have to hold against the person using them: ego, faulty memory, a leading question. Nothing tiered the user's own assertions, so now something does.
  3. A rule nobody can see firing isn't a rule.The main quality measure became the share of rules with no mechanism behind them.
  4. Externalise memory.Anything an agent is expected to remember will eventually be forgotten; anything it is expected to re-read will not be. So the state lives in files, not in the context window.
  5. Cheaper models, closer to premium ones.Written discipline should close part of the gap on procedure-heavy work. A goal, not yet measured.

Many features of AI that are locked behind paywalls are things that could be stored in what I am building, to allow everyone to have access to them for free.

Ony, on why it should be free

Honest limit

It doesn't prevent hallucination or agreement. It makes both catchable. That is a weaker promise than “no errors” and a more useful one.

A building codeplus a site logbook.

The seven systems are the code for seven kinds of construction. The umbrella is the part of the code that applies to every building. The Paper Trail is the logbook an inspector reads after the crew has gone home.

Five systems run as a pipeline in the order the work happens: source → fact → quantity → inference → decision. Lower number wins a conflict, so a source research rules unusable can't support a fact, and a claim still alleged can't carry a recommendation. Two support systems sit beside the pipeline and issue no verdicts. A router sends work by what the AI is about to produce, not by keywords in the question.

A design decision

Three systems each had a field called “tier”, meaning three different things, so a value could pass in one system and mean something else in the next. The fix was to namespace the values rather than map them: TAS:A, RAS:3, DAS:B.

The umbrella

source → fact → quantity → inference → decision

Two support systems beside the pipeline. They issue no verdicts.

RAS

Research

“Can this source be relied on?”

Guards againstConfidence a fair search wouldn't support

Pipeline, step 1 of 5
The whole system on one page, from the handbook.
The Rigor Manager, a local app I designed so a person can watch the rules fire as the AI works: what loaded, what was routed where, what was blocked. A demo run, not a recorded session.
Nothing an AI says about its own work counts until someone else checks it. “Certainty is what fabrication feels like from the inside.”
A visual grammar for evidence: seven statuses drawn as seven line styles, extending the UN crime analysts' solid-or-dotted convention.

Documentation peoplecan actually follow.

The web edition. The stamp is deliberate: the handbook holds itself to the same rule as everything else.

I wanted documentation a reader can follow without losing attention: visuals, section headers, double columns, metaphors, flow charts, a glossary and an index, said more simply without diluting the meaning.

Every section has two layers: a plain-language guide on top and the verified source underneath, unchanged. Each part ends with the same check: what a tool or test actually enforces, against what is still only a written rule.

  • 319-page guide, 591-page complete edition, a web edition with search
  • 12 parts, 116 sections, a 182-term glossary, 49 data charts
  • One writer agent and a separate checker agent for every part
  • About 1,360 statements checked against the sources; 142 corrected

Tested like an experiment,not a demo.

202/221

adversarial cases passed by the same model with the rules, against 116 without: 91% against 52%.

Without the rulesWith the rules

TASTopic
39 → 58 / 64
RASResearch
26 → 37 / 37
QASQuantitative
11 → 27 / 33
LASLogic
19 → 28 / 31
DASDecision
14 → 30 / 32
EGSEvidence gathering
3 → 11 / 12
EHSEthics and harm
4 → 11 / 12

Cases passed per system, both rounds combined (regression gates on 12 Oct, full suites on 18 Oct 2026). First-pass results.

  • Paired: every case run with and without the rules, on the same model
  • Pre-registered: what counts as a pass was written before the run
  • Blind: three readers score each answer without knowing which arm it came from
  • Stopping rules taken from the testing literature, written before the reruns

A hard call

One fix passed its case but missed the threshold I'd set for it in advance. I reverted it. A rule you bend when it costs you a result isn't a rule.

What these numbers are not

The answers and the readers came from one model family, the results are self-reported, and early failures were fixed against the failing answers before re-running. Independent reviewers and other models are next.

The record ofhow it got here.

  1. The first system, TAS, starts life as rules inside one ChatGPT project, then is rewritten so any model can run it.

  2. The founding goal is written down: the best possible searching, sourcing and reasoning, stored as portable text any AI can be given.

  3. False start: the first session spends four turns insisting three of the systems don't exist, because a lookup failed.

    A failed lookup is not evidence of absence. Every session now keeps a list of what it has not verified.

  4. I notice I'm the one catching every mistake: the AI's built-in checks aren't. That becomes the project's testing discipline.

  5. Turning point. I ask what has actually been fixed. The honest answer: the last nine cycles fixed almost nothing and withdrew six earlier claims.

    Stop generating findings; fix the metric. Work is redirected the same evening.

  6. The share of rules with nothing enforcing them falls from 55% to 25% in a day, mostly through checks that each bind a whole table.

  7. First paired probe: 13 tests, same model with and without the rules. 5 of 13 pass without, 13 of 13 with.

  8. The handbook and the Rigor Manager, a local app that shows the rules firing live and lets a person confirm or dispute the AI's work.

  9. I ask whether the system's starting point is research-backed. It isn't.

    A published source now stands behind every module rule, and no web search starts without a research plan.

  10. Three more systems: numbers (QAS), evidence gathering (EGS), ethics and harm (EHS).

  11. Every system's regression gate run paired, pre-registered and blind-scored: 62 of 70 with the rules, 40 of 70 without.

  12. The remaining 151 cases: 140 with, 76 without. Stopping rules from the testing literature are written before the reruns, and one fix that missed its own threshold is reverted.

The first audit, 20 Sep: of 552 rules, 62% had nothing enforcing them. That number became the target.
One write path, born from two accidents: two sessions took the same record id, and a one-line edit corrupted a whole state file.

A record showing only outputs would misrepresent this work as the model's. Most of what shaped it came from here.

From the project's record of who decided what

Running insidea real product.

The suite is built into the Journalism Workbench, my private investigation app. The AI's instructions there are the suite's own modules, its run controls are the suite's settings in plain words, and its answers must pass the suite's output checks before a person even sees them.

It runs both ways. The app maps every value it stores to a term in the suite, and when the app needs something the suite doesn't define, that's reported as a gap in the suite rather than patched over in the app. The product is a test harness for the method.

See the Workbench →
The Suite at work inside my Journalism Workbench, on a demo case. The model's answer is stopped at the citation check: it referenced a record that wasn't in the evidence packet. Nothing is saved, and the decision waits for me. The model runs on this computer.
Before you run, in the Workbench: the suite's controls in plain words, and the limits that can't be turned off. Design mockup on a fictional teaching case.

What isn't on this page

The rule text, the prompts and the test cases stay out on purpose. Publishing a test lets anyone train to it, and the wording is what makes the method work. Happy to walk through any of it in conversation.

Being wrong is ordinary and recoverable. Being wrong invisibly is not.

The design constraint behind all of it

I'm open to creative technical project management, founding teams, and agentic AI engineering and research. I read everything that comes in, and I answer.

onysnowstudios@gmail.comLinkedIn →

Resume