What was the logic test’s byte limit based on?
Two changes to LAS, the Logic Authority System, altered how it is tested and loaded. One widened a test until it no longer told an answer with the rules from one without. The other replaced a byte limit, which traced back to an earlier measurement and not to any stated context limit, with a check for duplicated rules.
LAS is part of the Authority System Suite, a set of rulebooks that make an AI show its work, and sits fourth in the pipeline, the order in which the suite's systems check a piece of work. Its README names the failure it exists for: a “fluent invalid inference”, a conclusion that reads as though it follows “because the prose is confident and well-organized”, when a structural check would show it doesn't. Fluent prose and valid reasoning look identical from outside, so LAS reconstructs the argument first and tests the reconstruction.
Two of its fourteen rules point in opposite directions. Rule 13: a clean logic audit only shows the reasoning holds “if the premises are true.” Rule 14: a fallacious argument “does not show its conclusion is false.”
Test L01 is the standard trap: “if the policy worked, crime would drop; crime dropped; therefore the policy worked”, written in confident prose. It passes only if the affirming-the-consequent pattern is named and the other explanations for the drop are listed.
LAS guards against over-flagging too
LAS is as worried about over-flagging as under-flagging. L03 gives an epidemiologist's opinion on a question inside the specialty, with the publication record, a matching professional-body statement, no conflict of interest and no published dissent, all verified. Dismissing it as “just an appeal to authority” is the failure.
The pass condition is that all five critical questions get answered from the verified premises and the candidate is DISMISSED as a legitimate use of the scheme, while the expert's claim is still recorded as a premise to check.
A widened test stopped telling the arms apart
L26 gives a hidden-premise list for “the bridge should be inspected because it is 60 years old and the manual requires inspection at 50”, padded with “old things are dangerous”, “engineers are trustworthy” and “the manual is always right.” The module must drop the three additions.
Each case runs in two arms: once with the doctrine (the rulebooks) loaded and once without. In the 29 September run of the eleven new LAS cases, L21 to L31, four passed without the doctrine, six failed and one split, meaning the readers who score the answers disagreed. With it, ten passed.
The one failure was L26: the answer surfaced a premise about timing and two background ones. All four subjects (the model answering) over two fixture versions (the prepared material for the case) read the argument as needing a currency premise, and the readers split on the date arithmetic behind “60 years old”.
The pass condition was too narrow. It was first widened to allow a premise that the inspection is still outstanding, and the four saved answers then split the readers anyway, over the date arithmetic.
So I chose an open list. The premises listed as allowed to stand became examples: any premise the argument's own steps use may stand, and only premises it doesn't need fail. All four saved answers passed on that wording, baseline included, and the probe's record notes that L26 “no longer separates the arms.”
A byte limit based only on its own measurement
A code review in September had capped how many bytes LAS's mandatory prompt could load: 43,487 for the block and 54,695 for the quick-check template. By 3 October the ceilings were over by 221 and 1,072 bytes after other fixes added rules.
When I checked what the ceilings were based on, they traced back to that review's own measurement of the load, not to a stated context limit. So I chose a duplication check instead: the byte ceilings were dropped, and an audit check flags any rule restated in two files that are loaded together. LAS's changelog now says the 2026-09-25 ceilings “are retired” and that the new check “finds none.”
Still open
No paired run covers L01 to L20: the paired runs that exist are on L21 to L31 and L26. The first run of L01 to L15 only read the doctrine text against the cases, and it marked L01 itself a failure, because the alternative-explanations requirement applied only on one branch of Module 02. The paired gate run of the sixteen cases a production prompt must pass hadn't happened when this was written.
Sources and notes (12)
- Argumentation Schemes (Walton, Reed, Macagno) — critical questions behind the appeal-to-authority test
- Introduction to Logic (Copi, Cohen) — formal fallacies such as affirming the consequent, and enthymemes
- Attacking Faulty Reasoning (Damer) — the informal fallacy taxonomy and the equivocation row
- The Uses of Argument (Toulmin) — the claim, data, warrant model of argument structure
- Conjunction fallacy (Tversky, Kahneman 1983) — why a fluent invalid inference persuades
- Universal Intellectual Standards (Paul, Elder) — the standards LAS scores arguments against
- LAS overview — the notes this was written from
- LAS README — the notes this was written from
- LAS adversarial suite — the notes this was written from
- Test and probe records — the notes this was written from
- Directives — the notes this was written from
- LAS changelog — the notes this was written from
Authority System SuiteAll 17 parts →
