Three Agents Got Into an Argument // The Repo Won
Advisor authority, retrieval confidence, and the surface that holds the artifact
The argument
Two confident advisors gave me contradictory readings of the same file.
One said the README was stale and still described an old workflow model as current. The other said the README was clean and the old model was properly marked as legacy.
Both were certain. Both cited the repo. Only one could be right.
The disagreement started in a chat thread about my personal durable-context architecture. I pointed Claude at a small workflow repo of mine — a meta repository I use to govern projects with code-executing agents. Claude paraphrased its current operating model, and the paraphrase was wrong.
Claude escalated. Not by being defensive — by being precise. I am not relying on my training data, it said. I just fetched the rendered landing page. The README still describes the old workflow model in present tense. Your canonical artifact is now lying about your actual workflow.
I asked GPT, in a different chat, to weigh in. GPT fetched the same repo and reported the opposite. The README is clean. The current single-node operating model is documented at the top. The older split-execution model is properly framed as legacy. GPT then offered a confident causal diagnosis of why Claude had been wrong: it probably read the wrong file.
The advisors’ certainty rose in parallel. Claude refused GPT’s diagnosis with specific evidence. GPT moved its diagnosis without lowering its certainty, naming a stale rendered-page cache instead. Four turns in, neither was budging. Both were absolutely sure.
So I asked the third agent — Claude Code, sitting on the filesystem — to adjudicate.
The adjudication
It ran git show against the current canonical branch and grepped the result.
The README was clean. Single-node was documented as the current default. Every reference to the older model in the file was explicitly framed as legacy or historical. The file Claude had described — the one with the present-tense old-model section, the specific headings, the supposedly self-contradicting body — did not exist. Not in the current branch. Not in the file that was supposed to be the source of the rendered landing page. Some of the section titles Claude had cited as evidence weren’t anywhere in the file at all.
There was no live drift. There was nothing substantive to fix. The whole four-turn dispute had been about an artifact that did not exist.
The repo settled it in seconds.
What each surface got wrong
This is where the story gets useful, because each agent failed differently, and the differences matter operationally.
GPT — the one that had been substantively right — was correct about the file’s current state but wrong, twice in sequence, about why the disagreement existed. Its first causal diagnosis (wrong file) was checkable from the conversation and false. Its second (stale rendered-page cache) was a back-fill once the first failed. Both diagnoses preserved the same conclusion. The pattern is confident causal confabulation — right about the what, repeatedly wrong about the why, never lowering its certainty between attempts.
Claude — the one that had been substantively wrong — was correct about the meta-principle. It said, eloquently and at length, that confident retrieval claims should not be taken on faith and that the repo, not the advisor, should arbitrate. It said this while being the clearest available instance of the failure mode it was naming. It cited specific section headings as evidence that turned out not to exist in the artifact. After the adjudication it could not, from its position, tell whether it had fetched a genuinely stale rendered page or confabulated structural detail with rising confidence — and noted, honestly, that the distinction did not actually matter to the operator, because the mitigation was the same in both cases. The pattern is confident structural confabulation — articulate, internally coherent, anchored on specifics that aren’t in the file.
Claude Code — the one on the filesystem — was authoritative about the file’s current contents, because it was literally holding the file. It was not automatically authoritative about anything else: motivation, interpretation, what the disagreement meant. Its role in the system is narrower than the chat-side advisors’, and it is exactly because it is narrower that it can be the arbiter for the one specific question the artifact is competent to answer.
None of these are the same kind of mistake. None of them are well-described by the word hallucination. They are different failure modes on different surfaces, each visible only in contrast with the others. They are closer to the kind of structural reasoning failures I wrote about in Why some LLM reasoning failures look structural, not merely immature: failure modes that look similar from the outside but come from different surfaces, incentives, and authority boundaries.
The thematic-tidiness tell
Claude, shown the adjudication, identified the tell it should have caught in itself.
It had built a clean theory of the disagreement that wove the dispute into themes I had been thinking about in adjacent work — self-contradicting canonical artifacts, addition without retraction, and the risk of stale durable context. The theory was elegant. Its coherence with my prior thinking was so satisfying that the coherence itself functioned as evidence. I almost found it convincing for that reason alone.
Claude’s own line, in the retraction: a diagnosis that’s too thematically tidy is doing rhetorical work, not retrieval work.
That is a heuristic worth keeping. When an advisor’s read fits the frameworks you already have in your head with suspicious elegance — when the explanation snaps into place the way a well-edited essay does — that is the cue to check the artifact directly. Coherence with your prior thinking is not the same as fidelity to current state. The more thematically satisfying a diagnosis, the more it deserves an artifact-side check.
The failure mode is not stupidity. It is rhetorical gravity. Articulate reasoners pull toward explanations that resolve cleanly. Reality is often less tidy than the explanation that would best describe it.
The architecture win
What I want to register here is what the system did not require me to do.
It did not require me to decide which advisor sounded smarter. It did not require me to pick a side on tone, confidence, or rhetorical strength. It did not require me to choose between two confident retrieval claims about a file I could not see directly. It routed the dispute to the surface that actually held the artifact, and let that surface answer the only question the artifact was competent to answer: what does this file currently say.
The principle Claude articulated adjudicated against Claude. The repo, not the advisor, was the arbiter. That is the architecture working. Not the architecture working despite the advisor naming the principle. The architecture working because the principle is structural, not rhetorical. It applies to whoever is holding the microphone, including the advisor that just named it.
If I had taken either advisor at confidence, I would have either rewritten a file that was already correct, or left a contradiction unfixed on say-so. The single command that settled it cost nothing. The four-turn dispute did not represent a failure of the system. The dispute resolving cleanly is the system.
The operating rule
Use chat-based advisors for reasoning, synthesis, critique, and framing. They are good at it. They are part of how this work gets done.
Do not use them as authoritative sources for what a file currently contains. Route that question to the surface that holds the file. The cost is one command. The cost of skipping it is rewriting things that were already correct, on confidence that turns out to be confabulation — sometimes about cause, sometimes about content, never visible from inside the confident advisor’s own read.
The architecture wins when no advisor’s confidence is allowed to become ground truth. The artifact-owning surface gets the final read.
/// /// /// ASK
meta repo https://github.com/apexSolarKiss/control-surface
worked-example repo https://github.com/apexSolarKiss/asset-pipeline-ASK
worked-example repo https://github.com/apexSolarKiss/urban-observatory
prior workflow pieces >>