Point a model at your repositories, ask why something broke, and you get an answer. Coherent, names a mechanism, shows you how it got there. The working is the problem rather than the reassurance. It looks the same whether it ends at a value in a record or at what usually happens, and only one of those is evidence.
I ran the same question at four levels of evidence, on two systems, across three models. The question was never find me a bug. It was always this already happened, work out how. The pass that worries me is the one just short of enough evidence. There is enough there to build a real hypothesis, and not enough to tell a real one from a plausible one. Two of its guesses sent me to look at things I had not checked. One was a dead end. The other was a real problem I had not known about, and nothing in either answer told me which was which.
A good guess costs the same to chase as a real one. You find out which it was at the end.
The four rungs, and what each one removes
I stopped thinking about this as adding context. Each rung takes away something the model would otherwise have to guess at, and that is the more useful way to look at it.
One. Repository access, broad. Point it at everything and ask. It removes nothing. The answer was structurally sensible, named components that were not involved, and arrived with no more hesitation than the correct one did three rungs later. If your team tried this once, got something confident and wrong, and decided the tooling is not there yet, this is probably where you stopped.
Two. Three bounded repositories, plus a written map. Which service talks to which, over what protocol, with what delivery and ordering guarantees. This removes rediscovery. What made it work was not the map. It was telling the model to treat the map as true and not go and check. That bought focus and gave up verification. Maps go stale quietly, and I have removed the step where it might have noticed.
Three. Plus traces and logs covering real executions. This removes guesswork about sequence and about what crossed a boundary. It is the closest it got without evidence of state, and it is the rung the top of this piece is about.
Four. Plus a small set of records showing the behaviour, customer fields removed and identifiers replaced, structure intact. This is the one that tells you which path the code actually took, and that turns out to be the whole game.
That last pass ran on Sonnet, which had been the weakest of the three lower down: vaguer, and closer to optimistic than the evidence supported. Opus and Fable pointed nearer the right area on thinner evidence. None of them got there. Then the one that had been worst at guessing was enough, once there was nothing left to guess.
I am not arguing that model choice is irrelevant. It changed how good the guessing was. It did not change whether there was guessing.
Why the records worked
The first defect I had already diagnosed, so I may have picked, at every rung, exactly the evidence that leads there. The second was a live bug on a project of my own, unsolved when I started. Old code, the kind that acquires a condition per edge case over years, so one entry point can take a large number of paths through it.
At rung three it reasoned forward. Here is what the code can do, here is what probably happened. At rung four it reasoned backward. Given these values in this record, this branch ran and these did not. Then it wrote a test for that path, ran it, confirmed the behaviour, and handed back a reproduction.
The reason is mundane. A branch you did not log is a branch you cannot rule out. Logging every branch in a service that has been accumulating conditions for years is not a decision anyone makes, because it costs storage, latency and review attention on every change, and the value is hypothetical until the day it is not. A record from an affected run settles it afterwards, because the values in it determine which conditions evaluated true. Not always. Two branches can converge on the same stored state, a later write can erase the evidence, and the deciding value may never have been persisted at all. Where none of those apply, the record tells you which branch ran.
Which is the job a log line would have done, if I had written one.
That is also the real difference between my two runs, and it is not technical. On the project I own, getting records was a query. Anywhere else it is a legal question somebody answered years ago without knowing they were answering it. That is a separate piece and I owe you one.
Where I could be wrong
Two runs, three models from one vendor. Not a benchmark.
Someone will say rung four hands it the diagnosis. I do not think so. Those are the same records a person opens, and opening them is not the same as knowing what they mean. The argument stands or falls on that: whether the records give you the answer, or give you what you need to find it.
What would falsify it: a defect where the branch taken is knowable from code and traces alone, with no conditional ambiguity. Rung four should add nothing.
What I keep ready now
A brief per service and a communication map, written for a model, regenerated on merge rather than maintained by hand. Delivery and ordering guarantees included, since no codebase states them.
A last-verified date on that map, because I am telling a model to trust it.
One field-aware extraction script that strips sensitive values and leaves field names, timestamps and joins intact. Written while nothing is on fire.
The habit of asking which branch it thinks ran, and what in the data says so. That question is the difference between rung three and rung four, and you can ask it from any rung.
Have you chased down a plausible side finding from a model and had it turn out to be real? Or turn out to be nothing, which costs exactly the same.
And if you are pointing a model at documentation you have told it to trust, how do you know the documentation is still true?