Generated Code Fails in a Specific Shape

Generated code does not fail randomly. After a year of shipping a lot of it, I can predict where it will break before I read the diff, and the prediction is usually right.

That predictability is useful. It means review can be targeted instead of exhaustive. Here is the pattern I have settled on, which is that the failures cluster in one place: the code is correct as a description of the happy path, and silent about the environment it will run in.

The signature

Every bug in this category shares a shape. The logic is right if you assume the function runs once, alone, on a machine where nothing else is happening and nothing fails.

Production violates all three of those assumptions constantly, and they are exactly the assumptions that are invisible in a code sample. A function that reads a balance, checks it, and writes it back is textbook correct. It is also wrong the first time two requests arrive together, and nothing in the code says so.

The version on top is the one I got. It reads well. Somebody reviewing it quickly sees a balance check and a decrement and moves on, because that is what the code is supposed to do. The version underneath is the same rule enforced inside the query, where the database can hold the line. No lock, no transaction, no race.

I did not catch this by being clever. I caught it because "what happens if two of these run at once" is one of three questions I now ask about anything that touches state.

Five places it concentrates

Concurrency, which is the big one. Read, decide, write. Check the seat count then insert the enrolment. Verify a coupon then apply it. All of these are correct in a single-threaded story and wrong under traffic. The fix is almost always to make the database enforce the condition rather than the application, either through the query filter or a constraint.

Retries and duplicates. Code assumes it will be called once. Payment confirmations arrive twice, one from the browser redirect and one from the provider webhook. Queues redeliver. Users double tap. Generated handlers rarely account for this because the second call is not visible in the description of the problem.

The failure path. Happy path complete, error path either missing or reduced to a catch that logs and moves on. What I look for specifically is partial failure. If this function does three writes and the second fails, what state is the system in. The generated answer to that question is usually silence.

Boundaries. Empty arrays, zero, null, the first page, the last page, the timezone at midnight, the amount that exactly equals the balance. Generated code handles the middle of the range confidently and the edges optimistically.

Version drift. APIs that were correct two years ago, arguments that moved, methods that were deprecated. This one is easy to catch because it fails loudly at runtime, which makes it the least dangerous item on the list.

Notice that four of the five are not about syntax or even about logic in isolation. They are about the gap between a function and a system.

Why the bugs land here

I find it useful to be concrete about the mechanism instead of hand waving about hallucination.

A model is producing the most plausible continuation of the code so far. Plausibility is calibrated on a vast amount of published code, and published code is overwhelmingly illustrative. Tutorials, documentation snippets, answers to narrow questions, example repos. That body of work is optimised for teaching one idea clearly, which means it deliberately strips out concurrency handling, retry logic and error paths, because those obscure the point being made.

So the training distribution is full of code that is correct as an illustration and incomplete as an implementation. The output inherits that. It is not making things up, it is faithfully reproducing a genre whose conventions include leaving out the hard parts.

That also explains the second thing I notice, which is that the confidence is uniform. Illustrative code does not hedge. There are no comments saying this part is subtle. So the generated version of the trickiest function in your system looks exactly as settled as the version that formats a date.

How I actually review with this in mind

I do not read the diff top to bottom any more. I search it.

First I find every place that reads state and then writes it, and I ask whether the condition can be pushed into the write. That single pass catches most of the concurrency class.

Then I find every entry point that can be triggered externally, and I ask whether calling it twice with the same input produces the same result. Webhooks, callbacks, queue consumers, retry-eligible endpoints. If the answer is no, it needs an idempotency record, not a comment saying this should not happen.

Then I look at each function that performs more than one write and ask what happens if it stops halfway. Sometimes the answer is that it is fine. Often the answer is a partially created record that nothing will ever clean up.

Finally I check the edges: zero, empty, exact equality, and the largest value someone might plausibly enter.

That is maybe ten minutes on a moderate diff, and it has a much better hit rate than reading every line with equal attention, because it goes where the bugs actually live.

The part that surprised me

I expected generated code to be wrong in obvious ways. Wrong function names, invented libraries, logic that does not compile. Those exist and they cost almost nothing, because the machine catches them immediately.

The expensive ones are the opposite. They compile, they pass the test you thought to write, they work in staging, and they fail under conditions your development environment structurally cannot reproduce. A single-user test environment cannot produce a race. A machine that never sleeps cannot produce a cold start. A clean database cannot produce the duplicate that arrives three seconds later.

Which means the review is not looking for mistakes in the code. It is looking for the assumptions the code is making about the world, and checking them against the world I actually deploy to.

That skill was always valuable. It is now the whole job.