The paper
Built on Broken Assumptions (BOBA): Why Coding Agents Cannot Be Left Alone, by Alejandro Cuadron and Arpit Gupta. The paper introduces a test set called BOBA-Bench and runs ten coding agents through it.
The finding in plain English
A coding agent is an AI tool that takes a request such as "add a discount code field to checkout", edits your code, runs some tests and tells you it is done. Most people who build with these tools do not stop at one request. They ask for a feature, then the next one on top of it, then a fix, then another feature. Each request builds on whatever the last one left behind.
The researchers measured what happens across that kind of run. Handed requests one at a time, the ten agents complete 95% of them on average, but when the same requests are chained 20 in a row, they complete 51% of the chains. Only 39% of chains got through all 20 steps with no request missed and no earlier test broken along the way.
The cause is narrow and easy to picture. An agent writes a change, writes or runs its own tests, sees them pass, and reports success. Sometimes those tests miss a mistake. The next request is then built on broken code, and it fails about twice as often. The mistake is rarely found later: only 14% to 33% of the tests that started failing were ever repaired by the agents' own later work.
The agents also cannot spot these mistakes by reading code. When the researchers asked each model whether a finished piece of work was correct, the models were right 53% of the time, barely better than a coin flip. That figure is measured on equal numbers of correct and incorrect implementations, so a reviewer that said "looks good" to everything would score 50%.
The good news is the cheapest fix the paper tested. When something outside the agent said "that's wrong" after a failed request and allowed one repair attempt, chain completion rose from 51% to 59%, and a mistake repaired this way stopped hurting the requests that came after it. The message did not explain what was wrong. It only said that the previous request had not been met.
How they did it
The team took the commit histories of 140 real open-source Python projects and cut them into 280 chains of 20 consecutive changes, 5,600 requests in all. Each request was written up as a plain specification based on the tests the original developers had added, and each came with hidden tests that the agent never saw. A request counts as done only if its hidden tests pass and nothing that passed before has broken.
Each agent worked through all 20 requests in order, every request in a fresh conversation, on the code the previous request left behind. Crucially, the agent was never told whether it had passed. That is what unattended use looks like: you say "next", and the agent carries on from wherever it got to. Every chain was attempted eight times per model to smooth out luck.
The ten models covered several companies, including Claude Opus 5.5, GPT-6 Astra, GLM-5.3 and Kimi K3. The best, Claude Opus 5.5, completed 66.7% of chains; the weakest in the test, Claude Opus 4.8, completed 39.2%. Every model showed the same drop from single requests to chains.
The researchers then separated two possible explanations. One is that agent-written code is simply a worse foundation than human-written code. To test that, they compared the next request built on the agent's own code after it had been checked and fixed against the same request built on the original developers' code. The gap averaged about one percentage point. Checked agent code is nearly as good a base as human code.
The other explanation is unchecked mistakes. Here the effect was large. After a single mistake slipped through, the next request's failure rate rose by 18 percentage points on average, and the odds of failure roughly doubled with every additional unverified mistake underneath. In their words, every request that follows is built on broken assumptions.
They also looked at why agents miss their own mistakes. Agents ran tests before 97% of submissions, so they were not skipping the step. The tests were the problem. A submission preceded by a passing local test run failed its hidden check 8.3% of the time, against 8.6% for all submissions, and 59% of all misses came straight after a clean local run. The authors put it simply: the agents test what they wrote, not what was asked. A green tick from the agent's own tests tells you almost nothing extra.
Finally, they tried using one AI to review another. Each of the ten models reviewed 2,180 finished implementations, half correct and half wrong. The results split into two styles, and neither works as a gate. Lenient reviewers accepted 77% to 95% of the wrong implementations; strict reviewers rejected 41% to 58% of the correct ones. Changing the harness (the software wrapper that runs the model, such as Codex CLI or OpenHands) did not help either.
Why it matters
If you build software with an agent, the useful number is how often a long sequence of requests ends up correct, since that is how most people actually work. Single-request scores, the ones most benchmarks publish, look close to perfect and hide this. Measured on chains, the best model in the test fails about a third of the time, and the average model about half the time.
For a solo founder or small team this has a direct cost. A broken step that you catch straight away costs one repair round. A broken step that you catch five features later costs you the time to find it, the features built on top of it, and possibly customer trust if it reaches production. The paper's own advice to users is short: check before you build on it, because checking after the fact is harder once the next request has already been built on the mistake.
It also changes how to read "AI reviews AI" setups. Many workflows add a second agent as a reviewer and treat that as the safety net. This study found that reviewers either waved most bad code through or rejected large amounts of good code. A second model is useful for many things, but on this evidence it is not a reliable check on whether code meets the request. We covered a related pattern in our piece on AI judges disagreeing about novelty.
And it gives a reason for some optimism. You do not need to understand the bug to help. A plain "that didn't work" was enough to recover part of the loss, and naming the failing test once did about as much as repeating the bare message three times.
Caveats
This is a single study, posted as a preprint and not yet peer reviewed. The projects are all Python, many of them are small, and "correct" means passing fixed tests, which says nothing about readability, security or whether the feature is what a user wanted. The written specifications were produced and audited by AI models and have not yet been reviewed by humans, which the authors flag themselves.
The 59% result uses a perfect signal: the "that's wrong" message came from hidden tests that were never mistaken. Your own checks will miss some problems. The authors did test noisy signals, and false alarms were cheap: rejecting one correct submission in ten cost at most 2 points of completion. The repair round also has a price. It raised the rate of chains where a previously passing test broke by 3 to 9 points and added 47 to 294 extra model calls per chain, which you pay for in tokens and time.
The paper states that the benchmark, data and code are published under Amazon's amazon-agi organisation on GitHub. When we checked on 18 November 2026, that repository address returned "not found", so the results could not yet be reproduced from the release.
How builders and ordinary people can use it today
The single habit that matters is this: after every step the agent says is done, run your own check before you ask for the next one. The agent's passing tests do not count as that check. Here is a workflow you can set up this week.
1. Write the measurement before the request
Before you ask for a feature, write down in one or two sentences how you will know it works. "A customer can enter SAVE10 at checkout and the total drops by 10%." If you code, turn that into a test yourself, or ask the agent to write the test in a separate request, then read the test before you let it write the feature. If you do not code, the check is a short manual script: open the page and then look at the result.
2. Keep checks out of the agent's reach
The paper's hidden tests worked because the agent could not see or change them. Keep your acceptance checks in a folder the agent is told not to touch, or run them from a separate script after the agent finishes. If the agent edits a test to make it pass, treat that as a failed step.
3. Put a checkpoint after every request
Commit your code (or duplicate your no-code project) after each step that passes your check. A commit is a saved snapshot you can return to. If step six turns out broken, you roll back to step five instead of untangling everything since.
4. Say "that's wrong" plainly, and name what failed
When your check fails, tell the agent so before giving it anything new. You can use close to the paper's wording: "Verification of your last change failed. The previous request is not met. Fix it so the request is satisfied." If you know which check failed, name it, for example "the SAVE10 test fails: the total does not change". The paper found that naming the failing test worked as well as about three rounds of the bare message. Give it one or two repair rounds. If it is still failing, roll back to the last checkpoint and rewrite the request.
5. Limit how far the agent runs alone
Decide a maximum number of requests the agent may chain without you checking. A reasonable starting rule (our estimate, not a figure from the paper): one request per check for anything touching payments, logins or customer data, and no more than three small requests between checks for low-risk work like copy or styling. In the study, mistakes were rarely repaired later, so the longer the unchecked run, the more work sits on top of any hidden error.
6. Keep a second AI out of the sign-off
A reviewing model can still be useful for explaining code or suggesting test cases you forgot. Do not let its approval replace running the check. In this study the best reviewer correctly separated a wrong implementation from its correct twin in fewer than one pair in five.
A worked example
Say you run a small online shop and want three things: a discount code field, a free-shipping threshold, and an order confirmation email that shows both. Done the unattended way, you ask for all three in sequence and check at the end. If the discount maths was quietly wrong in step one, the shipping threshold may be calculated on the wrong total and the email may show the wrong figures, and you now have three places to search. Done the checked way, you test the discount with two real codes after step one, commit, then ask for the shipping rule, test an order just under and just over the threshold, commit, then ask for the email and send yourself a test order. Each check takes a few minutes. Any mistake is caught at the step where it happened.
For more on why passing tests can hide bad agent behaviour, see Trial by Trace: agents that pass your test can still break the rules, and for how often agents meet every requirement of a task, our write-up of Co-Design Bench.
Sources
Cuadron and Gupta, "Built on Broken Assumptions (BOBA): Why Coding Agents Cannot Be Left Alone", arXiv:2611.02207, abstract page (submitted 1 Nov 2026, accessed 18 Nov 2026)
Full text (HTML v1), including Table 2, sections 4.1 to 5.4, Appendix C, F.6.4 and K.6 (accessed 18 Nov 2026)
amazon-agi organisation on GitHub (checked 18 Nov 2026; the BOBA-bench repository named in the paper was not yet public)

