If you run an AI agent, there is a good chance it keeps a lessons file. It might be a CLAUDE.md with a growing "gotchas" section, a skills folder the agent writes to itself, or the saved memories in ChatGPT or Claude. The idea is sensible: the agent makes a mistake, writes down a rule, and does better next time. A new preprint argues that this habit can lock mistakes in, and tests a cheaper design that kept the logs and threw the lessons away.
The paper
More Memory, Worse Agents: Error Reproduction and Anti-Persistence in LLM Agents by Ruhan Wang, Kishan Panaganti and Dongruo Zhou (Indiana University and Tencent HY), dated 19 October 2026. It appeared on Hugging Face's daily papers list It is a preprint and has not been peer reviewed.
The finding in plain English
Agents that improve without retraining usually store one of two things. They keep raw records of what happened (the task, the steps taken, whether it worked), or they keep lessons drawn from those records: rules, summaries, strategies or reusable skills. Lessons are compact and easy to reuse, which is why most agent memory systems save them.
The authors describe a loop they call error reproduction. A faulty rule gets into memory. The agent follows it on later tasks, so the logs from those tasks contain the same mistake. When the agent later reviews its logs to write new lessons, it reads that mistake back and writes a fresh rule that repeats it. In the paper's words, errors are "regenerated through a cycle of abstraction, execution, and re-abstraction". Deleting the original bad rule does not fix this, because the logs it shaped and the rules later derived from them are still there.
Their fix, called Anti-Persistence, keeps only the raw run records and outcomes. When a new task arrives, it pulls similar past runs and decides whether to use them as they are or write short guidance from them. That guidance is used for the one task and then discarded. Nothing the agent concludes about itself is stored.
How they did it
They tested on three standard agent benchmarks: ALFWorld (text-based household tasks), WebShop (finding and buying products on a simulated shop site) and ScienceWorld (text-based science experiments). They used three open models: Llama-4-Scout, Qwen3-Coder-Next and Gemma-4-31B-it. That gives nine benchmark and model combinations.
Each setup ran ten rounds of self-improvement on training tasks. After each round, memory was frozen and the agent was scored on held-out tasks it had not trained on, over three random seeds. They compared four memory designs (raw logs only, stored lessons only, both together, and Anti-Persistence) plus four published agent methods: ReAct, Reflexion, ExpeL and AutoRefine, which cleans up its stored lessons over time.
The guidance step has four modes. "None" passes past runs straight through with no extra model call. "Imitation" pulls out action patterns from successful runs. "Strategic" writes a higher-level plan. "Corrective" studies failed runs to produce warnings. A small selector picks a mode per task based on how each mode did on similar past tasks, with a small penalty for every extra model call.
The results
Averaged across all nine settings, held-out success was 59.9% for Anti-Persistence, against:
53.5% for raw logs only
49.0% for stored lessons only
49.3% for logs and lessons together
52.9% for Reflexion, the strongest of the published methods
It was the best performer in seven of the nine settings and had the lowest average number of model calls of the four memory designs. The authors put that partly down to the "none" mode, which reuses past runs without an extra call, and to shorter runs in some settings. Two further results are the ones to take away. Stored lessons, alone or alongside logs, did worse than plain logs in seven of nine settings. In a direct test where the only difference was whether generated guidance was kept or thrown away, throwing it away scored higher in all four settings tried. AutoRefine, which prunes and refines stored lessons, beat plain stored lessons in only one of six settings.
Why it matters
For anyone building agents, the result changes where effort goes. Many builders spend time curating a lessons file. This paper suggests the more valuable asset is a clean record of what actually happened, with advice written fresh each time from the relevant examples. It also suggests that "the agent writes its own rules" is risky without checks, because a wrong rule can survive its own deletion by contaminating the logs.
If you are weighing up a memory layer such as the one in our look at the Mem0 memory stack, this paper is a reason to check what kind of content it stores. Its evidence points one way: store observations, and be sceptical of stored conclusions.
Caveats
Preprint. No peer review yet, and no code link was visible in the version we read.
Benchmarks. ALFWorld, WebShop and ScienceWorld are simulated environments with clear pass or fail outcomes. Your sales agent or content pipeline has fuzzier success signals, which makes it harder to know which past runs were good.
Models. Three open-weight models. No frontier closed models such as Claude or GPT were tested, and stronger models may write better lessons.
Not a clean sweep. The authors say the preferred design stays "setting-dependent", it was not the top performer in two of nine settings, and they note that errors can still leak back through stored logs. They also say the AutoRefine comparison does not rule out better clean-up policies.
How builders can use it today
1. Audit your lessons file. Open your CLAUDE.md, memory folder or skills directory and sort every entry into one of three buckets:
Facts you wrote. Project paths, API names, house style, "we use pnpm". Keep these. You wrote them and can check them.
Rules the agent wrote about itself. "Always retry the API twice", "skip tests for small changes". These are the risky ones. For each, ask: which run produced this, and was that run actually correct?
Rules you cannot trace. If nobody remembers why it is there, move it to an archive file the agent does not load.
Watch for rules that are broad ("always", "never"), that contradict each other, or that showed up after a run you remember going badly.
2. Run an A/B test. Pick 10 to 20 real tasks the agent has not done before, with a clear pass or fail for each. Run them twice in fresh sessions: once with the agent-written lessons loaded, once with only your own facts plus a folder of past run logs. Same model, same settings. Score blind if you can. If the version without lessons does as well or better, retire the lessons. Our guide to automated evals for prompt tuning covers how to set up the scoring so you are not judging by feel.
3. Switch to logs plus fresh guidance. A minimal version of the paper's idea:
Save each run as a short record: task, key steps, outcome (pass or fail), date.
At the start of a new task, retrieve the three to five most similar records.
Either paste them in as examples, or ask the model to write a short task-specific checklist from them.
Use the checklist for this task only. Do not save it.
If you tune your agent's harness with a loop like the one in our Harness 2 write-up, which keeps an experience library of adopted edits and notes on when they worked, apply the same caution there: keep the before-and-after run records, and re-check a stored note against those runs before you reuse it.
How ordinary ChatGPT and Claude users can use it
You do not need to build anything. Open the memory or personalisation settings in whichever assistant you use and read what it has saved about you. Look for conclusions it drew on its own, such as a preference you stated once in passing or a "fact" that is out of date. Delete those, keep the stable facts you would happily write yourself, and for recurring jobs keep a short document of good past examples to paste in, rather than relying on the assistant's summary of what you like.