An agent picks up a failing build on Monday. It tries the obvious fix, fails, tries a second one, discovers that the test container needs a newer base image, and the suite passes. On Tuesday a near-identical job arrives and the agent tries the obvious fix again. Nothing it learned on Monday survived the night.
This is not a model problem. Every run starts blank. Session history ends with the session, and the memory layers most frameworks ship store what was said, or what the model chose to write down, not what actually worked. Most teams have no idea how often this happens inside their own automation, because nobody is counting repeated failures. The cost is real: the same tool calls, the same tokens, the same minutes, spent rediscovering a fix the system already paid for once.
We spent the last two months building the missing piece in the open: AgentExperience.NET, a preview library that gives an agent a memory of its own verified runs. It is the same approach we bring to every client system, and it rests on three rules. We introduced the first one in an earlier essay. This one is about the second, which is where the Tuesday mistake gets caught.
Rule one: a lesson exists only when your check says the run worked
A passing test suite, an exit code, a person's approval. Never the model's own account of itself. A run whose check failed is kept quarantined and never reused. This matters more than it sounds: a memory that stores the model's own claims just remembers its mistakes with confidence, which is worse than no memory. We have written before about why a model cannot grade its own homework, and a memory layer is the place where that principle either holds or quietly breaks.
Rule two: help arrives at the moment of failure
The usual way to use agent memory is to put a block of notes at the start of the task. That is the wrong moment and the wrong key. At the start, the agent does not yet know which of its tools is about to fail, and the words of the request are a poor predictor of which lesson applies. Two tickets can read alike and break differently, or read differently and break the same way.
So the library keys lessons on how a tool failed: the tool's name, the class of error, and a fingerprint of the normalized error line. When a tool fails the way an earlier verified run also failed, the agent's very next model call receives a one-line hint naming what fixed it that time. Not before, when it would be noise, and not at the end, when the agent has already wandered. The hint is capped in size, labelled as untrusted reference material rather than an instruction, and bound by the same tenant and scope rules as everything else in the store.
On the library's retrieval benchmark, exact matching on that failure signature finds 13 of 15 with zero false positives. Of everything in the design, this is the part we are most confident in: the failure the agent is standing in is a far better key than the text of its task.
Rule three: it measures itself
A random share of runs gets no lessons at all. The verified outcomes of the two groups are compared, and the verdict is one of three: helps, hurts, or no evidence yet, always with the interval. We think this is the rule worth fighting for, and the next essay is about why every AI pilot should have one.
What the measurement says today
We publish every result, including the ones that went against us.
- Synthetic task: positive, narrow. One run per model, 12 instances, and the working strategy reaches the block verbatim. It shows that a model acts on an injected lesson. It does not show that the library helps on real tasks.
- Realistic tasks: no overall benefit yet, and a wrong lesson hurts. In the only run that could be evaluated, lessons learned by Claude Haiku 4.5 were used by gpt-oss-120b on 11 services. Against no memory, no overall benefit (p = 0.50). Against a placebo block, the real lessons used fewer tool calls on 7 of the 8 services that differed (p = 0.035), so what a lesson says does matter. Lessons from other breakages did worse than no memory: 0.303 of trials verified, against 0.455. Every other real-task run was inconclusive or stopped early.
- Real work: no result yet. The run on real development sessions is pre-registered and not switched on.
A wrong lesson costing more than none is exactly what the third rule exists to catch. Without the holdout, that run would have looked like a success, because the agent still finished.
What this means for your agents
If your agents repeat mistakes, the first question is not which memory product to buy. It is whether you can say, in a way your own code can check, what "worked" means for a run. Once you can, a memory that only stores verified runs, and only speaks when the same failure recurs, becomes a small piece of engineering rather than a leap of faith. If you cannot, no memory layer will help, and it will probably hurt.
That is the first thing we settle in a free AI readiness assessment: what your agent's check is, how often it repeats itself today, and whether a memory would be worth measuring.
The library is free and open source. The Microsoft Agent Framework adapter is on NuGet today as a preview. The command-line tool and MCP server that bring the same loop to any other agent ship from the next preview; until then, the repository has everything, including every experiment report.