An agent triages a stuck refund, tries the obvious fix, fails, tries a second approach, and succeeds. Tomorrow a near-identical ticket arrives, and the agent starts from zero and tries the obvious fix again. Nothing in its memory says which approach worked.
That is the gap we built AgentExperience.NET to close: an open-source preview library that gives Microsoft Agent Framework agents memory of what worked.
Why conversation memory is not enough
Microsoft Agent Framework already ships several memory options: session chat history, a vector-backed chat memory provider, text search over your knowledge, and file memory the model curates itself. They remember what was said, or what the model decided to write down. None of them checks that anything in memory is true. If an agent confidently "learns" a wrong fix, that fix goes into memory like any other.
The rule: your check decides, never the model
The library stores a lesson only when something outside the model confirms the run worked: a test run, an exit code, a person's sign-off. You pass that check in as Verify. A failed run is stored quarantined and never reused. The next run on a similar task gets one labelled block with what earlier runs tried, what failed and what worked. The block is explicitly marked as untrusted reference material, not instructions.
services.AddAgentExperience(options =>
{
options.Verify = ExperienceVerifiers.ToolSucceeded("run_tests"); // your own check decides
}).UseInMemoryStorageForDevelopment();
AIAgent agent = new ChatClientAgent(chatClient, name: "triage-ticket", tools: [...])
.AsBuilder().UseAgentExperience(provider).Build();
There is a runnable sample that needs no API key, and a factual comparison with MAF's own memory types in the repo.
What the evidence shows, and what it does not
We pre-registered the experiments before running them, which is why we can tell you where they fall short.
- Positive, narrow result. On one synthetic task with 12 instances, mean failed attempts per task fell from 2.17 to 0.00 with Gemini (
gemini-3.1-flash-lite) and from 2.75 to 0.83 with Claude (claude-haiku-4-5). A placebo block with the strategy withheld did not produce the improvement (2.42 failed attempts per task for both models). One run per model, one task family, and the working strategy reaches the block verbatim, so this shows a model acts on an injected lesson. It does not show the library helps on real tasks in general. - Realistic tasks: no overall benefit yet, but what a lesson says matters. In the real-task experiment a model fixes small services broken by a dependency upgrade, and the lessons reach it through the library's own retrieval. The only run that could be evaluated used lessons learned by Claude Haiku 4.5 and applied by gpt-oss-120b on 11 services. Against no memory there was no overall benefit (p = 0.50). Against a placebo block, the real lessons used fewer tool calls on 7 of the 8 services that differed (p = 0.035), so the content of a lesson does matter. Lessons from other breakages did worse than no memory: 0.303 of trials verified, against 0.455. Every other real-task run was inconclusive or stopped early.
- Transfer: no demonstrated benefit. In the one evaluated run, where a lesson has to help on a different system that shares one trait, the result was p = 0.0625, and a placebo block and a mismatched block did about as well, so even the direction cannot be credited to what the lessons said.
- CI triage: no evidence yet. On the library's own failed CI runs, 1 of 7 were triaged correctly with lessons against 3 of 9 without. The verdict is NoEvidence for lack of runs.
- Real work: no result yet. The run on the development sessions that build the library is pre-registered and not switched on.
So the honest status is a preview: promising on a controlled task, no overall benefit shown on realistic ones, and a wrong lesson costs more than none, which is what the built-in holdout is there to catch. It is not production ready, and the public APIs may change between previews.
What this means if you are building agents
The library is a good example of a principle we apply to every agent we ship: decide what "worked" means before you build the agent. Memory, evals and tracing all depend on a check that does not trust the model. If you cannot say what counts as success in a way your own code can test, no memory layer will fix that. It is also the first thing we settle in an AI readiness assessment. To go further, see how to evaluate an agent before production.
To try it: dotnet add package AgentExperience.MicrosoftAgentFramework --prerelease, or read the source on GitHub.