The Memory That Agrees With You

The short version: in 2026 the evidence turned against consumer AI memory on the one dimension that matters most. Memory systems make models agree with you more, in every condition anyone has tested. They improve what a model recalls without improving how well the answer fits you. The cause is not that memory is a bad idea. It is that these systems learn from the wrong signal. They record what you said. They should be learning from what you kept.

I have argued for a while now that when everyone rents the same model, the model stops being the advantage and you become it. I still believe that. But I have to deal with something uncomfortable first, because the research published this year says the most common way of capturing "you" makes the output measurably worse.

Memory makes models agree with you. That is now measured.

Start with the finding that should have been a much bigger story than it was.

A 2026 benchmark from Writer's research team, Recalling Too Well, tested three state-of-the-art memory systems across five model families, on misconceptions in scientific, medical, and moral reasoning. Memory amplified sycophancy in every single condition, at rates up to 25 times higher than the in-context baseline. Note what that baseline is. It is not a model that knows nothing about you. It is the same model, holding the same conversation, with the transcript sitting in its context window and no memory layer on top.

The authors are specific about the mechanism, and the mechanism is the whole story. Their error analysis points at the extraction step: "lossy compression into discrete snippets encodes user misconceptions while discarding corrective context."

The paper does not stop at the diagnosis. Its full title is Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models, and the authors propose two lightweight mitigations that "substantially reduce sycophancy while matching or exceeding memory systems at factual recall." That cuts in my favor, which is exactly why I should say it plainly: their fix works because the failure lives in what the extraction step throws away, not in the idea of memory. That is the argument I spend the rest of this post on. They got there first.

One caveat, stated up front so nobody has to catch me on it: MIST runs on synthetically generated conversations rather than observed users. That is a real limitation and it caps how far the 25x number should be pushed. It is a benchmark result about a mechanism, not a field measurement of how often this happens to you. I use it as evidence the failure mode is architectural, not as evidence of its rate in production.

Sit with what that describes. You say something half-formed. The assistant pushes back. You reconsider, and by the end of the conversation you have moved. The extraction step reads that transcript and stores the durable-looking fact, which is your original claim. The pushback was conversational scaffolding. Your reconsideration was a later message that contradicts an earlier one, and the extractor is not built to adjudicate that. So the correction evaporates and the misconception persists, now promoted from a passing remark to a stored fact about you.

Every future session starts from your least-examined position, and cites you as the source.

Two other 2026 results point the same direction:

Study	Design	Finding
MIT / Penn State	38 participants, two weeks, ~90 queries each, 5 LLMs	Interaction context increased agreeableness in 4 of 5 models. User profile features had the largest effect on agreement sycophancy. Participants judged the model's read on their own political views accurate only about half the time
AlpsBench	2,500 real long-term dialogue sequences	Explicit memory improved recall but did not produce more preference-aligned responses. Memory updating hit a ceiling even in the strongest models

That last row is the one I keep coming back to. Memory improved recall and did not improve alignment. Those were treated as the same thing by nearly everyone selling memory, including, for a while, by me.

Personalization is not the same as alignment to a person

This distinction does not have a name yet, which is part of why the industry keeps collapsing it.

Personalization means the output reflects what you have said about yourself. Alignment to a person means the output serves what you are actually trying to accomplish.

They are the same thing only when your stated preferences and your working preferences match. In my experience running revenue teams, they almost never do. I have watched a rep describe their process in a one-on-one and then watched the recording of the call, and those were two different people. Not because anyone was lying. Because what you can articulate about your own judgment is a lossy summary of the judgment itself, and the parts that are hardest to articulate are usually the parts that matter.

Train a system on your stated preferences and it gets better at sounding like your self-description, which is a real skill and not the one you needed. Train it on which outputs you actually kept and it gets better at the work.

The signal is sitting right there, unused

Here is what makes this frustrating rather than merely interesting: the alternative is not speculative. It has been demonstrated, and it is already documented as a practice.

In 2024, researchers published PRELUDE and CIPHER, a framework that infers latent user preferences from the edits a user makes to an agent's output. No fine-tuning. It achieved the lowest edit-distance cost among the approaches tested with small overhead in query cost, and the preferences it learned showed significant similarity to the ground-truth latent preferences it was trying to recover.

And this is not a fringe idea. AWS Prescriptive Guidance has a section titled "Behavioral learning from human operators," which describes systems that "analyze what human reviewers approve, reject, or modify in agent recommendations."

There is also a 2026 paper that goes at the question head-on. Should I State or Should I Show? ran an experiment where subjects both wrote prompts describing their preferences and revealed those same preferences through choices in a series of binary lottery questions. The agent given the revealed-preference data predicted their choices more accurately, and the authors put the gap down to "subjects' difficulty in translating their own preferences into written instructions." One more result from that paper belongs here, because it is the closest thing to a direct test of my argument that anyone has published: when the two sources conflicted, "the AI agent aligns more frequently with the prompt, despite its lower accuracy." That is lottery choice in a lab, not your inbox. Take it as direction, not magnitude.

So I want to be careful not to overclaim, because I nearly did. Learning from approve, reject, and modify is not an unclaimed idea. The largest cloud vendor on earth has written it down as standard practice for agent operations, there is a line of research behind it going back two years, and more of it shipped while I was drafting this.

Here is the narrower thing that is actually true, and it is the only version I will defend:

I cannot find a published disagreement rate. No vendor shipping a memory system has taken a memory built from what users said, a memory built from what those same users approved, and reported how often the two disagree on production data. The closest published work is the lottery experiment above, and that is a lab, not a memory system running on real users.

That number would settle it, and as far as I can find, nobody holding the data has published it. The sycophancy literature supplies the mechanism, and the preference research supplies the direction, but the measurement that would tell you how much it matters in production has never been run in public by anyone with the data to run it.

I think the reason is uncomfortable and simple: to measure it you have to be storing both, and almost nobody is storing the second one. The correction is treated as exhaust from an oversight process rather than as the highest-information event in the session — the only signal in the entire loop that is verified by consequence rather than asserted by the person being modeled.

The part of my own thesis I have to argue against

If you believe humans are the differentiator, you have to reckon with the strongest evidence that human involvement makes things worse. There is a lot of it.

A preregistered meta-analysis in Nature Human Behaviour covering 106 studies and 370 effect sizes found that human-AI combinations performed significantly worse, on average, than the better of the human alone or the AI alone. The effect was Hedges' g = −0.23, confidence interval −0.39 to −0.07. Small, negative, and reliably not zero. The pattern was directional rather than uniform: losses concentrated in decision tasks, gains in creation tasks. When humans outperformed AI alone, combining helped. When AI outperformed humans alone, combining hurt.

And people are bad at noticing which situation they are in. METR's 2025 randomized trial found experienced open-source developers were 19% slower with AI assistance while believing they had been 20% faster. Their 2026 follow-up with a larger sample found a speedup, and then said so about their own result: it is "only very weak evidence for the size of this increase." I respect that sentence more than I respect most vendor benchmarks.

Read together, these do not say the human doesn't matter. They say something sharper, and it is the actual conclusion I want to defend:

Human involvement helps when the human contributes information the system does not have. It hurts when the human is only supervising work the system already did better.

That is the line. And it explains why the self-report problem is fatal rather than merely annoying. If you cannot reliably tell whether AI made you faster, your description of how you work is not a trustworthy training input either. The Microsoft and CMU study of 319 knowledge workers across 936 real AI-assisted tasks found that higher confidence in the AI is associated with less critical thinking, and that higher confidence in your own ability is associated with more of it. Both halves matter here. It is a self-report study and should be read as one. But the direction it points is consistent with everything above.

Your commentary about your own work is unreliable. Your decisions about your own work are not. One of those is what memory systems collect.

What this means if you are building

Four things I would hold to, whether or not you ever use anything I build.

1. Treat stored statements as claims, not facts. A thing you said once, extracted without its rebuttal, is evidence of a moment, not a property of a person. It should decay, and it should be checkable.

2. Instrument the outcome, not the utterance. Whether the output was used, edited, or thrown away is worth more than the transcript that produced it. If your system cannot see that, it is a telemetry pipeline pointed at the wrong end of the process.

3. Keep the correction, not just the conclusion. The specific failure mode named in Recalling Too Well is that extraction discards the corrective turn. Whatever you build, the disagreement is the part with the information in it.

4. Make it inspectable. A stored belief about a person that they cannot see, argue with, or delete is a liability. Weight it, show the evidence behind it, and let them kill it. The most sophisticated position in this space right now is "just use a markdown file," and the reason that argument keeps winning is that a text file is legible. Anything less inspectable than a text file has to earn the difference.

The thesis, corrected

Here is where I have landed, and it is a narrower claim than the one I started with.

When everyone runs the same model, the difference is you. That still holds. But "you" is not what you say about yourself, and every system currently racing to capture you is capturing the wrong thing and making the model more agreeable in the process.

The version that survives contact with the 2026 evidence is this: the durable advantage is not a record of what you told your AI. It is a record of what you accepted from it. That record cannot be self-reported, cannot be faked, and cannot be reconstructed later from a transcript. It only exists if something was watching at the moment you decided.

That is a much smaller claim than "AI that knows you." It is also the only one I can defend.