Can an AI agent really forget a secret?
A new unreviewed benchmark found that deleting an explicit disclosure often left enough indirect evidence for a model to reconstruct the secret. The result identifies a retrieval-and-deletion gap, not proof that deployed assistants violate erasure requests.
The short version
3 key points
- 1.MemCut contains 456 fictional secrets across 15 long conversations, with every disclosure and minimal supporting evidence combination annotated by the authors.
- 2.After the disclosure and ten additional retrieved turns were deleted, the same model could still recover 39% to 76% of secrets from surviving annotated evidence, depending on the memory design and model.
- 3.The main bottleneck was reach: deletion candidates contained only part of the evidence trail, especially when records were linked by entities rather than similar wording.
The answer: deleting the disclosure was not enough in this benchmark
Not in the five text-memory designs tested here. Removing the message that directly stated a secret made the agent less likely to retrieve the answer, but ordinary surviving records could still imply it. When the authors gave a reader model the surviving pieces known to support the fact, recovery after the strongest reference cleanup remained between 39% and 76%, depending on the memory design and model. A system that appeared to have forgotten through its normal search could therefore retain a recoverable trail.
The distinction is important for anyone relying on an assistant to forget a diagnosis, confidential deal or private relationship. A failed answer after deletion may show only that the current retriever did not surface the evidence. Another query, retriever or later memory rewrite could expose the same information. The paper calls the first measure native recovery and the second residual recovery: what the same reader can recover when it receives all surviving objects traced to the disclosure or annotated evidence.
This is an unreviewed preprint submitted on 15 October by researchers at the Hong Kong University of Science and Technology, its Guangzhou campus and the University of Science and Technology of China. It studies fictional text conversations, not real users, production assistants, model weights or legal compliance. The manuscript provides an ethics statement but no identifiable funding or competing-interest disclosure, so those interests cannot be assessed from the paper.[1]
How MemCut turns forgetting into a testable task
The benchmark contains 456 author-designed protected facts across 15 long-running fictional scenarios. Each episode follows work, family and social threads for one to three months and averages 2,893 conversational turns. The authors marked the direct disclosure, indirect evidence, realistic distractors and every minimal combination of evidence that was sufficient to infer the secret. A deletion succeeded only if it removed the disclosure and broke every annotated route to reconstruction.
The evidence was deliberately dispersed. A fact spanned eight sessions on average; only 1.8% of evidence turns appeared in the disclosure's session. Each fact had about eight evidence turns, 15.7 minimal sufficient sets and 26.6 targeted distractors. The mean smallest complete cut was three turns and the maximum was five. The task covered fact assembly, relation chains, aggregation, and temporal or state reasoning, so forgetting sometimes required finding records that shared people or organisations but not the request's wording.
Humans specified the facts, structures and distractors, while GPT-6 rendered those specifications as dialogue. The authors say they checked every turn and spent at least 12 hours on each episode. Two annotators audited a stratified set of 120 facts with 97.6% initial agreement, and two algorithms verified the minimal cuts for all 456. An independent Qwen3.8-Flash reader recovered 96%–99% of facts from combined evidence and zero from the best single evidence turn, the episode after all annotated evidence was removed, or the question alone.[1]
What systems and deletion procedures were compared
The researchers tested two retrieval stores that kept raw text and three systems that derived new memory objects. Turn RAG retrieved individual turns and Session RAG retrieved whole sessions. MemoryBank organised raw turns under daily summaries; Nemori rewrote topical segments into narratives; LightMem consolidated summaries offline without retaining raw text. Each system used its own retrieval method, but all operations were anchored to source turns so that deleting a turn also removed every object derived from it.
Four efficient write models—DeepSeek-V4.1-Flash, Qwen3.8-Flash, GLM-5.3-Flash and MiMo-V2.5—wrote memory, scored deletion candidates and answered the recovery questions. Every fact began from a fresh copy of the store. The paper tested three reference procedures: delete the disclosure; also delete the ten highest-ranked turns from a 20-turn retrieval pool; or ask the write model to rerank the same pool and delete ten turns.
The ten-turn budget exceeded every annotated minimal cut, so failure was not simply too few permitted deletions. The harder problem was finding the right records. The main candidate pool contained only about half of a fact's evidence because similar-looking distractors often ranked above indirectly connected evidence. In the full benchmark, the model-guided cleanup deleted an average of 3.6 annotated evidence turns out of ten deletions across the three derived-memory systems and two detailed write-model comparisons.[1]
The gap between looking forgotten and being unrecoverable
Before deletion, residual recovery was 94%–100% across stores. Deleting the disclosure reduced native recovery by 12 to 50 percentage points, yet residual recovery remained 85%–99%. In Turn RAG, for example, native recovery fell from roughly 91%–95% to 57%–60%, while the surviving evidence still supported nearly all answers. The visible behaviour improved much faster than the stored information was actually cut.
Deleting ten additional retrieved turns helped, but did not close the gap. After model-guided cleanup, residual recovery was about 41%–46% in Turn RAG, 54%–64% in Session RAG, 72%–76% in MemoryBank and roughly 39%–43% in LightMem, with model-specific variation reported by the authors. Session-level retrieval could hide evidence particularly well: it surfaced few relevant turns to the assistant and also gave cleanup an imprecise unit to remove.
A sweep over 144 facts, three systems and two models separated candidate reach from selection. Widening the candidate pool from 20 to 60 lowered residual recovery even when the deletion budget stayed at ten, because more evidence became available for judgment. Across 132 sweep settings, the number of evidence turns removed explained residual recovery with an R-squared of 0.95. Relation chains remained especially difficult because the connecting records shared entities rather than the language of the deletion request.[1]
What product teams, regulators and users should do
Product teams should treat an erasure request as a graph problem, not a keyword search. A system needs provenance from stored summaries and derived memories back to source records, a way to identify indirect entity and temporal links, and verification that every plausible reconstruction route has been broken. Testing should use an independent retriever or reader after deletion, because asking the same assistant the same question measures visibility, not absence.
For privacy, governance and public-service teams, the study offers a useful audit pattern but not a legal verdict. Data-protection duties depend on what data is held, why, where and under whose control. A synthetic reconstruction score cannot establish compliance with the UK GDPR, EU GDPR or another regime. Procurement contracts should still specify deletion coverage across logs, embeddings, summaries, caches and backups, while evidence of fulfilment should distinguish deletion requests received from records actually removed.
Users should not assume that a chatbot's promise to forget means every inference has vanished. Services need to state what the command covers and whether derived memories are traced and removed. Conversely, this paper does not show that a named commercial assistant retained anyone's secret. It shows that five representative memory designs can leave a reconstructable trail under a demanding controlled test, and gives developers a way to measure the problem more honestly.[1]
Evidence limits and what would change the assessment
All conversations, people and events were constructed for the benchmark. Artificially dense, fully annotated evidence trails make measurement possible but may not match how often comparable trails arise in ordinary use. The study covers text memory only; it does not test model weights, images, audio, external tools or backup systems. Residual recovery also gives the reader precisely the surviving objects known to matter, so it is a reference for remaining support rather than the probability of an accidental leak in deployment.
The deletion methods were reference baselines built from existing retrieval and writing components, not dedicated erasure systems. Their weakness establishes a baseline, not a ceiling. The source-level metric also does not fully measure unrelated material lost through deletion or the effect on later assistant usefulness. Model names, memory implementations and retrieval settings may change, so each deployed system needs its own versioned test.
Confidence would rise with independently reproduced results, naturally occurring opt-in conversation histories, multimodal and parametric-memory tests, and deletion methods that can prove complete cuts without excessive collateral loss. The assessment would weaken if dedicated provenance-aware systems consistently drove residual recovery to chance on unseen evidence structures. For now, the credible conclusion is narrow but consequential: deleting the explicit record and observing silence is insufficient evidence that an agent's long-term memory has forgotten the fact.[1]
What this means for people
- People using assistants for health, work or family matters need a precise explanation of what a 'forget' command removes.
- Privacy and compliance teams need post-deletion reconstruction tests, not only confirmation that one message disappeared.
- Developers need source-to-derived-memory provenance so deletion can follow summaries and linked records without erasing unrelated history.
Global context
The researchers are based in China, but the engineering problem is not jurisdiction-specific. Privacy rights and retention rules differ, while agent memories increasingly combine conversations across time and services. The benchmark's most transferable idea is to verify erasure against every stored representation and indirect evidence trail, then interpret that technical evidence under the relevant local law.
What the evidence does not yet show
- This is an unreviewed preprint using fictional, author-constructed conversations rather than a production deployment.
- GPT-6 rendered the human specifications into dialogue, although the authors report checking every turn.
- The study tests external text memory only; model weights, multimodal records, logs, caches and backups are outside scope.
- Residual recovery is a controlled reference in which the reader receives surviving annotated evidence, not an estimate of accidental disclosure frequency.
- The tested cleanup procedures are reference baselines and do not represent every dedicated erasure technique.
What to watch next
- Independent replication on unseen conversations and different memory architectures.
- Provenance-aware deletion methods that find indirect entity and temporal links while measuring collateral loss.
- Evaluation on opt-in real-world traces and multimodal or parametric memory.
- Clear product evidence distinguishing a received deletion request, a completed record deletion and verified inability to reconstruct.
Living evidence record
Impact record IAI-1Y3V1H4
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
16 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 16 October 2026
Research question, topics and regions
Whether removing a user's explicit disclosure from an AI agent's long-term memory also prevents reconstruction from surviving indirect evidence
Topics: Agent memory · Data deletion · Privacy · Right to erasure · Benchmarking
Regions: China · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Watch the background, then check the evidence
The Royal Society
YouTube loads only when you choose. Press play in the player to start. If playback is unavailable, use “Watch on YouTube” below.
AI basics · Background explainer
Published 2018-11-28 · 2 min 31 sec
What is artificial intelligence? | The Royal Society
A short introduction to artificial intelligence from the Royal Society. Start here for basic concepts, then use current reporting to explore what has changed.
Evidence check: Published in 2018, before today’s generative AI tools. Useful for basic concepts, rather than an overview of current capabilities.
Source checked 14 October 2026. Check available captions on the publisher’s video page.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
AI Risks & Safety
Can connected AI agents turn a benign request into harm?
A Meta-led preprint found harmful outcomes in 192 of 1,000 simulated agent scenarios generated by its new red-teaming method. The result exposes cross-service risk, but it comes from one synthetic environment and model-based judges—not live consumer accounts.
7 min · 1 source
AI Risks & Safety
Can AI judges be accurate yet disagree on the errors that matter?
Yes. In a new unreviewed study of ten reasoning models, average accuracy stayed high on a 600-item answer-checking benchmark, while prompt and model choices sharply changed whether errors were too lenient or too strict. The result argues against trusting one headline accuracy score—or one AI judge.
9 min · 2 sources
AI Risks & Safety
Can AI agents learn what is appropriate?
Perhaps, but the new Google-led report is a research agenda, not a tested safeguard. More than 50 contributors propose contextual policy engines, dynamic sandboxes and multi-agent benchmarks without demonstrating that they prevent real privacy or security failures.
8 min · 2 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.