Textual Memory
Long-context, persona, script, and conversation-memory benchmarks.
THE JOURNAL
15 postsLatest stories
48 Hours Left: CSIG AML Cycle 2 Project Submission Deadline Approaches
Key dates for Full evaluation, resuming existing tasks, participant and team registration, result review, and the official Cycle 2 leaderboard release.
AML Is in Full Swing: Participant FAQ, Part 2
Practical answers on application follow-ups, version registration, cross-track integration, endpoint issues, and model eligibility for Cycle 2.
AML Phase II Is Open: Participant FAQ
AML Cycle 2 public evaluation opened on September 20, 2026, at 00:00. This FAQ covers registration, models, Add/Search integration, recovery, and results.
Two Days to Go | CSIG-Hosted Agent Memory Challenge 2026 Cycle 2 Participation Guide
Two Days to Go | Agent Memory Challenge 2026 Cycle 2 Participation Guide
AML Official Analysis II | Coding Memory Leaderboards: How Do Industry and Open-Source Methods Perform?
Compared with textual memory, Coding Memory asks a more direct question: can experience accumulated during previous development actually help an Agent complete subsequent software engineering tasks?
AML Leading-Method Spotlight | FlowGrid AML Retriever: How a Markdown Decision Log Became an Agent Memory Retriever
There is no single answer to Agent Memory.
AML Leading-Method Spotlight | ChronoHybridMem: From Conversations to Verifiable Memory Evidence
There is no single answer to Agent Memory.
AML Leading Methods Explained | ActiveMemoryIndex: Why Does a System Without Memory Governance Rank First in Governance?
ActiveMemoryIndex explains its raw-record-first design, transparent ablations, the limits of item-count budgets, and the next steps for neighboring context and temporal reasoning.
Hosted by CSIG | RMB 150,000 Prize Pool: The Second Agent Memory Challenge Is Coming
The second Agent Memory Challenge will open on September 20, 2026, with Textual, Coding, and Multimodal tracks, unified evaluation, and an RMB 150,000 prize pool.
AML Leading Methods Explained | ReFind: Active Search Lets Agents Browse Original Chat Logs
ReFind preserves raw chat logs and moves memory computation to query-time active search. Its AML results and the InMind benchmark reveal both the strengths and limits of this approach.
AML Leading Methods Explained | From Similarity to Completeness: How InvMem Retrieves Truly Useful Long-Term Memories
InvMem combines fine-grained chunking, Dense and BM25 retrieval, Weighted RRF, and same-session neighboring expansion to turn relevant fragments into complete memory evidence.
AML Official Analysis, Part I: What Does the Textual Open-Source Leaderboard Reveal?
The top three are separated by just 0.22 points, yet excel in different ways. A closer look at seven capability areas reveals distinctive strengths, weaknesses, and open questions beyond the overall ranking.
The First AML Leaderboard Is Officially Released!
The first AML leaderboard publishes textual-track results from 67 memory frameworks, evaluated and reviewed through a unified pipeline. Explore rankings, capability profiles, and opportunities to contribute.
An Agent Only Starts to Understand You When It Has a Yesterday
Agents need more than the ability to think: they need the ability to remember. AML establishes a shared standard for comparing memory systems and invites the community to the first Agent Memory Challenge.
The Agent Memory Challenge Launches: The First Open Tournament for Memory Systems
AML brings multi-source data, a unified evaluation pipeline, capability profiles, and low-barrier integration together in an open, reproducible evaluation framework. The first Agent Memory Challenge is now open.
No posts in this category yet
Two Days to Go | CSIG-Hosted Agent Memory Challenge 2026 Cycle 2 Participation Guide
Agent Memory Leaderboard
After an Agent has spent months with a user and handled hundreds of tasks, what has it actually remembered?
Can it retrieve key facts when needed, recognize changes in information, update memories in time, filter outdated or conflicting content, and genuinely apply accumulated experience to the next task?
These are the questions that the Agent Memory Leaderboard (AML) continues to explore.
Hosted by the China Society of Image and Graphics (CSIG) and organized by Nanjing University, Zhejiang University, Datawhale, and the CSIG Enterprise Liaison and Standardization Working Committee, the second cycle of the open Agent memory challenge, Agent Memory Challenge 2026 II, will officially open at 00:00 on September 20, 2026.
Less than 2 days remain until evaluation opens.
Cycle 2 includes Textual Memory, Coding Memory, and Multimodal Memory tracks, open to both open-source methods and commercial products. Whether you are researching long-term memory methods, developing memory infrastructure, or already have a working commercial memory product, you can join the evaluation through the unified Add/Search interfaces.
Building on Cycle 1,
Continuing to Compare Memory Systems Under the Same Rules
The first AML leaderboard was published in August 2026. Cycle 1 attracted registrations from 136 teams, with 67 representative memory frameworks completing the first round of evaluation. The official website has received more than 300,000 visits to date.
The Agent Memory Challenge Has Officially Launched: The First Open Tournament for Memory Systems
The First AML Leaderboard Has Officially Been Released!
The first Hugging Face leaderboard also attracted widespread community attention and reached the top of the Spaces trending list.
The Three Tracks of Cycle 2
Cycle 2 of the challenge includes Textual, Coding, and Multimodal tracks.
Textual Memory
The Textual track evaluates facts, relationships, multi-hop composition, temporal events, personalization, rule execution, and memory governance across long conversations and cross-session histories.
Cycle 2 also incorporates Streaming continuous-memory capabilities: Add and Search may be interleaved as events unfold during evaluation. A system must not only retain information, but also use information available at the appropriate time and recognize updates, conflicts, and invalidation.
Coding Memory
The Coding track evaluates code location, module relationships, defect root causes, modification strategies, test verification, and lessons from failed attempts in historical engineering experience, as well as experience transfer, version differentiation, and noise filtering.
Full includes 150 real software engineering tasks, each run under relevant and noisy memory conditions, for a total of 300 evaluation units. Results are determined by actual code modifications and test verification, not merely retrieval similarity.
Multimodal Memory
Cycle 2 introduces a Multimodal Memory track, evaluating the writing, retrieval, and use of image-and-text memory evidence. Participating systems may process original images or use the caption in the request, but must correctly parse the image-and-text content array and disclose whether they use original images, OCR, or captions, and whether they return image evidence. During evaluation, systems must not only remember event information, but also capture key information implicit in images.
The Two Participant Divisions of Cycle 2
The challenge is open globally to universities, research institutes, companies, open-source communities, independent research teams, and individuals. Individuals, teams, and cross-organization teams are all welcome.
Participants may select one or more tracks, but may choose only one participant division:
The Open-source Methods leaderboard is suitable for research-paper methods, research projects, and open-source systems. Participants need to provide a public repository, a fixed Commit, dependencies, and running instructions, and accurately disclose the original work, authors, and modifications made for this submission.
The Commercial Products leaderboard is suitable for commercial APIs, closed-source products, and systems that do not disclose their internal implementation. Participants do not need to publish internal code, but must fix their product or API version and declare models, interfaces, capacity, and key operating conditions.
Participants in both divisions must deploy stable, accessible Add/Search APIs themselves. Registration for this challenge is free. The platform covers the unified Answer, Evaluation, and evaluation-orchestration costs; participants cover their own services, databases, model calls, storage, bandwidth, and computing resources.
Participation Workflow
The participation workflow can be summarized as follows:
Choose your tracks and division, deploy your Add/Search APIs, and submit an Evaluation Access Request with complete materials online;
After approval, bind your system version, interfaces, and target tracks;
Run Smoke separately for each track;
After Smoke passes, start Full;
Keep your service, version, and evaluation state stable during Full. If a task is interrupted, resume it according to the platform's guidance;
Review your private results and cooperate with the review process. Approved results are then placed on the corresponding public leaderboard.
Add/Search Interface Specification
A participating system needs to provide two HTTP POST interfaces:
Add: receives historical content written by the platform;
Search: retrieves relevant memories within the specified user_id scope.
Before formal integration, pay particular attention to these five points:
First, strict isolation. user_id defines the memory-isolation boundary; data from different user_id values must not be mixed in retrieval. Different session_id values under the same user may still require cross-session retrieval.
Second, read-after-write visibility. A synchronous Add may return HTTP 200 only after the content has been persisted and is immediately retrievable by subsequent Search requests. Merely putting the request into a background queue does not mean the write has completed.
Third, ensure idempotency. The platform reuses the same logical request when retrying. Participating services should use request_id for idempotency to avoid duplicate writes contaminating the results.
Fourth, return the standard structure. The top-level field of a successful Search response is a data array; return {"data": []} when there are no relevant memories. The returned order is the evidence-priority order, and the number of items must not exceed the top_k for that request.
Fifth, Search returns memory evidence. The platform uniformly handles the final answer or coding task. Participating systems must not disguise answers to the current question, option letters, or external gold labels as memories.
For complete fields, multimodal content arrays, error handling, and examples, refer to the API Guide and Documentation.
Key Dates
September 20, 00:00: Cycle 2 officially opens and begins accepting registrations, access requests, and evaluation tasks.
September 20–October 31: Complete material review, interface integration, Smoke, Full, and any necessary rechecks.
October 31, 23:59: Deadline for registration and submission of complete materials.
November 4, 23:59: Evaluation for this cycle stops.
Mid-November: Results are reviewed and the official leaderboard is published.
Full evaluation typically takes approximately 0.5–2 days. Actual duration is also affected by interface responses, error retries, and platform scheduling. Submitting a task does not mean completing evaluation; allow sufficient time for queuing, running, and handling exceptions.
For this challenge, each AML Key may run at most 30 Smoke evaluations and 2 Full evaluations per track. Smoke evaluations must be at least 1 hour apart, and the second Full evaluation may be initiated only 30 days after the first Full has successfully completed. Teams planning two Full evaluations should complete their first round early.
Prizes
The combined prize pool across the three tracks is RMB 150,000, reserved for Open-source Methods leaderboard teams that comply with the rules and pass review.
Each track offers:
First Prize: 1 winner, RMB 20,000;
Second Prize: 2 winners, RMB 8,000 each;
Third Prize: 3 winners, RMB 3,000 each;
Best Technical Innovation: 1 winner, RMB 5,000.
Commercial Products leaderboard entries are not eligible for this cycle's prizes, but approved official results will be displayed on a separate leaderboard.
Long-term memory is gradually moving from an additional Agent capability to infrastructure for continuous work, personalized interaction, and experience reuse. Evaluation is valuable not only for producing an overall score, but also for helping every method understand its strengths, boundaries, and next steps.
If you already have a system that can write and retrieve memories, now is the time to put it through a unified evaluation workflow!
Registration and Evaluation:
https://agentmemories.ai/evaluation
Challenge Overview:
https://agentmemories.ai/competition
Participation Rules:
https://agentmemories.ai/rules
Interface Documentation:
https://agentmemories.ai/api-guide
Challenge Inquiries: