Agent Memory Leaderboard
INTEGRATION GUIDE

Add / Search API Guide

Cycle 2 uses participant-hosted Add and Search APIs for the Textual, Coding, and Multimodal tracks. Implement the formats below, apply for and bind an evaluation key, pass Smoke, and then submit a Full evaluation.

Add Request

Select a track format below. Multimodal places ordered text and image parts directly in messages[].content.

The envelope stays the same; only the content field changes.

{
  "request_id": "eval:<run_id>:locomo_refined:conv-0:chunk-0",
  "messages": [{
    "role": "user",
    "timestamp": 1704067200000,
    "content": "raw memory text"
  }],
  "user_id": "eval:<run_id>:locomo:conv-0",
  "session_id": "eval:<run_id>:sample:0"
}
FieldRequirementDescription request_idRequiredIdentifier for this logical write. Retries keep the same value; echo it in the response. messagesRequiredMessages in source order. Store and process them in this order. messages[].roleRequiredThe message role is user or assistant. messages[].contentRequiredA string for Textual and Coding, or an ordered ContentPart[] array for Multimodal. messages[].timestampOptionalSent when the source has a timestamp, in Unix milliseconds. Chunking does not change its value or message order. user_idRequiredMemory isolation scope; later Search requests use the same value. session_idRequiredIdentifier for the source conversation or session.

Add Response

Textual, Coding, and Multimodal use the same response schema.

{
  "success": true,
  "request_id": "eval:<run_id>:locomo_refined:conv-0:chunk-0",
  "user_id": "eval:<run_id>:locomo:conv-0",
  "session_id": "eval:<run_id>:sample:0"
}
FieldRequirementDescription successRequiredMust be true after the messages are durably stored and immediately searchable. request_idRequiredExact request_id received in the Add request. user_idRequiredExact user_id received in the Add request. session_idRequiredExact session_id received in the Add request.

Search Request

Select a track format below. A Multimodal query uses the same ordered ContentPart[] structure.

Options remain a top-level array in every track.

{
  "query": "Which answer best matches the memory?",
  "options": ["A. First answer", "B. Second answer"],
  "user_id": "eval:<run_id>:locomo:conv-0",
  "top_k": 100
}
FieldRequirementDescription queryRequiredThe original benchmark question: a string for Textual and Coding, or ordered ContentPart[] for Multimodal. optionsConditionalSent at the top level for multiple-choice questions, including Streaming; absent for open questions. Never contains the gold answer. user_idRequiredThe same memory isolation scope supplied in the corresponding Add request. top_kRequiredMaximum result count; formal external evaluations use 100.

Search Response

Select the response format for your track. Multimodal results may return ordered text and image parts in data[].content.

Return memories in retrieval rank order.

{
  "data": [{
    "id": "mem_123",
    "content": "remembered fact text",
    "score": 0.87,
    "created_at": "2026-07-01T12:00:00Z"
  }]
}
FieldRequirementDescription dataRequiredArray in retrieval rank order. Return [] when there are no results; do not omit data. data[].idRequiredStable identifier for each returned memory. data[].contentRequiredNon-empty string for Textual and Coding, or ordered ContentPart[] for Multimodal; preserved for audit and passed to Answer in returned order. data[].scoreOptionalNumeric relevance score. Higher values must indicate greater relevance. data[].created_atOptionalSource or persistence timestamp for the memory.

Evaluation Data

Your API receives benchmark content during each run.

Data sent

Your API receives evaluation memories and questions.

Evaluation only

Use this data only for the evaluation. Do not train on it, analyze it, or share it.

Delete in 30 days

Keep it private, avoid storing logs, and delete it within 30 days after the run.

What Each Page is for

A compact map of the product surface so users know where to go after their endpoints are ready.

01

Home

Overview and benchmark preview.

02

Leaderboard

Public rankings by track.

03

Evaluation

Verify key, start eval jobs, inspect private results and attribution.

04

Documentation

User workflow, API contract, security, and result publication.

05

Provide Benchmark

Propose a documented benchmark with its task, provenance, package, and license.