NEW

SurrealDB 4 is here, introducing SurrealDB Earth and SurrealDS!

Learn more

Agent memory in one SurrealQL query: vectors, graph, text

Engineering
AI

Nov 9, 20267 min read

Tobie Morgan Hitchcock

Tobie Morgan Hitchcock

Show all postsAgent memory in one SurrealQL query: vectors, graph, text

Our newsletter

Get tutorials, AI agent recipes, webinars, and early product updates in your inbox every two weeks

A single SurrealQL query can search records by meaning, by their connections and by the exact words they contain, inside one transaction. One SELECT can combine HNSW vector search, full-text matches, filters on document fields and graph traversal. Any application that searches this way can use the pattern, and this post builds an AI agent's memory with it, because an agent needs all three kinds of recall at once: what a memory means, what it connects to and what was said.

An agent's memory answers three questions about a topic: what is close to it in meaning, what is connected to it, and what was said about it in specific words. A fourth question, about when, who and what kind, narrows the answer. SurrealDB answers each one with a feature of the same database:

  • What does it mean? An HNSW vector index and the KNN operator, used below as embedding <|3,COSINE|> $q.

  • What was said? A full-text index with an analyzer, used below as text @@ 'migration'.

  • What does it connect to? RELATE edges and graph traversal, used below as ->mentions->entity.

  • When, who and what kind? Ordinary document fields: kind, speaker and said_at.

Many stacks keep these in four separate systems and join the results in application code. In SurrealDB they are four kinds of index on the same records.

Define one table of memories, one table of entities and two relation tables.

Each memory stores an embedding: an array of numbers that represents the meaning of its text, produced by an embedding model. Texts with similar meanings get arrays that are close together. Each number captures one aspect of the meaning, so real models use long arrays to tell fine differences apart: OpenAI's text-embedding-3-small produces 1,536 numbers per text and text-embedding-3-large produces 3,072, while Voyage AI's voyage-3.5, the provider Anthropic recommends for embeddings, produces 1,024 by default. Arrays that long are impossible to read in a blog post, so the examples here use embeddings of only three numbers, written by hand. With a real model, only the DIMENSION value of the index and the length of the array change.

-- Memories: what was said, by whom, when, and what it means
DEFINE TABLE memory SCHEMAFULL;
DEFINE FIELD text ON memory TYPE string;
DEFINE FIELD kind ON memory TYPE 'turn' | 'document';
DEFINE FIELD speaker ON memory TYPE string;
DEFINE FIELD said_at ON memory TYPE datetime;
DEFINE FIELD embedding ON memory TYPE array<float, 3>;

-- Vector index: finds memories by meaning
DEFINE INDEX memory_embedding ON memory
	FIELDS embedding HNSW DIMENSION 3 DIST COSINE;

-- Full-text index: finds memories by the words they contain.
-- The analyzer splits text into words, lowercases them and reduces
-- each word to its stem, so "migrations" matches "migration".
DEFINE ANALYZER memo_text
	TOKENIZERS blank, punct
	FILTERS lowercase, ascii, snowball(english);
DEFINE INDEX memory_text ON memory
	FIELDS text FULLTEXT ANALYZER memo_text BM25 HIGHLIGHTS;

-- Entities: the people, systems and projects that memories mention
DEFINE TABLE entity SCHEMAFULL;
DEFINE FIELD name ON entity TYPE string;
DEFINE FIELD kind ON entity TYPE string;

-- Graph edges: from a memory to an entity it mentions...
DEFINE TABLE mentions TYPE RELATION IN memory OUT entity;
-- ...where each memory can mention each entity only once
DEFINE INDEX unique_mention ON mentions FIELDS in, out UNIQUE;

-- ...and from one entity to another
DEFINE TABLE relates_to TYPE RELATION IN entity OUT entity;
DEFINE FIELD label ON relates_to TYPE string;

The full-text index ranks matches with BM25, a standard relevance formula used by most search engines. BM stands for "best matching", and 25 is the number of the formula in the series that introduced it. The DEFINE INDEX reference covers the index options, and the DEFINE ANALYZER reference covers tokenizers and filters.

The example data is the memory of an agent that helps a small engineering team. The team is running a project, the ledger migration, which moves the data of its billing service off a database that is reaching the end of its support. In the queries below:

  • entity:ada is a person, Ada, who owns the ledger migration.

  • entity:ledger is the ledger migration project.

  • entity:billing is the billing service, which the migration changes.

  • entity:postgres is the Postgres 14 database the migration depends on.

Create the four entities and connect them with RELATE:

-- The people, systems and projects
CREATE entity:ada CONTENT { name: 'Ada', kind: 'person' };
CREATE entity:billing CONTENT { name: 'Billing service', kind: 'system' };
CREATE entity:ledger CONTENT { name: 'Ledger migration', kind: 'project' };
CREATE entity:postgres CONTENT { name: 'Postgres 14', kind: 'system' };

-- How they relate to each other
RELATE entity:ada->relates_to->entity:ledger SET label = 'owns';
RELATE entity:ledger->relates_to->entity:billing SET label = 'changes';
RELATE entity:ledger->relates_to->entity:postgres SET label = 'depends on';

Then store four memories and link each one to the entities it mentions. Three are turns from a conversation with the agent, and one is a passage from a document:

CREATE memory:m1 CONTENT {
	text: 'Ada said the ledger migration must finish before the invoice run on Friday',
	kind: 'turn',
	speaker: 'user',
	said_at: d'2026-11-02T09:00:00Z',
	embedding: [0.9, 0.1, 0.0],
};

CREATE memory:m2 CONTENT {
	text: 'The billing service retries failed payments three times with backoff',
	kind: 'document',
	speaker: 'docs',
	said_at: d'2026-10-20T09:00:00Z',
	embedding: [0.8, 0.3, 0.1],
};

CREATE memory:m3 CONTENT {
	text: 'Lunch order for the offsite is vegetarian pizza',
	kind: 'turn',
	speaker: 'user',
	said_at: d'2026-11-01T12:00:00Z',
	embedding: [0.0, 0.2, 0.9],
};

CREATE memory:m4 CONTENT {
	text: 'Postgres 14 reaches end of life so the ledger migration moves data first',
	kind: 'turn',
	speaker: 'user',
	said_at: d'2026-11-03T10:00:00Z',
	embedding: [0.7, 0.2, 0.2],
};

-- Which entities each memory mentions
RELATE memory:m1->mentions->entity:ada;
RELATE memory:m1->mentions->entity:ledger;
RELATE memory:m2->mentions->entity:billing;
RELATE memory:m4->mentions->entity:postgres;
RELATE memory:m4->mentions->entity:ledger;

The KNN operator <|3,COSINE|> returns the three memories nearest to a query vector, and vector::distance::knn() gives the distance of each. The full-text operator @0@ matches the analyzed text, and search::score(0) and search::highlight() give the BM25 score and the matched words.

-- The embedding of the agent's question
LET $q = [0.85, 0.15, 0.05];

-- By meaning: the three nearest memories
SELECT id, text, vector::distance::knn() AS distance
FROM memory
WHERE embedding <|3,COSINE|> $q
ORDER BY distance;

-- By words: memories that contain "migration"
SELECT id, text,
	search::score(0) AS score,
	search::highlight('<b>', '</b>', 0) AS hit
FROM memory
WHERE text @0@ 'migration'
ORDER BY score DESC;

The vector search ranks the ledger memory, the billing document and the Postgres memory, and leaves out the lunch order. The full-text search finds the two memories that contain the word.

[
  {
    "id": "memory:m1",
    "distance": 0.0037,
    "text": "Ada said the ledger migration must finish before the invoice run on Friday"
  },
  {
    "id": "memory:m2",
    "distance": 0.0185,
    "text": "The billing service retries failed payments three times with backoff"
  },
  {
    "id": "memory:m4",
    "distance": 0.0272,
    "text": "Postgres 14 reaches end of life so the ledger migration moves data first"
  }
]
[
  {
    "id": "memory:m1",
    "score": 0.645,
    "hit": "Ada said the ledger <b>migration</b> must finish before the invoice run on Friday"
  },
  {
    "id": "memory:m4",
    "score": 0.645,
    "hit": "Postgres 14 reaches end of life so the ledger <b>migration</b> moves data first"
  }
]

Put every condition in the same WHERE clause and traverse the graph in the projection. The query below finds memories that are near the question vector, contain the word "migration", are turns of the conversation and were said after 1 November 2026. For each one it then follows the mentions and relates_to edges. It runs between BEGIN and COMMIT, so every read comes from one consistent view of the data.

BEGIN;
LET $q = [0.85, 0.15, 0.05];
SELECT
	id,
	text,
	speaker,
	said_at,
	vector::distance::knn() AS distance,
	-- The entities this memory mentions
	->mentions->entity.name AS mentions,
	-- The entities those entities relate to
	->mentions->entity->relates_to->entity.name AS connected_to,
	-- Who owns what this memory mentions
	->mentions->entity<-(relates_to WHERE label = 'owns')<-entity.name AS owners
FROM memory
WHERE embedding <|3,COSINE|> $q      -- by meaning
	AND text @@ 'migration'          -- by words
	AND kind = 'turn'                -- by kind
	AND said_at > d'2026-11-01T00:00:00Z'  -- by time
ORDER BY distance;
COMMIT;

The result is the agent's recall, one object per memory:

[
  {
    "id": "memory:m1",
    "text": "Ada said the ledger migration must finish before the invoice run on Friday",
    "speaker": "user",
    "said_at": "2026-11-02T09:00:00Z",
    "distance": 0.0037,
    "mentions": ["Ledger migration", "Ada"],
    "connected_to": ["Postgres 14", "Billing service", "Ledger migration"],
    "owners": ["Ada"]
  },
  {
    "id": "memory:m4",
    "text": "Postgres 14 reaches end of life so the ledger migration moves data first",
    "speaker": "user",
    "said_at": "2026-11-03T10:00:00Z",
    "distance": 0.0272,
    "mentions": ["Postgres 14", "Ledger migration"],
    "connected_to": ["Postgres 14", "Billing service"],
    "owners": ["Ada"]
  }
]

Each result answers all three questions:

  • What it means: the memory is close to the question vector, and the distance shows how close.

  • What was said: the text, speaker and said_at fields, filtered by the full-text match.

  • What it connects to: the entities it mentions, the systems they touch and the person who owns the project.

The billing document is not in the result, because it is not a turn and does not contain the word. The lunch order was never close enough in meaning to appear.

Yes. A transaction sees its own writes in the vector index, the full-text index and the graph. The query below stores a new memory, links it to two entities and recalls it before committing:

BEGIN;
CREATE memory:m5 CONTENT {
	text: 'Ada moved the ledger migration cutover to Monday',
	kind: 'turn',
	speaker: 'user',
	said_at: d'2026-11-04T08:00:00Z',
	embedding: [0.88, 0.12, 0.02],
};
RELATE memory:m5->mentions->entity:ada;
RELATE memory:m5->mentions->entity:ledger;

-- The new memory is already visible to all three kinds of search
SELECT id, text,
	vector::distance::knn() AS distance,
	->mentions->entity.name AS mentions
FROM memory
WHERE embedding <|2,COSINE|> [0.85, 0.15, 0.05]
	AND text @@ 'cutover'
ORDER BY distance;
COMMIT;

The SELECT returns the new memory, at a distance of 0.0014, with mentions of ["Ledger migration", "Ada"].

The transaction also keeps a memory and its links together when a write fails. An agent usually finds the entities in a text with a language model, and two phrases in one text can resolve to the same entity: below, "the ledger cutover" and "the cutover" both resolve to the ledger migration. The unique_mention index refuses the second edge, and the whole transaction is cancelled with it:

BEGIN;
CREATE memory:m6 CONTENT {
	text: 'The ledger cutover needs a read-only window on the billing service',
	kind: 'turn',
	speaker: 'user',
	said_at: d'2026-11-05T08:00:00Z',
	embedding: [0.8, 0.25, 0.05],
};
RELATE memory:m6->mentions->entity:ledger;
RELATE memory:m6->mentions->entity:billing;
-- "the cutover" also resolved to the ledger migration
RELATE memory:m6->mentions->entity:ledger;
COMMIT;
Error output
Database index `unique_mention` already contains [memory:m6, entity:ledger], with record `mentions:doegt5zm2zhb80w0wcoy`

Afterwards, SELECT * FROM memory:m6 returns an empty array, and so does a search for its edges. The memory, both of its edges and its entries in the vector and full-text indexes were never written, so the agent can deduplicate the entities and try again without first cleaning up half a memory.

Because SurrealDB has vector, full-text, document and graph indexes on the same records, it can be the memory of an agent, as the queries above show. A memory built by hand quickly needs more design decisions than these queries make, though. It has to decide how text becomes entities and edges, what happens when two names refer to the same thing, how a new fact replaces an old one without losing the history, how to answer what was true at a past date, and how to keep each user's or agent's memories apart. And not every team that needs an agent's memory also wants to run a database.

SurrealDB Agent Memory makes those decisions and runs the database for you. It is built on SurrealDB and adds:

  • Ingestion of conversations, text and documents, with extraction of entities and relationships.

  • Reconciliation and supersession, which match new facts to the entities already stored and record a correction as a new value that replaces the old one, with the full history kept.

  • A tri-temporal model, which records when each fact was true, when it was known and when it was stored, so an agent can ask what was true on any past date.

  • Hybrid recall, which fuses vector similarity, BM25 and graph traversal into one ranking.

  • Scoping, which keeps each user's, agent's or team's memories apart.

The Agent Memory documentation covers each of these, and the introduction to SurrealDB Agent Memory covers the product.

The queries in this post show the primitives that Agent Memory is built on. Write them yourself when you want to design the schema, the extraction and the ranking for your own application. Use Agent Memory when you want an agent to have a memory without building and running one.

Related posts

Our newsletter

Get tutorials, AI agent recipes, webinars, and early product updates in your inbox every two weeks