# Data model and schema Tables, relations, and indexes in SurrealDB. SurrealDB Agent Memory stores all state in SurrealDB. This page describes the key tables, their fields, and the indexes that power retrieval. Schema migrations are bundled in the SurrealDB Agent Memory binary and applied automatically when a Context is created or upgraded. ## Schema namespaces | Namespace | Database | Contents | |---|---|---| | `spectron` | `metadata` | Control plane: Context registry, API keys | | `spectron` | `_jobqueue` | Async ingestion job queue | | `` | `` | All authoritative knowledge and experiential memory data for one Context | Each Context is bound to its own `(namespace, database)` pair. Tables are never shared across Contexts. ## Scope on stored records Scoped tables carry **`scope_sets`** - content-addressed OR-of-AND clauses referencing hierarchical **`scopes`** paths. Entity and keyword index records also store derived **`visible_scope_sets`** so name/type lookups cannot leak facts the caller cannot read whole. See [Contexts and scope](/docs/agent-memory/mental-model/contexts-and-scope.md) for the visibility rules. ## Control plane tables ### `context` The Context registry. Records each Context's binding and embedded configuration. | Field | Type | Notes | |---|---|---| | `id` | string | Context identifier (e.g. `acme-prod`) | | `namespace` | string | Bound SurrealDB namespace | | `database` | string | Bound SurrealDB database | | `config` | object | Embedded config (models, providers, limits) | | `created_at` | datetime | | ### `api_key` Management keys (not per-Context). Used for creating and managing Contexts. ### `context_api_key` Per-Context end-user keys, stored as Argon2 hashes. Each key binds to a principal and optionally attenuates that principal's grants. | Field | Type | Notes | |---|---|---| | `context` | `record` | The Context this key belongs to | | `name` | string | Human-readable label, unique per Context | | `hash` | string | Argon2 hash of the secret | | `principal_id` | option\ | The principal this key acts as. Stored as a plain string because the `principal` row lives in the Context's own database, not the control plane | | `grants` | option\ | Attenuating per-verb pattern map (`grants.*` is `array`). Absent ⇒ inherit the principal's grants | | `strict` | bool | Opt into hard out-of-region errors instead of empty results. Defaults to `false` | | `valid_until` | option\ | Mint-time expiry. Absent ⇒ no expiry | | `created_at` | datetime | | | `last_used_at` | option\ | Vestigial - no longer written. Last use is derived as `max(used_at)` over `context_api_key_usage`; the column stays defined so older rows deserialise | ### `context_api_key_usage` Append-only key-use log: one row per successful data-plane key validation, throttled to at most one row per key per minute. This is what backs the derived `lastUsedAt` on the self-service key listing - it records recent activity, not a per-request audit trail. | Field | Type | Notes | |---|---|---| | `key` | `record` | The key that validated | | `context` | `record` | | | `used_at` | datetime | | ## Knowledge tables (Authoritative pillar, per Context) ### `document` Source files and their processing state. ```surql DEFINE TABLE document SCHEMAFULL; DEFINE FIELD title ON document TYPE string; DEFINE FIELD mime_type ON document TYPE string; DEFINE FIELD source ON document TYPE string; DEFINE FIELD storage_key ON document TYPE string; DEFINE FIELD content_hash ON document TYPE string; DEFINE FIELD size_bytes ON document TYPE int; DEFINE FIELD observed_at ON document TYPE option; DEFINE FIELD scope ON document TYPE array>; DEFINE FIELD version ON document TYPE int DEFAULT 1; DEFINE FIELD status ON document TYPE string DEFAULT "queued" ASSERT $value IN ["queued", "extracting", "chunking", "embedding", "keywording", "ready", "failed"]; DEFINE FIELD error ON document TYPE option; DEFINE FIELD processing_started_at ON document TYPE option; DEFINE FIELD processing_completed_at ON document TYPE option; DEFINE FIELD created_at ON document TYPE datetime DEFAULT time::now() READONLY; DEFINE FIELD updated_at ON document TYPE datetime DEFAULT time::now(); DEFINE INDEX content_hash_index ON document FIELDS content_hash; DEFINE INDEX scope_index ON document FIELDS scope; DEFINE INDEX status_index ON document FIELDS status; ``` ### `knowledge_chunk` Text segments extracted from documents, with vector embeddings. ```surql DEFINE TABLE knowledge_chunk SCHEMAFULL; DEFINE FIELD document ON knowledge_chunk TYPE record; DEFINE FIELD text ON knowledge_chunk TYPE string; DEFINE FIELD embedding ON knowledge_chunk TYPE option>; DEFINE FIELD position ON knowledge_chunk TYPE int; DEFINE FIELD section ON knowledge_chunk TYPE option; DEFINE FIELD char_start ON knowledge_chunk TYPE int; DEFINE FIELD char_end ON knowledge_chunk TYPE int; DEFINE FIELD token_count ON knowledge_chunk TYPE option; DEFINE FIELD simhash ON knowledge_chunk TYPE option; DEFINE FIELD duplicate_of ON knowledge_chunk TYPE option>; DEFINE FIELD scope ON knowledge_chunk TYPE set>; DEFINE FIELD created_at ON knowledge_chunk TYPE datetime DEFAULT time::now() READONLY; DEFINE INDEX embedding_index ON knowledge_chunk FIELDS embedding HNSW DIMENSION 3072 DIST COSINE TYPE F32; DEFINE INDEX document_index ON knowledge_chunk FIELDS document; DEFINE INDEX scope_index ON knowledge_chunk FIELDS scope; ``` Embedding width is fixed deployment-wide at **3072** (`gemini-embedding-2`). Optional **`simhash`** / **`duplicate_of`** support near-duplicate suppression on recall (see **`includeDuplicates`** on `/query`). ### `keyword` RAKE-extracted keyphrases from documents, stored as nodes in the keyword graph. ```surql DEFINE TABLE keyword SCHEMAFULL; DEFINE FIELD text ON keyword TYPE string; DEFINE FIELD normalised ON keyword TYPE string; DEFINE FIELD embedding ON keyword TYPE option>; DEFINE FIELD scope ON keyword TYPE set> DEFAULT []; DEFINE FIELD document_count ON keyword TYPE int DEFAULT 0; DEFINE FIELD created_at ON keyword TYPE datetime DEFAULT time::now() READONLY; DEFINE FIELD updated_at ON keyword TYPE datetime DEFAULT time::now(); DEFINE INDEX embedding_index ON keyword FIELDS embedding HNSW DIMENSION 3072 DIST COSINE TYPE F32; DEFINE INDEX normalised_index ON keyword FIELDS normalised UNIQUE; ``` ### `knowledge_has_keyword` Relation edge from a document to its keywords. ```surql DEFINE TABLE knowledge_has_keyword TYPE RELATION IN document OUT keyword SCHEMAFULL; DEFINE FIELD score ON knowledge_has_keyword TYPE float; DEFINE FIELD scope ON knowledge_has_keyword TYPE set> DEFAULT []; DEFINE FIELD created_at ON knowledge_has_keyword TYPE datetime DEFAULT time::now() READONLY; DEFINE INDEX unique_pair ON knowledge_has_keyword FIELDS in, out UNIQUE; ``` Structured authoritative facts extracted from documents (or ingested via `POST /facts` with `infer: "triples"`) are stored as **`entity`**, **`attribute`**, and **`relates_to`** records with `source.kind = "document"`. ## Experiential pillar tables (per Context) ### `scope_attribute` Canonical key-value scope tags shared across all scoped records. ```surql DEFINE TABLE scope_attribute SCHEMAFULL; DEFINE FIELD key ON scope_attribute TYPE string; DEFINE FIELD value ON scope_attribute TYPE string; DEFINE INDEX key_value ON scope_attribute FIELDS key, value UNIQUE; ``` ### `session` First-class conversation records. ```surql DEFINE TABLE session SCHEMAFULL; DEFINE FIELD scope ON session TYPE set>; DEFINE FIELD metadata ON session TYPE option FLEXIBLE; DEFINE FIELD created_at ON session TYPE datetime DEFAULT time::now() READONLY; DEFINE INDEX scope_index ON session FIELDS scope; ``` ### `turn` Individual messages within a session. ```surql DEFINE TABLE turn SCHEMAFULL; DEFINE FIELD session ON turn TYPE record; DEFINE FIELD role ON turn TYPE string ASSERT $value IN ["user", "assistant", "system", "tool"]; DEFINE FIELD content ON turn TYPE string; DEFINE FIELD seq ON turn TYPE int; DEFINE FIELD created_at ON turn TYPE datetime DEFAULT time::now() READONLY; DEFINE INDEX session_seq ON turn FIELDS session, seq UNIQUE; ``` ### `entity` Named things tracked in experiential memory. ```surql DEFINE TABLE entity SCHEMAFULL; DEFINE FIELD id ON entity TYPE array; DEFINE FIELD name ON entity TYPE string; DEFINE FIELD type ON entity TYPE string; DEFINE FIELD memory_category ON entity TYPE string ASSERT $value IN ["identity", "knowledge", "context"]; DEFINE FIELD scope ON entity TYPE set>; DEFINE FIELD embedding ON entity TYPE option>; DEFINE FIELD resolves_to ON entity TYPE option>; DEFINE FIELD source_turn ON entity TYPE option>; DEFINE FIELD created_at ON entity TYPE datetime DEFAULT time::now() READONLY; DEFINE FIELD updated_at ON entity TYPE datetime DEFAULT time::now(); DEFINE INDEX embedding_index ON entity FIELDS embedding HNSW DIMENSION 3072 DIST COSINE TYPE F32; DEFINE INDEX type_index ON entity FIELDS type; DEFINE INDEX scope_index ON entity FIELDS scope; DEFINE INDEX resolves_to_index ON entity FIELDS resolves_to; ``` ### `attribute` Key-value properties on entities, with supersession chains. ```surql DEFINE TABLE attribute SCHEMAFULL; DEFINE FIELD entity ON attribute TYPE record; DEFINE FIELD key ON attribute TYPE string; DEFINE FIELD value ON attribute TYPE string; DEFINE FIELD memory_category ON attribute TYPE string ASSERT $value IN ["identity", "knowledge", "context"]; DEFINE FIELD scope ON attribute TYPE set>; DEFINE FIELD supersedes ON attribute TYPE option>; DEFINE FIELD superseded_by ON attribute TYPE option>; DEFINE FIELD source_turn ON attribute TYPE option>; DEFINE FIELD valid_from ON attribute TYPE option; DEFINE FIELD valid_until ON attribute TYPE option; DEFINE FIELD created_at ON attribute TYPE datetime DEFAULT time::now() READONLY; DEFINE INDEX entity_key ON attribute FIELDS entity, key; DEFINE INDEX scope_index ON attribute FIELDS scope; DEFINE INDEX temporal_index ON attribute FIELDS valid_from, valid_until; ``` ### `relates_to` Graph edges between entities. ```surql DEFINE TABLE relates_to TYPE RELATION IN entity OUT entity SCHEMAFULL; DEFINE FIELD label ON relates_to TYPE string; DEFINE FIELD memory_category ON relates_to TYPE string ASSERT $value IN ["identity", "knowledge", "context"]; DEFINE FIELD scope ON relates_to TYPE set>; DEFINE FIELD source_turn ON relates_to TYPE option>; DEFINE FIELD valid_from ON relates_to TYPE option; DEFINE FIELD valid_until ON relates_to TYPE option; DEFINE FIELD created_at ON relates_to TYPE datetime DEFAULT time::now() READONLY; DEFINE INDEX label_index ON relates_to FIELDS label; DEFINE INDEX scope_index ON relates_to FIELDS scope; DEFINE INDEX temporal_index ON relates_to FIELDS valid_from, valid_until; ``` ### `instruction` Behavioural directives for the agent. ```surql DEFINE TABLE instruction SCHEMAFULL; DEFINE FIELD label ON instruction TYPE string; DEFINE FIELD description ON instruction TYPE string; DEFINE FIELD active ON instruction TYPE bool DEFAULT true; DEFINE FIELD scope ON instruction TYPE set>; DEFINE FIELD source_turn ON instruction TYPE option>; DEFINE FIELD created_at ON instruction TYPE datetime DEFAULT time::now() READONLY; DEFINE INDEX scope_index ON instruction FIELDS scope; ``` ### `uncertainty` Ambiguous or unresolved information from extraction. ```surql DEFINE TABLE uncertainty SCHEMAFULL; DEFINE FIELD about ON uncertainty TYPE string; DEFINE FIELD reason ON uncertainty TYPE string; DEFINE FIELD scope ON uncertainty TYPE set>; DEFINE FIELD source_turn ON uncertainty TYPE record; DEFINE FIELD resolved ON uncertainty TYPE bool DEFAULT false; DEFINE FIELD created_at ON uncertainty TYPE datetime DEFAULT time::now() READONLY; DEFINE INDEX scope_index ON uncertainty FIELDS scope; ``` ### `memory_chunk` Raw text segments from turns, embedded for semantic recall. ```surql DEFINE TABLE memory_chunk SCHEMAFULL; DEFINE FIELD session ON memory_chunk TYPE record; DEFINE FIELD text ON memory_chunk TYPE string; DEFINE FIELD embedding ON memory_chunk TYPE option>; DEFINE FIELD scope ON memory_chunk TYPE set>; DEFINE FIELD source_turn ON memory_chunk TYPE option>; DEFINE FIELD turn ON memory_chunk TYPE option>; DEFINE FIELD position ON memory_chunk TYPE option; DEFINE FIELD char_start ON memory_chunk TYPE option; DEFINE FIELD char_end ON memory_chunk TYPE option; DEFINE FIELD token_count ON memory_chunk TYPE option; DEFINE FIELD role ON memory_chunk TYPE option; DEFINE FIELD simhash ON memory_chunk TYPE option; DEFINE FIELD duplicate_of ON memory_chunk TYPE option>; DEFINE FIELD created_at ON memory_chunk TYPE datetime DEFAULT time::now() READONLY; DEFINE INDEX embedding_index ON memory_chunk FIELDS embedding HNSW DIMENSION 3072 DIST COSINE TYPE F32; DEFINE INDEX session_index ON memory_chunk FIELDS session; DEFINE INDEX scope_index ON memory_chunk FIELDS scope; ``` Optional **`role`**, **`simhash`**, and **`duplicate_of`** support role-aware retrieval and near-duplicate suppression (see **`includeDuplicates`** on `/query`). **`char_start` / `char_end` / `token_count`** locate each sub-chunk in the turn’s text when a long passage is split for embedding. ### `decision_trace` Audit record for every retrieval and storage operation. | Field | Description | |---|---| | `query` | The original query or operation description | | `tier` | Resolution tier: `direct`, `cache`, `hybrid`, `full_context` | | `scope` | The scope used for this operation | | `sources` | IDs of contributing memory/knowledge chunks | | `token_cost` | Tokens consumed by any LLM call | | `duration_ms` | Total duration | | `api_key_id` | The key that made the request | | `session_id` | Associated session (if any) | | `created_at` | Timestamp | ## Migrations Schema is bundled into the SurrealDB Agent Memory binary and applied automatically when a Context is created or upgraded. See [Migration status](/docs/agent-memory/reference/management-api.md#migration-status) for the fleet-wide catch-up view.