Uploading documents

The knowledge layer holds authoritative material - manuals, policies, product data, and files your agents should treat as curated sources. Documents enter through an asynchronous upload pipeline: bytes land in object storage, then SurrealDB Agent Memory extracts, chunks, embeds, and indexes structured state in SurrealDB.

Document extraction uses the same structured schema and reconciler as conversational ingest (Storing memories). Facts from a PDF and facts from a turn land in one substrate; prefer documents for long curated sources and /facts for short-lived moments.

Supported formats

Text-first formats

These MIME types are accepted on POST /api/v1/{context_id}/documents under every ingestion profile:

Multimodal formats

The upload endpoint also accepts image, audio, and video MIME types. The default Context profile is MultimodalFull, so OCR, transcription, captioning, and modality-native embeddings run when providers are configured. Dial the Context down to StandardMultimodal, TextPlusKeyword, or TextOnly to skip richer stages:

Under TextOnly, multimodal uploads may be accepted but image/audio/video processing stages are skipped. See Multimodal content for profile details.

Uploading a document

The upload endpoint is asynchronous. It returns 202 Accepted with a document id and initial status; processing continues on the worker tier.

REST

POST /api/v1/{context_id}/documents
Content-Type: multipart/form-data

file=<binary>
metadata={"title":"Returns Policy","scopes":[["org/acme/team/eng"]],"labels":["team=eng"]}

The metadata part is JSON, and both it and every field in it are optional. title and source default to the filename. mimeType overrides the Content-Type of the file part; without either, the type is inferred from the bytes. scopes is a DNF selector (OR of conjunctive slash-path clauses). labels are descriptive key=value tags (same validation as fact ingest - keys must not start with _; count caps return 409). Optional observedAt (RFC 3339) sets the known time of facts derived from this document - essential for page-by-page or episode-by-episode ingest where later plot points must stay hidden until the reader reaches them (spoiler-safe narrative memory). observedAtHeader names a field in a leading --- front-matter block whose value (RFC 3339 or YYYY-MM-DD) is read as observedAt when you do not send observedAt yourself. A missing or unparseable field is not an error: facts are then dated to ingest time. Omit scopes to tag the document with the caller's full memory:write region.

Response:

{
  "id": "01hx9…",
  "status": "queued",
  "contentHash": "4f3c…",
  "deduplicated": false
}

The id is the bare document id, without a document: prefix, and contentHash is the BLAKE3 hash as plain hex. When the upload sets observedAt or detects one through observedAtHeader, the response also carries the effective observedAt.

The upload size limit is set for the deployment, 4 GiB by default.

CLI

agent-memory documents upload ./returns-policy.pdf \
  --scope org/acme/team/eng \
  --label team=eng \
  --url "$SPECTRON_URL" \
  --api-key "$SPECTRON_API_KEY" \
  --context-id "$SPECTRON_CONTEXT_ID"

Use the generated OpenAPI clients (Python, TypeScript) for application code - method names follow the spec.

Polling for status

Poll GET /api/v1/{context_id}/documents/{id} until status is ready or failed.

Pipeline stages

Oversized chunk persists (SurrealDB transaction write-set or WebSocket message caps) are classified as permanent size errors and dead-letter without spending minutes on re-parse retries.

Content addressing and deduplication

Every document is identified by a BLAKE3 hash of its raw bytes. Re-uploading identical content returns the existing document with deduplicated: true and skips reprocessing.

When a second uploader in a different scope hits the same hash, SurrealDB Agent Memory unions their scope clause onto the existing document (and related index records) instead of trapping them with a deduplicated id they cannot read. Each union emits a document.scope_widen audit event; operators can watch the documents.scope_clause_count histogram (values above ~32 clauses on one document warrant investigation).

Outbound links

During ingest, SurrealDB Agent Memory extracts outbound hyperlinks from the raw bytes and stores them as typed knowledge_links_to edges:

These edges feed hybrid_graph reranking (document-link density) and citation-style navigation between corpus documents.

Scope and labels

Documents and their chunks inherit the caller's resolved write scope from the API key when you omit explicit scope on upload - required for scoped keys to recall their own uploads.

You can narrow tagging with scopes on POST /documents, agent-memory documents upload --scope …, or MCP upload - the path must lie within the caller's memory:write region (out-of-region scope returns 403). A document's scope is fixed at upload - reprocess keeps the existing scope and labels and ignores any scopes or labels in its metadata.

Optional labels (key=value strings) are stamped on the document, chunks, and sections. They follow the same validation rules as fact ingest and are not copied onto reconciled graph records.

Document management

• •

  GET /documents/{id} - status and metadata

• •

  GET /documents - list with filters

• •

  GET /documents/{id}/chunks - parsed segments

• •

  GET /documents/{id}/raw - original bytes

• •

  DELETE /documents/{id} - remove document, chunks, graph edges, and object-store bytes

See REST API for request shapes.