> Full SurrealDB documentation index: https://surrealdb.com/docs/llms.txt

# Uploading documents

How to ingest documents into the SurrealDB Agent Memory knowledge layer.

The **knowledge** layer holds **authoritative** material - manuals, policies, product data, and files your agents should treat as curated sources. Documents enter through an asynchronous upload pipeline: bytes land in object storage, then SurrealDB Agent Memory extracts, chunks, embeds, and indexes structured state in SurrealDB.

Document extraction uses the same structured schema and reconciler as conversational ingest ([Storing memories](/docs/agent-memory/memory-and-knowledge/ingest/experiential/remember.md)). Facts from a PDF and facts from a turn land in one substrate; prefer documents for long curated sources and `/facts` for short-lived moments.

## Supported formats

### Text-first formats

These MIME types are accepted on **`POST /api/v1/{context_id}/documents`** under every ingestion profile:

| Format | MIME type |
| --- | --- |
| Plain text | `text/plain` |
| Markdown | `text/markdown` |
| CSV | `text/csv` |
| JSON | `application/json` |
| HTML | `text/html` |
| PDF | `application/pdf` |

### Multimodal formats

The upload endpoint also accepts image, audio, and video MIME types. The default Context profile is **`MultimodalFull`**, so OCR, transcription, captioning, and modality-native embeddings run when providers are configured. Dial the Context down to **`StandardMultimodal`**, **`TextPlusKeyword`**, or **`TextOnly`** to skip richer stages:

| Format | MIME types (examples) |
| --- | --- |
| Images | `image/png`, `image/jpeg`, `image/webp`, `image/gif` |
| Audio | `audio/wav`, `audio/mpeg`, `audio/ogg`, `audio/flac`, `audio/aac` |
| Video | `video/mp4`, `video/webm`, `video/quicktime` |

Under **`TextOnly`**, multimodal uploads may be accepted but image/audio/video processing stages are skipped. See [Multimodal content](/docs/agent-memory/memory-and-knowledge/ingest/authoritative/multimodal-content.md) for profile details.

## Uploading a document

The upload endpoint is asynchronous. It returns **`202 Accepted`** with a document id and initial status; processing continues on the worker tier.

### REST

```http
POST /api/v1/{context_id}/documents
Content-Type: multipart/form-data

file=<binary>
metadata={"title":"Returns Policy","scopes":[["org/acme/team/eng"]],"labels":["team=eng"]}
```

The `metadata` part is JSON, and both it and every field in it are optional. **`title`** and **`source`** default to the filename. **`mimeType`** overrides the `Content-Type` of the `file` part; without either, the type is inferred from the bytes. **`scopes`** is a DNF selector (OR of conjunctive slash-path clauses). **`labels`** are descriptive `key=value` tags (same validation as fact ingest - keys must not start with `_`; count caps return **`409`**). Optional **`observedAt`** (RFC 3339) sets the known time of facts derived from this document - essential for **page-by-page or episode-by-episode ingest** where later plot points must stay hidden until the reader reaches them ([spoiler-safe narrative memory](/docs/agent-memory/cookbooks/patterns/spoiler-safe-narrative-memory.md)). **`observedAtHeader`** names a field in a leading `---` front-matter block whose value (RFC 3339 or `YYYY-MM-DD`) is read as `observedAt` when you do not send `observedAt` yourself. A missing or unparseable field is not an error: facts are then dated to ingest time. Omit **`scopes`** to tag the document with the caller's full `memory:write` region.

> [!NOTE]
> **`observedAt` is caller-supplied metadata** - SurrealDB Agent Memory does not scan document body text to infer a narrative timeline, and reads front matter only when you name a field with **`observedAtHeader`**. For serial fiction or journals, split uploads per chapter or episode and set **`observedAt`** (or ingest turns with **`observed_at`**) explicitly. Bulk single-file upload stamps derived facts at ingest time unless you provide the field.

Response:

```json
{
  "id": "01hx9…",
  "status": "queued",
  "contentHash": "4f3c…",
  "deduplicated": false
}
```

The `id` is the bare document id, without a `document:` prefix, and `contentHash` is the BLAKE3 hash as plain hex. When the upload sets `observedAt` or detects one through `observedAtHeader`, the response also carries the effective `observedAt`.

The upload size limit is set for the deployment, 4 GiB by default.

### CLI

```bash
surreal memory documents upload ./returns-policy.pdf \
  --scope org/acme/team/eng \
  --label team=eng \
  --url "$SURREAL_MEMORY_URL" \
  --api-key "$SURREAL_MEMORY_API_KEY" \
  --context-id "$SURREAL_MEMORY_CONTEXT_ID"
```

Use the generated OpenAPI clients ([Python](/docs/agent-memory/integrations/sdks/python.md), [TypeScript](/docs/agent-memory/integrations/sdks/javascript-and-typescript.md)) for application code - method names follow the spec.

## Polling for status

Poll **`GET /api/v1/{context_id}/documents/{id}`** until **`status`** is **`ready`** or **`failed`**.

## Pipeline stages

| Status | Description |
| --- | --- |
| `queued` | Waiting to enter the pipeline |
| `extracting` | Reading content from the uploaded bytes |
| `chunking` | Splitting content into overlapping segments |
| `embedding` | Generating dense vectors for chunks and sections |
| `keywording` | RAKE keyword extraction (every profile except `TextOnly`) |
| `extracting_nodes` | Extracting entities, attributes, relations and actions with a language model (every profile except `TextOnly`, when an LLM is attached) |
| `ready` | Fully indexed and available for retrieval |
| `failed` | Pipeline error; inspect `error` on the document record |

Oversized chunk persists (SurrealDB transaction write-set or WebSocket message caps) are classified as permanent size errors and dead-letter without spending minutes on re-parse retries.

## Content addressing and deduplication

Every document is identified by a [BLAKE3](https://github.com/BLAKE3-team/BLAKE3) hash of its raw bytes. Re-uploading identical content returns the existing document with **`deduplicated: true`** and skips reprocessing.

When a **second uploader in a different scope** hits the same hash, SurrealDB Agent Memory **unions their scope clause** onto the existing document (and related index records) instead of trapping them with a deduplicated id they cannot read. Each union emits a **`document.scope_widen`** audit event; operators can watch the **`documents.scope_clause_count`** histogram (values above ~32 clauses on one document warrant investigation).

## Outbound links

During ingest, SurrealDB Agent Memory extracts outbound hyperlinks from the raw bytes and stores them as typed **`knowledge_links_to`** edges:

| Source | Link kind |
| --- | --- |
| Markdown `[]()` syntax | `markdown_link` |
| HTML `<a href="…">` attributes | `html_link` |
| PDF page link annotations | `pdf_annotation` |

These edges feed **`hybrid_graph`** reranking (document-link density) and citation-style navigation between corpus documents.

## Scope and labels

Documents and their chunks inherit the caller's **resolved write scope** from the API key when you omit explicit scope on upload - required for scoped keys to recall their own uploads.

You can **narrow** tagging with **`scopes`** on **`POST /documents`**, **`surreal memory documents upload --scope …`**, or MCP **`upload`** - the path must lie within the caller's `memory:write` region (out-of-region scope returns **`403`**). A document's scope is **fixed at upload** - reprocess keeps the existing scope and labels and ignores any `scopes` or `labels` in its metadata.

Optional **`labels`** (`key=value` strings) are stamped on the document, chunks, and sections. They follow the same validation rules as fact ingest and are **not** copied onto reconciled graph records.

> [!NOTE]
> **MCP `upload`** accepts optional **`scopes`** and **`labels`** arguments with the same semantics as REST - scoped keys produce scope-tagged documents and chunks visible under `memory:read` within that region.

## Document management

- **`GET /documents/{id}`** - status and metadata
- **`GET /documents`** - list with filters
- **`GET /documents/{id}/chunks`** - parsed segments
- **`GET /documents/{id}/raw`** - original bytes
- **`DELETE /documents/{id}`** - remove document, chunks, graph edges, and object-store bytes

See [REST API](/docs/agent-memory/reference/rest-api.md) for request shapes.
