Embedded databases

An embedded database runs the SurrealDB engine inside your Python process, so there is no server to start or connect to. This page shows how to install the engine, how to open in-memory and on-disk databases, and which SDK features an embedded connection supports. Embedded databases suit tests, command-line tools, desktop applications and other programs that keep their data on the same machine.

API references

Installing the embedded engine

The engine is a separate package, surrealdb-embedded, which the embedded extra installs at the version that matches the SDK:

pip install 'surrealdb[embedded]'

Without the extra, an embedded URL raises UnsupportedEngineError, with a message that names the extra to install. WebSocket and HTTP connections work without it. SDK 2.x included the engine in the surrealdb package itself, so a project that upgrades must add the extra: see Migrating to 3.x.

See Installation for the other extras and the supported Python versions.

URL schemes

The scheme of the URL chooses where the embedded database keeps its data.

file:// and surrealkv:// open the same storage engine, SurrealKV. A path after :// is relative to the working directory of the process, so surrealkv://data/app stores the data in data/app. A third slash makes the path absolute, as in surrealkv:///var/lib/app/data.

In-memory databases

An in-memory database is kept in the memory of the process. Each connection to mem:// opens a database of its own, so two connections do not share data, and the data is lost when the connection closes. This suits tests, which can each start with an empty database.

from surrealdb import RecordID, Surreal

with Surreal("mem://") as db:
    db.use("surrealdb", "docs")
    db.create(RecordID("person", "tobie"), {"name": "Tobie"})
    print(db.select(RecordID("person", "tobie")).execute())

{'id': RecordID(table_name=person, record_id='tobie'), 'name': 'Tobie'}

close() shuts the engine down. connect() after close() opens the connection again, with a new, empty in-memory database.

On-disk databases

An on-disk database keeps its data in a directory, so the data is still there when the program opens the directory again:

from surrealdb import RecordID, Surreal

with Surreal("surrealkv://data/app") as db:
    db.use("surrealdb", "docs")
    db.upsert(RecordID("person", "tobie"), {"name": "Tobie"})

with Surreal("surrealkv://data/app") as db:
    db.use("surrealdb", "docs")
    print(db.select(RecordID("person", "tobie")).execute())

{'id': RecordID(table_name=person, record_id='tobie'), 'name': 'Tobie'}

Versioned databases

A database opened with surrealkv+versioned:// keeps the earlier versions of each record. .version() on a select() builder reads the records as they were at a point in time. It takes a datetime or a Datetime, and a datetime without a time zone counts as UTC.

from datetime import datetime, timezone

from surrealdb import RecordID, Surreal

with Surreal("surrealkv+versioned://data/history") as db:
    db.use("surrealdb", "docs")
    tobie = RecordID("person", "tobie")

    db.upsert(tobie, {"name": "Tobie"})
    before = datetime.now(timezone.utc)
    db.upsert(tobie, {"name": "Tobias"})

    current = db.select(tobie).execute()
    earlier = db.select(tobie).version(before).execute()

current holds the record with the name Tobias, and earlier holds the record as it was at before, with the name Tobie. The builder sends a SELECT statement with a VERSION clause, so SELECT * FROM person:tobie VERSION d"..." in a query() gives the same result.

An in-memory database and a database opened with surrealkv:// do not keep versions, and the engine refuses a versioned read on them with an error saying that the datastore does not support versioned queries.

No sign-in

An embedded connection needs no sign-in. The engine enforces authentication, and the session of the connection starts as an owner with full access, so call use() and start to query.

The engine has no root user, so do not call signin() with root credentials: it raises NotAllowedError. invalidate() makes the session anonymous, and an anonymous session has no access. To get an owner session back, close the connection and open it again. A reopened in-memory database is empty.

What embedded connections support

An embedded connection has the same methods as a WebSocket connection, but some of them need a server. Those raise UnsupportedFeatureError before anything reaches the engine.

To apply several statements as one transaction on an embedded connection, use batch(), or send BEGIN and COMMIT with the statements in one query(). See Transactions.

Connection options are ignored

An embedded connection ignores every option of Surreal() and AsyncSurreal(), such as timeout, reconnect, auth and version_check. It runs no server version check. with db.timeout(seconds): is accepted and has no effect, because requests do not leave the process.

The embedded engine

The surrealdb-embedded package of SDK 3.0.0 runs SurrealDB 4.0. version() returns the version of the engine:

from surrealdb import Surreal

with Surreal("mem://") as db:
    print(db.version())

surrealdb-4.0.0

The engine is built with a smaller set of features than the SurrealDB server. It includes:

• •

  the crypto::* functions, including password hashing

• •

  the geo::*, encoding::* and parse::url::* functions, and string::slug

• •

  ORDER BY ... COLLATE, string::html::sanitize and Unicode regular expressions

• •

  full-text and HNSW indexes, with the search::* functions

• •

  DEFINE API endpoints, with the api::* functions

• •

  text patches in RETURN DIFF

• •

  background maintenance tasks

• •

  file buckets, with DEFINE BUCKET and the file::* functions

It leaves out ISO GQL and GraphQL, JavaScript functions, the http::* functions, and the PostgreSQL and MySQL wire protocols.

Moving on-disk data from an earlier engine

The SurrealDB 4.0 engine of SDK 3.0.0 cannot use a file:// or surrealkv:// directory that an earlier engine wrote. To keep the data, export it with a SurrealDB server that can read the directory, then load the export into a new directory. Point SDK 3.0.0 at the new directory, keep the old one until you have checked the new one, and make a copy of the old one before you start. In-memory databases are not affected.

How you export depends on the engine that wrote the directory:

• •

  SDK 2.x: the engine of SDK 2.x was SurrealDB 2.3, so the directory holds a SurrealDB 2.x datastore. Start a SurrealDB 2.x server on the directory, and make a 3.x-compatible export of it, either with the V3 compatible export in SurrealDB Studio, which needs SurrealDB 2.6 or later, or with surreal v2 export --v3 from a SurrealDB 3.x binary, 3.0.3 or later. The 4.0 binary has no v2 command. Upgrading from 2.x to 3.x describes both.

• •

  A SurrealDB 3.x engine: Surreal() and AsyncSurreal() raise ConnectionUnavailableError when they open the directory, with a message saying that it holds a SurrealKV v0.x database written by SurrealDB 3.x. The message suggests SURREAL_EXPERIMENTAL_UPGRADE_FROM_3X=true, but the embedded engine does not read SURREAL_* environment variables, so that setting has no effect. Start a SurrealDB 3.x server on the directory, and export the database with the surreal export command of that 3.x binary.

Then open a new directory with SDK 3.0.0 and run the export through query(), because import_() is not available on embedded connections:

import pathlib

from surrealdb import Surreal

dump = pathlib.Path("export.surql").read_text()

with Surreal("surrealkv://data/app-4") as db:
    db.use("surrealdb", "docs")
    db.query(dump).execute()

The engine runs every statement of the export. If some of them fail, execute() raises the error of the first one, and the other statements are still applied. To see the result of every statement, use query_raw(dump) instead, which reports each one with a status of OK or ERR.

Exports and imports lists the changes that affect a restore on SurrealDB 4.0, such as access methods that must be defined again.

Building from source

Building surrealdb-embedded from source needs Rust 1.95 or later. In a clone of the SDK repository, build it with maturin:

uv run maturin develop --release --manifest-path embedded/Cargo.toml

Learn more

• •

  Connecting to SurrealDB for the remote connection types and their options

• •

  Surreal API reference for every method of a connection

• •

  Transactions for batch() and transactions without a server

• •

  Errors for UnsupportedFeatureError and the other errors