NEW

SurrealDB 4 is here, introducing SurrealDB Earth and SurrealDS!

Learn more

1/4

SurrealDB 4. The database and memory for every agent

Release
Engineering
Featured
Company
Community

Nov 3, 20269 min read

Tobie Morgan Hitchcock

Tobie Morgan Hitchcock

Show all postsSurrealDB 4. The database and memory for every agent

Our newsletter

Get tutorials, AI agent recipes, webinars, and early product updates in your inbox every two weeks

SurrealDB 4 is generally available today. It is the database and memory for every agent. And with it, SurrealDB Earth: serverless SurrealDB, built on Cloudflare, that scales to as many agents and users as you need, wherever they are. SurrealDB 4, the engine underneath, returns vectors, graphs, documents and full text in one query, in one transaction, and speaks PostgreSQL and MySQL over their own wire protocols, so users and agents can write the SQL they already know.

  • SurrealDB 4 is the database engine: a rebuilt query core, PostgreSQL SQL and MySQL support over their wire protocols, mql.js 1.0 and more than 240 fixes that no 3.x release contains. It runs embedded, on a laptop, in the browser, on your own servers or in SurrealDB Cloud.

  • SurrealDB Earth is serverless SurrealDB, built on Cloudflare, that scales to as many agents and users as you need, wherever they are. Every Earth database is its own isolated instance, with dedicated storage and commit log.

  • SurrealDS is distributed storage backed by object-storage for large SurrealDB clusters, in SurrealDB Cloud multi-node clusters and self-hosted Enterprise Edition deployments.

  • Agent Memory, launched in October, is a separate, managed service built on the same engine. Memory in one query is built into the engine, and Agent Memory is the managed way to get it.

Earth is where you run it, and SurrealDB 4 is what you run.

If you already work in Postgres and do not want to touch SurrealQL yet, this is where you can start. SurrealDB does not need PostgreSQL installed: the listener lets Postgres clients talk to SurrealDB. On your machine, start SurrealDB 4 with the PostgreSQL listener on, then connect with psql and select the PostgreSQL dialect:

surreal db start --user root --pass secret --postgres-bind 127.0.0.1:5432 memory
psql "host=127.0.0.1 port=5432 user=root password=secret dbname=main/main options='-c dialect=pgsql'" -c 'SELECT 1;'

Both wire protocol listeners are off until you pass --postgres-bind or --mysql-bind. The PostgreSQL and MySQL protocol references list the connection options.

Most agents today are short-lived. Their state is rebuilt from a conversation history, a vector store or a set of external systems each time they run. As agents become persistent software, they need somewhere durable to keep:

  • long-term memory

  • the entities they know about and how those entities relate

  • current and historical state

  • tasks and intermediate results

  • documents and embeddings

  • permissions and identity

  • state shared transactionally with other agents

That state has to outlive whichever model or process happens to be running the agent at the time. SurrealDB is built to hold it, and SurrealDB Agent Memory, launched in October, is the managed product built on the same engine for exactly this job.

SurrealDB Earth runs SurrealDB serverless on Cloudflare's global points of presence. Instead of provisioning a server and sizing it in advance, your application or agent creates databases on demand, and Earth decides where and how they run. Every Earth database is its own isolated instance, with dedicated storage and commit log, keeping each tenant's data separate. Each one is fully transactional and spins up and down on demand. Earth is web-native, so it connects over HTTP and WebSocket.

A database starts with its first query, in ~300ms from cold, and scales to zero instantly when idle, so you pay for requests and storage, with no idle server costs. Its data stays durable in Cloudflare's storage.

That makes fine-grained isolation practical. A SaaS platform can give each customer a database, an application can give each project one, and an AI platform can give an agent a persistent database of its own when it needs one. Each one's data is isolated, and removing a customer removes their database.

Scale is one reason to choose Earth, and not the only one. At peak, Earth handles 10+ billion rows per second, fully transactional.

Start free on Earth with no card, see the full Earth pricing.

SurrealDS is distributed storage for workloads that grow beyond a single machine and runs on object-backed storage. It separates compute from durable storage, a pattern common in analytical databases and rare in transactional ones. Query nodes keep the state and caches they need to run queries quickly, while durable database pages live in shared object storage such as Amazon S3. Compute and storage each grow on their own, with no manual sharding.

The design assumes that nodes, disks and networks will fail. A write is acknowledged only once it is on two disks in two zones, and a replacement node recovers from the surviving copy and object storage rather than from the machine that failed. The protocol is model-checked. The aim is predictable behaviour when infrastructure stops being healthy, not only availability when everything is fine.

Together with Earth, it gives SurrealDB one range:

  • One agent: one lightweight database on Earth.

  • Many agents: many isolated databases, each its own Earth instance.

  • Large agent platforms: shared SurrealDS clusters, where many agents read and write the same data transactionally.

The data model is the same at each step. SurrealDS is available in SurrealDB Cloud multi-node clusters and self-hosted Enterprise Edition deployments.

Memory in one query is built into the engine. An agent's memory is several kinds of data at once: what something means, what it is connected to, and what was actually said. SurrealDB returns all three in one statement, inside one transaction:

SELECT
    text,
    ->mentions->person.name AS people,
    search::score(1) AS relevance
FROM note
WHERE embedding <|5,40|> $question
    AND text @1@ 'contract renewal'
ORDER BY relevance DESC;

The example query finds the five notes nearest in meaning to the question, keeps those that mention a contract renewal, follows the graph to the people each note mentions, and ranks the results by relevance.

PostgreSQL and MySQL clients connect to SurrealDB 4 over their own wire protocols. SurrealDB 4 adds PostgreSQL SQL, selected for each connection, and the MySQL protocol with MySQL SQL. Each listener is off until you turn it on, and a connection picks its dialect. That SQL runs through the same query planner as SurrealQL and reaches the same records, so an agent can write the SQL it was trained on.

The PostgreSQL dialect (select it with SET dialect = 'pgsql', or with options=-c dialect=pgsql when you connect) supports joins, grouping, aggregates, inserts with ON CONFLICT, schema changes, transactions, non-recursive WITH, COPY in text and CSV, and pgvector columns. The MySQL dialect supports joins, grouping, aggregates, non-recursive WITH and INSERT ... ON DUPLICATE KEY UPDATE.

This is a defined subset of SQL. Window functions, recursive CTEs and FULL OUTER JOIN are refused with a named error (0A000), and a few statements, such as CREATE SCHEMA in the PostgreSQL dialect, are accepted but change nothing. The PostgreSQL and MySQL protocol references list what is supported, what is refused and where behaviour differs.

For Node.js applications written for the MongoDB driver, mql.js 1.0 brings that driver's API to SurrealDB, including aggregation pipelines and Mongoose models. mongosh, Compass and MongoDB drivers in other languages cannot connect.

A database scales furthest when each query costs less, so SurrealDB 4 rebuilds the query core to do less work for every query. Our benchmarks page goes into full details on the full results and how they were measured. We'll continue to expand our benchmarks for distributed database comparisons, vector search (HNSW throughput and recall), full-text search (BM25), graph query performance benchmarks, and more.

Some reliability work is not dramatic: finding edge cases, improving error handling, tightening correctness. It is a major part of this release. SurrealDB 4 includes more than 240 fixes to the database that no 3.x release contains, including more than 90 community bug reports on GitHub.

Rebuilding a query core can quietly change years of behaviour, so the change was made behind a safety net: while the new engine was built, every query test ran on both the earlier engine and the new one, and failed if the two disagreed. Today more than 4,500 SurrealQL test files run in CI.

SurrealDB also includes many other features such as:

  • ISO GQL support, for graph pattern matching

  • Schema migrations: existing rows convert as they are read, and the table is rewritten in the background

  • Filtered, exact and permission-aware vector search

  • Python SDK 3

SurrealDB 4 is recommended for new (greenfield) database deployments. For existing deployments running 3.x, we recommend upgrading to SurrealDB 4.1 once it becomes available. Our internal testing of the migration process has shown promising results, but we want to validate it further against a broader range of real-world workloads and deployment scenarios from our community before recommending it for production migrations. This additional validation will help us deliver a more robust and reliable upgrade experience in 4.1.

For those who want to test upgrading from 3.x to SurrealDB 4, we've introduced an experimental in-place upgrade process designed to make the transition as seamless as possible, without requiring an export and import of your data. This requires explicit opt-in and should only be attempted after taking and verifying a backup, ideally using a copy of your data. The upgrade is one-way, meaning migrated datastores cannot be reopened with 3.x.

Before testing the upgrade, we recommend updating to the latest 3.3 release, running the new surreal db upgrade-check command to identify compatibility issues, and updating your SDKs.

A few additional changes to be aware of:

  • RocksDB and IndexedDB have been replaced by SurrealKV. Existing RocksDB datastores are automatically converted on first use, requiring additional time and disk space. Existing SurrealKV datastores also migrate to the new storage format.

  • Some indexes need rebuilding to preserve query correctness, while others can be rebuilt to benefit from performance improvements in SurrealDB 4.

  • SurrealML has been removed. Existing models must be exported, if needed, and removed before upgrading.

For detailed instructions, including how to enable the experimental upgrade, compatibility considerations and rollback options, see our 3.x to 4.0 upgrade guide.

Start free on SurrealDB Earth or install SurrealDB 4 to run it locally. See everything in the release. For deploying SurrealDS clusters, talk to our team.

The goal is one database that grows from a single lightweight workload to the infrastructure behind millions of users, devices and agents. SurrealDB 4 is the biggest step towards it so far. Thank you to everyone who reported a bug, opened a pull request or answered a question on Discord this year: a large part of this release is yours.

Related posts

Our newsletter

Get tutorials, AI agent recipes, webinars, and early product updates in your inbox every two weeks