---
title: "SurrealDB Scalability: the details"
description: "SurrealDB scales with separate compute and storage layers, a cost-based query planner, and distributed deployments on SurrealDB Cloud Scale and Enterprise."
url: https://surrealdb.com/blog/surrealdb-scalability
date: 2023-05-01
authors: "Hugh Kaznowski & Tobie Morgan Hitchcock"
---

# SurrealDB Scalability: the details

![SurrealDB Scalability: the details](https://cdn.surrealdb.com/cl93hl203ggs73c8trdg.auto)

SurrealDB is a multi-paradigm database that allows you to perform document, graph, temporal, spatial, vector, and text operations within an ACID environment.

This post was first written in 2023, when SurrealDB was approaching 1.0 and much of the scaling story was still ahead of us. Enough has changed since that a light edit would have left more wrong than right, so it has been rewritten against the 3.x releases. The shape of the argument is Hugh's; the details are current.

## What is SurrealDB, technically

SurrealDB works with a compute layer that processes queries and operates on a storage layer. Which storage engine sits underneath is a deployment choice rather than a fixed part of the design.

A single node runs on RocksDB, or our own storage engine [SurrealKV](/docs/running/file-backed). RocksDB is a key-value store originally from Facebook, forked from Google's LevelDB LSM tree and optimised to reduce write amplification (improving SSD lifetime), space amplification, and compute utilisation. It is an industry-standard engine, used heavily in many prominent products.

For a distributed deployment, the storage layer is a distributed key-value store rather than a local one. That is where the scaling properties in this post come from.

## How does SurrealDB achieve scalability?

The future of databases, particularly in the cloud, involves separating the storage from the compute layers. We do not want to force that decision on you: running in embedded mode opts out of the modularity entirely, and a single file-backed node remains a perfectly good way to run SurrealDB.

But where storage and compute scale independently, you move closer to paying for what you use. If you hold five petabytes but run one query a minute, you should not be paying for compute to match the data. If you serve thousands of queries a second against a gigabyte, you should be able to grow the compute without growing the storage.

## How does the storage layer scale?

In a distributed deployment, the storage layer partitions data into ranges and spreads those ranges across nodes, replicating each one. Partitions are kept small, which keeps conflicting transactions cheap under high traffic and keeps distributed locks rare, because the balancing works to reduce them.

There are two supported routes to this today. [SurrealDB Cloud Scale](https://surrealdb.com/pricing/scale) runs the distributed storage layer for you, and [SurrealDB Enterprise](https://surrealdb.com/enterprise) runs it in your own environment. Both are the production answer for multi-node high availability.

## How does the compute layer scale?

The compute layer is stateless. Messaging between nodes is allowed to be fallible, so adding and removing instances needs no coordination. Scaling is a matter of starting or stopping SurrealDB instances, with no health checks or membership protocol of its own beyond what the storage layer already provides.

## Yes, but is it webscale?

Scalability is a genuinely complex problem, and the reason people ask about it is rarely idle. They want to reason about the system and be confident it behaves as they expect under load. The "MongoDB Is Web Scale" video is funny precisely because it does not engage with any of that, so the rest of this post tries to.

## Size of data

The most obvious question is how much data can be held and processed. In a distributed deployment, storage scales horizontally: ranges become partitions, partitions are distributed across nodes, and the distribution is rebalanced based on observed usage. The behaviour is predictable, which matters more than a headline number.

## Data access patterns and performant queries

A common issue with any OLTP database is how the data is laid out. A schema that matches the domain ("user", "post") may not match how the application actually reads it ("recommended users", "top posts", "recent user posts"). Filtering the primary source-of-truth table on every read is the usual trap:

```surql
SELECT * FROM user WHERE user.connections CONTAINS $this_user.id AND ...
```

Moving that computation into a secondary table is often the better shape.

![Secondary Tables](https://cdn.surrealdb.com/w(1600)q(80)/ch6mrg434uvc7389j560.auto)

A secondary table is close to what a secondary index does. An index maps constraints to a primary key entry, where the primary index stores the table data by document ID. A secondary table is more flexible: it can hold only the fields you need, which reduces how much is read and garbage collected, and it can hold information that is not in the primary data at all, such as values already resolved from a join.

![Primary index, secondary index, secondary table](https://cdn.surrealdb.com/w(1600)q(80)/ch7rkmk34uvc7389j7h0.auto)

## How a query planner works

A query planner reads a query and decides how to turn it into reads and writes: which index to use, which predicates can be simplified away, what order to do the work in.

SurrealDB has one. It picks indexes, and [`EXPLAIN`](/docs/reference/query-language/statements/select) shows what it chose:

```surql
DEFINE INDEX colour_idx ON user FIELDS favourite_colour;

SELECT * FROM user WHERE favourite_colour = 'Red' EXPLAIN;
```

```surql title="Output"
{
	children: [
		{
			attributes: {
				access: "= 'Red'",
				direction: 'Forward',
				index: 'colour_idx'
			},
			context: 'Db',
			operator: 'IndexScan'
		}
	],
	context: 'Db',
	operator: 'SelectProject'
}
```

That was not true when this post was first written, and the original argued that complex record IDs were a way to work around the absence. The technique is still worth knowing, but for a better reason: it gives you an access path you chose yourself, rather than one a planner inferred.

Planners work from estimates. If the planner expects a predicate to match a large share of the table, it may prefer a full scan over an index, and tuning that back can be fiddly. A record ID that encodes the access pattern is not an estimate, so its performance does not move:

```surql
CREATE user:one CONTENT { name: 'Mx One', favourite_colour: 'Red' };
CREATE user:hugh CONTENT { name: 'Hugh', favourite_colour: 'Blue' };
CREATE user:other CONTENT { name: 'Someone Else', favourite_colour: 'Red' };

CREATE user_colour:['Red', user:one] CONTENT { user: user:one };
CREATE user_colour:['Red', user:other] CONTENT { user: user:other };
CREATE user_colour:['Blue', user:hugh] CONTENT { user: user:hugh };

SELECT * FROM user_colour:['Red', NONE]..;
```

```surql title="Output"
[
	{ id: user_colour:['Red', user:one], user: user:one },
	{ id: user_colour:['Red', user:other], user: user:other }
]
```

The range scan reads exactly the entries under `'Red'` and nothing else. As with any index, there is a trade: you pay on create and update to make reads faster. A secondary table lets you choose when to pay it, by batching or deferring the write.

## Data locality

Data locality usually means one of two things: region affinity or edge storage.

Region affinity is keeping data near the users who read it, so European users are served from a European region. Distributed deployments rebalance partitions automatically, and SurrealDB Cloud exposes the region your instance runs in.

![Region Affinity](https://cdn.surrealdb.com/w(1600)q(80)/ch6mrg434uvc7389j54g.auto)

Edge storage is different: holding data on the client rather than in a data centre. It is a hard fit for an ACID OLTP system, because that data is far from everything else, which brings latency and availability of the client into the transaction path. SurrealDB does run embedded, in a browser through WebAssembly or inside an application process, which covers many of the cases people reach for edge storage to solve.

![Edge](https://cdn.surrealdb.com/w(1600)q(80)/ch6mrg434uvc7389j54g.auto)

## Fault tolerance

A fault-tolerant system handles network failures, disk failures, client failures and internal errors. The exception is Byzantine failures, where a cluster member reports false information.

Distributed deployments use a multi-raft consensus algorithm for fault tolerance. Tolerating n failures, where a failure can be a machine, a rack, a network or a whole region, requires 2n+1 copies. To survive two data centres going down, you need 2 × 2 + 1 = 5.

Single-node fault tolerance, such as power loss or a transaction terminated part-way, is handled by the write-ahead log. A commit is written to the log before the client is told it succeeded, so a failure can be replayed from the last good snapshot plus the un-flushed transactions in the log.

## Conclusion

SurrealDB is in production use, and the scaling story is no longer a plan: a cost-based planner with real indexes, distributed storage available on Cloud Scale and Enterprise, and a stateless compute layer you can grow independently of the data.

If you have any questions about scalability or reliability, [we would like to hear them](https://surrealdb.com/contact). If you want to try SurrealDB, get started with [SurrealDB Cloud Scale](https://surrealdb.com/docs/manage/instances).

## Massive thank you to the Discord community

We decide what to focus on based on user feedback, and the original version of this post was driven by users sharing their concerns and experience. A massive thank you to these people for sharing their perspectives:

- amaster507#1406
- nerdo#4825
- emmagamma#5637
- BitShift#1597

The list is incomplete, as many others contributed too. We are grateful to you even if you missed the list above.