Gen 3 architecture. TAPIR consensus, compute-storage separation, and object storage - the foundation beneath the context layer.
GEN 3 ARCHITECTURE
Storage redesigned from first principles
Most distributed databases were designed for provisioned disks attached to compute nodes. The storage engine places transactional data directly in cloud object storage and runs the database as stateless, elastic compute on top.
Gen 1: Single machine
Compute and storage on one box. Scaling means a bigger box. MySQL, Postgres self-hosted.
Gen 2: Proprietary separation
Storage separated into proprietary cloud tiers. Data locked in. Single vendor. Aurora, AlloyDB.
Gen 3: Object storage
Data in commodity object storage. Stateless compute on top. Scale independently.
Client
Load balancer layer
Request routed to SurrealDB TAPIR node, or SurrealDB read-proxy node
The storage engine uses TAPIR (Transactional Application Protocol for Inconsistent Replication). Each availability zone has its own write node, writes scale horizontally, and transactions commit once a quorum acknowledges. Lower latency than leader-based replication.
1 transaction → 1 TAPIR decision
Every transaction is encapsulated within a TAPIR consensus decision. Transaction consensus is performed at the writing node, allowing writes to horizontally scale across all write nodes in a cluster.
TAPIR transaction
(majority commit)
TAPIR consensus
Storage enginedata storage
Storage enginedata storage
Storage enginedata storage
WRITE PATH
How a write transaction flows
From client request through the compute layer, into TAPIR quorum consensus, and down to durable object storage.
SurrealDB Node
Query engine
Transaction engine
Storage engine(local disk and block storage)
Transaction engine
Periodic storage data compaction and compacted data storage
Quorum commit: transaction is committed once a TAPIR quorum acknowledges the writes.
Durable log: committed writes are safe via a replicated write-ahead-log, with asynchronous durability to object storage for fast node recovery and secondary-region disaster recovery.
Catch-up: failed node restarts and replays transaction log from object storage (for reduced cost) and from cross-availability zone (for recent transactions).
Node crash: a new node catches up with the cluster, regardless of the existing local state, using transaction-log replay before joining the quorum.
Outcome:
Transaction commit succeeds with quorum majority
Client receives transaction success confirmation
SurrealDB TAPIR node in Availability Zone C catches up with cluster after restart
No split-brain / consistent ordering
Significant reduction in cross-availability zone traffic
IN PRACTICE
What Gen 3 storage delivers
Storage costs drop
Object storage costs a fraction of provisioned disk. The total dataset can far exceed the local storage of any running instance.
Cross-region replication
Data flows through object storage, not between nodes. The most expensive line item in traditional distributed databases is structurally reduced.
Scale to zero
Compute is stateless. Scale down to zero when idle - data persists in object storage. When demand returns, nodes restore from the durable store in seconds.
Instant recovery
A crashed node restores from object storage and replays the transaction log. Recovery time is independent of dataset size.
Instant branching
Clone a petabyte-scale database in seconds for development, testing, or experimentation. Git-like workflows for production data.
Built-in disaster recovery
Object storage is the durable tier (99.999999999% durability). No separate backup infrastructure. No snapshot schedules.
THE FULL STACK
From object storage to agent memory
No specialist agent database owns its storage layer. No memory middleware controls how data is persisted. The storage engine is the reason the context layer can scale down to zero, branch instantly, and recover from failures by design. No other product in the market owns this vertical.
TAPIR (Transactional Application Protocol for Inconsistent Replication) is a consensus protocol where each availability zone has its own write node. Transactions commit once a quorum of zones acknowledges. This eliminates the single-leader bottleneck and provides lower latency than traditional Raft or Paxos-based systems.
The storage engine supports Amazon S3, Google Cloud Storage, and Azure Blob Storage. Data is stored in commodity object storage, making it portable across cloud providers.
Because the storage engine separates compute from storage, the database engine is stateless. When no queries are running, compute instances can shut down entirely - data persists in object storage. When demand returns, new compute nodes restore state from object storage and replay the recent transaction log to rejoin the cluster. Cold-start time depends on the size of the transaction log delta, not the size of the dataset. You only pay for the compute you use.
The storage engine can clone a petabyte-scale dataset in seconds by creating a logical branch that shares the underlying object storage data. Changes on the branch are isolated. This enables Git-like workflows for databases - branch for testing, experiment freely, merge or discard.
GET STARTED
Build on the storage layer no one else has
From object storage to agent memory. Scale down to zero, branch instantly, recover by design. This is what the context layer is built on.
