Releases

Bug fixes:

• Fixed HNSW index compaction write conflicts

Features & improvements:

• Improved consistency for set and array operations

• Reduced import overhead and added benchmarks

• Added query planner and executor improvements

• Added GraphQL subscriptions

• Updated IndxDB to v0.12.0

Bug fixes:

• Fixed invalid time formatting strings causing a panic

• Fixed +=/-= silent no-ops with incompatible types

• Fixed START clause performance regression with pushdown KV skipping

• Fixed SurrealValue::from_value for all serde_json::Value variants

• Fixed BM25 search::score() returning 0 after index compaction

• Propagated actual query errors instead of misleading Connection uninitialised

• Fixed race condition in distributed task lease acquisition

• Fixed UPSERT SET with IF expressions always evaluating to the ELSE branch on existing records

• Fixed LIMIT producing fewer rows when WHERE is only partially resolved by the index

• Fixed subqueries not respecting nested fields with AS key.key

• Fixed index compaction KeyAlreadyExists and TransactionConflict errors

Features & improvements:

• Improved idempotency for transaction operations

• Updated SurrealKV to 0.21.0

• Added missing exports for API, BUCKET, MODULE, and CONFIG definitions

• Added NaiveDate implementation for SurrealValue in the SDK

• Fixed ORDER BY performance regressions with compound indexes

• Added root level field comments to GraphQL schema generation

• Implemented v2 subcommand to run the legacy binary

• Added retry logic for initial datastore transactions on concurrent startup

Bug fixes:

• Returned None for records that don't exist instead of a confusing error

• Fixed write-write conflicts on IP keys during IndexAppending

• Fixed serialization error of RecordIdKeyType::Object

• Ensured transaction timeouts were respected for queries

• Applied new executor bug fixes and optimizations

• Fixed DEFINE FUNCTION parsing from disk

• Fixed permission checks for DELETE live events

• Fixed compound unique indexing

Features & improvements:

• Added functions to set parts of datetimes

• Updated timestamp code to use boxed traits

• Made deriving SurrealValue more convenient when using the SDK

• Added error chaining infrastructure

• Added --tables-exclude flag to the surreal export command

Bug fixes:

• Fixed record IDs not being dereferenced when a field is computed on them

• Prevented creating indexes on computed fields

• Patch fixes for version errors and /sql endpoint

• Allow durations to be multiplied and divided by numbers, and incremented and decremented like numbers

• Fixed permission check condition for root users

• Improved environment variable values

• Parallelized index compaction across distinct indexes

• Fixed GraphQL literals for string enums

• Improved WASM compatibility

• Fixed error serialization

• Fixed new planner and deduplication logic

After numerous alpha and beta releases over more than a year of development, we are releasing SurrealDB v3.0.0! 🎉. The changes for this release pertain to those over all the alpha and beta versions before 3.0.0, starting with 3.0.0-alpha.1 and the last 3.0.0-beta.4 and finally the changes between the final beta version and stable 3.0.0.

Items in the release notes marked as 🆕 are the newest changes from the final beta and 3.0.0.

To migrate from a 2.x version to 3.0, please see the following page which goes into further detail on the breaking changes and how to use migration diagnostics to obtain a list of actions for your particular schema.

Performance and optimisations:

• 🆕 Internal streaming execution (new query planner)

• 🆕 Improve scanner for batched retrievals

• 🆕 Enable concurrent writes on HNSW vector index

• 🆕 Add support for skipping keys and values at storage engine layer

• 🆕 Implement multi-get fetches for graph edge scans

• Improved GROUP and ORDER BY execution correctness

• Improved edge lookups: Only fetch edge keys when values are unused

• Added criterion benchmarks for executor and functions

• Fetch ns, db, table, and fields once

• Reduced heap modifications in MemoryOrderedLimit

• Optimise index scans (descending) order and limit batch sizes

• Yield during CF/GC collection

• Optimise some array methods

• Invert expression value relation and move ast types out of value.

• Integrated CRUD benchmarking suite

• Enable query planner on DELETE, UPDATE and UPSERT statements

• Implemented reverse scan for table and range iterator

• Added memory limit threshold check to all statements

• Reduced storage and serialisation cost

• Optimised index iterators: stop scanning after LIMIT

• Added support for multiple writers in Full-Text search implementation

• Added support for range scans on compound indexes (prefix + range)

• Optimised START/LIMIT execution: correct semantics with WHERE/ORDER and early-stop iteration

• Added HTTP connection pooling to maximise throughput

• Removed Force Index mechanism and unify index building across all platforms

• Split AST & Logical Plan into separate components

• Modified allocation tracking behaviour and implementation

Features, additions and syntax improvements:

• Unified lookups for graph & references

• 🆕 Improve GraphQL integration and implementation

• 🆕 New datastore scheme with parameters

• Implemented permissions support for DEFINE API

• Hash-based vector deduplication for HNSW indexes

• Overhauled DEFINE API middleware logic

• Introduced Regex size limit configuration

• Added Filesystem support

• Added cbor encoding and decoding functions

• Added support for distributed sequences

• Added SQL base64 encoding/decoding with & without padding

• Added support for INFO FOR as a subquery

• Added support for filtering down graph selections with a range selection

• Exposed QueryType for live and kill responses

• Added RPC live statement options

• Added LQ Killed notification

• Allowed passing fields to RPC live

• Added support for partial array types in schemafull tables

• Introduced ALTER FIELD, update ALTER TABLE

• Enhanced Full-Text Search with OR Boolean Operation

• Added support for converting to/from protobufs and flatbuffers

• Introduced COUNT indexes for constant-time count() queries

• Introduced composable HTTP router factory for embedders

• Added Support of standard deviation and variance in foreign tables

• Use surrealdb_types::* in SDK and as core's public interface.

• Extended mock syntax to support full range

• Added support for variables in closures and live queries

• Allow access to $input parameter in EVENTS

• Added support for multi-session support in RPC protocol

• Added refresh token support to the RPC protocol and SDK

• New Surrealism plugins

• Added support for client-side transactions

• Memory-bounded LRU cache for HNSW vectors; remove deprecated MTREE index

• Moved strict mode to DB definition, added strict object type

• Added ALTER INDEX statement with PREPARE REMOVE for safe index decommissioning

• Added Default NS & DB

• Introduced a None Type

• Added record::is::edge() function

• Added string::capitalize() function

• Added array::sequence() function

• Added Value::Set / Set type

• Fixed CBOR and add type::of function.

• Added Infinity float value

• Added method syntax for record::is_edge()

• Added array functions for natural and/or lexical sorting

• Added set literal syntax

• Improved parameterisation of statements

• Added Finnish to snowball filter

Bug fixes:

• 🆕 Fix HNSW index memory explosion

• 🆕 Add support for versioned count queries

• 🆕 Reimplement ID generator to resolve concurrency freezes

• 🆕 Fix schemafull index validation for sub-field paths on structured field types

• 🆕 Fix count() AS optimisation ignoring field aliases with GROUP ALL

• 🆕 Fix WebSocket client memory leak in pending request cleanup

• Fixed async event retries to correctly roll back failed executions

• Fixed JWT decoding issues

• Fixed concurrent index builds dropping updates under heavy write load

• Fixed backwards compatibility issues in authentication limits

• Fixed HNSW layer popping logic

• Fixed edge cases in ONLY, OMIT, and SELECT VALUE

• Fixed edge cases in FETCH and ORDER BY

• Fixed ORDER BY behaviour for sets and Unicode characters

• Fixed INSERT RELATION preserving extra fields when using table variants

• Fixed parameterized clause validation

• Fixed Linux Docker builder issues with OpenSSL and ONNX Runtime linking

• Fixed datastore shutdown failures

• Fixed supported SDK version reporting

• Removed unsupported VERSION clause from CREATE and INSERT

• Ensured api::invoke omits execution context where appropriate

• Fixed race condition in Scanner causing duplicate range fetches

• Fixed panic in HNSW vector cache eviction under async contexts

• Fixed LIVE SELECT not returning UUID

• Fixed GraphQL parsing: excluded incorrectly treated as included

• Fixed bug in casting logic

• Fixed idiom continuity not broken by parentheses

• Fixed table serialisation bug

• Fixed LIVE DIFF response structure

• Fixed array slice iterating over own items

• Fixed arithmetic wrap panic in duration functions

• Fixed index out of bounds error

• Fixed overflow and panics in math functions

• Improved ALL/ANYINSIDE behaviour with strings and arrays of strings

• Improved over-fetch documents when LIMIT and START used together

• Fixed Tokeniser fail with successive punctuation characters

• Fixed the default endpoint for exports and imports

• Fixed Graph clauses ->(SELECT ... issue in Rust SDK

• Fixed select with start and limit for arrays

• Fixed allowing null bytes in strings

• Fixed duplicate ID in GraphQL

• Fixed a bug with conversions to union types like array<int> | bool

• Fixed rand::time and other time-related issues

• Canonicalised allowed paths

• Normalised decimal values to fix indexing results

• Fixed concurrent indexing on quoted table names

• Fixed indexing on filtered count results

• Ensured arguments for object idiom methods are computed

• Fixed geometry parsing bug

• Reworked subqueries/covered expressions

• Allowed idiom functions to be restricted by capabilities

• Fixed regex_non_string.surql language-test for UUIDs

• Fixed memory usage and performance of Garbage Collection on old databases

• Fixed method calls on negative numbers

• Fixed JSON multipoint parsing

• Fixed START clause with WHERE; safer optimisation

• Fixed LIVE SELECT parsing with type::table()

• Limit fields in live query delete notifications

• Fixed invalid table view results with sum() returning 0

• Std index shouldn't fail if value already exists

• Fixed potential deadlock in distributed sequence allocation

• Replaced BTreeDocIds with SeqDocIds in M-Tree, remove ANALYSE, use Strand for RecordKey::String

• Enabled referencing fields in default value for defined fields.

• Removed dec when casting decimal to a string, allow string casting from None and Null

• Restricted supported types on id field

• Fixed incorrect ORDER BY LIMIT results on multi-column index scans

• Added notification for caller when a connection is dropped in Rust SDK

• Fixed ignored insert with unique index creating a record when index violated

• Fixed strings, identifiers and escape sequences

• Fixed Windows file path handling in bucket operations

• Fixed WebSocket limits configurable and enforce max message size

• Fixed possible invalid formatting of idioms

• Fixed aliases not being respected in views

• Fixed invalid formatting of statement expressions within general expressions

• Ensured subqueries and blocks have consistent outputs

• Added CONTAINS/INSIDE planning consistent with substring semantics

• Ensured destructuring with computed fields returns correct output

• Improve behaviour of group by and improve performance and robustness of views

• Fixed formatting bugs related to empty identifiers

• fixed the info rpc

• Ensured all statements return a value

• Fixed TableCheck output for TYPE RELATION / NORMAL

• Fixed SELECT with SPLIT so ORDER BY and START/LIMIT apply after splitting

• Added exact hint when path is deprecated

• Fixed Number hash to use consistent value hashing

• Fixed OOM crash in COUNT index on large datasets by batching key retrieval

• Fixed provided vars not being attached to context in surrealism 'sql_with_vars' function

• Fixed literal objects allow unknown keys

Workflow, observability and developer experience improvements:

• 🆕 Improve error communication and serialisation

• Added /data and /logs directories to the Docker container

• Ensured a checksum is generated for Windows binaries

• Added debugger setup instructions

• Improved metrics consistency

• Overwrite by default during data import

• Improve context cancellation checking with adaptive back-off strategy for better OOM protection

• Improved Surrealism Wasm

• Implemented slow logs

• Tokio Console integration

• Added support for log files, json log format, and advanced log span filtering

• Added configurable slow-query logging with parameter allow/deny filtering and normalised single-line output

• Included event errors in output

Storage/index format and protocol:

• Normalised numeric key encoding to unify ordering

• Introduced DecimalLexEncoder for Number

• Used IndexId for Index keys

• Stored keys using IDs for Namespace/Database

• Improved key-value storage engine layer

Security/networking:

• Reject IPs resolving to forbidden net targets

• DNS resolution on Context::check_allowed_net

• Configurable user agent for HTTP requests

Bug fixes:

• Fixed missing and invalid permissions in v3 export

Bug fixes:

• Fixed write-write conflicts in deferred index during concurrent CRUD operations

• Improved context cancellation checking with adaptive back-off strategy

Features & improvements:

• Added --tables-exclude flag to surreal export command

Bug fixes:

• Made v2 exports compatible with v3

• Fixed deferred index queue accounting for concurrent transactions

• Replaced nanoid with thread-local RNG to resolve concurrency freezes

Bug fixes:

• JavaScript engine fixes and improvements

• Fixed HNSW index memory explosion

Security Advisories:

• Denial of Service through scripting function memory edge case

Features:

• Implemented v3 compatible export

Bug fixes:

• Fixed HNSW layer pruning logic

• Updated in-memory caching to avoid storing entries with zero weights

Features:

• Added DEFER keyword for background index building and maintenance (Experimental)

• Added an auth_limit on defined APIs, functions, fields and events, that limits execution to the permissions of the creating user instead of the invoking user.

Security Advisories:

• Confused Deputy Privilege Escalation through Future Fields and Functions

Note: Storable closures in future fields and functions are now disabled by default for security reasons. To enable the previous behaviour, set the SURREAL_CAPS_ALLOW_INSECURE_STORABLE_CLOSURES environment variable. This change addresses the security vulnerability described in Advisory.

Features:

• Added CONTAINS/INSIDE planning consistent with substring semantics; use indexes only for flattened or set-based operators

• Made WebSocket limits configurable

• Added 32-bit floats by default for HNSW vector indexes

• Exposed and tune RocksDB BlobDB options via environment variables

• Aligned RocksDB blob defaults with upstream and make blob compression optional

• Added configurable slow-query logging with parameter allow/deny filtering and normalised single-line output

• Added disk space limits and read-only/deletion-only modes for RocksDB

• Added notification caller when a connection is dropped

• Added COUNT Index Support with Compaction System and Enable Concurrent Index Building for WASM

• Added minimal implementation for 2.0 → 3.0 migration tool

• Ensured reverse scan is used only with compatible datastore

Bug fixes:

• Fixed SELECT with SPLIT so ORDER BY and START/LIMIT apply after splitting

• Fixed OOM crash in COUNT index on large datasets by batching key retrieval

• Fixed insert ignore with unique index still creates a record when index violated

Breaking changes:

• CONTAINS/INSIDE planning made consistent with substring semantics - see language test example in this file with details on using the CONTAINSANY operator with indexes

Bug fixes:

• Fixed incorrect ORDER BY LIMIT results on multi-column index scans

• Fixed broken docs.rs

Bug fixes:

• Added HTTP connection pooling to maximise throughput

• Optimised START/LIMIT execution: correct semantics with WHERE/ORDER and early-stop iteration

• Optimised memory usage by dropping mimalloc

Bug fixes:

• Fixed START clause with WHERE conditions and improve optimisation safety

• Fixed allowing null bytes in strings

• Fixed Live Queries document issue

Improvements:

• Add configurable user agent to HTTP requests

• Support range scans on compound indexes (prefix + range)

• Yield during Change feed garbage collection

• Implemented slow logs for queries

• Added range scans support on compound indexes (prefix + range)

Security advisories:

• Unauthorized Data Exposure via LIVE Query Subscriptions

Bug fixes:

• Ensure session variables are attached to DEFINE API execution context

• The standard index should not fail if a value already exists

Bug fixes:

• Implements DNS resolution on Context::check_allowed_net

Workflow improvements:

• Added support for log files, json log format, and advanced log span filtering

• Improved metrics consistency

• Added support for Docker container timezone customisation

• Ensured errors are caught when checking permissions for LQ

• Ensured NS/DB is set; Ensure transaction is committed or cancelled

Security advisories:

• Bypass of deny-net flags via DNS resolution

Bug fixes:

• TIKV client update to 0.3.0-surreal.3

Workflow improvements:

• Improved metrics consistency

• Reject resolution of IP addresses resolving to denied IP address when doing HTTP request

Bug fixes:

• Fixed memory usage and performance of Garbage Collection on old databases

Workflow improvements:

• Improved import memory usage

• Context cancellation check in prepare mock

• Changed default RocksDB configuration options

• Avoid using commit pool in wasm for skv in-memory

• Set default WS buffer size to 0, reactive RocksDB mapped R/W

Bug fixes:

• Fixed UPDATE permission to CREATE record

• Ensured caches of dependent resources are refreshed on TABLE OVERWRITE

Aligned arbitrary and experimental capabilities

This release resolves an issue identified in v2.3.0 that can corrupt the database when an UPDATE statement is stored within a function. If you experience this issue when migrating to v2.3.0 please kindly remove those function definitions, upgrade to v2.3.1 and then add them back.

Bug fixes:

• Fixed invalid revision on UPDATE statement

• Restored GraphQL, accidentally made inaccessible in the previous release

• Fixes invalid * value for SURREAL_CAPS_ALLOW_ARBITRARY_QUERY and SURREAL_CAPS_ALLOW_EXPERIMENTAL

NB: We identified an issue in this release that can corrupt the database when an UPDATE statement is stored within a function. We released v2.3.1 to address this. If you defined functions with UPDATE statements in v2.3.0 please kindly remove those function definitions, upgrade to v2.3.1 and then add them back.

Features:

• Added support for filtering down graph selections with a range selection

• Improved the parsing behaviour to accept any type of expression or start of idiom path

• Added Finnish to the snowball filter

• Implemented reverse scan for table and range iterator

• Ensured documents are not over-fetched when LIMIT and START are used together

• Added array functions for natural and/or lexical sorting

• Added /data and /logs directories to the Docker container, and set up Docker volumes for those directories by default

• Enabled the query planner on DELETE, UPDATE and UPSERT statements

• Implemented TryFrom<RecordIdKey> for basic types

• Added booleans as literal types

• Added support for ORDER BY DESC on indexes (RocksDB + TiKV)

• Implemented kind regex

• Improved ALL/ANYINSIDE behaviour with strings and arrays of strings

Bug fixes:

• Fixed rand::time and other time-related issues

• Fix the JSON response format

• Made GraphQL avoid duplicate ID fields

• Fixed sub-allocation in TrackAlloc

• Fixed the default endpoint for exports and imports

• Made several flow/timeout improvements

• Fixed tokeniser failure with successive punctuation characters

• Fixed overflow and panics in math functions

• Fixed possible index out of bounds on compound indexes

• Fixed arithmetic wrap panic in duration functions

• Fixed indexing on filtered count results

• Fixed concurrent indexing on quoted table names

• Fixed select with start and limit for arrays

• Enforced a limit to string::replace with regular expressions

• Ensured slicing an array does not iterate over its items

Deprecations:

The following Rust SDK APIs are now deprecated. In addition, some Capabilities methods were renamed and the previous names marked as deprecated.

• IntoQuery::into_query

• IntoEndpoint::into_endpoint

• IntoFn::into_fn

• IntoExportDestination::into_export_destination

• QueryResult::query_result

• QueryResult::stats

• QueryStream::query_stream

• IntoResource::into_resource

• CreateResource::into_resource

• proto::ResponseAction

• remote::WsNotification

Workflow improvements:

• Made the default Rust version explicit

• Allowed publishing multiple nightly crates per day

• Added Trivy scanner runs on new Docker images

• Added more language tests

• Refactored dependencies to use workspace dependencies

Bug fix:

• Fixed Live Queries document issue

Security advisories:

• Unauthorized Data Exposure via LIVE Query Subscriptions

Bug fixes:

• Ensure session variables are attached to DEFINE API execution context

• The standard index should not fail if a value already exists

Bug fixes:

• TIKV client update to 0.3.0-surreal.3

• Implements DNS resolution on Context::check_allowed_net

Workflow improvements:

• Added support for log files, json log format, and advanced log span filtering

• Improved metrics consistency

• Added support for Docker container timezone customisation

• Ensured errors are caught when checking permissions for LQ

• Ensured NS/DB is set; Ensure transaction is committed or cancelled

Security advisories:

• Bypass of deny-net flags via DNS resolution

Workflow improvements:

• Improved metrics consistency

• Reject resolution of IP addresses resolving to denied IP address when doing HTTP request

Workflow improvements:

• Improved import memory usage

• Context cancellation check in prepare mock

• Changed default RocksDB configuration options

Bug fixes:

• Fixed escape issue on index specification

• [Fixed invalid "*" value for SURREAL_CAPS_[ALLOW_ARBITRARY_QUERY | ALLOW_EXPERIMENTAL]](https://github.com/surrealdb/surrealdb/pull/5890)

• Fixed select broken with index on count / where

Bug fixes:

• Check limit to string::replace with regular expressions

• Fixes long running FOR loops, adds script timeouts, improves HTTP redirect security, and optimises task yielding for better performance

• Implements SURREAL_FILE_ALLOWLIST

• Improve error handling on value conversion (RPC / HTTP/SQL endpoint) (#5647)

• Introduces Regex size limit configuration (#5638)

• Backport Allocation tracking - 2.2.x (#5662)

• Add memory limit threshold check to all statements (#5704) (#5736)

• Add deal-store datasets to the upgrade test (#5738)

• Update branch name and pin Rust version for clippy and tests (#5768)

Security Advisories:

• Uncaught exception in Net module leads to database crash

• Server-takeover via SurrealQL injection on backup import

• Memory exhaustion via nested functions and scripts

• Local file read of 2-column TSV files via analyzers

• Memory exhaustion via string::replace using regex

• No JavaScript script function default timeout could facilitate DoS

• CPU exhaustion via custom functions result in total DoS

• Bypass of deny-net flags via redirect results in SSRF

Features:

• Allowed configuring experimental features in Rust SDK and CLI

• Introduced CLI Export customisation

Bug fixes:

• Ensured idiom fetching uses the LQ context

• Refactored capabilities to reduce allocations

• Ensured crates integrity

• Improved DEFINE API implementation

• Fixed DEFINE API stored key

• Improved array::fill

• Ensured NONE values are maintained in arrays

Features:

• Added shortest path queries

• Added support for graph clauses

• Added object::is_empty() function

• Added variable support for shortest path queries

• Added support of keys only logic with index iterators

• Added SURREAL_COUNT_BATCH_SIZE env variable

• Added type::is::range() method and idiom .is_range() method

• Added support for memory threshold limits

• Added environment variables to disable tracing and metrics

• Added support for datetimes for rand::time function

• Added support of compound indexes on AND queries

• Enabled allocation tracking

• Added DEFAULT ALWAYS clause

• Implemented FETCH for live queries

Bug fixes:

• Fixed Index is corrupted - TreeNode out of bounds

• Enabled ANSI code support on Windows

• Fixed (-$param) failed bug in Parser

• Fixed index_hnsw bench

• Allowed escape characters inside mathematical angle bracket escaped idents

• Fixed allocator on Windows

• Fixed parse buffer not properly growing

• Reintroduced id selection behaviour

• Fixed design for live queries cache

• Fixed array::transpose() not adding NONE at non-existent indexes when transposing arrays of varying length

• Fixed concurrent index creation with overwrite

• Ensure INSERT IGNORE does not throw error

Optimisations:

• Introduced language testing suite

• Added cross-transaction caching to database and table statements

• Disallow mismatched field types

• Skip processing documents when START is specified

• Added minor optimisations for SELECT count() FROM table GROUP ALL queries

• Implemented priority queue for queries with ORDER and LIMIT

• Improved error messages for HTTP database functions

• Reworked query streaming and error reporting

• Disabled PARALLEL option

• Added file URL handling and tests for store initialisation

• Added default limit for max open files with rocksdb storage engine

• Removed -log trace from Docker instructions

• Increased Rust memory allocation, parallelism information and RocksDB block cache size

• Added initial support for improved retryable transaction errors

• Simplified field processing logic

• Optimised transaction multi-get function interface

• Optimised RocksDB key-value gets

• Added stdout and stderr log splitting

• Added tests for other SDKs in CI

• Moved all experimental features to experimental capabilities

• Streamlined FoundationDB features

• Improved RocksDB block cache configuration

• Remove naive concurrency rate limiter in WebSocket

• Added dedicated threadpool for storage engine operations

• Skip strategy down to scanner for START queries

• Allow chaining path operations

• Implement support for RPC protocol versioning

• Switch to tokio::mpsc from async_channel in WebSocket

• Allow configurable background flush syncing

SurrealKV:

• Expose surrealkv config options as env variable

• Enable data durability configuration on RocksDB and SurrealKV

Workflow improvements:

• Fixed nightly version prefix

• Added Tx::count(), Tr::count(), and kvs::count() methods

• Fixed make check and cargo make ci-check-wasm commands

• Made all count/group use transaction::count method

• Documented core as internal only and remove doc(hidden)

• Added support for minimum number of tokio worker threads

• Removed development panic backtrace on transaction drop

• Fixed clippy warnings

• Added Compute args for idiom methods

• Ensured Windows builds have a default allocator

• Optimised Options and Auth core engine internals

• Added error logging for make sql and make repl

• Added Goodbye message to terminal before exiting

• Improved Issue templates

• Fixed GitHub issue labeling

Experimental:

• Changed bearer key storage and verification to only use hashes

• Implement refresh tokens for record users

• Added functions to GraphQL

• Added Record references to GraphQL

Breaking changes:

• array::transpose() now adds NONE at non-existent indexes when arrays vary in length.

Bug fixes and improvements:

• Added nu-ansi-term dependency

• Fixed Live Queries document issue

Security advisories:

• Unauthorized Data Exposure via LIVE Query Subscriptions

Bug fixes:

• TIKV client update to 0.3.0-surreal.3

• Implements DNS resolution on Context::check_allowed_net

Workflow improvements:

• Added support for log files, json log format, and advanced log span filtering

• Improved metrics consistency

• Added support for Docker container timezone customisation

• Ensured errors are caught when checking permissions for LQ

Security advisories:

• Bypass of deny-net flags via DNS resolution

Workflow improvements:

• Reject resolution of IP addresses resolving to denied IP address when doing HTTP request

Workflow improvements:

• Improved import memory usage

• Context cancellation check in prepare mock

• Changed default RocksDB configuration options

Bug fixes:

• Check limit to string::replace with regular expressions (#5636)

• Fixes long running FOR loops, adds script timeouts, improves HTTP redirect security, and optimises task yielding for better performance

• Fix import injection security bug

• Backport JS scripting mem exhaustion fix

• Implements SURREAL_FILE_ALLOWLIST

• Improve error handling on value conversion (RPC / HTTP/SQL endpoint) (#5647)

• Introduces Regex size limit configuration

• Backport Allocation tracking (#5663)

• Add is_beyond_threshold_check (#5667)

• Add deal-store datasets to the upgrade test

• Add memory limit threshold check to all statements (#5704) (#5737)

Security advisories:

• Uncaught exception in Net module leads to database crash

• Server-takeover via SurrealQL injection on backup import

• Memory exhaustion via nested functions and scripts

• Local file read of 2-column TSV files via analyzers

• Memory exhaustion via string::replace using regex

• No JavaScript script function default timeout could facilitate DoS

• CPU exhaustion via custom functions result in total DoS

• Bypass of deny-net flags via redirect results in SSRF

Bug fixes:

• Support escape sequences inside idents escaped by mathematical angle brackets

Security advisories:

• Silent failure to overwrite table definition of relation type

Bug fixes:

• Added support for surrealkv+versioned in the binary

• Improved server termination handling, cleanup and responsiveness to pings

• Improved concurrency with long running queries

• Fixed parallel iteration support

• Ensured the Rust SDK can run queries against v2.0 servers

Bug fixes:

• Permission check should not generate an error when the table does not exist

• Document running in Docker as a non-root user

Bug fixes:

• Fixed behaviour on NONE and NULL targets

• Fixed live queries issue via the rpc method

• Simplified unnecessary versioned puts

• Check permissions on keys-only optimisation

SurrealKV:

• Disable versions by default

• Update SurrealKV to 0.6.1

Workflow improvements:

• Moved sub-crates into their own directories

• Ignore flaky transaction conflict tests

• Address Cargo Deny findings

• Fixed core and SDK paths

• Fixed benchmark workflow

Features:

• Recursive idiom/graph traversal

• Add cross-transaction caching to the document processing layer

• Added dictionary-based lemmatisation to DEFINE ANALYZER statement

• More static values now possible after DEFAULT clause

• type::field() and type::fields() no longer restricted to just SELECT statements

• Now possible to configure what to export

• Allow data to be imported when server starts

• Added custom storage engine shutdown behaviour

• Versioned records now available in exports

• Enabled configuration of data in exports

• array::fold() and array::reduce() methods

• New string::distance and string::similarity functions for various algorithms

• Improvements to experimental bearer access

• New capabilities to limit HTTP and WebSocket APIs

• (Rust SDK) Allow constructing Datetime, better UUID support

• (SurrealKV) Versioning enabled on edges and relations

• Object Improvements

• Accept computation in VERSION clause

• SDK export config

• Add AND EXPUNGE clause to certain REMOVE statements

Performance improvements:

• Split collecting/processing + multithreading

• Remove execution-time parsing from RPC protocol

• SurrealKV now used for in-memory storage engine

• In-memory parallel sorting

• Efficient incremental ordering

• Added cross-transaction caching to document processing layer

• Query streaming for imports

• Improved performance when reduced document scopes not needed

Bug fixes:

• Fixed incorrect error message

• Fixed field not being removed when set to NONE

• Fixed panic caused by invalid UTF-8

• Fixed panic during parallel query calls in scripting functions

• Fixed bug in random ordering when using files as storage

• Fixed bytes serialisation

• Fixed issue with lacking unicode escape sequences

• Fixed incorrect future CBOR decoding

• Fixed overly nested graph query output

• Fixed unexpected behaviour for optional operator

• Fixed OTEL endpoint

• Fixed ignoring of INDEX IGNORE

• Fixed panic when ordering by rand

• Fixed --auth-level flag not requiring necessary --username and --password

• Fixed like operator parsing error

• Fixed invalid exports containing array<any>

• Fixed string-like record ID parsing

• Gracefully handle Option<T> in rand::time()

• Allow customising record exporting

• Better handling of unexpected authentication errors

• Improve error messages for negative arithmetic + try_reserve

• Ensure LIMIT and PARALLEL can be used together

Workflow improvements:

• Benchmark to compare indexed with non-indexed queries

• Improved error message when permissions insufficient

• RocksDB storage engine config options now shown on startup

• Tests to prevent regressions after permission fixes

• Use new runner types in benchmark jobs

• Update dependencies with known security vulnerabilities

• Update rquickjs to newest version

Features with breaking changes:

• Improved UPSERT behaviour

Bug fixes:

• Check limit to string::replace with regular expressions

• Fixes long running FOR loops, adds script timeouts, improves HTTP redirect security, and optimises task yielding for better performance

• Backport JS scripting mem exhaustion fix

• Introduces Regex size limit configuration

• Improve error handling on value conversion (RPC / HTTP/SQL endpoint)

• Update rquickjs to newest version

Security advisories:

• Uncaught exception in Net module leads to database crash

• Server-takeover via SurrealQL injection on backup import

• Memory exhaustion via nested functions and scripts

• Memory exhaustion via string::replace using regex

• No JavaScript script function default timeout could facilitate DoS

• CPU exhaustion via custom functions result in total DoS

• Bypass of deny-net flags via redirect results in SSRF

Performance improvements:

• Added lower and upper bounds for RocksDB iterators

• Added with or without index benchmark

• Improved behaviour and performance of reduced document scopes

Bug fixes:

• Fixed panic when calling Thing::try_from("") - with empty value

• Fixed parsing of functions and models in selectors

• Fixed clippy warnings in release build

• Fixed micro second parsing

Workflow improvements:

• Ensure unstable code is also checked in CI

• Update versions actively supported for security fixes

Security advisories:

• Improper Authorisation in Select Permissions

• Uncaught Exception Handling Parsing Errors on Empty Strings

Features:

• Allowed transactions during imports

• Deprecated DELETE on field permissions

• Implemented Ord for Duration and Table

• Improved Number ordering accuracy

• Made max_value_size configurable in surrealkv

Parser improvements:

• Update reblessive

• Fixed digit object name export

Bug fixes:

• Fixed record IDs not being used as Object keys

• Fixed ID type conversion

• Fixed thing parsing function

• Removed truthiness check on Datetime that requires after epoch dates

• Reintroduced meta:: scope as deprecated methods

• Fixed idiom path processing

• Fixed value conversion to a vector: NONE

Workflow improvements:

• Update SurrealKV version to 0.3.7

• Update rocksdb 3rd party dependency

• Create major.minor and major tags when pushing to Docker

• Update surrealcs dependency

Bug fixes:

• Fixed fields being parsed as idioms.

• Fixed error message panics

• Fixed Incorrect lowercase filter in analyzer

• Fixed mismatch in record id for edge tables

• Fixed immediate insert subqueries

• Fixed single check for rpc insert

• Fixed decimal parsing

• Fixed failed authenticate resulting in session reset

• Fixed version patching

• Fixed query planner ignoring isolated predicates in WHERE clause

• Fixed inconsistent numeric object key

• Fixed ULID/UUID gen, programmatically generating ranges, RETURN inside FOR/IF and improves arithmetic operations

• Fixed CBOR Range encoding

• Fixed parsing bug in graph for bidirectional relationships

• Fixed compilation issue on 32bits platforms (other than wasm32)

• Fixed setting a range id on a document's id, in and out fields

• Fixed max key size for surrealkv store

• Included request id in CBOR encoding failure message

• Fixed panic when the query file size is too large.

• Fixed bug with number ordering

Improvements:

• Improved log messages for authentication errors

  • Explicitly check for token expiration error

• Added GraphQL description exactly matches the comments set by the user

• Added configuration capabilities for SurrealCS connection pool size

• Added test for including an ID in the content body

• Added minor improvements to core engine

• Added VERSION clause to all queries

• Added additional ULID and UUID utilities

• Added DTrace profiling support

Bug fixes:

• Fixed infinite loop in parser

After numerous alpha and beta releases over months of development, we are releasing SurrealDB v2.0.0! 🎉. The changes for this release pertain to those in between the final 2.0.0-beta-3 and other alpha versions.

NOTE: Because of changes in the underlying way SurrealDB stores data, steps are required to migrate to 2.0.0 from previous versions. Please use the following steps for any existing databases:

• If you are moving from 1.x to 2.x, use the surreal fix command.

Features:

• Enable FoundationDB transaction timeout customisation

• Implement missing modulo operator in new parser

• Restructure the parser around the concept of token gluing.

• Added DEFINE ACCESS statement to grant access to resources.

• Deprecated DEFINE SCOPE in favour of DEFINE ACCESS ... TYPE RECORD.

• Deprecated DEFINE TOKEN in favour of DEFINE ACCESS ... TYPE JWT.

• Deprecated DEFINE TOKEN ON SCOPE in favour of DEFINE ACCESS ... TYPE RECORD ... WITH JWT.

• Added customisable algorithm and key when issuing tokens for records users with the WITH ISSUER clause.

• Added customisable token and session duration to DEFINE USER and DEFINE ACCESS.

• Removed session expiration when not explicitly defined.

• Changed the INFO statement to redact secrets defined via DEFINE ACCESS.

• Changed HTTP request headers expected by SurrealDB to require the surreal- prefix.

• Removed the --auth flag in the CLI to enable it by default. Added --unauthenticated.

• Removed the --enable-auth-level flag in CLI to enable the behaviour by default. Defaults to root.

• Changed authentication to expect level via the --auth-level flag or surreal-auth-* headers.

• Changed the default --bind address in the CLI from 0.0.0.0 to 127.0.0.1.

• Added the --no-identification-headers flag in the CLI to limit information leakage.

• Added type::range function.

• Added TEMPFILES clause to the SELECT statement.

• Introduce resource creation checks for DEFINE statement to avoid duplicates.

• Added string functions for IP string::is::ip, string::is::ipv4, string::is::ipv6.

• Introduce resource creation checks for DEFINE statement to avoid duplicates.

• Added string functions for IP string::is::ip,string::is::ipv4,string::is::ipv6.

• Added string functions for HTML: string::html::encode, string::html::sanitize.

• Added new math functions: math::acos, math::acot, math::asin, math::atan, math::clamp, math::cos, math::cot, math::deg2rad, math::lerp, math::lerpangle, math::ln, math::log, math::log2, math::log10, math::rad2deg, math::sign, math::sin, math::tan.

• Support batch import with INSERT statement. This also extends to relationships between tables using the INSERT RELATION statement

• Allow bridge connectivity in all docker images built

• Added define JWT access at the root level

• Allow authenticating with a token from the CLI

• Added Asynchronous building for indexes

• Improved HNSW persistence

• Added Range value support

• Added ability to SELECT with version, CREATE with version and INSERT with version on SurrealKV

• Added ENFORCED keyword to DEFINE TABLE

• SCHEMAFULL tables are now TYPE NORMAL by default

• Added literal types to the query language

• Added the .chain() method

• Added typed LET statements

• Added typed function returns

• Added record::exists() function

• Added new functions value::diff() and value:::patch()

• Array function changes and additions: array::all(), array::any(), array::filter_index(), array::find_index() functions can now take a value or a closure. Also added array::filter() and array::find() functions.

• Upgraded SurrealKV to v0.3

• More flexible create API for Rust SDK

• Query depth limits can now be configured

• Added GraphQL config

• Indexing operators can now take expressions

• Support for mixed number types on range queries

• Improved capabilities logic with specific allows able to override general denies

• "false" string no longer eagerly evaluated as false

• time::epoch constant now available

• Added geo::is::valid() function

• Method call syntax now works on integers

• Improved GraphQL record fetching

• Added AUTHENTICATE clause on RECORD access

• Enabled publishing SurrealDB ports in Docker for bridge networking connectivity

• Allow generating ULID and UUID based on a timestamp

• Allow unsetting namespace and database on RPC protocol

• Moved fetch resolution to a compute method

• Added ALTER TABLE statement

• Moved AUTHENTICATE clause to the root of DEFINE ACCESS statement to allow the clause to be used in every access method

• Added OVERWRITE clause for defined resources

• Added array::windows() function

• Added support for idiom destructuring in SurrealQL

• Added string::is::record function

• Added Method chaining for Database functions

• Added anonymous functions

• Added typed function returns

• Function return types now shown in INFO statement

• Type any can now act as an optional type

• Added time::is::leap_year() function

• Added environment variables to configure HTTP endpoint limits & JavaScript runtime limits

• Introduced savepoint capabilities

• Added experimental VERSION clause to INFO FOR DB and INFO FOR TABLE

• Added additional chained methods to number values

Performance improvements:

• SELECT/count optimisation on table range

• Improvements to node membership processing and garbage collection

• Improvements to select count table scan

• Avoid potentially some methods twice

• Stabilise and use parser2 for parsing.

• KNN filtering with limit and KNN distance function.

Bug fixes:

• Fixed incorrectly parsed identifier characters

• Fixed record<T> casting from string

• Connection ID now stored inside session data

• Fixed certain identifier characters not being parsed correctly

• Fixed remote fields added to output result inside SELECT * queries

• Revise RecordAccess structure to aid v2 alpha upgrade

• Fixed GHSA-gh9f-6xm2-c4j2: Improper Authentication when Changing Databases as Scope User

• Fixed argument check for array::clump

• Fixed aborting task leading to an error

• Fixed non-enforced field permissions on WHERE statements

• Fixed failing search when search analyzer lacks tokenisers

• Fixed core dump when performing INSERT

• Fixed discarding without error when using RELATE statement on edge that already exists

• Fixed modification of arrays with JSON Patch semantics

• Fixed selecting directly from a graph edge failing to select all primary edges

• Fixes to issues with composite indexes

• Fixed INSERT not working in subqueries

• Fixed KNN brute force error when an index exists.

• Fixed Redefining HNSW index error.

• Check that a field exists before index creation.

• Fixed missing serialisation for math::inf.

• Fixed polygon serialisation.

• Fixed panic and infinite loop in parser.

• Fixed URL parsing function when a port is present.

• Fixed display formats for access methods

• Fixed parser error to allow ns as an alias for NAMESPACE

• Added missing datetime prefix

• Removed duplicated badge for docker pulls count in README.md

• Fixed export for sql::filter::Filter type in surrealdb-core

• Updated deprecated version of Node.js

• Fixed ELSE IF pretty formatting

• Changed geometry point values to floats in CBOR

• Fixed authentication checks on root user websocket RPC session

• Ensured a transaction is always available when live queries are evaluated

• Reduced read lock duration while processing connection messages

• Ensured execution halts after the RETURN statement

• Ensured the server shuts down properly

• Fixed querying complex edge table

• Removed unnecessary backticks from INFO FOR statement output

• Fixed live query panic issue

• Fixed validation on schemafull RELATE

• Fixed division by zero bug in math::mean() function

• Improved string::is::datetime and string::is::uuid

• Parse like (~) operator correctly

• Removed faulty flatten in graph traversal

• Fixed record<T> casting from string

• Removed ASSERT for type option<T> and value NONE

• Fixed aud field on a JWT claim decoding

• Removed unneeded return in matching

• Disabled default audience validation for JWKS

• Fixed errors in supply chain security workflow

• Enabled check for single and multiple token audiences

• Removed panic on invalid range scan

• Added table to reserved keywords

• Concurrent indexing no longer fails if record updated during build

• Unreachable code now returns an error

• DEFINE EVENT defaults to true when no WHEN clause specified

• Fixed bug in which NONE and NULL both < 1 and > 1

• Fixed inconsistency on queries using equality with or without index

• Create on rpc which returned an array now returns an object. Similar change made to Rust SDK

• Rust SDK fix retrieving multiple columns from the same response

• Improved consistency for surrealdb.query() and surrealdb.value() in JavaScript runtime

• Fixed ENFORCED keyword not displaying in INFO statement

• Fixed GHSA-64f8-pjgr-9wmr: Untrusted Query Object Evaluation in RPC API

• Improved math::mean() to support returning floats

• Improved namespace/database validation for GraphQL

• Ensured TIMEOUT clauses are processed correctly

• Fixed importing from 1.x

• Deprecated missing format inference on RPC protocol

Workflow improvements:

• Deprecated file://

• Added surreal fix command

• Exports no longer use explicit transactions

• /lib path renamed to /sdk

• Rename the next workflow to alpha

• Improved structured output for INFO statement

• Added database upgrade test from 1.5.x to 2.0.x

• Improved InfoStructure implementation on Ident to return the raw value

• Improved placeholder text when filling a bug report

• Separate the supply chain security actions to its workflow

• Improve Cargo local testing

• Limit Cargo build jobs during tests with coverage

• Improved type safety in router API

• Improved hashing mechanism to aid MTree benchmarking

• Removed unnecessary cloning of DefineStatementAnalyzer

• Simplified string formatting

• Refactored transaction, caching, and key-value store interfaces

• Imported entry enum in src/sql/mod.rs

• Extracted duplicated code for storing interfaces to initialize_store

• Resolved a TODO: map_err to inspect error

• Improved pull request template

• Added Improvements to kvs module layer

• Updated rustc version to 1.80.0

• Enabled the ml feature in nightly and alpha releases

• Removed limit to Cargo jobs

• Restored bench-sdk-surrealkv

• Upgraded to http/hyper 10

• Upgrade upstream jsonwebtoken crate

• Bumped SurrealKV version to 0.3.1

• Added aliases for complex feature flag combinations

• Improved RocksDB range scans

• Improved Serialisation for sql::Value

• Improved parser error type

• Added Support for SurrealCS

• Bumped async graphql version

• Improved transparent API

• Fixed cargo dependency loading issue

• Improved the logic for migrating IDs in surreal fix

API improvements:

• Decoupled capabilities from the public API

Surrealkv API improvements:

• Changed disk format for compaction in Surrealkv

• Removed global static Datastore

• Added versioning to SELECT * FROM table queries

Parser improvements:

• Added support for the s string prefix in parser

• Fixed parser panic when a dot follows a string in idiom-allowed contexts

• Ensured variables after the FETCH clause are parsed

• Ensured UUIDs are parsed correctly

• Fixed floating point parsing bug

• Updates reblessive dependency to newest version

• Removed flume as a dependency in favour of async-channels

• Fixed warnings and simplified connection trait

Breaking changes:

• Functions beginning with meta:: renamed to record::

• Functions string::endsWith() and string::startsWith() renamed to string::ends_with() and string::starts_with()

• Changed UPDATE statement behaviour of creating records in favour of new UPSERT statement.

Experimental features:

• Added BEARER access type and its basic grant management

Features:

• Created v1, v1-dev, v1.5 and v1.5-dev Docker tags

Bug fixes:

• Fixed an issue with object keys starting with a digit not being properly exported

Bug fixes:

• Fixed GHSA-64f8-pjgr-9wmr: Untrusted Query Object Evaluation in RPC API

Bug fixes:

• Allowed fields production to end a subquery

• Fixed GHSA-gh9f-6xm2-c4j2: Improper Authentication when Changing Databases as Scope User

Features:

• Added an --alpha flag to surreal upgrade.

Bug fixes:

• Fixed KNN brute force error when an index exists.

• Fixed redefining HNSW index error.

Bug fixes:

• Fixed incorrect computations on aggregations within foreign tables

• Fixed surreal upgrade --nightly

Bug fixes:

• Make temporary table active only if the temporary directory is set

• Fixes compilation issue related to temporary-directory when the feature sql2 is not enabled

Features:

• Added an implementation of HNSW in memory.

• Added a REBUILD INDEX statement.

• Added variable support in CONTENT clause for RELATE statements.

• Added more information to INFO FOR SCOPE.

• Added a relate method to the RPC protocol.

• Added an INFO STRUCTURE statement.

• Stabilised sql2 and jwks features.

• Introduced an on-disk temporary table.

• Added a run method to the RPC protocol.

• Implemented limits for parsing depth in the new parser.

• Implemented reblessive into the parser to prevent any overflows.

• Increased the Minimum Supported Rust Version (MSRV) to 1.77.

Bug fixes:

• Made the query planner recognise the exact operator (==).

• Fixed math:min in foreign tables.

• Fixed typo in function math:sum: was math::sun.

• Made record IDs more flexible in the new parser.

• Reverted changefeed polling frequency back to 10 seconds.

• Fixed a problem with date-time parsing.

• Fixed response content-type on /rpc endpoint.

• Fixed decimal decoding.

• Made the query planner support the IN operator.

• Made the JWKS implementation more RFC 7517 compliant.

• Fixed wrong count when using COUNT with a subquery.

• Fixed the content type header on a CBOR HTTP response.

• Fixed $value being NONE for DELETE events.

• Fixed array::group in a group by query.

Performance improvements:

• Added a query planner strategy for record links.

• Made TreeCache more efficient on writes.

• Reduced byte size of datetime and UUID types using CBOR format.

Bug fixes:

• Fix problems with if in identifiers after remove and define statements.

• Fix $value being NONE for DELETE events.

• Fix CBOR headers.

• Fix wrong count when using COUNT with a subquery.

• Fix IN operator should be recognised by the query planner.

• Fix response content-type on /rpc endpoint.

• Fix problems with if in identifiers after remove and define statements.

• Fix $value being NONE for DELETE events.

• Fix CBOR headers

• Fix wrong count when using COUNT with a subquery

• Fix IN operator should be recognised by the query planner

• Fix response content-type on /rpc endpoint

When using the Rust SDK, make sure surrealdb-core is up-to-date. If not, you might get back Revision errors as a result.

Features:

• Added a warning message about debug builds in CLI start.

• Moved JWKS cache storage to memory.

• Overhauled force implementation.

• Added DEFINE TABLE ... RELATION.

• Allowed RPC calls to be made over an HTTP connection, not just WebSocket.

• Allowed the highlighter to only highlight the matching searched token rather than the whole term.

• Added an INCLUDE ORIGINAL option to change-feeds.

• Added an insert method to the Rust SDK to allow bulk inserts.

Bug fixes:

• Fixed export generating unparsable code with the new parser.

• Eliminated a potential panic in MsgPack format implementation.

• Fixed string::is::longitude RegEx.

• Improved CBOR decoding.

• Fixed relation type parsing.

• Fixed handling of empty array on index.

• Allowed legacy headers in CORS.

• Allowed surreal upgrade to detect when upgrading to the same version and return early.

• Fixed certain environment variables to allow configuration at runtime.

• Reduced the byte size of datetime and uuid types using CBOR format.

• Fixed array::group in a group by query.

Performance improvements:

• Improved query aggregation handling.

Bulk insert support in the Rust SDK

This was one of the frequently requested features. While this was already possible using the query method, this release adds an insert method that makes it more convenient.

db.insert(Resource::from("person"))
    .content(vec![
        User {
            id: thing("person:tobie")?,
            name: "Tobie",
            settings: Settings {
                active: true,
                marketing: false,
            },
        },
        User {
            id: thing("person:jaime")?,
            name: "Jaime",
            settings: Settings {
                active: true,
                marketing: true,
            },
        },
    ])
    .await?;

Bug fixes:

• Add JWT ID claim to tokens issued by SurrealDB.

• Consistently enforce session expiration.

Features:

• Introduced IF NOT EXISTS clause on DEFINE statements.

• Implemented IF EXISTS for additional REMOVE statements.

• Changed the KNN operator from <2> to <|2|> in the new parser and added support in the old parser for the new syntax.

• Implemented WebSocket session expiration.

• Added support for tables and geometries in CBOR.

• Added support for parsing decimal numbers with scientific notation.

• Added support for printing a warning in the CLI when using an outdated version of SurrealDB.

• Added Surreal::wait_for to make it possible for the client to wait for certain events.

• Added SurrealKV as an unstable feature.

• Added more error message improvements for the new parser.

Bug fixes:

• More consistent handling of negative numbers in record IDs.

• Removed the unimplemented backup command from the CLI.

• Fixed document not being available to delete permissions clause.

• Ensured objects properties are recognised by the query planner.

• Implemented the union strategy on unique indexes.

Performance improvements:

• Added compile-time feature for flamegraph and pprof profiling.

IF NOT EXISTS clause on DEFINE statements

DEFINE statements now allow you to include an IF NOT EXISTS clause which ensures that the define statement is only run if the resource doesn't already exist. If it exists, you will get an error. Learn more about this in the documentation.

DEFINE DATABASE IF NOT EXISTS example

New KNN syntax

The KNN operator now supports a new syntax using <| and |> instead of < and >. Learn more about this in the documentation

SELECT id FROM point WHERE point_b <|2|> [2, 3, 4, 5]

Waiting for client events

It's now possible to initialise the connection and run setup actions like authentication and selecting the database concurrently with other queries by making it possible to wait for the SDK to connect or select the database to use before allowing other queries to execute.

Something like this would be susceptible to a race condition before:

// A static, global instance of the client
static DB: Lazy<Surreal<Any>> = Lazy::new(|| {
    // Connect and setup the database connection in the background
    tokio::spawn(async {
        // This example is for demonstration purposes only.
        // If the `unwrap` panics the client will be left hanging,
        // waiting for the setup to happen when using `Surreal::wait_for`.
        setup_db().await.unwrap();
    });
    // Initialise a new unconnected instance of the client
    Surreal::init()
});

// Connect to the server and run setup actions
async fn setup_db() -> surrealdb::Result<()> {
    // Connect to the server
    DB.connect("ws://localhost:8000").await?;

    // Signin as a namespace, database, or root user
    DB.signin(Root {
        username: "root",
        password: "root",
    }).await?;

    // Select a specific namespace / database
    DB.use_ns("namespace").use_db("database").await?;

    Ok(())
}

// Whether this would run successfully or return an error would depend on whether
// `DB::setup` was able to execute before getting to this point.
DB.create(Resource::from("person")).await?;

Now it's possible to make the client wait for either the connection to be established

use surrealdb::opt::WaitFor::Connection;

// This will wait until the connection is established.
// As soon as it is, it will return and allow the client to continue.
DB.wait_for(Connection).await;

// This will still likely return an error.
// While we know that the connection is already established at this point,
// there is no guarantee that the database has already been selected.
// This query requires the database to be selected first.
DB.create(Resource::from("person")).await?;

or the database to be selected.

use surrealdb::opt::WaitFor::Database;

// This will wait until the database is selected.
// As soon as it is, it will return and allow the client to continue.
DB.wait_for(Database).await;

// At this point, the database is guaranteed to be selected already
// so this query should run successfully.
DB.create(Resource::from("person")).await?;

Bug fixes:

• Ensure relevant indexes are triggered when using IN in a SELECT query.

• Ensure the query planner accepts Datetime and Uuid values.

• Restore cosine distance on MTree indexes.

• Fix regression in index data serialisation.

• Ensure rquickjs builds don't stall on macOS.

Bug fixes:

• Fix an issue with WHERE clause on queries resolving record links or graph edges.

• Fix MATH::SQRT_2 not parsing.

• Fix a panic in span rendering.

• Fix CLI output not displaying properly sometimes.

Features:

• Bump MSRV to 1.75.

• In-memory index store.

• Show execution time in CLI.

• knn brute force.

• Implement support for remote JSON Web Key Sets.

• Add support for LIVE SELECT in the SDK and CLI.

• Add IF EXISTS to REMOVE TABLE statement.

• Add READONLY keyword to DEFINE FIELD statement.

• Add alias -V and --version flags for surreal version command.

• Define types for subfields when defining array fields.

• Add string::semver::compare, string::semver::major, string::semver::minor, string::semver::patch, string::semver::inc::major, string::semver::inc::minor, string::semver::inc::patch, string::semver::set::major, string::semver::set::minor, string::semver::set::patch methods.

Bug fixes:

• Make record id string parsing never fail early.

• Prevent overflow in math::power().

• Fix error message pointing to wrong character.

• Respect alias for dynamic field queries with type::field.

• Remove min/1000 in tx:delp.

• Replace custom JWT parser causing decoding issues.

• Issue with scoring on complex queries.

• Limit recursion depth when parsing nested RELATE statements.

• Fix a bug where a non-empty list parser would parse empty lists.

• Ensure an attempt to set a protected variable returns an error.

• Fix duration addition in timeout causing overflow.

• Panic invoking parameters and functions without a database.

• Fix a bug where the kind parser would eat whitespace.

• Fix WebSocket notification sending format.

• Implement missing errors for missing clauses on DEFINE-statements.

• Ensure advanced DEFINE PARAM parameters are computed correctly.

• Ensure path idioms are correct when looping over.

• Fix json failing to parse with trailing whitespace.

• Fix the five second delay in Wasm after initial connection.

• Add context to live query notifications.

• Fix the modulo operator on sql2 and parser1.

• Improve the js-surql value conversion for numbers.

• Implement revision types for client/server communication.

• Fix builtin error pointing to the wrong part.

• Fix a panic when invalid builtin function names are passed to the executor.

Performance improvements:

• Ensure compression is only enabled when response is a certain size.

• In MTree large vector improvement.

Use JWKS to dynamically configure your token definitions

-- Specify the namespace and database for the token
USE NS abcum DB app_vitalsense;

-- Set the name of the token
DEFINE TOKEN token_name
  -- Use this token provider for database authorization
  ON DATABASE
  -- Specify the JWKS specification used to verify the token
  TYPE JWKS
  -- Specify the URL where the JWKS object can be found
  VALUE "https://example.com/.well-known/jwks.json"
;

Define a READONLY field

DEFINE FIELD created ON resource VALUE time::now() READONLY;

Define types for subfields when defining array fields

DEFINE FIELD foo ON bar TYPE array<number>
-- Where <number> is a subfield of the array type.
-- It now defines the number subfield automatically instead of having to run this manually:
DEFINE FIELD foo.* ON bar TYPE number

Bug fixes:

• Fix WebSocket notification sending format.

• Fix missing custom claims from token parameter.

• Fix URL encoding in JS functions.

• Fix panic when invoking parameters and functions without a database.

Features:

• The type::is::record() function now accepts a second optional table argument, validating the record being stored on the passed table.

• Add time::micros(), time::millis() and time::from::nanos functions.

• Add type::is::none() function.

• Add object::entries(), object::from_entries(), object::len(), object::keys() and object::values() functions.

• Clean paths in the start command and honour ~.

• CLI: Split results by comment.

• Add surreal sql welcome message.

• Add Debugging env flag: SURREAL_INSECURE_FORWARD_SCOPE_ERRORS.

• Add SURREAL_ROCKSDB_KEEP_LOG_FILE_NUM environment variable (default 20).

• Support auth levels for basic auth (behind feature flag)

• Add remainder/modulo operator.

• Implement string prefixes: s, r, d and u.

• Add ability to cast string to a Thing/Record ID.

• Analyzers to support functions.

• Support of subfields for embedding indexing.

• Add live query API to Rust SDK.

• Add Query::with_stats() to return query statistics along with the results.

• Permissions are now always displayed for visibility

• Add a --beta flag to surreal upgrade to make installing the latest beta release easier.

Bug fixes:

• Fix stack overflow in graph traversal.

• Bugfix - parse error for invalid leading whitespace.

• Fix memory leak caused by OTEL callbacks.

• Fix wrong function name export and function name parsing.

• The position of the LIMIT and ORDER clauses are now interchangeable.

• Fix index plan for idiom param value.

• Fix bug where error offset could underflow.

• Query results should be consistent and representative.

• Indexes used with the operators CONTAINS [ ANY | ALL ].

• Forward custom thrown errors in SIGNIN and SIGNUP queries.

• Fix ORDER BY RAND() failing to parse when selecting specific fields.

• Fix identifiers which look like numbers failing to parse.

• Change math::median indexing for even length arrays.

• Pass IP & Origin onto session used by scope queries.

• Fix possible corruption of MTree and incomplete knn.

• Allow array::flatten() to be used as an aggregate function.

• Make SELECT ONLY deterministic.

• Optional function arguments should be optional.

• Default table permissions should be NONE

• Bugfix: Fix inconsistent record parsing

• Fix time regression in surrealdb.wasm binaries.

• Fix computing futures in query conditions.

• Fix issue with scoring on complex queries.

• Fix ML support on Windows and enable the feature in Windows binaries.

• Replace the custom JWT parser causing decoding issues.

• Ensure compression is only enabled when response is a certain size.

• Respect alias for dynamic field queries with type::field().

• Prevent overflow in math::power().

• Fix error message pointing to wrong character.

• Expand logic for static value validation to improve DEFAULT clause handling.

• Fallback to a string when record ID parsing fails.

• Ensure an attempt to set a protected variable returns an error.

• Fix duration addition in timeout causing overflow.

• Fix a bug where a non-empty list parser would parse empty lists.

• Limit recursion depth when parsing nested RELATE statements.

• Ensure REMOVE statement does not only remove the first 1000 keys.

• Fix BTree deletion bug.

• Replace close method on live::Stream with a Drop trait implementation.

Performance improvements:

• Enable compression on the HTTP connector.

• Make REMOVE [ TABLE | DATABASE | NAMESPACE ] faster for TiKV and FoundationDB.

• Repetitive expressions and idioms are no longer re-evaluated.

• Improve performance of CREATE statements, and record insertion.

• Improve RocksDB performance and configuration, introducing SURREAL_ROCKSDB_THREAD_COUNT, SURREAL_ROCKSDB_WRITE_BUFFER_SIZE, SURREAL_ROCKSDB_TARGET_FILE_SIZE_BASE, SURREAL_ROCKSDB_MAX_WRITE_BUFFER_NUMBER, SURREAL_ROCKSDB_MIN_WRITE_BUFFER_NUMBER_TO_MERGE, SURREAL_ROCKSDB_ENABLE_PIPELINED_WRITES, SURREAL_ROCKSDB_ENABLE_BLOB_FILES, SURREAL_ROCKSDB_MIN_BLOB_SIZE environment variables.

• Improve SpeeDB performance and configuration, introducing SURREAL_SPEEDB_THREAD_COUNT, SURREAL_SPEEDB_WRITE_BUFFER_SIZE, SURREAL_SPEEDB_TARGET_FILE_SIZE_BASE, SURREAL_SPEEDB_MAX_WRITE_BUFFER_NUMBER, SURREAL_SPEEDB_MIN_WRITE_BUFFER_NUMBER_TO_MERGE, SURREAL_SPEEDB_ENABLE_PIPELINED_WRITES, SURREAL_SPEEDB_ENABLE_BLOB_FILES, SURREAL_SPEEDB_MIN_BLOB_SIZE environment variables.

• Improve WebSocket performance, introduce SURREAL_WEBSOCKET_MAX_FRAME_SIZE, SURREAL_WEBSOCKET_MAX_MESSAGE_SIZE, SURREAL_WEBSOCKET_MAX_CONCURRENT_REQUESTS environment variables.

• Use specific memory allocators depending on OS.

• Fix memory leak in WebSocket implementation.

• The type::is::record() function now accepts a second optional table argument, validating the record being - stored on the passed table.

• Add time::micros(), time::millis() and time::from::nanos functions.

• Add type::is::none() function.

• Add object::entries(), object::from_entries(), object::len(), object::keys- () and object::values() functions.

• Clean paths in the start command and honour ~.

• CLI: Split results by comment.

• Add surreal sql welcome message.

• Add Debugging env flag: SURREAL_INSECURE_FORWARD_SCOPE_ERRORS.

• Add SURREAL_ROCKSDB_KEEP_LOG_FILE_NUM environment variable (default 20).

• Support auth levels for basic auth (behind feature flag)

• Add remainder/modulo operator.

• Implement string prefixes: s, r, d and u.

• Add ability to cast string to a Thing/Record ID.

• Analyzers to support functions.

• Support of subfields for embedding indexing.

• Add live query API to Rust SDK.

• Add Query::with_stats() to return query statistics along with the results.

• Permissions are now always displayed for visiblity

• Add a --beta flag to surreal upgrade to make installing the latest beta release easier.

Bug fixes:

• Fix stack overflow in graph traversal.

• Bugfix - parse error for invalid leading whitespace.

• Fix memory leak caused by OTEL callbacks.

• Fix wrong function name export and function name parsing.

• The position of the LIMIT and ORDER clauses are now interchangable.

• Fix index plan for idiom param value.

• Fix bug where error offset could underflow.

• Query results should be consistent and representative.

• Indexes used with the operators CONTAINS [ ANY | ALL ].

• Forward custom thrown errors in SIGNIN and SIGNUP queries.

• Fix ORDER BY RAND() failing to parse when selecting specific fields.

• Fix identifiers which look like numbers failing to parse.

• Change math::median indexing for even length arrays.

• Pass IP & Origin onto session used by scope queries.

• Fix possible corruption of MTree and incomplete knn.

• Allow array::flatten() to be used as an aggregate function.

• Make SELECT ONLY deterministic.

• Optional function arguments should be optional.

• Default table permissions should be NONE

• Bugfix: Fix inconsistant record parsing

• Fix time regression in surrealdb.wasm binaries.

• Fix computing futures in query conditions.

• Fix issue with scoring on complex queries.

• Fix ML support on Windows and enable the feature in Windows binaries.

• Replace the custom JWT parser causing decoding issues.

• Ensure compression is only enabled when response is a certain size.

• Respect alias for dynamic field queries with type::field().

• Prevent overflow in math::power().

• Fix error message pointing to wrong character.

• Expand logic for static value validation to improve DEFAULT clause handling.

• Fallback to a string when record ID parsing fails.

• Ensure an attempt to set a protected variable returns an error.

• Fix duration addition in timeout causing overflow.

• Fix a bug where a non-empty list parser would parse empty lists.

• Limit recursion depth when parsing nested RELATE statements.

• Ensure REMOVE statement does not only remove the first 1000 keys.

• Fix BTree deletion bug.

• Replace close method on live::Stream with a Drop trait implementation.

Performance improvements:

• Enable compression on the HTTP connector.

• Make REMOVE [ TABLE | DATABASE | NAMESPACE ] faster for TiKV and FoundationDB.

• Repetitive expressions and idioms are not anymore re-evaluated.

• Improve performance of CREATE statements, and record insertion.

• Improve RocksDB performance and configuration, introducing SURREAL_ROCKSDB_THREAD_COUNT, SURREAL_ROCKSDB_WRITE_BUFFER_SIZE, SURREAL_ROCKSDB_TARGET_FILE_SIZE_BASE, SURREAL_ROCKSDB_MAX_WRITE_BUFFER_NUMBER, SURREAL_ROCKSDB_MIN_WRITE_BUFFER_NUMBER_TO_MERGE, SURREAL_ROCKSDB_ENABLE_PIPELINED_WRITES, SURREAL_ROCKSDB_ENABLE_BLOB_FILES, SURREAL_ROCKSDB_MIN_BLOB_SIZE environment variables.

• Improve SpeeDB performance and configuration, introducing SURREAL_SPEEDB_THREAD_COUNT, SURREAL_SPEEDB_WRITE_BUFFER_SIZE, SURREAL_SPEEDB_TARGET_FILE_SIZE_BASE, SURREAL_SPEEDB_MAX_WRITE_BUFFER_NUMBER, SURREAL_SPEEDB_MIN_WRITE_BUFFER_NUMBER_TO_MERGE, SURREAL_SPEEDB_ENABLE_PIPELINED_WRITES, SURREAL_SPEEDB_ENABLE_BLOB_FILES, SURREAL_SPEEDB_MIN_BLOB_SIZE environment variables.

• Improve WebSocket performance, introduce SURREAL_WEBSOCKET_MAX_FRAME_SIZE, SURREAL_WEBSOCKET_MAX_MESSAGE_SIZE, SURREAL_WEBSOCKET_MAX_CONCURRENT_REQUESTS environment variables.

• Use specific memory allocators depending on OS.

• Fix memory leak in Websocket implementation.

Get realtime updates in your Rust application with the Live Query API

v1.1.0 introduces a new Live Query API to the Rust SDK, for powerful realtime updates in your Rust applications.

// Select the namespace/database to use
db.use_ns("namespace").use_db("database").await?;

// Listen to all updates on a table
let mut stream = db.select("person").live().await?;

// Listen to updates on a range of records
let mut stream = db.select("person").range("jane".."john").live().await?;

// Listen to updates on a specific record
let mut stream = db.select(("person", "h5wxrf2ewk8xjxosxtyc")).live().await?;

// The returned stream implements `futures::Stream` so we can
// use it with `futures::StreamExt`, for example.
while let Some(result) = stream.next().await {
  handle(result);
}

// Handle the result of the live query notification
fn handle(result: Result<Notification<Person>>) {
  match result {
    Ok(notification) => println!("{notification:?}"),
    Err(error) => eprintln!("{error}"),
  }
}

Object functions

It was previously impossible to iterate over objects, so we introduced some new functions to make working with object data structures easier.

LET $fruits = {
  apple: {
    name: "Apple",
    stock: 20,
  },
  banana: {
    name: "Banana",
    stock: 40,
  },
};

LET $num_fruit_types = object::len($fruits);
LET $num_fruit_total = math::sum((
  SELECT VALUE stock FROM object::values($fruits)
));

RETURN "We have " + <string> $num_fruit_types + " type of fruits.";
RETURN "We have " + <string> $num_fruit_total + " pieces of fruit in total.";

String prefixes

Strings can optimistically be parsed as Record IDs, Datetimes or as a UUID, if the content matches such a value. With string prefixes you get to decide what value a string holds.

-- Interpeted as a record ID, because of the structure with the semicolon:
RETURN "5:20";
-- Forcefully parsed as just a string
RETURN s"5:20";

-- This will be a record ID.
RETURN r"person:john";
-- This fails, as it's not a valid record ID.
RETURN r"I am not a record ID";

-- Example for a date and a UUID.
RETURN d"2023-11-28T11:41:20.262Z";
RETURN u"8c54161f-d4fe-4a74-9409-ed1e137040c1";

Deterministic SELECT ONLY

The ONLY clause is sometimes not deterministic. Selecting from an array, table or range with the ONLY clause now requires you to limit the result to 1.

-- Fails, not limited to 1 result and result can contain multiple outputs.
SELECT * FROM ONLY table_name;

-- Works! Resource gets returned, or NONE if no resource was found.
SELECT * FROM ONLY table_name LIMIT 1;

Optional function arguments

Optional function arguments on custom functions are now actually optional.

DEFINE FUNCTION fn::create::resource($required: string, $optional: option<string>) {
  // The $required argument is a string
  // The $optional argument is a string or NONE
};

// Previously you needed to pass NONE for optional arguments
fn::create::resource("Required argument", NONE);

// Now you simply omit the argument
fn::create::resource("Required argument");

Bug fixes:

• Support connecting to beta servers from the Rust SDK.

Bug fixes:

• Add a patch for GHSA-x5fr-7hhj-34j3.

• Tables defined without explicit permissions have NONE instead of FULL permissions.

• Table permissions are always explicitly displayed with the INFO FOR DB statement.

After numerous beta releases crammed into just months of development, we are releasing SurrealDB v1.0.0! 🎉

Here follow some of our 1.0.0 highlights:

Introducing new type validation methods in SurrealQL

v1.0.0 introduces new type validation methods. These new methods allow you to check which type any sort of value is on the go.

LET $value = "I am a string";

RETURN [
  type::is::string($value),
  type::is::record($value),
  type::is::number($value),
];

Guaranteed single item results with the new ONLY keyword

It was previously difficult to select, create, update or delete just a single record, so we simplified it a bit.

// Will return an object, as we can guarantee just a single record is requested
SELECT * FROM ONLY person:tobie;

// This will throw an error, as multiple records are being created and returned
CREATE ONLY person:tobie, person:jaime;

Get realtime updates in your application with SurrealDB Live Queries

v1.0.0 introduces Live Queries. This powerful technology allows you to write applications where you can serve realtime updates to your frontend.