Release v2.1.0
Released on Nov 21st, 2024
Features:
- Recursive idiom/graph traversal
- Add cross-transaction caching to the document processing layer
- Added dictionary-based lemmatization to DEFINE ANALYZER statement
- More static values now possible after DEFAULT clause
- type::field() and type::fields() no longer restricted to just SELECT statements
- Now possible to configure what to export
- Allow data to be imported when server starts
- Added custom storage engine shutdown behaviour
- Versioned records now available in exports
- Enabled configuration of data in exports
- array::fold() and array::reduce() methods
- New string::distance and string::similarity functions for various algorithms
- Improvements to experimental bearer access
- New capabilities to limit HTTP and WebSocket APIs
- (Rust SDK) Allow constructing Datetime, better UUID support
- (SurrealKV) Versioning enabled on edges and relations
- Object Improvements
- Accept computation in VERSION clause
- SDK export config
- Add AND EXPUNGE clause to certain REMOVE statements
Performance improvements:
- Split collecting/processing + multithreading
- Remove execution-time parsing from RPC protocol
- SurrealKV now used for in-memory storage engine
- In-memory parallel sorting
- Efficient incremental ordering
- Added cross-transaction caching to document processing layer
- Query streaming for imports
- Improved performance when reduced document scopes not needed
Bug fixes:
- Fixed incorrect error message
- Fixed field not being removed when set to NONE
- Fixed panic caused by invalid UTF-8
- Fixed panic during parallel query calls in scripting functions
- Fixed bug in random ordering when using files as storage
- Fixed bytes serialization
- Fixed issue with lacking unicode escape sequences
- Fixed incorrect future CBOR decoding
- Fixed overly nested graph query output
- Fixed unexpected behaviour for optional operator
- Fixed OTEL endpoint
- Fixed ignoring of INDEX IGNORE
- Fixed panic when ordering by rand
- Fixed —auth-level flag not requiring necessary —username and —password
- Fixed like operator parsing error
- Fixed invalid exports containing array
- Fixed string-like record ID parsing
- Gracefully handle
Option<T>
inrand::time()
- Allow customising record exporting
- Better handling of unexpected authentication errors
- Improve error messages for negative arithmetic + try_reserve
- Ensure
LIMIT
andPARALLEL
can be used together
Workflow Improvements:
- Benchmark to compare indexed with non-indexed queries
- Improved error message when permissions insufficient
- RocksDB storage engine config options now shown on startup
- Tests to prevent regressions after permission fixes
- Use new runner types in benchmark jobs
- Update dependencies with known security vulnerabilities
- Update rquickjs to newest version
Features with breaking changes:
Release v2.0.4
Released on Oct 8th, 2024
Performance improvements:
- Added lower and upper bounds for RocksDB iterators
- Added with or without index benchmark
- Improved behaviour and performance of reduced document scopes
Bug fixes:
- Fixed panic when calling
Thing::try_from("")
- with empty value - Fixed parsing of functions and models in selectors
- Fixed clippy warnings in release build
- Fixed micro second parsing
Workflow Improvements:
Security advisories:
Release v2.0.3
Released on Oct 3rd, 2024
Features:
- Allowed transactions during imports
- Deprecated DELETE on field permissions
- Implemented Ord for Duration and Table
- Improved Number ordering accuracy
- Made
max_value_size
configurable in surrealkv
Parser Improvements:
Bug fixes:
- Fixed record IDs not being used as Object keys
- Fixed ID type conversion
- Fixed thing parsing function
- Removed truthiness check on Datetime that requires after epoch dates
- Reintroduced
meta::
scope as deprecated methods - Fixed idiom path processing
- Fixed value convertion to a
vector: NONE
Workflow Improvements:
Release v2.0.2
Released on Sep 25th, 2024
Bug fixes:
- Fixed fields being parsed as idioms.
- Fixed error message panics
- Fixed Incorrect lowercase filter in analyzer
- Fixed mismatch in record id for edge tables
- Fixed immediate insert subqueries
- Fixed single check for rpc insert
- Fixed decimal parsing
- Fixed failed authenticate resulting in session reset
- Fixed version patching
- Fixed query Planner Ignoring Isolated predicates in WHERE Clause
- Fixed inconsitent numeric object key
- Fixed ULID/UUID gen, programmatically generating ranges, RETURN inside FOR/IF and improves arithmetic operations
- Fixed CBOR Range encoding
- Fixed parsing bug in graph for bidirectional relationships
- Fixed compilation issue on 32bits platforms (other than wasm32)
- Fixed seting a range id on a document’s id, in and out fields
- Fixed max key size for surrealkv store
- Included request id in CBOR encoding failure message
- Fixed panic when the query file size is too large.
- Fixed bug with number ordering
Improvements:
- Improved log messages for authentication errors
- AddedGraphQL description exactly match the comments set by the user
- Added configuration capabilities for SurrealCS connection pool size
- Added test for including an ID in the content body
- Added minor improvements to core engine
- Added
VERSION
clause to all queries - Added additional ULID and UUID utilities
- Added DTrace profiling support
Release v2.0.1
Released on Sep 18th, 2024
Bug Fixes:
- Fixed infinite loop in parser
Release v2.0.0
Released on Sep 17th, 2024
After numerous alpha and beta releases over months of development, we are releasing SurrealDB v2.0.0! 🎉. The changes for this release pertain to those in between the final 2.0.0-beta-3
and other alpha versions.
NOTE: Because of changes in the underlying way SurrealDB stores data, steps are required to migrate to 2.0.0
from previous versions. Please use the following steps for any existing databases:
- If you are moving from
1.x
to2.x
, use thesurreal fix
command.
Features:
- Enable FoundationDB transaction timeout customization
- Implement missing modulo operator in new parser
- Restructure the parser around the concept of token gluing.
- Added
DEFINE ACCESS
statement to grant access to resources. - Deprecated
DEFINE SCOPE
in favor ofDEFINE ACCESS ... TYPE RECORD
. - Deprecated
DEFINE TOKEN
in favor ofDEFINE ACCESS ... TYPE JWT
. - Deprecated
DEFINE TOKEN ON SCOPE
in favor ofDEFINE ACCESS ... TYPE RECORD ... WITH JWT
. - Added customizable algorithm and key when issuing tokens for records users with the
WITH ISSUER
clause. - Added customizable token and session duration to
DEFINE USER
andDEFINE ACCESS
. - Removed session expiration when not explicitly defined.
- Changed the
INFO
statement to redact secrets defined viaDEFINE ACCESS
. - Changed HTTP request headers expected by SurrealDB to require the
surreal-
prefix. - Removed the
--auth
flag in the CLI to enable it by default. Added--unauthenticated
. - Removed the
--enable-auth-level
flag in CLI to enable the behavior by default. Defaults to root. - Changed authentication to expect level via the
--auth-level
flag orsurreal-auth-*
headers. - Changed the default
--bind
address in the CLI from0.0.0.0
to127.0.0.1
. - Added the
--no-identification-headers
flag in the CLI to limit information leakage. - Added
type::range
function. - Added
TEMPFILES
clause to theSELECT
statement. - Introduce resource creation checks for
DEFINE
statement to avoid duplicates. - Added string functions for IP
string::is::ip
,string::is::ipv4
,string::is::ipv6
. - Added string functions for HTML:
string::html::encode
,string::html::sanitize
. - Added new math functions:
math::acos
,math::acot
,math::asin
,math::atan
,math::clamp
,math::cos
,math::cot
,math::deg2rad
,math::lerp
,math::lerpangle
,math::ln
,math::log
,math::log2
,math::log10
,math::rad2deg
,math::sign
,math::sin
,math::tan
. - Support batch import with
INSERT
statement. This also extends to relationships between tables using theINSERT RELATION
statement - Allow bridge connectivity in all docker images built
- Added define JWT access at the root level
- Allow authenticating with a token from the CLI
- Added Asynchronous building for indexes
- Improved HNSW persistence
- Added Range value support
- Added ability to SELECT with version, CREATE with version and INSERT with version on SurrealKV
- Added ENFORCED keyword to DEFINE TABLE
- SCHEMAFULL tables are now TYPE NORMAL by default
- Added literal types to the query language
- Added the .chain() method
- Added typed LET statements
- Added typed function returns
- Added record::exists() function
- Added new functions
value::diff()
andvalue:::patch()
- Array function changes and additions:
array::all()
,array::any()
,array::filter_index()
,array::find_index()
functions can now take a value or a closure. Also addedarray::filter()
andarray::find()
functions. - Upgraded SurrealKV to v0.3
- More flexible create API for Rust SDK
- Query depth limits can now be configured
- Added GraphQL config
- Indexing operators can now take expressions
- Support for mixed number types on range queries
- Improved capabilities logic with specific allows able to override general denies
- “false” string no longer eagerly evaluated as false
- time::epoch constant now available
- Added geo::is::valid() function
- Method call syntax now works on integers
- Improved GraphQL record fetching
- Added
AUTHENTICATE
clause onRECORD
access - Enabled publishing SurrealDB ports in Docker for bridge networking connectivity
- Allow generating
ULID
andUUID
based on a timestamp - Allow unsetting namespace and database on RPC protocol
- Moved fetch resolution to a compute method
- Added
ALTER TABLE
statement - Moved
AUTHENTICATE
clause to the root ofDEFINE ACCESS
statement to allow the clause to be used in every access method - Added
OVERWRITE
clause for defined resources - Added
array::windows()
function - Added support for idiom destructuring in SurrealQL
- Added
string::is::record
function - Added Method chaining for Database functions
- Added anonymous functions
- Added typed function returns
- Function return types now shown in
INFO
statement - Type
any
can now act as an optional type - Added
time::is::leap_year()
function - Added environment variables to configure HTTP endpoint limits & JavaScript runtime limits
- Introduced savepoint capabilities
- Added experimental
VERSION
clause toINFO FOR DB
andINFO FOR TABLE
- Added additional chained methods to number values
Performance improvements:
- SELECT/count optimization on table range
- Improvements to node membership processing and garbage collection
- Improvements to select count table scan
- Avoid potentially some methods twice
- Stabilise and use
parser2
for parsing. - KNN filtering with limit and KNN distance function.
Bug fixes:
- Fixed incorrectly parsed identifier characters
- Fixed
record<T>
casting from string - Connection ID now stored inside session data
- Fixed certain identifier characters not being parsed correctly
- Fixed remote fields added to output result inside SELECT * queries
- Revise
RecordAccess
structure to aid v2 alpha upgrade - Fixed GHSA-gh9f-6xm2-c4j2: Improper Authentication when Changing Databases as Scope User
- Fixed argument check for
array::clump
- Fixed aborting task leading to an error
- Fixed non-enforced field permissions on WHERE statements
- Fixed failing search when search analyzer lacks tokenizers
- Fixed core dump when performing INSERT
- Fixed discarding without error when using RELATE statement on edge that already exists
- Fixed modification of arrays with JSON Patch semantics
- Fixed selecting directly from a graph edge failing to select all primary edges
- Fixes to issues with composite indexes
- Fixed INSERT not working in subqueries
- Fixed KNN brute force error when an index exists.
- Fixed Redefining HNSW index error.
- Check that a field exists before index creation.
- Fixed missing serialisation for
math::inf
. - Fixed polygon serialization.
- Fixed panic and infinite loop in parser.
- Fixed URL parsing function when a port is present.
- Fixed display formats for access methods
- Fixed parser error to allow
ns
as an alias forNAMESPACE
- Added missing datetime prefix
- Removed duplicated badge for docker pulls count in
README.md
- Fixed export for sql::filter::Filter type in surrealdb-core
- Updated deprecated version of Node.js
- Fixed ELSE IF pretty formatting
- Changed geometry point values to floats in CBOR
- Fixed authentication checks on root user websocket RPC session
- Ensured a transaction is always available when live queries are evaluated
- Reduced read lock duration while processing connection messages
- Ensured execution halts after the
RETURN
statement - Ensured the server shuts down properly
- Fixed querying complex edge table
- Removed unnecessary backticks from
INFO FOR
statement output - Fixed Live Query Panic issue
- Fixed validation on schemafull
RELATE
- Fixed division by zero bug in
math::mean()
function - Improved
string::is::datetime
andstring::is::uuid
- Parse like (
~
) operator correctly - Removed faulty flatten in graph traversal
- Fixed
record<T>
casting from string - Removed
ASSERT
for typeoption<T>
and valueNONE
- Fixed
aud
field on aJWT
claim decoding - Removed unneeded return in matching
- Disabled default audience validation for JWKS
- Fixed errors in supply chain security workflow
- Enabled check for single and multiple token audiences
- Removed panic on invalid range scan
- Added
table
to reserved keywords - Concurrent indexing no longer fails if record updated during build
- Unreachable code now returns an error
DEFINE EVENT
defaults totrue
when no WHEN clause specified- Fixed bug in which
NONE
andNULL
both < 1 and > 1 - Fixed inconsistency on queries using equality with or without index
Create
on rpc which returned an array now returns an object. Similar change made to Rust SDK- Rust SDK fix retrieving multiple columns from the same response
- Improved consistency for
surrealdb.query()
andsurrealdb.value()
in JavaScript runtime - Fixed
ENFORCED
keyword not displaying in INFO statement - Fixed GHSA-64f8-pjgr-9wmr: Untrusted Query Object Evaluation in RPC API
- Improved
math::mean()
to support returning floats - Improved namespace/database validation for GraphQL
- Ensured
TIMEOUT
clauses are processed correctly - Fixed importing from
1.x
- Deprecated missing format inference on RPC protocol
Workflow improvements:
- Deprecated
file://
- Added
surreal fix
command - Exports no longer use explicit transactions
- /lib path renamed to /sdk
- Rename the next workflow to alpha
- Improved structured output for
INFO
statement - Added database upgrade test from
1.5.x
to2.0.x
- Improved
InfoStructure
implementation onIdent
to return the raw value - Improved placeholder text when filling a bug report
- Seperate the supply chain security actions to its workflow
- Improve Cargo local testing
- Limit Cargo build jobs during tests with coverage
- Improved type safety in router API
- Improved hashing mechanism to aid MTree benchmarking
- Removed unnecessary cloning of
DefineStatementAnalyzer
- Simplified string formatting
- Refactored transaction, caching, and key-value store interfaces
- Imported entry enum in
src/sql/mod.rs
- Extracted duplicated code for storing interfaces to initialize_store
- Resolved a TODO: map_err to inspect error
- Improved pull request template
- Added Improvements to kvs module layer
- Updated rustc version to
1.80.0
- Enabled the
ml
feature in nightly and alpha releases - Removed limit to Cargo jobs
- Restored
bench-sdk-surrealkv
- Upgraded to http/hyper 10
- Upgrade upstream
jsonwebtoken
crate - Bumped SurrealKV version to 0.3.1
- Added aliases for complex feature flag combinations
- Improvemed RocksDB range scans
- Improved Serialisation for
sql::Value
- Improved parser error type
- Added Support for SurrealCS
- Bumped async graphql version
- Improved transparent API
- Fixed cargo dependency loading issue
- Improved the logic for migrating IDs in
surreal fix
API improvements:
Surrealkv API improvements:
- Changed disk format for compaction in Surrealkv
- Removed global static Datastore
- Added versioning to
SELECT * FROM table
queries
Parser improvements:
- Added support for the
s
string prefix in parser - Fixed parser panic when a dot follows a string in idiom-allowed contexts
- Ensured variables after the
FETCH
clause are parsed - Ensured UUIDs are parsed correctly
- Fixed floating point parsing bug
- Updates reblessive dependency to newest version
- Removed flume as a dependency in favour of async-channels
- Fixed warnings and simplified connection trait
Breaking changes:
- Functions beginning with
meta::
renamed torecord::
- Functions
string::endsWith()
andstring::startsWith()
renamed tostring::ends_with()
andstring::starts_with()
- Changed
UPDATE
statement behaviour of creating records in favour of newUPSERT
statement.
Experimental features:
Release v1.5.6
Released on Nov 12th, 2024
Features:
- Created
v1
,v1-dev
,v1.5
andv1.5-dev
Docker tags
Bug fixes:
Release v1.5.5
Released on Sep 11th, 2024
Bug fixes:
- Fixed GHSA-64f8-pjgr-9wmr: Untrusted Query Object Evaluation in RPC API
Release v1.5.4
Released on Jul 10th, 2024
Bug fixes:
- Allowed fields production to end a subquery
- Fixed GHSA-gh9f-6xm2-c4j2: Improper Authentication when Changing Databases as Scope User
Release v1.5.3
Released on Jun 14th, 2024
Features:
- Added an
--alpha
flag tosurreal upgrade
.
Bug fixes:
Release v1.5.2
Released on Jun 6th, 2024
Bug fixes:
- Fixed incorrect computations on aggregations within foreign tables
- Fixed
surreal upgrade --nightly
Release v1.5.1
Released on May 23rd, 2024
Bug fixes:
- Make temporary table active only if the temporary directory is set
- Fixes compilation issue related to temporary-directory when the feature sql2 is not enabled
Release v1.5.0
Released on May 14th, 2024
Features:
- Added an implementation of HNSW in memory.
- Added a
REBUILD INDEX
statement. - Added variable support in
CONTENT
clause forRELATE
statements. - Added more information to
INFO FOR SCOPE
. - Added a relate method to the RPC protocol.
- Added an
INFO STRUCTURE
statement. - Stabilised
sql2
andjwks
features. - Introduced an on-disk temporary table.
- Added a run method to the RPC protocol.
- Implemented limits for parsing depth in the new parser.
- Implemented reblessive into the parser to prevent any overflows.
- Increased the Minimum Supported Rust Version (MSRV) to 1.77.
Bug fixes:
- Made the query planner recognise the exact operator (
==
). - Fixed
math:min
in foreign tables. - Fixed typo in function
math:sum
: wasmath::sun
. - Made record IDs more flexible in the new parser.
- Reverted changefeed polling frequency back to 10 seconds.
- Fixed a problem with date-time parsing.
- Fixed response content-type on
/rpc
endpoint. - Fixed decimal decoding.
- Made the query planner support the IN operator.
- Made the JWKS implementation more RFC 7517 compliant.
- Fixed wrong count when using
COUNT
with a subquery. - Fixed the content type header on a CBOR HTTP response.
- Fixed
$value
beingNONE
forDELETE
events. - Fixed
array::group
in a group by query.
Performance improvements:
- Added a query planner strategy for record links.
- Made
TreeCache
more efficient on writes. - Reduced byte size of datetime and UUID types using CBOR format.
Release v1.4.2
Released on Apr 20th, 2024
Bug fixes:
- Fix problems with if in identifiers after remove and define statements.
- Fix $value being NONE for DELETE events.
- Fix CBOR headers
- Fix wrong count when using COUNT with a subquery
- Fix IN operator should be recognised by the query planner
- Fix response content-type on /rpc endpoint
When using the Rust SDK, make sure surrealdb-core
is up-to-date. If not, you might get back Revision
errors as a result.
Release v1.4.0
Released on Apr 9th, 2024
Features:
- Added a warning message about debug builds in CLI start.
- Moved JWKS cache storage to memory.
- Overhauled force implementation.
- Added
DEFINE TABLE ... RELATION
. - Allowed RPC calls to be made over an HTTP connection, not just WebSocket.
- Allowed the highlighter to only highlight the matching searched token rather than the whole term.
- Added an
INCLUDE ORIGINAL
option to change-feeds. - Added an
insert
method to the Rust SDK to allow bulk inserts.
Bug fixes:
- Fixed export generating unparsable code with the new parser.
- Eliminated a potential panic in MsgPack format implementation.
- Fixed
string::is::longitude
RegEx. - Improved CBOR decoding.
- Fixed relation type parsing.
- Fixed handling of empty array on index.
- Allowed legacy headers in CORS.
- Allowed
surreal upgrade
to detect when upgrading to the same version and return early. - Fixed certain environment variables to allow configuration at runtime.
- Reduced the byte size of datetime and uuid types using CBOR format.
- Fixed
array::group
in a group by query.
Performance improvements:
- Improved query aggregation handling.
Bulk insert support in the Rust SDK
This was one of the frequently requested features. While this was already possible using the query
method, this release adds an insert
method that makes it more convenient.
db.insert(Resource::from("person")) .content(vec![ User { id: thing("person:tobie")?, name: "Tobie", settings: Settings { active: true, marketing: false, }, }, User { id: thing("person:jaime")?, name: "Jaime", settings: Settings { active: true, marketing: true, }, }, ]) .await?;
Release v1.3.1
Released on Mar 15th, 2024
Bug fixes:
- Add JWT ID claim to tokens issued by SurrealDB.
- Consistently enforce session expiration.
Release v1.3.0
Released on Mar 12th, 2024
Features:
- Introduced
IF NOT EXISTS
clause onDEFINE
statements. - Implemented
IF EXISTS
for additionalREMOVE
statements. - Changed the KNN operator from
<2>
to<|2|>
in the new parser and added support in the old parser for the new syntax. - Implemented WebSocket session expiration.
- Added support for tables and geometries in CBOR.
- Added support for parsing decimal numbers with scientific notion.
- Added support for printing a warning in the CLI when using an outdated version of SurrealDB.
- Added
Surreal::wait_for
to make it possible for the client to wait for certain events. - Added SurrealKV as an unstable feature.
- Added more error message improvements for the new parser.
Bug fixes:
- More consistent handling of negative numbers in record IDs.
- Removed the unimplemented
backup
command from the CLI. - Fixed document not being available to delete permissions clause.
- Ensured objects properties are recognized by the query planner.
- Implemented the union strategy on unique indexes.
Performance improvements:
- Added compile-time feature for flamegraph and pprof profiling.
IF NOT EXISTS
clause on DEFINE
statements
DEFINE
statements now allow you to include an IF NOT EXISTS
clause which ensures that the define statement is only run if the resource doesn’t already exist. If it exists, you will get an error. Learn more about this in the documentation
DEFINE DATABASE IF NOT EXISTS example
New KNN syntax
The KNN operator now supports a new syntax using <|
and |>
instead of <
and >
. Learn more about this in the documentation
SELECT id FROM point WHERE point_b <|2|> [2, 3, 4, 5]
Waiting for client events
It’s now possible to initialise the connection and run setup actions like authentication and selecting the database concurrently with other queries by making it possible to wait for the SDK to connect or select the database to use before allowing other queries to execute.
Something like this would be susceptible to a race condition before:
// A static, global instance of the client static DB: Lazy<Surreal<Any>> = Lazy::new(|| { // Connect and setup the database connection in the background tokio::spawn(async { // This example is for demonstration purposes only. // If the `unwrap` panics the client will be left hanging, // waiting for the setup to happen when using `Surreal::wait_for`. setup_db().await.unwrap(); }); // Initialise a new unconnected instance of the client Surreal::init() }); // Connect to the server and run setup actions async fn setup_db() -> surrealdb::Result<()> { // Connect to the server DB.connect("ws://localhost:8000").await?; // Signin as a namespace, database, or root user DB.signin(Root { username: "root", password: "root", }).await?; // Select a specific namespace / database DB.use_ns("namespace").use_db("database").await?; Ok(()) } // Whether this would run successfully or return an error would depend on whether // `DB::setup` was able to execute before getting to this point. DB.create(Resource::from("person")).await?;
Now it’s possible to make the client wait for either the connection to be established
use surrealdb::opt::WaitFor::Connection; // This will wait until the connection is established. // As soon as it is, it will return and allow the client to continue. DB.wait_for(Connection).await; // This will still likely return an error. // While we know that the connection is already established at this point, // there is no guarantee that the database has already been selected. // This query requires the database to be selected first. DB.create(Resource::from("person")).await?;
or the database to be selected.
use surrealdb::opt::WaitFor::Database; // This will wait until the database is selected. // As soon as it is, it will return and allow the client to continue. DB.wait_for(Database).await; // At this point, the database is guaranteed to be selected already // so this query should run successfully. DB.create(Resource::from("person")).await?;
Release v1.2.2
Released on Mar 7th, 2024
Bug fixes:
- Ensure relevant indexes are triggered when using IN in a SELECT query.
- Ensure the query planner accepts
Datetime
andUuid
values. - Restore cosine distance on MTree indexes.
- Fix regression in index data serialisation.
- Ensure
rquickjs
builds don’t stall on macOS.
Release v1.2.1
Released on Feb 16th, 2024
Bug fixes:
- Fix an issue with
WHERE
clause on queries resolving record links or graph edges. - Fix MATH::SQRT_2 not parsing.
- Fix a panic in span rendering.
- Fix CLI output not displaying properly sometimes.
Release v1.2.0
Released on Feb 13th, 2024
Features:
- Bump MSRV to 1.75.
- In-memory index store.
- Show execution time in CLI.
- knn brute force.
- Implement support for remote JSON Web Key Sets.
- Add support for
LIVE SELECT
in the SDK and CLI. - Add
IF EXISTS
toREMOVE TABLE
statement. - Add
READONLY
keyword toDEFINE FIELD
statement. - Add alias
-V
and--version
flags forsurreal version
command. - Define types for subfields when defining array fields.
- Add
string::semver::compare
,string::semver::major
,string::semver::minor
,string::semver::patch
,string::semver::inc::major
,string::semver::inc::minor
,string::semver::inc::patch
,string::semver::set::major
,string::semver::set::minor
,string::semver::set::patch
methods.
Bug fixes:
- Make record id string parsing never fail early.
- Prevent overflow in
math::power()
. - Fix error message pointing to wrong character.
- Respect alias for dynamic field queries with type::field.
- Remove min/1000 in
tx:delp
. - Replace custom JWT parser causing decoding issues.
- Issue with scoring on complex queries.
- Limit recursion depth when parsing nested RELATE statements.
- Fix a bug where a non-empty list parser would parse empty lists.
- Ensure an attempt to set a protected variable returns an error.
- Fix duration addition in timeout causing overflow.
- Panic invoking parameters and functions without a database.
- Fix a bug where the kind parser would eat whitespace.
- Fix WebSocket notification sending format.
- Implement missing errors for missing clauses on DEFINE-statements.
- Ensure advanced
DEFINE PARAM
parameters are computed correctly. - Ensure path idioms are correct when looping over.
- Fix json failing to parse with trailing whitespace.
- Fix the five second delay in Wasm after initial connection.
- Add context to live query notifications.
- Fix the modulo operator on sql2 and parser1.
- Improve the js-surql value conversion for numbers.
- Implement revision types for client/server communication.
- Fix builtin error pointing to the wrong part.
- Fix a panic when invalid builtin function names are passed to the executor.
Performance improvements:
- Ensure compression is only enabled when response is a certain size.
- In MTree large vector improvement.
Use JWKS to dynamically configure your token definitions
-- Specify the namespace and database for the token USE NS abcum DB app_vitalsense; -- Set the name of the token DEFINE TOKEN token_name -- Use this token provider for database authorization ON DATABASE -- Specify the JWKS specification used to verify the token TYPE JWKS -- Specify the URL where the JWKS object can be found VALUE "https://example.com/.well-known/jwks.json" ;
Define a READONLY
field
DEFINE FIELD created ON resource VALUE time::now() READONLY;
Define types for subfields when defining array fields
DEFINE FIELD foo ON bar TYPE array<number> -- Where <number> is a subfield of the array type. -- It now defines the number subfield automatically instead of having to run this manually: DEFINE FIELD foo.* ON bar TYPE number
Release v1.1.1
Released on Jan 16th, 2024
Bug fixes:
- Fix WebSocket notification sending format.
- Fix missing custom claims from token parameter.
- Fix URL encoding in JS functions.
- Fix panic when invoking parameters and functions without a database.
Release v1.1.0
Released on Jan 9th, 2024
Features:
- The
type::is::record()
function now accepts a second optional table argument, validating the record being - stored on the passed table. - Add
time::micros()
,time::millis()
andtime::from::nanos
functions. - Add
type::is::none()
function. - Add
object::entries()
,object::from_entries()
,object::len()
,object::keys- ()
andobject::values()
functions. - Clean paths in the start command and honour
~
. - CLI: Split results by comment.
- Add
surreal sql
welcome message. - Add Debugging env flag:
SURREAL_INSECURE_FORWARD_SCOPE_ERRORS
. - Add
SURREAL_ROCKSDB_KEEP_LOG_FILE_NUM
environment variable (default 20). - Support auth levels for basic auth (behind feature flag)
- Add remainder/modulo operator.
- Implement string prefixes:
s
,r
,d
andu
. - Add ability to cast string to a Thing/Record ID.
- Analyzers to support functions.
- Support of subfields for embedding indexing.
- Add live query API to Rust SDK.
- Add
Query::with_stats()
to return query statistics along with the results. - Permissions are now always displayed for visiblity
- Add a
--beta
flag tosurreal upgrade
to make installing the latest beta release easier.
Bug fixes:
- Fix stack overflow in graph traversal.
- Bugfix - parse error for invalid leading whitespace.
- Fix memory leak caused by OTEL callbacks.
- Fix wrong function name export and function name parsing.
- The position of the
LIMIT
andORDER
clauses are now interchangable. - Fix index plan for idiom param value.
- Fix bug where error offset could underflow.
- Query results should be consistent and representative.
- Indexes used with the operators
CONTAINS [ ANY | ALL ]
. - Forward custom thrown errors in
SIGNIN
andSIGNUP
queries. - Fix
ORDER BY RAND()
failing to parse when selecting specific fields. - Fix identifiers which look like numbers failing to parse.
- Change math::median indexing for even length arrays.
- Pass IP & Origin onto session used by scope queries.
- Fix possible corruption of MTree and incomplete knn.
- Allow
array::flatten()
to be used as an aggregate function. - Make
SELECT ONLY
deterministic. - Optional function arguments should be optional.
- Default table permissions should be
NONE
- Bugfix: Fix inconsistant record parsing
- Fix time regression in
surrealdb.wasm
binaries. - Fix computing futures in query conditions.
- Fix issue with scoring on complex queries.
- Fix ML support on Windows and enable the feature in Windows binaries.
- Replace the custom JWT parser causing decoding issues.
- Ensure compression is only enabled when response is a certain size.
- Respect alias for dynamic field queries with
type::field()
. - Prevent overflow in
math::power()
. - Fix error message pointing to wrong character.
- Expand logic for static value validation to improve
DEFAULT
clause handling. - Fallback to a string when record ID parsing fails.
- Ensure an attempt to set a protected variable returns an error.
- Fix duration addition in timeout causing overflow.
- Fix a bug where a non-empty list parser would parse empty lists.
- Limit recursion depth when parsing nested
RELATE
statements. - Ensure
REMOVE
statement does not only remove the first 1000 keys. - Fix BTree deletion bug.
- Replace close method on
live::Stream
with a Drop trait implementation.
Performance improvements:
- Enable compression on the HTTP connector.
- Make
REMOVE [ TABLE | DATABASE | NAMESPACE ]
faster for TiKV and FoundationDB. - Repetitive expressions and idioms are not anymore re-evaluated.
- Improve performance of
CREATE
statements, and record insertion. - Improve RocksDB performance and configuration, introducing
SURREAL_ROCKSDB_THREAD_COUNT
,SURREAL_ROCKSDB_WRITE_BUFFER_SIZE
,SURREAL_ROCKSDB_TARGET_FILE_SIZE_BASE
,SURREAL_ROCKSDB_MAX_WRITE_BUFFER_NUMBER
,SURREAL_ROCKSDB_MIN_WRITE_BUFFER_NUMBER_TO_MERGE
,SURREAL_ROCKSDB_ENABLE_PIPELINED_WRITES
,SURREAL_ROCKSDB_ENABLE_BLOB_FILES
,SURREAL_ROCKSDB_MIN_BLOB_SIZE
environment variables. - Improve SpeeDB performance and configuration, introducing
SURREAL_SPEEDB_THREAD_COUNT
,SURREAL_SPEEDB_WRITE_BUFFER_SIZE
,SURREAL_SPEEDB_TARGET_FILE_SIZE_BASE
,SURREAL_SPEEDB_MAX_WRITE_BUFFER_NUMBER
,SURREAL_SPEEDB_MIN_WRITE_BUFFER_NUMBER_TO_MERGE
,SURREAL_SPEEDB_ENABLE_PIPELINED_WRITES
,SURREAL_SPEEDB_ENABLE_BLOB_FILES
,SURREAL_SPEEDB_MIN_BLOB_SIZE
environment variables. - Improve WebSocket performance, introduce
SURREAL_WEBSOCKET_MAX_FRAME_SIZE
,SURREAL_WEBSOCKET_MAX_MESSAGE_SIZE
,SURREAL_WEBSOCKET_MAX_CONCURRENT_REQUESTS
environment variables. - Use specific memory allocators depending on OS.
- Fix memory leak in Websocket implementation.
Get realtime updates in your Rust application with the Live Query API
v1.1.0
introduces a new Live Query API to the Rust SDK, for powerful realtime updates in your Rust applications.
// Select the namespace/database to use db.use_ns("namespace").use_db("database").await?; // Listen to all updates on a table let mut stream = db.select("person").live().await?; // Listen to updates on a range of records let mut stream = db.select("person").range("jane".."john").live().await?; // Listen to updates on a specific record let mut stream = db.select(("person", "h5wxrf2ewk8xjxosxtyc")).live().await?; // The returned stream implements `futures::Stream` so we can // use it with `futures::StreamExt`, for example. while let Some(result) = stream.next().await { handle(result); } // Handle the result of the live query notification fn handle(result: Result<Notification<Person>>) { match result { Ok(notification) => println!("{notification:?}"), Err(error) => eprintln!("{error}"), } }
Object functions
It was previously impossible to iterate over objects, so we introduced some new functions to make working with object data structures easier.
LET $fruits = { apple: { name: "Apple", stock: 20, }, banana: { name: "Banana", stock: 40, }, }; LET $num_fruit_types = object::len($fruits); LET $num_fruit_total = math::sum(( SELECT VALUE stock FROM object::values($fruits) )); RETURN "We have " + <string> $num_fruit_types + " type of fruits."; RETURN "We have " + <string> $num_fruit_total + " pieces of fruit in total.";
String prefixes
Strings can optimistically be parsed as Record IDs, Datetimes or as a UUID, if the content matches such a value. With string prefixes you get to decide what value a string holds.
-- Interpeted as a record ID, because of the structure with the semicolon: RETURN "5:20"; -- Forcefully parsed as just a string RETURN s"5:20"; -- This will be a record ID. RETURN r"person:john"; -- This fails, as it's not a valid record ID. RETURN r"I am not a record ID"; -- Example for a date and a UUID. RETURN d"2023-11-28T11:41:20.262Z"; RETURN u"8c54161f-d4fe-4a74-9409-ed1e137040c1";
Deterministic SELECT ONLY
The ONLY
clause is sometimes not deterministic. Selecting from an array, table or range with the ONLY
clause now requires you to limit the result to 1
.
-- Fails, not limited to 1 result and result can contain multiple outputs. SELECT * FROM ONLY table_name; -- Works! Resource gets returned, or NONE if no resource was found. SELECT * FROM ONLY table_name LIMIT 1;
Optional function arguments
Optional function arguments on custom functions are now actually optional.
DEFINE FUNCTION fn::create::resource($required: string, $optional: option<string>) { // The $required argument is a string // The $optional argument is a string or NONE }; // Previously you needed to pass NONE for optional arguments fn::create::resource("Required argument", NONE); // Now you simply omit the argument fn::create::resource("Required argument");
Release v1.0.2
Released on Dec 21st, 2023
Bug fixes:
- Support connecting to beta servers from the Rust SDK.
Release v1.0.1
Released on Dec 14th, 2023
Bug fixes:
- Add a patch for GHSA-x5fr-7hhj-34j3.
- Tables defined without explicit permissions have NONE instead of FULL permissions.
- Table permissions are always explicitly displayed with the INFO FOR DB statement.
Release v1.0.0
Released on Sep 13th, 2023
After numerous beta releases crammed into just months of development, we are releasing SurrealDB v1.0.0! 🎉
Here follow some of our 1.0.0 highlights
Introducing new type validation methods in SurrealQL
v1.0.0
introduces new type validation methods. These new methods allow you to check which type any sort of value is on the go.
LET $value = "I am a string"; RETURN [ type::is::string($value), type::is::record($value), type::is::number($value), ];
Guaranteed single item results with the new ONLY keyword
It was a previously difficult to select, create, update or delete just a single record, so we simplified it a bit.
// Will return an object, as we can guarantee just a single record is requested SELECT * FROM ONLY person:tobie; // This will throw an error, as multiple records are being created and returned CREATE ONLY person:tobie, person:jaime;
Get realtime updates in your application with SurrealDB Live Queries
v1.0.0
introduces Live Queries. This powerful technology allows you to write applications where you can serve realtime updates to your frontend.
// You can initiate the live query in your application LET $lq = LIVE SELECT * FROM person WHERE age > 18; // And you can dispose the live query once it is no longer needed KILL $lq;
// SurrealDB SDKs allow you to easily initiate live queries, and to process the received messages. const lq = await db.live('person', function ({ action, result }) { if (action == 'CREATE') { renderPersonInFrontend(result); } }); // Once you no longer need the live feed of updates, you can dispose the live query await db.kill(lq);
Efficiently index and search your data with SurrealDB Full Text Search
With Full Text Search, you can efficiently store and index data, and search through it.
// We can define an analyzer on our database, // Then, define an index to dictate which content we want to index (yep, that easy) DEFINE ANALYZER simple TOKENIZERS blank,class FILTERS snowball(english); DEFINE INDEX content ON article FIELDS content SEARCH ANALYZER simple BM25 HIGHLIGHTS; // We can then create some content CREATE article SET content = "Join us at SurrealDB World, as we unveil our version 1.0.0 to the world!"; CREATE article SET content = "We will absolutely be at Surreal World!"; // And lastly, select some data from the article table. // We use the special "@num@" operator, where we define a reference. // We can then reference back in the search::* functions. SELECT *, search::score(1) AS score, search::offsets(1) AS offsets, search::highlight('<b>', '</b>', 1) AS highlighted FROM article WHERE content @1@ 'world' ORDER BY score DESC;
Allow or deny capabilities when starting your SurrealDB instance
All capabilities are disabled by default. This means that by default, you are not able to use any methods, embedded scripting functions, make outbound network calls, or access the database anonymously. Down below follows a set of examples to showcase how one can configure capabilities.
Capabilities are further documented in the Capabilities documentation.
# Allow all capabilities user@localhost % surreal start --allow-all # Allow all functions, except for custom functions user@localhost % surreal start --allow-funcs --deny-funcs fn # Allow all capabilities, but deny guest/anonymous access to the database user@localhost % surreal start --allow-all --deny-guests
Revamped root users
It is now possible to define multiple root users in SurrealDB. This change did require some changes in the way that you start your database however.
With this change, you will now only initially have to provide the --user
and --pass
flags to create the initial root user, but once the first root user exists, they will no longer by utilized.
For more information, check out the Authentication guide, and the surreal start and DEFINE USER
documentation.
# When you initially start the database, you can create the first root user. user@localhost % surreal start --auth --user root --pass root file:database.db # In the future, you don't need to specify the --user and --pass flags anymore user@localhost % surreal start --auth file:database.db
// Afterwards, you are able to create multiple root users. DEFINE USER tobie ON ROOT PASSWORD "SecurePassword!" ROLES OWNER; DEFINE USER jaime ON ROOT PASSWORD "SecurePassword!" ROLES EDITOR; DEFINE USER john ON ROOT PASSWORD "SecurePassword!" ROLES VIEWER;
Strict typing in SurrealQL
v1.0.0
introduces a more strict and powerful typing system. It makes things more simple to understand, and it goes a long way in preventing all kinds of weird bugs in your schemas!
// Where you previously had to manually assert that a field does not contain NONE or NULL DEFINE FIELD age ON person TYPE number ASSERT $value != NONE AND $value != NULL; // You can now simply set the type and be ensured that only a number will be stored in that field. DEFINE FIELD age ON person TYPE number; // To make types optional, you can use the newly introduced "option<type>" type. // This guarantees the field to be either empty (NONE), or a number. DEFINE FIELD age ON person TYPE option<number>; // We also introduces some new types! DEFINE FIELD ratings ON movie TYPE array<number>; DEFINE FIELD best_ratings ON movie TYPE array<number, 5>; DEFINE FIELD unique_ratings ON movie TYPE set<number, 5>; // And we made sure that the record type is also in-line with this new format DEFINE FIELD author ON book TYPE record<person>; // Lastly, it is now possible to set a union type. DEFINE FIELD title ON article TYPE string | number; // They can also be used in "nested" types DEFINE FIELD description ON article TYPE option<string | number>; DEFINE FIELD author ON article TYPE record<person | admin>;
Set a DEFAULT
value on field definitions
// You previously needed to involve logic with the "VALUE" clause to set a default value. DEFINE FIELD enabled ON user TYPE bool VALUE $value OR $before OR true; // You can now simply use the DEFAULT clause for this. DEFINE FIELD enabled ON user TYPE bool DEFAULT true; // As an added bonus, when you only define a VALUE clause with a simple value, // then that will also act as the default value. DEFINE FIELD constant ON demo VALUE 123; // This will act the same as the following DEFINE FIELD constant ON demo VALUE 123 DEFAULT 123;
PERMISSIONS
on global parameters and custom functions
Scope and anonymous users previously had access to every defined global parameter and function. You can now define these resources with a PERMISSIONS
clause to protect them.
DEFINE PARAM $perms_none VALUE 'Nobody can see me' PERMISSIONS NONE; DEFINE PARAM $perms_full VALUE 'Everybody can see me' PERMISSIONS FULL; DEFINE PARAM $perms_scope VALUE 'Only admins can see me' PERMISSIONS WHERE $scope = 'admin'; DEFINE FUNCTION fn::perms::none() { RETURN 'Nobody can invoke me'; } PERMISSIONS NONE; DEFINE FUNCTION fn::perms::full() { RETURN 'Everybody can invoke me'; } PERMISSIONS FULL; DEFINE FUNCTION fn::perms::scope() { RETURN 'Only admins can invoke me'; } PERMISSIONS WHERE $scope = 'admin';
FOR
, BREAK
and CONTINUE
statements
// Here, we loop through an array with three numbers. // For any number under two, we skip the iteration // For any number above two, we break the loop // The result is that only when the number is two, we will create the "person:two" record. FOR $num IN [1, 2, 3] { IF $num < 2 { CONTINUE; }; IF $num > 2 { BREAK; }; CREATE person:two; }; // Here, we select the id for every person in the database, and we gift them a ticket to SurrealDB World. FOR $person IN (SELECT VALUE id FROM person) { CREATE gift CONTENT { recipient: $person, type: "ticket", event: "SurrealDB World" }; };
THROW
statement
Did something unexpected happen, and do you want to throw an error to the client? Now you can!
LET $message = "Some error message"; THROW "Failed to perform action: " + $message;
COMMENT
away on resource definitions!
// Give extra context about a certain resource DEFINE TABLE user COMMENT "This table will store users!";
Less bulky IF ELSE
statements
We found the IF ELSE
statement to be a bit bulky at times. Now, when you use a block ({}
) as the body of the statement, you can skip out on the THEN
and END
keywords!
// Previously IF something = true THEN { RETURN 123; } END; // Now IF something = true { RETURN 123; };
More features for our embedded scripting functions
With fetch()
, query()
, value()
and basically every SurrealQL function now being available within the embedded scripting functions, they are a very powerful extension to SurrealQL, and can be used to solve complex problems otherwise impossible!
Read more about them in the Embedded scripting functions documentation.
function() { const page = await fetch('https://google.com'); const people = await surrealdb.query('SELECT * FROM person'); const auth = await surrealdb.value('auth'); const uuid = surrealdb.functions.rand.uuid.v4(); }
Support for FLEXIBLE
fields on SCHEMAFULL
tables
SCHEMAFULL
and SCHEMALESS
functionality can now be used together, suitable for capturing schema-free log data.
// We want to define a SCHEMAFULL table DEFINE TABLE person SCHEMAFULL; // But we want one field to allow any content DEFINE FIELD settings on person FLEXIBLE TYPE object; // We can then set the field value without a schema CREATE person:test CONTENT { settings: { nested: { object: { thing: 'test' } } } };
Support code blocks and advanced expressions
It is now possible to run blocks of code, with support for an arbitrary number of statements, including LET
and RETURN
statements. This allows for writing advanced custom logic, and allowing for more complicated handling of data operations.
DEFINE FIELD average_sales ON metrics VALUE { LET $sales = (SELECT VALUE quantity FROM sales); LET $total = math::sum($sales); LET $count = count($sales); RETURN ($total / $count); };
Define custom functions with DEFINE FUNCTION statements
SurrealDB now supports the ability to define global database-wide custom functions, which allow for complicated or repeated user-defined code, to be run seamlessly within any query across the database. Custom functions support typed arguments, and multiple nested queries with custom logic.
-- Define a global function which can be used in any query DEFINE FUNCTION fn::get_person($first: string, $last: string, $birthday: string) { LET $person = SELECT * FROM person WHERE [first, last, birthday] = [$first, $last, $birthday]; RETURN IF $person[0].id THEN $person[0] ELSE CREATE person SET first = $first, last = $last, birthday = $birthday END; }; -- Call the global custom function, receiving the returned result LET $person = fn::get_person('Tobie', 'Morgan Hitchcock', '2022-09-21');
Release v0.3.0
Released on Dec 14th, 2021
Features:
- Enable query and session parameters to be defined on a JSON-RPC connection
- Ensure subqueries can access encoding parent query and grand-parent queries
- Add diff-match-patch functionality when updating document records
- Separate authentication levels for Namespace and Database specific access
- Authentication scope definition and setup, with user-defined authentication logic for each scope
Pre-defined aggregate analytics views
Aggregate views let you pre-compute analytics queries as data is written to SurrealDB. Similarly to an index, a table view lets you select, aggregate, group, and order data, with support for moving averages, time-based windowing, and attribute-based counting. Pre-defined aggregate views are efficient and performant, with only a single record modification being made for every write.
-- Drop all writes to the reading table. We don't need every reading. DEFINE TABLE reading DROP; -- Define a table as a view which aggregates data from the reading table DEFINE TABLE temperatures_by_month AS SELECT count() AS total, time::month(recorded_at) AS month, math::mean(temperature) AS average_temp FROM reading GROUP BY city ; -- Add a new temperature reading with some basic attributes CREATE reading SET temperature = 27.4, recorded_at = time::now(), city = 'London', location = (-0.118092, 51.509865) ;
Release v0.2.0
Released on Jan 21st, 2021
Features:
- Parameters can be used to store values or result sets
- Nested subquery functionality, with scoped parameters
- Nested field query notation allowing nested arrays and objects to be queried
- Mathematical operators for complex mathematical calculations in queries
- Advanced functions for working with arrays, strings, time, validation, parsing, and counting
Release v0.1.0
Released on Dec 8th, 2019
Features:
- Multi-tenancy data separation, with namespaces and databases
- Schemafull or schemaless tables with limitless document fields
- Multi-table, multi-row, serialisable ACID transactions
- Table fields, table change events, table indexes, and data constraints
- Advanced data model including empty values, strings, numbers, objects, arrays, durations, and datetimes