Releases
SurrealDB has been built from the ground up to be the ultimate database for developers who want to build tomorrow's applications. On this page you can view the release notes for the different historic versions of SurrealDB, and for imminent future releases.
Release v1.0.0
13 September, 2023
View the blog post release announcement for full information.
Release v1.0.0-beta.12
12 September, 2023
Features
EXPLAIN
command now also explains why it sometimes falls back to a full table iteration.- Add
$before
and$after
params toRETURN
clauses. - Add live queries to
INFO FOR TB
statement. - Support of range queries in the query planner.
Bug fixes
-
Majorly improved error messages. -
When a scope fails to authenticate, we now give a detailed explanation as to why. -
Ensure parameters are computed before being stored on WebSocket. -
Ensure DEFAULT
clause is displayed onDEFINE FIELD
statements. -
Ensure scope SIGNUP
andSIGNIN
works with guest access disabled. -
Fix authentication issues with LIVE SELECT
statements. -
Support FOR
statements inside code blocks. -
When a scope fails to authenticate, we now give a detailed explanation as to why. -
Cluster bootstrapping now warns and continues instead of fail-stopping -
Ensure Live Queries are killed correctly. -
Ensure that scripting functions can be used within SELECT
-statement predicates. -
Avoid panics when displaying errors. -
Prevent infinite parser recursion in stmts & binary exprs. -
Fix panic on commit when defining db and tb in strict mode. -
Fix decimal deserialisation. -
Fix change feeds enabled at the database-level. -
Allow for multiple indexes on a single field.
Release v1.0.0-beta.11
05 September, 2023
Features
- Moved every
is::*
function tostring::is::*
. - Introduce new
type::is::array
,type::is::bool
,type::is::bytes
,type::is::collection
,type::is::datetime
,type::is::decimal
,type::is::duration
,type::is::float
,type::is::geometry
,type::is::int
,type::is::line
,type::is::null
,type::is::multiline
,type::is::multipoint
,type::is::multipolygon
,type::is::number
,type::is::object
,type::is::point
,type::is::polygon
,type::is::record
,type::is::string
,type::is::uuid
type validation methods. - Added
X-Forwarded-For
header to the accepted headers for the--client-ip
option. - Allow all functions to be invoked by default.
- Allow any type of value to be used in
THROW
statements. - Introduce a new
ONLY
keyword.
Bug fixes
-
Ensure that errors within scope logic are properly handled and do not cause a panic. -
Ensure that capabilities are properly applied to embedded scripting functions. -
Fix various niche issues caught by fuzzer that could cause a panic. -
Fix relative file paths.
Introducing new type validation methods in SurrealQL
v1.0.0-beta.11
introduces new type validation methods. These new methods allow you to check which type any sort of value is on the go.
LET $value = "I am a string";
RETURN [
type::is::string($value),
type::is::record($value),
type::is::number($value),
];
Guaranteed single item results with the new ONLY
keyword
It was a bit rough previously to select, create, update or delete just a single record, so we simplified it a bit.
// Will return an object, as we can guarantee just a single record being selected here.
SELECT * FROM ONLY person:tobie;
// This will throw an error, as multiple records will be created and returned.
CREATE ONLY person:tobie, person:jaime;
THROW
dynamic error messages
Previously, only strings could be passed to the THROW
statement. We updated this so that the THROW
statement now accepts any type of value.
LET $message = "Some error message";
THROW "Failed to perform action: " + $message;
Release v1.0.0-beta.10
01 September, 2023
If upgrading from 1.0.0-beta.9
, make sure you follow the upgrading guide.
Features
- Introduce Live queries, which allow you to write application with realtime updates pushed to your users.
- Introduce Full Text Search, which allows you to efficiently index and search through databases.
- Introduce Change feeds, which allow you to retrieve an ordered sequence of changes that happened to your database, enabling you to easily replicate changes.
- Introduce strict types into SurrealQL. With this change, types of fields on records can always be guaranteed.
- Implemented the HTTP
PUT
andPATCH
methods for complete tables. - Implemented the
insert
method in the websocket protocol to allow for bulk insertion of records. - Implemented missing JSON Patch operations (move, copy and test).
- Introduce a new
surreal validate [file]
CLI command to validate SurrealQL queries. - Introduce a new
--json
flag to thesurreal sql
CLI command to return JSON instead of SurrealQL objects. - Introduce a new
--endpoint
flag to thesurreal version
CLI command to check the version of a remote instance. - Introduce a new
--query-timeout
flag to thesurreal start
CLI command to limit the maximum execution time of queries. - Introduce a new
--client-ip
flag to thesurreal start
CLI command to change the behaviour for detecting client IP-addresses. - Allow multiline queries in the CLI by ending the line with a
\
character, then pressing enter. - Added a SpeeDB storage engine implementation.
- Add authentication support for local engines
- Introduce the ability to allow and deny certain capabilities of SurrealDB.
- Introduce the ability to create multiple root-level users.
- Introduce basic RBAC functionality.
- Introduce the full
fetch
-suite into our embedded scripting functions. - Introduce a new
query
function into our embedded scripting functions, enabling the execution of surrealql within them. - Introduce a new
value
function into our embedded scripting functions, enabling the retrieval of variables from the current execution context. - Expose all built-in SurrealQL methods to the embedded scripting functions.
- Introduce a new
UNSET
clause to remove properties from a document. - Introduce a new
DEFAULT
clause on field definitions. - Introduce a new
OMIT
clause to omit certain fields from a selection. - Introduce a new
COMMENT
clause on all resource definitions, allowing to add a description to such. - The
WHEN
clause on events is now optional. - Allow segmenting custom function names with multiple
::
splits. - Allow
DEFINE
andREMOVE
statements to be used within subqueries. - Introduce the ability to reference a object keys by a string, variable or document property.
- Introduce the ability to set permissions on
PARAM
s and customFUNCTION
s. - Introduce a simplified
IF-ELSE
syntax when using a block as the body. - Change the
INFO
statement to use full words instead of abbreviations. - Introduce the new
FOR
,CONTINUE
,BREAK
andTHROW
statements. - Introduce new
array::slice
,array::join
,array::clump
,array::find
,array::boolean_and
,array::boolean_or
,array::boolean_xor
,array::boolean_not
,array::logical_and
,array::logical_or
,array::logical_xor
,array::matches
,array::retain
,array::transpose
,array::first
,array::last
andarray::at
array methods. - Introduce new
string::contains
,string::distance::hamming
,string::distance::levenshtein
,string::similarity::fuzzy
,string::similarity::jaro
,string::similarity::smithwaterman
string methods. - Introduce new
vector::add
,vector::angle
,vector::cross
,vector::dot
,vector::divide
,vector::magnitude
,vector::multiply
,vector::normalize
,vector::project
,vector::subtract
,vector::distance::chebyshev
,vector::distance::euclidean
,vector::distance::hamming
,vector::distance::mahalanobis
,vector::distance::manhattan
,vector::distance::minkowski
,vector::similarity::cosine
,vector::similarity::jaccard
,vector::similarity::pearson
,vector::similarity::spearman
vector methods - Introduce new
time::ceil
,time::min
andtime::max
time methods. - Introduce a new
time::EPOCH
constant, containing the unix epoch timestamp as a datetime. - Introduce
type::field()
andtype::fields()
methods to dynamically select properties from a resource.
Bug fixes
-
Ensure duration addition and subtraction does not panic. -
Ensure custom functions can write to the database without being wrapped in a transaction. -
Ensure edge record deletes do not cascade to related records. -
Ensure http::*
functions parse response bodies as JSON instead of SurrealQL. -
Fixed HTTP errors sometimes being unable to be deserialised. -
Allow exports larger than 10,240 bytes for local engines. -
Fixed an issue where array::remove()
would cause a panic. -
Ensure string::slice()
properly handles UTF-8 values. -
Ensure large duration do not cause a panic. -
Prevent infinite recursion with futures causing an overflow. -
Ensure chained future
values have access to current document context. -
Ensure $input
is available to array fields inSET
clause -
Make computation depth 4X deeper, and configurable via the SURREAL_MAX_COMPUTATION_DEPTH
environment variable. -
Ensure JSON-like text is trimmed before parsing. -
Ensure Idiom paths result in writeable transactions where necessary. -
Visibly warn if failed to deserialize websocket response. -
Throw an error for invalid patches instead of panicking. -
Ensure errors do not hide other results. If any query in a set of queries were to fail in the CLI, only the error would be shown. This is now fixed by including the error in an array of results. -
Ensure duration is bigger than zero before truncating and rounding datetime. -
Pass on the error that causes a transaction to fail. -
Allow custom functions to be defined with an empty body. -
Limit output size of all string::*
functions to prevent memory exhaustion. -
Allow \
escaped characters to be used in embedded scripting functions. -
Fixed not being able to select from an array by index, directly after an array filter. -
Allow deletions of records on DROP
tables. -
Allow keywords as ident prefix. -
Ensure permissions are enforced on edge in
andout
fields. -
Ensure records can be INSERT
ed on tables with permissions. -
Ensure foreign tables are deleted fully when removed -
Support whitespace between function keyword and argument list for embedded scription functions. -
Ensure ON DUPLICATE KEY UPDATE
clause is displayed onINSERT
statement. -
Limit parser depth to prevent exhaustion and overflows. -
Prevent being able to create a record with an empty ID. -
Throw an error when a record is created with a mismatched record ID. -
Support arbitrary number of arrays in array::concat()
. -
Support ON DUPLICATE KEY UPDATE
when a unique index entry exists.
Performance improvements
-
Parsing nested objects and blocks now has an O(n) complexity instead of O(2^n), resulting in a 3666x improvement. -
The JSON parser is now 153 times faster.
Get realtime updates in your application with SurrealDB Live Queries
v1.0.0-beta.10
introduces Live Queries. This powerful technology allows you to write applications where you can serve realtime updates to your frontend.
// You can initiate the live query in your application
LET $lq = LIVE SELECT * FROM person WHERE age > 18;
// And you can dispose the live query once it is no longer needed
KILL $lq;
// SurrealDB SDKs allow you to easily initiate live queries, and to process the received messages.
const lq = await db.live('person', function({ action, result }) {
if (action == "CREATE") {
renderPersonInFrontend(result);
}
});
// Once you no longer need the live feed of updates, you can dispose the live query
await db.kill(lq);
Efficiently index and search your data with SurrealDB Full Text Search
With Full Text Search, you can efficiently store and index data, and search through it.
// We can define an analyzer on our database,
// Then, define an index to dictate which content we want to index (yep, that easy)
DEFINE ANALYZER simple TOKENIZERS blank,class FILTERS snowball(english);
DEFINE INDEX content ON article FIELDS content SEARCH ANALYZER simple BM25 HIGHLIGHTS;
// We can then create some content
CREATE article SET content = "Join us at SurrealDB World, as we unveil our version 1.0.0 to the world!";
CREATE article SET content = "We will absolutely be at Surreal World!";
// And lastly, select some data from the article table.
// We use the special "@num@" operator, where we define a reference.
// We can then reference back in the search::* functions.
SELECT
*,
search::score(1) AS score,
search::offsets(1) AS offsets,
search::highlight('<b>', '</b>', 1) AS highlighted
FROM article WHERE
content @1@ 'world'
ORDER BY score DESC;
Allow or deny capabilities when starting your SurrealDB instance
All capabilities are disabled by default. This means that by default, you are not able to use any methods, embedded scripting functions, make outbound network calls, or access the database anonymously. Down below follows a set of examples to showcase how one can configure capabilities.
Capabilities are further documented in the Capabilities documentation.
# Allow all capabilities
user@localhost % surreal start --allow-all
# Allow all functions, except for custom functions
user@localhost % surreal start --allow-funcs --deny-funcs fn
# Allow all capabilities, but deny guest/anonymous access to the database
user@localhost % surreal start --allow-all --deny-guests
Revamped root users
It is now possible to define multiple root users in SurrealDB. This change did require some changes in the way that you start your database however.
With this change, you will now only initially have to provide the --user
and --pass
flags to create the initial root user, but once the first root user exists, they will no longer by utilized.
For more information, check out the Authentication guide, and the surreal start
and DEFINE USER
documentation.
# When you initially start the database, you can create the first root user.
user@localhost % surreal start --auth --user root --pass root file:database.db
# In the future, you don't need to specify the --user and --pass flags anymore
user@localhost % surreal start --auth file:database.db
// Afterwards, you are able to create multiple root users.
DEFINE USER tobie ON ROOT PASSWORD "SecurePassword!" ROLES OWNER;
DEFINE USER jaime ON ROOT PASSWORD "SecurePassword!" ROLES EDITOR;
DEFINE USER john ON ROOT PASSWORD "SecurePassword!" ROLES VIEWER;
Strict typing in SurrealQL
v1.0.0-beta.10
introduces a more strict and powerful typing system.
It makes things more simple to understand, and it goes a long way in preventing all kinds of weird bugs in your schemas!
// Where you previously had to manually assert that a field does not contain NONE or NULL
DEFINE FIELD age ON person TYPE number ASSERT $value != NONE AND $value != NULL;
// You can now simply set the type and be ensured that only a number will be stored in that field.
DEFINE FIELD age ON person TYPE number;
// To make types optional, you can use the newly introduced "option<type>" type.
// This guarantees the field to be either empty (NONE), or a number.
DEFINE FIELD age ON person TYPE option<number>;
// We also introduces some new types!
DEFINE FIELD ratings ON movie TYPE array<number>;
DEFINE FIELD best_ratings ON movie TYPE array<number, 5>;
DEFINE FIELD unique_ratings ON movie TYPE set<number, 5>;
// And we made sure that the record type is also in-line with this new format
DEFINE FIELD author ON book TYPE record<person>;
// Lastly, it is now possible to set a union type.
DEFINE FIELD title ON article TYPE string | number;
// They can also be used in "nested" types
DEFINE FIELD description ON article TYPE option<string | number>;
DEFINE FIELD author ON article TYPE record<person | admin>;
Set a DEFAULT
value on field definitions
// You previously needed to involve logic with the "VALUE" clause to set a default value.
DEFINE FIELD enabled ON user TYPE bool VALUE $value OR $before OR true;
// You can now simply use the DEFAULT clause for this.
DEFINE FIELD enabled ON user TYPE bool DEFAULT true;
// As an added bonus, when you only define a VALUE clause with a simple value,
// then that will also act as the default value.
DEFINE FIELD constant ON demo VALUE 123;
// This will act the same as the following
DEFINE FIELD constant ON demo VALUE 123 DEFAULT 123;
PERMISSIONS
on global parameters and custom functions
Scope and anonymous users previously had access to every defined global parameter and function. You can now define these resources with a PERMISSIONS
clause to protect them.
DEFINE PARAM $perms_none VALUE 'Nobody can see me' PERMISSIONS NONE;
DEFINE PARAM $perms_full VALUE 'Everybody can see me' PERMISSIONS FULL;
DEFINE PARAM $perms_scope VALUE 'Only admins can see me' PERMISSIONS WHERE $scope = 'admin';
DEFINE FUNCTION fn::perms::none() {
RETURN 'Nobody can invoke me';
} PERMISSIONS NONE;
DEFINE FUNCTION fn::perms::full() {
RETURN 'Everybody can invoke me';
} PERMISSIONS FULL;
DEFINE FUNCTION fn::perms::scope() {
RETURN 'Only admins can invoke me';
} PERMISSIONS WHERE $scope = 'admin';
FOR
, BREAK
and CONTINUE
statements
// Here, we loop through an array with three numbers.
// For any number under two, we skip the iteration
// For any number above two, we break the loop
// The result is that only when the number is two, we will create the "person:two" record.
FOR $num IN [1, 2, 3] {
IF $num < 2 {
CONTINUE;
};
IF $num > 2 {
BREAK;
};
CREATE person:two;
};
// Here, we select the id for every person in the database, and we gift them a ticket to SurrealDB World.
FOR $person IN (SELECT VALUE id FROM person) {
CREATE gift CONTENT {
recipient: $person,
type: "ticket",
event: "SurrealDB World"
};
};
THROW
statement
Did something unexpected happen, and do you want to throw an error to the client? Now you can!
IF !is::string($variable) {
THROW "Expected a string, but got something else."
}
COMMENT
away on resource definitions!
// Give extra context about a certain resource
DEFINE TABLE user COMMENT "This table will store users!";
Less bulky IF-ELSE
statements
We found the IF-ELSE
statement to be a bit bulky at times.
Now, when you use a block ({}
) as the body of the statement, you can skip out on the THEN
and END
keywords!
// Previously
IF something = true THEN {
RETURN 123;
} END;
// Now
IF something = true {
RETURN 123;
};
More features for our embedded scripting functions
With fetch()
, query()
, value()
and basically every SurrealQL function now being available within the embedded scripting functions, they are a very powerful extension to SurrealQL, and can be used to solve complex problems otherwise impossible!
Read more about them in the Embedded scripting functions documentation.
function() {
const page = await fetch('https://google.com');
const people = await surrealdb.query('SELECT * FROM person');
const auth = await surrealdb.value('auth');
const uuid = surrealdb.functions.rand.uuid.v4();
}
Release v1.0.0-beta.9
01 April, 2023
If upgrading from 1.0.0-beta.8
, make sure you follow the upgrading guide.
Features
- Add WebSocket binary protocol
- Don’t treat
NONE
andNULL
as the same - Allow
SELECT
statements toSTART AT 0
- Add
not()
function for negating a value - Add support for mathematical constants
- Add functionality for open telemetry tracing
- Add support for SQL parameters in HTTP REST endpoints
- Log release version identifier when starting the server
- Add
string::is::url()
function for checking if a string is a URL - Implement inclusive and unbounded record rangese
- Support negative offsets in SQL string::slice() function
- Add
time::timezone()
function for getting the local timezone offset - Add
is::datetime()
function for checking if a value is a datetime - Add ability to set global parameters using
DEFINE PARAM
statements - Prevent invalid aggregate functions being used in
GROUP BY
clauses - Check expressions for
SPLIT ON
,GROUP BY
, andORDER BY
clauses - Enable fetching fields and values from within complex Record IDs
- Allow parameters in
LIMIT
andSTART
clauses inSELECT
statements - Add
parse::url::scheme()
function for parsing a url protocol - Add
time::format()
function for formatting of datetimes - Add support for
FETCH
clauses in SQLRETURN
statements - Add
rand::uuid::v4()
andrand::uuid::v7()
functions for creating different UUID types - Add Null Coalescing Operator and Ternary Conditional Operator
- Enable current input to be retrieved in
ON DUPLICATE KEY UPDATE
clauses with$input
parameter - Add
math::pow()
function and**
operator - Ensure command-line exits with non-zero code on errors
- Add
IN
andNOT IN
operators as aliases toINSIDE
andNOT INSIDE
- Add command-line argument flag to disable SurrealDB banner at server startup
- Enable calling SurrealQL functions from within JavaScript scripting runtime
- Add support for
FLEXIBLE
fields onSCHEMAFULL
tables - Add additional array functions for array checking, and manipulation:
-
array::all()
,array::any()
,array::pop()
-array::add()
,array::append()
,array::insert()
,array::prepend()
,array::push()
-array::remove()
,array::reverse()
,array::group()
,array::complement()
Bug fixes
-
Enable searching within Record IDs using the CONTAINS operator -
Ensure date strings are not treated as datetimes -
Limit computation depth in functions, futures, and subqueries -
Ensure SQL queries are parsed completely or fail -
Ensure all valid unicode characters are parsed without failing -
Ensure nested non-defined objects are not stored in SCHEMAFULL
tables -
Ensure equals comparator function never reeaches unreachable code -
Ensure cancelled context does not prevent FETCH of records -
Ensure GROUP BY fields with functions are output correctly -
Ensure system parameters are not able to be overridden -
Ensure record is only deleted after permissions have been checked -
Ensure double quote characters are always escaped properly -
Ensure RocksDB range scans are inclusive at the start -
Ensure uncaught JavaScript exceptions are caught in JavaScript runtime -
Do not run permissions on DEFINE EVENT
andDEFINE TABLE
queries -
Ensure invalid datetimes to not panic
Performance improvements
-
Limit computation depth in functions, futures, and subqueries -
Ensure PERMISSIONS clauses are not run for ROOT / NS / DB users
Enable returning a single field from SELECT
statements
SurrealDB now allows you to fetch a single field from a SELECT
statement or from a subquery, returning an array of values, rather than an array of objects.
// We can now select an array of just a single field
SELECT VALUE name FROM person;
Add support for FETCH
clauses at the end of RETURN
statements
SurrealDB now allows fetching remote fields in RETURN
statements, in addition to SELECT
statements.
// We can now fetch fields from RETURN statements
RETURN person:tobie FETCH account, connections;
Add Null Coalescing operator and Ternary Conditional operator
SurrealDB now supports the Null Coalescing operator (??
), and the Ternary Conditional operator (?:
) for simpler logic calculations.
// Check whether either of two values are truthy
SELECT * FROM NULL ?: 0 ?: false ?: 10; -- Will return 10
// Check whether either of two values are truthy and not NULL.
SELECT * FROM NULL ?? 0 ?? false ?? 10; -- Will return 0
Allow fetching fields from within complex Record IDs
Fields within complex Record IDs can now be retrieved and used within SELECT
queries.
// Create a record with a complex Record ID
CREATE recording:{ location: 'London', date: time::now() } SET temperature = 23.7;
// Select a field from within the Record ID
SELECT id.location, temperature FROM recording;
Support for FLEXIBLE
fields on SCHEMAFULL
tables
SCHEMAFULL
and SCHEMALESS
functionality can now be used together, suitable for capturing schema-free log data.
// We want to define a SCHEMAFULL table
DEFINE TABLE person SCHEMAFULL;
// But we want one field to allow any content
DEFINE FIELD settings on person FLEXIBLE TYPE object;
// We can then set the field value without a schema
CREATE person:test CONTENT {
settings: {
nested: {
object: {
thing: 'test'
}
}
}
};
Support code blocks and advanced expressions
It is now possible to run blocks of code, with support for an arbitrary number of statements, including LET
and RETURN
statements. This allows for writing advanced custom logic, and allowing for more complicated handling of data operations.
DEFINE FIELD average_sales ON metrics VALUE {
LET $sales = (SELECT VALUE quantity FROM sales);
LET $total = math::sum($sales);
LET $count = count($sales);
RETURN ($total / $count);
};
Enable access to the user input in DEFINE FIELD
statements
DEFINE FIELD
statements now have access to an $input
variable in the VALUE
and ASSERT
clauses.
DEFINE FIELD name ON TABLE person
-- Prevent any changes, if the input value is not title-case
ASSERT IF $input THEN $input = /^[A-Z]{1}[a-z]+$/ ELSE true END
-- If the input value is title-case, then add a prefix to the field
VALUE IF $input THEN 'Name: ' + $input ELSE $value END
;
-- This record update will fail
UPDATE person:test CONTENT { name: 'jaime' };
-- This record update will succeed
UPDATE person:test REPLACE { name: 'Tobie' };
Define global parameters with DEFINE PARAM
statements
SurrealDB now supports the ability to define global database-wide parameters, which can be useful to store certain commonly-used variables.
-- Define a global parameter which will be accessible to all queries.
DEFINE PARAM $STRIPE VALUE "https://api.stripe.com/payments/new";
-- Use the defined global parameter in all queries on the database.
DEFINE EVENT payment ON TABLE order WHEN $event = 'CREATE' THEN http::post($STRIPE, $value);
// We can also override the parameter for certain queries if we want
LET $STRIPE = "https://api-dev.stripe.com/payments/new";
// And fetching this parameter will fetch the locally defined parameter
RETURN http::post($STRIPE, $value);
Define custom functions with DEFINE FUNCTION
statements
SurrealDB now supports the ability to define global database-wide custom functions, which allow for complicated or repeated user-defined code, to be run seamlessly within any query across the database. Custom functions support typed arguments, and multiple nested queries with custom logic.
-- Define a global function which can be used in any query
DEFINE FUNCTION fn::get_person($first: string, $last: string, $birthday: string) {
LET $person = SELECT * FROM person WHERE [first, last, birthday] = [$first, $last, $birthday];
RETURN IF $person[0].id THEN
$person[0]
ELSE
CREATE person SET first = $first, last = $last, birthday = $birthday
END;
};
-- Call the global custom function, receiving the returned result
LET $person = fn::get_person('Tobie', 'Morgan Hitchcock', '2022-09-21');
Add support for selecting ranges of records
SurrealDB supports the ability to query a range of records, using the record ID. The record ID ranges, retrieve records using the natural sorting order of the record IDs. These range queries can be used to query a range of records in a timeseries context.
-- Select all person records with IDs between the given range
SELECT * FROM person:1..1000;
-- Select all records for a particular location, inclusive
SELECT * FROM temperature:['London', NONE]..=['London', time::now()];
-- Select all temperature records with IDs less than a maximum value
SELECT * FROM temperature:..['London', '2022-08-29T08:09:31'];
-- Select all temperature records with IDs greater than a minimum value
SELECT * FROM temperature:['London', '2022-08-29T08:03:39']..;
-- Select all temperature records with IDs between the specified range
SELECT * FROM temperature:['London', '2022-08-29T08:03:39']..['London', '2022-08-29T08:09:31'];
Rust API with embedded and remote connections over HTTP and WebSockets
SurrealDB now has a native Rust API for connecting to local database instances, and remote database servers over both HTTP and WebSockets with a binary protocol and a custom serialization format.
static DB: Lazy<Surreal<Client>> = Lazy::new(Surreal::init);
#[tokio::main]
async fn main() -> surrealdb::Result<()> {
// Connect to the database
DB.connect::<Wss>("cloud.surrealdb.com").await?;
// Select a namespace + database
DB.use_ns("test").use_db("test").await?;
// Create or update a specific record
let tobie: Option<Record> = DB
.update(("person", "tobie"))
.content(Person { name: "Tobie" })
.await?;
Ok(())
}
Release v1.0.0-beta.8
30 September, 2022
Features
- Improve HTTP request error messages
- Add support for dynamic expressions in Record IDs
- Add support for
PERMISSIONS
clauses to be separated by commas or spaces - Allow deep-merging in
UPDATE ... MERGE
statements - Add debug and trace logging for authentication attempts
- Make validation and parsing functions more robust with certain edge cases
- Ignore empty or blank lines when using the SurrealDB SQL REPL
- Use a dedicated executor thread for CPU-intensive functions
- Ensure server listens to, and gracefully exits, on SIGINT/SIGTERM signals
- Add
duration
functions for calculating durations as specific units - Add support for calculating the duration between two datetime values
- Improve error message when automatically creating a table without authorization
- Add support for uppercase and lowercase object keys in JWT authentication tokens
- Allow namespaced claim aliases in JWT token using full domain-specific key names
- Add support for retrieving JWT authentication token contents using
$token
parameter - Add support for different content return types on
/signup
and/signin
HTTP routes - Add
session::token()
function for retrieving the contents of the JWT authentication token - Ensure
NONE
andNULL
values are not automatically cast to another type when updating records - Add
/health
HTTP endpoint for checking the database and data storage engine status - Add
crypto::bcrypt::generate()
andcrypto::bcrypt::compare()
functions - Improve error messages for unique indexes, when inserting records which already exist in the index
- Add
meta::tb()
andmeta::id()
functions for retrieving separate parts of a SurrealDB Record ID - Add support for using 3rd party authentication JWTs with
DEFINE TOKEN ... ON SCOPE ...
Bug fixes
-
Add support for escaped characters and unicode characters in strings -
Ensure datetimes work correctly in Eastern timezones -
Ensure is::uuid()
parses valid UUIDs correctly -
Ensure LET
statements throw errors correctly on failure -
Ensure Record IDs are parsed correctly when defined as a string -
Fix bug where escaped characters were not supported in regex values -
Ensure datetimes with milliseconds or nanoseconds are parsed correctly -
Ensure datetimes with partial timezones are correctly calculated -
Ensure time::month()
returns the month of the specified datetime -
Ensure FETCH
clauses fetch the respective data correctly -
Handle connection errors properly when WebSocket clients disconnect improperly -
Ensure HTTP session is not verified multiple times when requesting an invalid HTTP route -
Use Accept
header instead ofContent-Type
header for client content negotiation -
Fix key scan range iteration in RocksDB, which caused SurrealDB to randomly crash -
Ensure authenticated session data is stored after successful scope signup / signin -
Fix bug where http
functions would panic when an invalid URI was specified -
Ensure correct transaction type (optimistic / pessimistic) was initiated when using TiKV distributed storage engine -
Ensure math::mean()
,math::median()
, andmath::sqrt()
work correctly with empty or zero values -
Fix bug where MultiPoint
,MultiLine
, andMultiPolygon
geometry values were not formatted correctly -
Fix bug where defined fields with empty values would be set on the root object, losing the object structure
Performance improvements
-
Miscellaneous performance optimizations and code cleanup -
Limit maximum allowed runtime and memory in JavaScript functions -
Ensure crypto
andrand
functions do not allow unbounded resource usage -
Ensure read-only transactions are used when write functionality is not needed when using TiKV distributed storage engine
Embedding the results of a subquery
SurrealDB now allows you to use the resulting Record IDs of a subquery, and embed them within another record, without having to specify the id
field specifically.
// It is now possible to define a new Record ID as normal
LET $friend = (CREATE person SET name = 'Jaime');
// There is no need to select the `id` field using `$friend.id`
CREATE person SET name = "Tobie", friend = $friend;
Embedded Record IDs in CREATE
and RELATE
statements
SurrealDB now supports the ability to set a specific reocrd ID in the record content itself when creating records using CREATE
and RELATE
statements.
// It is now possible to define a new Record ID as normal
CREATE person:test SET name = "Tobie";
// Or by specifying the Record ID within the record data part of the query
CREATE person SET id = person:test, name = "Tobie";
// In addition the ID can be specified in a CONTENT object
CREATE person CONTENT { id: person:test, name: "Tobie" };
// Numbers can be used, and will be converted into a Record ID
CREATE user CONTENT { id: 1, name: "Robert" };
// Strings can be used, and will be converted into a Record ID
CREATE city CONTENT { id: "London", name: "London" };
// UUIDs can be used, and will be converted into a Record ID
CREATE city CONTENT { id: "8e60244d-95f6-4f95-9e30-09a98977efb0", name: "London" };
// Complex values can be used, and will be converted into a Record ID
CREATE temperature CONTENT { id: ["London", time::now()], name: "London" };
// Complex values can be used, and will be converted into a Record ID
RELATE person:tobie->knows->person:jaime SET id = knows:cofounder;
Support for deep-merge record updates
SurrealDB now supports deep-merging record changes, ensuring that only specified object fields are modified and overwritten.
// Create a record
CREATE person:test SET name.initials = 'TMH', name.first = 'Tobie', name.last = 'Morgan Hitchcock';
// Use the MERGE clause to deep merge objects, without overwriting any fields
UPDATE person:test MERGE {
name: {
title: 'Mr',
initials: NONE,
},
tags: ['Rust', 'Golang', 'JavaScript'],
};
Dynamic expressions in complex Record IDs
Complex Record IDs now support dynamic expressions, allowing parameters, and function expressions to be used as values within the IDs. This is useful in a timeseries context, or for ensuring locality between specific records in a table.
// Set a new parameter
LET $now = time::now();
// Create a record with a complex ID using an array
CREATE temperature:['London', $now] SET
location = 'London',
date = $now,
temperature = 23.7
;
// Create a record with a complex ID using an object
CREATE temperature:{ location: 'London', date: $now } SET
location = 'London',
date = $now,
temperature = 23.7
;
// Select a range of records up to the current time
SELECT * FROM temperature:['London', '2022-08-29T08:03:39']..['London', time::now()];
Release v1.0.0-beta.7
29 August, 2022
- Add a Windows
amd64
release build - Add support for Objects and Arrays as Record IDs
- Add support for querying records using Record ID ranges
- Add SQL
session
functions for retrieving session variables - Make
--ns
and--db
arguments optional in command-line REPL - Return an error when the specified datastore is not able to be initiated
- Enable root authentication for client libraries using WebSocket protocol
- Ensure
math::sum()
returns a number instead of aNONE
value, when called on a non-array value - Add ACID compliant, persistent, on-disk storage implementation, with multiple concurrent writers using RocksDB
Complex Record IDs
SurrealDB now supports the ability to define record IDs using arrays and objects. These values sort correctly, and can be used to store values or recordings in a timeseries context.
// Create a record with a complex ID using an array
CREATE temperature:['London', '2022-08-29T08:03:39'] SET
location = 'London',
date = '2022-08-29T08:03:39',
temperature = 23.7
;
// Create a record with a complex ID using an object
CREATE temperature:{ location: 'London', date: '2022-08-29T08:03:39' } SET
location = 'London',
date = '2022-08-29T08:03:39',
temperature = 23.7
;
// Select a specific record using a complex ID
SELECT * FROM temperature:['London', '2022-08-29T08:03:39'];
Record ID ranges
SurrealDB now supports the ability to query a range of records, using the record ID. The record ID ranges, retrieve records using the natural sorting order of the record IDs. With the addition of complex record IDs above, this can be used to query a range of records in a timeseries context.
// Select all person records with IDs between the given range
SELECT * FROM person:1..1000;
// Select all temperature records with IDs between the given range
SELECT * FROM temperature:['London', '2022-08-29T08:03:39']..['London', '2022-08-29T08:09:31'];
Release v1.0.0-beta.6
13 August, 2022
- Add command-line SurrealQL REPL for quick querying of a database
- Log username at server startup when root authentication is enabled
- Enable SurrealDB server to be configured using environment variables
- Implement config definition key and value caching within a transaction
- Add
array::sort
functions for sorting of arrays and array fields - Ensure an error is returned when selecting from a non-existent table in strict mode
- Allow polymorphic remote record constraints in
DEFINE FIELD
statements - Fix bug with SQL export, where
DEFINE INDEX
statements were not exported - Fix bug where multi-yield path expressions with multiple alias outputs were returning nested arrays
- Fix bug where aliased field was not output when fetching a multi-yield expressions with a final alias yield
SurrealDB REPL
SurrealDB now supports the ability to start a command-line REPL to query a local or remote database from the terminal.
user@localhost % surreal sql --conn http://localhost:8000 --user root --pass root --ns test --db test
Release v1.0.0-beta.5
01 August, 2022
- Temporarily disable HTTP response compression
- Improve
surreal import
andsurreal export
cli commands - Fix bug where
GROUP BY
fields with an aliasAS
name were not output correctly - Fix SQL syntax parsing bug when specifying
DEFINE INDEX
statements withUNIQUE
constraints
Release v1.0.0-beta.4
28 July, 2022
- Add new strict mode to SurrealDB server
- Ensure default table permissions are set to
NONE
notFULL
- Fix bug when defining
NS
andDB
without first selecting aNS
orDB
- Fix bug with
VALUE
andASSERT
clauses inDEFINE FIELD
statements when fields are not present in query
Strict mode
SurrealDB now supports the ability to startup in strict mode. When running in strict mode, all NAMESPACE
, DATABASE
, and TABLE
definitions will not be created automatically when data is inserted. Instead if the selected namespace, database, or table has not been specifically defined, then the query will return an error.
user@localhost % surreal start --strict --log debug memory
Release v1.0.0-beta.3
24 July, 2022
- Enable years as a unit in durations (
1y
) - Log root authentication configuration status on server startup
- Ensure CORS headers are set on all HTTP responses even when request fails with an error
- Improve syntax for defining futures:
fn::future ->
changed to<future>
- Improve syntax for defining embedded functions:
fn::script -> () =>
changed tofunction()
- Ensure root authentication is completely disabled when
-p
or--pass
cli arguments are not specified
Release v1.0.0-beta.2
20 July, 2022
- Improve command-line logging output
- Enable new
--log
argument for specifying server log level - Hide default randomly-generated server password
- Ensure correct version is displayed when running
surreal version
command
Release v1.0.0-beta.1
18 July, 2022
- Entire SurrealDB codebase re-written in Rust
- Single-node, in-memory storage for development use
- Highly-available, highly-scalable distributed storage for production use
- Improved SurrealQL query language with faster parsing, and embedded inspection of types
- Performance improvements with data parsing, and serialization and deserialization of records
- Added support for casting and converting between different data types
- Added a new data type for storing values which should only be computed in the future when selected for output
- Embedded JavaScript functions for writing complex functions and triggers, with runtime context isolation
- Addition of nested GeoJSON data types, including
Point
,Line
,Polygon
,MultiPoint
,MultiLine
,MultiPolygon
, andCollection
values
Futures
Values which should be computed only when outputting data, can be stored as futures. These values are stored in SurrealDB as SurrealQL code, and are calculated only when output as part of a SELECT
clause.
Casting
In SurrealDB, all data values are strongly typed. Values can be cast and converted to other types using specific casting operators. These include bool
, int
, float
, string
, number
, decimal
, datetime
, and duration
casts.
UPDATE product SET
name = "SurrealDB",
launch_at = "2021-11-01",
countdown = <future> { launch_at - time::now() }
;
UPDATE person SET
waist = <int> "34",
height = <float> 201,
score = <decimal> 0.3 + 0.3 + 0.3 + 0.1
;
RELATE statements
The RELATE
statement adds graph edges between records in SurrealDB. It follows the convention of vertex -> edge -> vertex
or noun -> verb -> noun
, enabling the addition of metadata to the edge record.
INSERT statements
The INSERT
statement resembles the traditional SQL statement, enabling users to get started quickly. It supports the creation of records using a VALUES
clause, or by specifying the record data as an object.
-- Add a graph edge between user:tobie and article:surreal
RELATE user:tobie->write->article:surreal
SET time.written = time::now()
;
-- Add a graph edge between specific users and developers
LET $from = (SELECT users FROM company:surrealdb);
LET $devs = (SELECT * FROM user WHERE tags CONTAINS 'developer');
RELATE $from->like->$devs UNIQUE
SET time.connected = time::now()
;
INSERT INTO company {
name: 'SurrealDB',
founded: "2021-09-10",
founders: [person:tobie, person:jaime],
tags: ['big data', 'database']
};
INSERT IGNORE INTO company (name, founded)
VALUES ('SurrealDB', '2021-09-10')
ON DUPLICATE KEY UPDATE tags += 'developer tools'
;
Expressions
SurrealQL supports fetching data using dot notation .
, array notation []
, and graph semantics ->
. SurrealQL enables records to link to other records and traverses all embedded links or graph connections as desired. When traversing and fetching remote records SurrealQL enables advanced filtering using traditional WHERE
clauses.
-- Select a nested array, and filter based on an attribute
SELECT emails[WHERE active = true] FROM person;
-- Select all 1st, 2nd, and 3rd level people who this specific person record knows, or likes, as separate outputs
SELECT ->knows->(? AS f1)->knows->(? AS f2)->(knows, likes AS e3 WHERE influencer = true)->(? AS f3) FROM person:tobie;
-- Select all person records (and their recipients), who have sent more than 5 emails
SELECT *, ->sent->email->to->person FROM person WHERE count(->sent->email) > 5;
-- Select other products purchased by people who purchased this laptop
SELECT <-purchased<-person->purchased->product FROM product:laptop;
-- Select products purchased by people in the last 3 weeks who have purchased the same products that we purchased
SELECT ->purchased->product<-purchased<-person->(purchased WHERE created_at > time::now() - 3w)->product FROM person:tobie;
Embedded JavaScript functions
Javascript functions can be used for more complex functions and triggers. Each Javascript function iteration runs with its own context isolation - with the current record data passed in as the execution context or this
value.
CREATE film SET
ratings = [
{ rating: 6, user: user:bt8e39uh1ouhfm8ko8s0 },
{ rating: 8, user: user:bsilfhu88j04rgs0ga70 },
],
featured = function() {
return this.ratings.filter(r => {
return r.rating >= 7;
}).map(r => {
return { ...r, rating: r.rating * 10 };
});
}
;
Release v0.3.0
14 December, 2021
- Enable query and session parameters to be defined on a JSON-RPC connection
- Ensure subqueries can access encoding parent query and grand-parent queries
- Add diff-match-patch functionality when updating document records
- Separate authentication levels for Namespace and Database specific access
- Authentication scope definition and setup, with user-defined authentication logic for each scope
Pre-defined aggregate analytics views
Aggregate views let you pre-compute analytics queries as data is written to SurrealDB. Similarly to an index, a table view lets you select, aggregate, group, and order data, with support for moving averages, time-based windowing, and attribute-based counting. Pre-defined aggregate views are efficient and performant, with only a single record modification being made for every write.
-- Drop all writes to the reading table. We don't need every reading.
DEFINE TABLE reading DROP;
-- Define a table as a view which aggregates data from the reading table
DEFINE TABLE temperatures_by_month AS
SELECT
count() AS total,
time::month(recorded_at) AS month,
math::mean(temperature) AS average_temp
FROM reading
GROUP BY city
;
-- Add a new temperature reading with some basic attributes
CREATE reading SET
temperature = 27.4,
recorded_at = time::now(),
city = 'London',
location = (-0.118092, 51.509865)
;
Release v0.2.0
21 January, 2021
- Parameters can be used to store values or result sets
- Nested subquery functionality, with scoped parameters
- Nested field query notation allowing nested arrays and objects to be queried
- Mathematical operators for complex mathematical calculations in queries
- Advanced functions for working with arrays, strings, time, validation, parsing, and counting
Release v0.1.0
08 December, 2019
- Multi-tenancy data separation, with namespaces and databases
- Schemafull or schemaless tables with limitless document fields
- Multi-table, multi-row, serialisable ACID transactions
- Table fields, table change events, table indexes, and data constraints
- Advanced data model including empty values, strings, numbers, objects, arrays, durations, and datetimes