SurrealDB 2.0

Bug fixes:

• •

  Check limit to string::replace with regular expressions

• •

  Fixes long running FOR loops, adds script timeouts, improves HTTP redirect security, and optimises task yielding for better performance

• •

  Backport JS scripting mem exhaustion fix

• •

  Introduces Regex size limit configuration

• •

  Improve error handling on value conversion (RPC / HTTP/SQL endpoint)

• •

  Update rquickjs to newest version

Security advisories:

• •

  Uncaught exception in Net module leads to database crash

• •

  Server-takeover via SurrealQL injection on backup import

• •

  Memory exhaustion via nested functions and scripts

• •

  Memory exhaustion via string::replace using regex

• •

  No JavaScript script function default timeout could facilitate DoS

• •

  CPU exhaustion via custom functions result in total DoS

• •

  Bypass of deny-net flags via redirect results in SSRF

Performance improvements:

• •

  Added lower and upper bounds for RocksDB iterators

• •

  Added with or without index benchmark

• •

  Improved behaviour and performance of reduced document scopes

Bug fixes:

• •

  Fixed panic when calling Thing::try_from("") - with empty value

• •

  Fixed parsing of functions and models in selectors

• •

  Fixed clippy warnings in release build

• •

  Fixed micro second parsing

Workflow improvements:

• •

  Ensure unstable code is also checked in CI

• •

  Update versions actively supported for security fixes

Security advisories:

• •

  Improper Authorisation in Select Permissions

• •

  Uncaught Exception Handling Parsing Errors on Empty Strings

Features:

• •

  Allowed transactions during imports

• •

  Deprecated DELETE on field permissions

• •

  Implemented Ord for Duration and Table

• •

  Improved Number ordering accuracy

• •

  Made max_value_size configurable in surrealkv

Parser improvements:

• •

  Update reblessive

• •

  Fixed digit object name export

Bug fixes:

• •

  Fixed record IDs not being used as Object keys

• •

  Fixed ID type conversion

• •

  Fixed thing parsing function

• •

  Removed truthiness check on Datetime that requires after epoch dates

• •

  Reintroduced meta:: scope as deprecated methods

• •

  Fixed idiom path processing

• •

  Fixed value conversion to a vector: NONE

Workflow improvements:

• •

  Update SurrealKV version to 0.3.7

• •

  Update rocksdb 3rd party dependency

• •

  Create major.minor and major tags when pushing to Docker

• •

  Update surrealcs dependency

Bug fixes:

• •

  Fixed fields being parsed as idioms.

• •

  Fixed error message panics

• •

  Fixed Incorrect lowercase filter in analyzer

• •

  Fixed mismatch in record id for edge tables

• •

  Fixed immediate insert subqueries

• •

  Fixed single check for rpc insert

• •

  Fixed decimal parsing

• •

  Fixed failed authenticate resulting in session reset

• •

  Fixed version patching

• •

  Fixed query planner ignoring isolated predicates in WHERE clause

• •

  Fixed inconsistent numeric object key

• •

  Fixed ULID/UUID gen, programmatically generating ranges, RETURN inside FOR/IF and improves arithmetic operations

• •

  Fixed CBOR Range encoding

• •

  Fixed parsing bug in graph for bidirectional relationships

• •

  Fixed compilation issue on 32bits platforms (other than wasm32)

• •

  Fixed setting a range id on a document's id, in and out fields

• •

  Fixed max key size for surrealkv store

• •

  Included request id in CBOR encoding failure message

• •

  Fixed panic when the query file size is too large.

• •

  Fixed bug with number ordering

Improvements:

• •

  Improved log messages for authentication errors

  • ◦

    Explicitly check for token expiration error

• •

  Added GraphQL description exactly matches the comments set by the user

• •

  Added configuration capabilities for SurrealCS connection pool size

• •

  Added test for including an ID in the content body

• •

  Added minor improvements to core engine

• •

  Added VERSION clause to all queries

• •

  Added additional ULID and UUID utilities

• •

  Added DTrace profiling support

Bug fixes:

• •

  Fixed infinite loop in parser

After numerous alpha and beta releases over months of development, we are releasing SurrealDB v2.0.0! 🎉. The changes for this release pertain to those in between the final 2.0.0-beta-3 and other alpha versions.

NOTE: Because of changes in the underlying way SurrealDB stores data, steps are required to migrate to 2.0.0 from previous versions. Please use the following steps for any existing databases:

• •

  If you are moving from 1.x to 2.x, use the surreal fix command.

Features:

• •

  Enable FoundationDB transaction timeout customisation

• •

  Implement missing modulo operator in new parser

• •

  Restructure the parser around the concept of token gluing.

• •

  Added DEFINE ACCESS statement to grant access to resources.

• •

  Deprecated DEFINE SCOPE in favour of DEFINE ACCESS ... TYPE RECORD.

• •

  Deprecated DEFINE TOKEN in favour of DEFINE ACCESS ... TYPE JWT.

• •

  Deprecated DEFINE TOKEN ON SCOPE in favour of DEFINE ACCESS ... TYPE RECORD ... WITH JWT.

• •

  Added customisable algorithm and key when issuing tokens for records users with the WITH ISSUER clause.

• •

  Added customisable token and session duration to DEFINE USER and DEFINE ACCESS.

• •

  Removed session expiration when not explicitly defined.

• •

  Changed the INFO statement to redact secrets defined via DEFINE ACCESS.

• •

  Changed HTTP request headers expected by SurrealDB to require the surreal- prefix.

• •

  Removed the --auth flag in the CLI to enable it by default. Added --unauthenticated.

• •

  Removed the --enable-auth-level flag in CLI to enable the behaviour by default. Defaults to root.

• •

  Changed authentication to expect level via the --auth-level flag or surreal-auth-* headers.

• •

  Changed the default --bind address in the CLI from 0.0.0.0 to 127.0.0.1.

• •

  Added the --no-identification-headers flag in the CLI to limit information leakage.

• •

  Added type::range function.

• •

  Added TEMPFILES clause to the SELECT statement.

• •

  Introduce resource creation checks for DEFINE statement to avoid duplicates.

• •

  Added string functions for IP string::is::ip, string::is::ipv4, string::is::ipv6.

• •

  Introduce resource creation checks for DEFINE statement to avoid duplicates.

• •

  Added string functions for IP string::is::ip,string::is::ipv4,string::is::ipv6.

• •

  Added string functions for HTML: string::html::encode, string::html::sanitize.

• •

  Added new math functions: math::acos, math::acot, math::asin, math::atan, math::clamp, math::cos, math::cot, math::deg2rad, math::lerp, math::lerpangle, math::ln, math::log, math::log2, math::log10, math::rad2deg, math::sign, math::sin, math::tan.

• •

  Support batch import with INSERT statement. This also extends to relationships between tables using the INSERT RELATION statement

• •

  Allow bridge connectivity in all docker images built

• •

  Added define JWT access at the root level

• •

  Allow authenticating with a token from the CLI

• •

  Added Asynchronous building for indexes

• •

  Improved HNSW persistence

• •

  Added Range value support

• •

  Added ability to SELECT with version, CREATE with version and INSERT with version on SurrealKV

• •

  Added ENFORCED keyword to DEFINE TABLE

• •

  SCHEMAFULL tables are now TYPE NORMAL by default

• •

  Added literal types to the query language

• •

  Added the .chain() method

• •

  Added typed LET statements

• •

  Added typed function returns

• •

  Added record::exists() function

• •

  Added new functions value::diff() and value:::patch()

• •

  Array function changes and additions: array::all(), array::any(), array::filter_index(), array::find_index() functions can now take a value or a closure. Also added array::filter() and array::find() functions.

• •

  Upgraded SurrealKV to v0.3

• •

  More flexible create API for Rust SDK

• •

  Query depth limits can now be configured

• •

  Added GraphQL config

• •

  Indexing operators can now take expressions

• •

  Support for mixed number types on range queries

• •

  Improved capabilities logic with specific allows able to override general denies

• •

  "false" string no longer eagerly evaluated as false

• •

  time::epoch constant now available

• •

  Added geo::valid() function

• •

  Method call syntax now works on integers

• •

  Improved GraphQL record fetching

• •

  Added AUTHENTICATE clause on RECORD access

• •

  Enabled publishing SurrealDB ports in Docker for bridge networking connectivity

• •

  Allow generating ULID and UUID based on a timestamp

• •

  Allow unsetting namespace and database on RPC protocol

• •

  Moved fetch resolution to a compute method

• •

  Added ALTER TABLE statement

• •

  Moved AUTHENTICATE clause to the root of DEFINE ACCESS statement to allow the clause to be used in every access method

• •

  Added OVERWRITE clause for defined resources

• •

  Added array::windows() function

• •

  Added support for idiom destructuring in SurrealQL

• •

  Added string::is::record function

• •

  Added Method chaining for Database functions

• •

  Added anonymous functions

• •

  Added typed function returns

• •

  Function return types now shown in INFO statement

• •

  Type any can now act as an optional type

• •

  Added time::is::leap_year() function

• •

  Added environment variables to configure HTTP endpoint limits & JavaScript runtime limits

• •

  Introduced savepoint capabilities

• •

  Added experimental VERSION clause to INFO FOR DB and INFO FOR TABLE

• •

  Added additional chained methods to number values

Performance improvements:

• •

  SELECT/count optimisation on table range

• •

  Improvements to node membership processing and garbage collection

• •

  Improvements to select count table scan

• •

  Avoid potentially some methods twice

• •

  Stabilise and use parser2 for parsing.

• •

  KNN filtering with limit and KNN distance function.

Bug fixes:

• •

  Fixed incorrectly parsed identifier characters

• •

  Fixed record<T> casting from string

• •

  Connection ID now stored inside session data

• •

  Fixed certain identifier characters not being parsed correctly

• •

  Fixed remote fields added to output result inside SELECT * queries

• •

  Revise RecordAccess structure to aid v2 alpha upgrade

• •

  Fixed GHSA-gh9f-6xm2-c4j2: Improper Authentication when Changing Databases as Scope User

• •

  Fixed argument check for array::clump

• •

  Fixed aborting task leading to an error

• •

  Fixed non-enforced field permissions on WHERE statements

• •

  Fixed failing search when search analyzer lacks tokenisers

• •

  Fixed core dump when performing INSERT

• •

  Fixed discarding without error when using RELATE statement on edge that already exists

• •

  Fixed modification of arrays with JSON Patch semantics

• •

  Fixed selecting directly from a graph edge failing to select all primary edges

• •

  Fixes to issues with composite indexes

• •

  Fixed INSERT not working in subqueries

• •

  Fixed KNN brute force error when an index exists.

• •

  Fixed Redefining HNSW index error.

• •

  Check that a field exists before index creation.

• •

  Fixed missing serialisation for math::inf.

• •

  Fixed polygon serialisation.

• •

  Fixed panic and infinite loop in parser.

• •

  Fixed URL parsing function when a port is present.

• •

  Fixed display formats for access methods

• •

  Fixed parser error to allow ns as an alias for NAMESPACE

• •

  Added missing datetime prefix

• •

  Removed duplicated badge for docker pulls count in README.md

• •

  Fixed export for sql::Filter type in surrealdb-core

• •

  Updated deprecated version of Node.js

• •

  Fixed ELSE IF pretty formatting

• •

  Changed geometry point values to floats in CBOR

• •

  Fixed authentication checks on root user websocket RPC session

• •

  Ensured a transaction is always available when live queries are evaluated

• •

  Reduced read lock duration while processing connection messages

• •

  Ensured execution halts after the RETURN statement

• •

  Ensured the server shuts down properly

• •

  Fixed querying complex edge table

• •

  Removed unnecessary backticks from INFO FOR statement output

• •

  Fixed live query panic issue

• •

  Fixed validation on schemafull RELATE

• •

  Fixed division by zero bug in math::mean() function

• •

  Improved string::is::datetime and string::is::uuid

• •

  Parse like (~) operator correctly

• •

  Removed faulty flatten in graph traversal

• •

  Fixed record<T> casting from string

• •

  Removed ASSERT for type option<T> and value NONE

• •

  Fixed aud field on a JWT claim decoding

• •

  Removed unneeded return in matching

• •

  Disabled default audience validation for JWKS

• •

  Fixed errors in supply chain security workflow

• •

  Enabled check for single and multiple token audiences

• •

  Removed panic on invalid range scan

• •

  Added table to reserved keywords

• •

  Concurrent indexing no longer fails if record updated during build

• •

  Unreachable code now returns an error

• •

  DEFINE EVENT defaults to true when no WHEN clause specified

• •

  Fixed bug in which NONE and NULL both < 1 and > 1

• •

  Fixed inconsistency on queries using equality with or without index

• •

  Create on rpc which returned an array now returns an object. Similar change made to Rust SDK

• •

  Rust SDK fix retrieving multiple columns from the same response

• •

  Improved consistency for surrealdb.query() and surrealdb.value() in JavaScript runtime

• •

  Fixed ENFORCED keyword not displaying in INFO statement

• •

  Fixed GHSA-64f8-pjgr-9wmr: Untrusted Query Object Evaluation in RPC API

• •

  Improved math::mean() to support returning floats

• •

  Improved namespace/database validation for GraphQL

• •

  Ensured TIMEOUT clauses are processed correctly

• •

  Fixed importing from 1.x

• •

  Deprecated missing format inference on RPC protocol

Workflow improvements:

• •

  Deprecated file://

• •

  Added surreal fix command

• •

  Exports no longer use explicit transactions

• •

  /lib path renamed to /sdk

• •

  Rename the next workflow to alpha

• •

  Improved structured output for INFO statement

• •

  Added database upgrade test from 1.5.x to 2.0.x

• •

  Improved InfoStructure implementation on Ident to return the raw value

• •

  Improved placeholder text when filling a bug report

• •

  Separate the supply chain security actions to its workflow

• •

  Improve Cargo local testing

• •

  Limit Cargo build jobs during tests with coverage

• •

  Improved type safety in router API

• •

  Improved hashing mechanism to aid MTree benchmarking

• •

  Removed unnecessary cloning of DefineStatementAnalyzer

• •

  Simplified string formatting

• •

  Refactored transaction, caching, and key-value store interfaces

• •

  Imported entry enum in src/sql/mod.rs

• •

  Extracted duplicated code for storing interfaces to initialize_store

• •

  Resolved a TODO: map_err to inspect error

• •

  Improved pull request template

• •

  Added Improvements to kvs module layer

• •

  Updated rustc version to 1.80.0

• •

  Enabled the ml feature in nightly and alpha releases

• •

  Removed limit to Cargo jobs

• •

  Restored bench-sdk-surrealkv

• •

  Upgraded to http/hyper 10

• •

  Upgrade upstream jsonwebtoken crate

• •

  Bumped SurrealKV version to 0.3.1

• •

  Added aliases for complex feature flag combinations

• •

  Improved RocksDB range scans

• •

  Improved Serialisation for sql::Value

• •

  Improved parser error type

• •

  Added Support for SurrealCS

• •

  Bumped async graphql version

• •

  Improved transparent API

• •

  Fixed cargo dependency loading issue

• •

  Improved the logic for migrating IDs in surreal fix

API improvements:

• •

  Decoupled capabilities from the public API

Surrealkv API improvements:

• •

  Changed disk format for compaction in Surrealkv

• •

  Removed global static Datastore

• •

  Added versioning to SELECT * FROM table queries

Parser improvements:

• •

  Added support for the s string prefix in parser

• •

  Fixed parser panic when a dot follows a string in idiom-allowed contexts

• •

  Ensured variables after the FETCH clause are parsed

• •

  Ensured UUIDs are parsed correctly

• •

  Fixed floating point parsing bug

• •

  Updates reblessive dependency to newest version

• •

  Removed flume as a dependency in favour of async-channels

• •

  Fixed warnings and simplified connection trait

Breaking changes:

• •

  Functions beginning with meta:: renamed to record::

• •

  Functions string::endsWith() and string::startsWith() renamed to string::ends_with() and string::starts_with()

• •

  Changed UPDATE statement behaviour of creating records in favour of new UPSERT statement.

Experimental features:

• •

  Added BEARER access type and its basic grant management