Skip to content

Observability

Tokio console

Available since: v3.0.0

Telemetry (OTLP) and GET /metrics (Prometheus pull) tell you what the database is doing from the outside (requests, statements, resources). The Tokio console shows what the Tokio runtime is doing on the inside: which async tasks exist, how long they spend polled versus waiting, and where scheduling stalls appear. It is aimed at contributors and advanced operators debugging performance or deadlocks, not at replacing production metrics.

Tokio console support is not part of the released surreal binaries or Docker images, so using it means building SurrealDB from source with the support included, and then enabling it with environment variables. Enable it only on trusted hosts, for example local development or a dedicated staging instance.

Console support needs two things at build time: the tokio-console feature of the surrealdb-server crate, and Tokio's unstable APIs, which are switched on with the tokio_unstable compiler flag. From a clone of the SurrealDB repository, build the surreal binary with both:

RUSTFLAGS="--cfg tokio_unstable" cargo build --release --bin surreal --features surrealdb-server/tokio-console

The binary is written to target/release/surreal. The feature is named through surrealdb-server/ because the top-level package does not pass it on.

A build that lacks either part does not start the console:

  • Without the feature, which includes every released binary, the server logs SURREAL_TOKIO_CONSOLE_ENABLED is set but this build was compiled without the tokio-console feature and starts without the console.

  • With the feature but without tokio_unstable, the server panics at startup with task tracing requires Tokio to be built with RUSTFLAGS="--cfg tokio_unstable"!.

The server exposes data to the console subscriber; you still need the Tokio console client on your machine.

  1. Install Rust and Cargo if you do not already have them.

  2. Install the CLI: cargo install --locked tokio-console

  3. Start the binary you built with console support enabled (see below), then run tokio-console in another terminal. By default it connects to 127.0.0.1:6669.

Set the following before starting surreal:

  • SURREAL_TOKIO_CONSOLE_ENABLED - set to true to turn on the subscriber.

  • SURREAL_TOKIO_CONSOLE_SOCKET_ADDR - optional; overrides the default listen address 127.0.0.1:6669.

  • SURREAL_TOKIO_CONSOLE_RETENTION - optional; overrides the default 6s (six seconds) for how long to retain data for completed events.

The socket listens only on the address you configure; keep it on loopback unless you fully understand the exposure (the stream describes internal task behaviour).

A screenshot of the Tokio console showing active SurrealDB tasks.

A screenshot of the Tokio console showing poll times by percentile.

Typical uses for the tokio console are: spotting tasks that rarely get polled, comparing poll percentiles after a change, and correlating spikes in load with specific async workloads.

Was this page helpful?