SurrealDB can be monitored by enabling the built in observability.
To enable observability, the SURREAL_TELEMETRY_PROVIDER environment variable has to be set to otlp. If set to anything else, no observability will be available.
If enabled, SurrealDB will send metrics and/or traces to an OpenTelemetry Collector. Configuration of the collector is done via environment variables. The most important one is OTEL_EXPORTER_OTLP_ENDPOINT. By default this is set to localhost. It should be set to the GRPC endpoint of your OTEL collector. For example if your OTEL collector named my-collector is running in Kubernetes in the monitoring namespace the following can be used:
OTEL_EXPORTER_OTLP_ENDPOINT="http://my-collector.monitoring.svc.cluster.local:4317"
Metrics can be disabled (even if SURREAL_TELEMETRY_PROVIDER is set to otlp) by setting the SURREAL_TELEMETRY_DISABLE_METRICS environment variable to true. Similarly traces can be disabled by setting SURREAL_TELEMETRY_DISABLE_TRACING to true.
Metrics are gathered every minute and sent to the collector. The following metrics are present:
| Name | Instrument | Explanation |
|---|---|---|
rpc.server.duration | histogram | Measures duration of inbound RPC requests in milliseconds |
rpc.server.active_connections | counter | The number of active WebSocket connections |
rpc.server.response.size | histogram | Measures the size of HTTP response messages |
http.server.duration | histogram | The HTTP server duration in milliseconds |
http.server.active_requests | counter | The number of active HTTP requests |
http.server.request.size | histogram | Measures the size of HTTP request messages |
http.server.response.size | histogram | Measures the size of HTTP response messages |
The metrics are shown here in the form required by the OpenTelemetry Metrics Semantic Conventions with a . separator. When ingested into Prometheus the . separator will be replaced with an _. For example rpc.server.active.connections will be transformed into rpc_server_active_connections.
Available since: v3.0.0-alpha.7
The tokio console tool can also be used to observe the low-level async tasks for the database.
To use it, install the Cargo package manager for Rust, use the command cargo install --locked tokio-console to install tokio console, and then tokio-console to run it.
The following environment variables are used to manage the console:
SURREAL_TOKIO_CONSOLE_ENABLED set to true to enable the consoleSURREAL_TOKIO_CONSOLE_SOCKET_ADDR to override the default address at 127.0.0.1:6669SURREAL_TOKIO_CONSOLE_RETENTION to override the default 6s (six seconds) value for length of time to retain data for completed events.