SurrealDB can be monitored by enabling the built in observability.
To enable observability, the SURREAL_TELEMETRY_PROVIDER environment variable has to be set to otlp. If set to anything else, no observability will be available.
If enabled, SurrealDB will send metrics and/or traces to an OpenTelemetry Collector. Configuration of the collector is done via environment variables. The most important one is OTEL_EXPORTER_OTLP_ENDPOINT. By default this is set to localhost. It should be set to the GRPC endpoint of your OTEL collector. For example if your OTEL collector named my-collector is running in Kubernetes in the monitoring namespace the following can be used:
OTEL_EXPORTER_OTLP_ENDPOINT="http://my-collector.monitoring.svc.cluster.local:4317"
Metrics can be disabled (even if SURREAL_TELEMETRY_PROVIDER is set to otlp) by setting the SURREAL_TELEMETRY_DISABLE_METRICS environment variable to true. Similarly traces can be disabled by setting SURREAL_TELEMETRY_DISABLE_TRACING to true.
Metrics are gathered every minute and sent to the collector. The following metrics are present:
| Name | Instrument | Explanation |
|---|---|---|
rpc.server.duration | histogram | Measures duration of inbound RPC requests in milliseconds |
rpc.server.active_connections | counter | The number of active WebSocket connections |
rpc.server.response.size | histogram | Measures the size of HTTP response messages |
http.server.duration | histogram | The HTTP server duration in milliseconds |
http.server.active_requests | counter | The number of active HTTP requests |
http.server.request.size | histogram | Measures the size of HTTP request messages |
http.server.response.size | histogram | Measures the size of HTTP response messages |
The metrics are shown here in the form required by the OpenTelemetry Metrics Semantic Conventions with a . separator. When ingested into Prometheus the . separator will be replaced with an _. For example rpc.server.active.connections will be transformed into rpc_server_active_connections.
Available since: v3.0.0-alpha.7
The tokio console tool can also be used to observe the low-level async tasks for the database.
To use it, install the Cargo package manager for Rust, use the command cargo install --locked tokio-console to install tokio console, and then tokio-console to run it.
The following environment variables are used to manage the console:
SURREAL_TOKIO_CONSOLE_ENABLED set to true to enable the consoleSURREAL_TOKIO_CONSOLE_SOCKET_ADDR to override the default address at 127.0.0.1:6669SURREAL_TOKIO_CONSOLE_RETENTION to override the default 6s (six seconds) value for length of time to retain data for completed events.SurrealDB contains a large number of flags for the surreal start command and environment variables to modify the output from the database server. A quick overview of some of these flags is as follows:
--log to set the logging level (info, warn, trace, etc.).--log-format to set the format (text or json).--log-socket to send logs to a certain host:port.--log-file-enabled to output logs to a file, followed by --log-file-path to specify the path to save logs to to.--log-file-rotation to specify whether to create a new file for logs at a certain interval (daily, hourly, or never).--slow-log-threshold to specify a duration after which a query will be logged as a slow query.--slow-log-param-allow and --slow-log-param-deny to indicate which parameters are to be calculated before logging. For example, if $table_name has the value “person”, allowing table_name will change SELECT * FROM $table_name to SELECT * FROM person in the logs.