• Start

Enterprise observability

Metrics, logs, and traces in SurrealDB Enterprise versus Community — OTLP opt-ins, audit and slow-query pipelines, and SurrealDS cluster instruments.

SurrealDB Community and Enterprise share one unified observability implementation in the server: the same labelled metric families, the same GET /metrics path, and the same OTLP push pipeline when enabled. Enterprise adds optional durability and cluster depth on top:

  • SurrealDS cluster instruments (surrealdb.ds.*) — consensus, networking, recovery, and garbage collection when you run the distributed datastore path.

  • Audit and slow-query pipelines — bounded queues, file sinks with rotation and optional hash chaining, and pipeline self-metrics (surrealdb.audit.*, surrealdb.slow_query.*) when those sinks are configured.

  • OpenTelemetry logs for audit and slow-query — the same redacted records can be emitted as OTel log records, but only when you opt in with SURREAL_AUDIT_OTEL_EXPORT=true and/or SURREAL_SLOW_QUERY_OTEL_EXPORT=true (default false so sensitive payloads stay on the file sink unless you choose otherwise).

Throughout this page, markers mean:

  • [C] — Registered in every build that exposes metrics (Community and Enterprise).

  • [E] — Registered only when running a SurrealDB Enterprise binary with the relevant feature enabled.

For Prometheus pull behaviour, naming rules, the public PUBLIC_METRICS allowlist, and multi-tenant guidance common to both editions, see Observability (metrics and Prometheus). For OTLP push, intervals, and legacy HTTP/RPC instruments, see Telemetry (OTLP).

HTTP / WS / Executor → ExecutionObserver → FanOutObserver
├── MetricsObserver (labelled surrealdb.*) [C]
├── DsMetrics (surrealdb.ds.*) [E]
├── AuditObserver (audit → file + optional OTel) [E]
└── SlowQueryObserver (slow-query → file + optional OTel) [E]

SdkMeterProvider + SdkLoggerProvider

┌───────────────┼────────────────┐
▼ ▼ ▼
Prometheus text OTLP metrics push OTLP logs / traces push
exporter (/metrics)

ExecutionObserver short-circuits when no observer in the chain has work, so a process with neither /metrics nor OTLP configured avoids hot-path cost. Build flavour is carried on the OpenTelemetry resource as service.edition (community or enterprise), for example on target_info in Prometheus — filter dashboards on that attribute rather than on meter scope names.

PipelineTriggerTransportAuthenticationWhat is exposed
Prometheus pullSURREAL_METRICS_ENABLED=true (default)GET /metrics on the main HTTP port (text/plain; version=0.0.4)Root credentials unlock the full surface. Anonymous scrapers receive only the public allowlist (aggregate process gauges).Every instrument the running binary registers.
OTLP pushSURREAL_TELEMETRY_PROVIDER=otlpgRPC to OTEL_EXPORTER_OTLP_ENDPOINT (default http://localhost:4317). Push cadence: OTEL_METRIC_EXPORT_INTERVAL (ms, default 60000).TLS and client authentication are the collector’s responsibility.Same metrics as an authenticated Prometheus view, plus traces and (when configured) audit and slow-query log records.

Disabling pieces:

SwitchDefaultEffect
SURREAL_METRICS_ENABLEDtrueWhen false, /metrics is not mounted (404).
SURREAL_TELEMETRY_PROVIDERunsetSet to otlp to start OTLP exporters.
SURREAL_TELEMETRY_DISABLE_METRICSfalseSkips only the OTLP metrics reader; logs and traces can still push.
SURREAL_TELEMETRY_DISABLE_TRACINGfalseSkips only the OTLP trace exporter.
ConcernCommunityEnterprise
Labelled surrealdb.* statement, query, transaction, RPC, HTTP, session, auth, network, live query, process, storageYes [C]Yes [C]
Slow-query counter (surrealdb.slow_query.*) vs thresholdSURREAL_SLOW_QUERY_METRIC_THRESHOLD_MS (default 1000 ms; 0 disables)Same [C]
SurrealDS surrealdb.ds.* cluster metricsNot emittedEmitted when the DS runtime is active and a metrics reader exists [E]
Durable audit / slow-query file sinksNot availableSURREAL_AUDIT_*, SURREAL_SLOW_QUERY_* [E]
Pipeline self-metrics (surrealdb_audit_*, surrealdb_slow_query_*)Not registeredRegistered when the matching pipeline is configured [E]
OTel log export for audit / slow-query recordsN/A (no enterprise pipelines)Opt-in per pipeline: SURREAL_AUDIT_OTEL_EXPORT, SURREAL_SLOW_QUERY_OTEL_EXPORT [E]
Per-tenant rollup scope surrealdb.tenantReserved for future low-cardinality rollupsSame; no instruments are registered there yet. Use proxy-side filtering on dimensional families for customer-visible views today.

These apply to every build:

VariableDefaultPurpose
SURREAL_METRICS_ENABLEDtrueMount /metrics.
SURREAL_TELEMETRY_PROVIDERunsetotlp enables OTLP push.
SURREAL_TELEMETRY_DISABLE_METRICSfalseSkip OTLP metrics only.
SURREAL_TELEMETRY_DISABLE_TRACINGfalseSkip OTLP traces only.
SURREAL_SLOW_QUERY_METRIC_THRESHOLD_MS1000Statements slower than this also increment the slow-query counter; 0 disables that counter.
SURREAL_PROCESS_METRICS_REFRESH_INTERVAL5Seconds between process memory/CPU gauge refreshes (minimum 1).
OTEL_EXPORTER_OTLP_ENDPOINThttp://localhost:4317OTLP gRPC endpoint.
OTEL_METRIC_EXPORT_INTERVAL60000OTLP metric push interval (ms).
VariableDefaultNotes
SURREAL_AUDIT_SINKnonenone or file.
SURREAL_AUDIT_FILE_PATHRequired for file; parent directory must exist; file mode 0600 on Unix.
SURREAL_AUDIT_FSYNC_EVERY0Mid-stream fsync cadence; rotation and graceful shutdown always flush.
SURREAL_AUDIT_HASH_CHAINfalseTamper-evident chaining; requires SURREAL_AUDIT_FSYNC_EVERY=1.
SURREAL_AUDIT_INCLUDE_SQLfalseInclude full SQL on statement records.
SURREAL_AUDIT_OVERFLOWblockdrop or block when the queue is full; compliance deployments usually keep block.
SURREAL_AUDIT_OTEL_EXPORTfalseAlso emit each record as an OTel log (opt-in).
VariableDefaultNotes
SURREAL_SLOW_QUERY_SINKnoneSame pattern as audit.
SURREAL_SLOW_QUERY_THRESHOLD_MSRequired when the sink is enabled.
SURREAL_SLOW_QUERY_INCLUDE_SQLtrueSQL in slow-query records (turn off for throughput-sensitive deployments).
SURREAL_SLOW_QUERY_OVERFLOWdropDefaults to drop (triage data); audit defaults to block.
SURREAL_SLOW_QUERY_OTEL_EXPORTfalseOTel log export for slow-query records (opt-in).

Redaction (SURREAL_AUDIT_REDACT_* / SURREAL_SLOW_QUERY_REDACT_*) runs on the executor thread before the queue so the file sink and OTel path see the same scrubbed text. Full knob lists live with the rest of the server environment variables.

Large clusters expose saturation through surrealdb.ds.* and logs. Typical operator levers include inbound byte budgets, handler caps, read/write operation caps, and consensus timeouts. If you deploy on Kubernetes, pair metrics with the environment-variable reference in your platform runbook.

Records flow in parallel:

  1. File sink — optional NDJSON with rotation; optional hash chain for tamper evidence.

  2. OpenTelemetry logs — only when the matching SURREAL_*_OTEL_EXPORT flag is true.

Severity rules differ slightly between audit event kinds and slow-query (slow-query is always treated as a degraded path — typically WARN). Monitor surrealdb_audit_dropped, surrealdb_slow_query_dropped, surrealdb_audit_append_errors, and surrealdb_slow_query_append_errors: any sustained non-zero rate means lost or failed records.

When you front /metrics with a proxy or expose metrics to customers:

  1. Always safe without authentication — matches the built-in public allowlist (surrealdb_build_info, process gauges, target_info, otel_scope_info).

  2. Customer-visible — only after the proxy filters dimensional series to that customer’s namespace / database. Strip user before showing outside trusted operator contexts.

  3. Operator-onlysurrealdb.ds.*, pipeline self-metrics, surrealdb_auth_*, storage backend detail, RPC method detail, and similar. Scrape these on a private interface with root credentials.

Legacy series names were collapsed into labelled families in SurrealDB 3.1. The same migration table applies to Community and Enterprise; see Observability (metrics and Prometheus) for the summary, Telemetry (OTLP) for wire-format and deprecation notes, and the Metrics reference for the full migration table.

curl http://127.0.0.1:8000/metrics
curl -u root:root http://127.0.0.1:8000/metrics

Enterprise-only metrics (surrealdb_ds_*, audit/slow-query pipeline gauges) appear only in the authenticated view when those subsystems are active.

  • Audit logging — record shape, rotation, overflow semantics, hash chaining, redaction.

  • Slow-query logging — triage pipeline mirroring the audit observer.

  • Configuration — all SURREAL_AUDIT_* and SURREAL_SLOW_QUERY_* variables and the compliance checklist.

Was this page helpful?