Skip to content

monitoring

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

monitoring

README.adoc

Monitoring

User Guide

XTDB provides prebuilt Grafana dashboards for monitoring. We assume you already have a Grafana instance running with Prometheus configured to scrape your XTDB cluster or nodes (e.g. via Kubernetes).

If so, you can import the dashboards from the public-dashboards/ directory into Grafana:

  1. Navigate to Dashboards → New → New dashboard.

  2. Click Import dashboard.

  3. Under Upload dashboard JSON file, upload the desired file from public-dashboards/.

    1. Select your Prometheus data source.

    2. Click Import.

XTDB provides two dashboards:

  • XTDB Cluster Monitoring – an overview of cluster health and performance.

  • XTDB Node Debugging – detailed insights into individual node behavior.

XTDB Developer Guide

For local development, a monitoring stack can be started via Docker Compose. This setup runs:

  • Prometheus – metrics collection

  • Grafana – visualization, preloaded with dashboards and a Prometheus data source

  • Tempo – distributed tracing for OpenTelemetry traces

Grafana is preconfigured to:

  • Use Prometheus as the default data source

  • Include Tempo as a tracing data source

  • Automatically load dashboards from monitoring/public-dashboards/

Prometheus is configured to scrape metrics from localhost:8081. This assumes the monitored service:

  • Exposes metrics at :8081/metrics

  • Is reachable from inside the Prometheus container at:

    • 172.17.0.1:8081

Tempo is configured to:

  • Accept OTLP traces on port 4317 (gRPC) and 4318 (HTTP)

  • Store traces locally for 48 hours

  • Expose the Tempo API on port 3200

Start the stack with:

docker-compose up -d

If you then go to http://localhost:3000/dashboards - you can log in (admin/admin) and work on/use the dashboards.

To view traces, use Grafana’s Explore view with the Tempo data source at http://localhost:3000/explore.

Configuring XTDB Tracing

To enable OpenTelemetry tracing in your XTDB node, configure it under the tracer section:

(require '[xtdb.node :as xtn])

(def node
  (xtn/start-node
    {:tracer {:enabled? true
              :endpoint "http://localhost:4318/v1/traces"
              :service-name "xtdb-dev"}}))

Or in YAML:

tracer:
  enabled: true
  endpoint: "http://localhost:4318/v1/traces"
  serviceName: "xtdb-dev"
  queryTracing: true
  transactionTracing: true

Configuration options:

  • enabled? (boolean) – Enable OpenTelemetry tracing. Default: false

  • endpoint (string) – OTLP HTTP endpoint for sending traces. Default: "http://localhost:4318/v1/traces"

  • service-name (string) – Service name for traces. Default: "xtdb"

  • query-tracing? (boolean) – Enable tracing for queries. Default: true

  • transaction-tracing? (boolean) – Enable tracing for transactions. Default: true

Traces will be sent to Tempo and can be viewed in Grafana’s Explore view.

Exporting Dashboards

When updating or creating dashboards, export them to both the developer and public dashboards directories.

  • For dev-dashboards:

    1. Click Share.

    2. Open the Export tab.

    3. Click Save to file.

  • For public-dashboards:

    1. Click Share.

    2. Open the Export tab.

    3. Select Export for sharing externally.

    4. Click Save to file.

Replace the existing files in the relevant directories with your updated versions.

Scraping Multiple Nodes

To scrape multiple XTDB nodes or services:

  1. Ensure each node exposes its /metrics endpoint on a discoverable port.

  2. Update prometheus.yml with either static targets or service discovery.

Example (prometheus.yml):

scrape_configs:
  - job_name: 'xtdb-nodes'
    static_configs:
      - targets: ['172.17.0.1:8081', '172.17.0.1:8082']