Each Wiki server can expose its metrics in the Prometheus format at /metrics. It's off by default, works on every license tier and in any license state, and isn't served in setup or recovery mode. Each server reports its own numbers, so scrape every server.
1. Turn it on
Choose one, or both:
| Variable | What |
|---|---|
| METRICS_ADDR | A separate listener, such as :9090 or 10.0.0.5:9090, serving /metrics without authentication. It must differ from PORT. Bind it to a private interface or firewall it. |
| METRICS_TOKEN | Also serves /metrics on the server's main port to requests with Authorization: Bearer <token>. At least 16 printable characters without spaces, for example openssl rand -hex 32. |
# prometheus.yml
scrape_configs:
- job_name: wiki
static_configs:
- targets: ["wiki-1.internal:9090", "wiki-2.internal:9090"]
# or, on the main port with a token
- job_name: wiki-token
authorization: { credentials_file: /etc/prometheus/wiki-token }
static_configs:
- targets: ["wiki-1.internal:8080"]
/metrics is never reachable through the web app's address: Prometheus scrapes the servers directly. A scrape isn't rate limited and isn't counted in the metrics.
2. Metrics
Every metric name starts with depotex_, the name of the platform the Wiki is built on.
| Metric | What |
|---|---|
| depotex_collab_connections | Live editing connections this server serves now |
| depotex_collab_updates_total, depotex_collab_update_bytes_total | Edits that browsers sent and that were stored, and their size in bytes |
| depotex_collab_closed_total{code} | Live editing connections the server closed, by close code |
| depotex_collab_presence_dropped_total | Cursor and presence messages dropped because a queue was full |
| depotex_wiki_doc_compactions_total, depotex_wiki_doc_compaction_failures_total{reason} | Page documents whose stored edits were merged, and merges that failed |
| depotex_wiki_draft_snapshots_total, depotex_wiki_draft_snapshot_failures_total{reason} | Draft snapshots taken (the draft history of a page), and snapshots that failed |
| depotex_backup_runs_total{kind, outcome}, depotex_backup_verifications_total{outcome} | Backups taken and checked, by result |
| depotex_backup_duration_seconds{kind}, depotex_backup_last_size_bytes | How long backups take, and the size of the last one |
| depotex_backup_last_success_timestamp_seconds | When the last backup succeeded |
| depotex_backup_cleanup_deleted_total | Backups the cleanup job deleted |
| depotex_backup_restore_runs_total{outcome}, depotex_backup_restore_duration_seconds | Restores and how long they took |
| depotex_backup_test_restore_runs_total{outcome}, depotex_backup_test_restore_last_success_timestamp_seconds | Test restores, and when the last one passed |
| depotex_ratelimit_requests_total{class, result} | Requests checked against a budget, per kind of traffic and result: allowed, limited (refused) or dry_run (over, but let through by a dry run). A request counts once per budget it is checked against |
| depotex_ratelimit_fallback_total{reason} | Checks a server made in its own memory although Redis is configured: redis_error, backoff (Redis skipped for a few seconds after an error) or canceled |
| depotex_ratelimit_redis_up | 1 while the server reaches Redis for rate limits, 0 while it counts on its own. Missing on servers without Redis |
| depotex_ratelimit_check_duration_seconds{store} | How long a rate limit check took, in Redis or in memory |
| depotex_read_cache_hits_total{cache}, depotex_read_cache_misses_total{cache} | Reads of settings answered from this server's memory, and reads that went to the database |
| depotex_read_cache_checks_total{result}, depotex_read_cache_notices_total | How often the server checked whether its cached settings changed, and change notices it received |
| depotex_oauth_workload_exchanges_total{grant, endpoint, outcome} | CI identity tokens exchanged for access tokens (see OAuth applications) |
| go_*, process_* | The server process: memory, goroutines, CPU, open files |
Metrics never carry who made a request, or which page or space it was about (no user, token, page or address labels). To see who was limited, use the Activity tab of Rate limits.
3. Alerting examples
# No successful backup for more than a day (with the scheduled backup on)
- alert: WikiBackupStale
expr: time() - depotex_backup_last_success_timestamp_seconds > 26 * 3600
for: 15m
# Pages' drafts can't be snapshotted
- alert: WikiDraftSnapshotsFailing
expr: sum(rate(depotex_wiki_draft_snapshot_failures_total[15m])) > 0
for: 30m
# A server lost Redis: rate limits are per server until it's back
- alert: WikiRateLimitRedisDown
expr: depotex_ratelimit_redis_up == 0
for: 5m