From 6164ce70bc36aca1f90162b1e6fa20a79b794636 Mon Sep 17 00:00:00 2001 From: Kennan Hunter Date: Thu, 30 Jul 2026 21:38:00 -0400 Subject: [PATCH 1/4] document OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE behavior --- docs/configuration/pgdog.toml/otel.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/configuration/pgdog.toml/otel.md b/docs/configuration/pgdog.toml/otel.md index 39eae335..cba17214 100644 --- a/docs/configuration/pgdog.toml/otel.md +++ b/docs/configuration/pgdog.toml/otel.md @@ -79,6 +79,7 @@ corresponding `pgdog.toml` values (or supply a value when the setting is omitted | `OTEL_METRIC_EXPORT_INTERVAL` | Push interval in milliseconds. Equivalent to `push_interval`. | | `OTEL_SERVICE_NAME` | Sets the `service.name` resource attribute. Takes precedence over any value set via `OTEL_RESOURCE_ATTRIBUTES`. _Default:_ `pgdog`. | | `OTEL_RESOURCE_ATTRIBUTES` | Comma-separated `key=value` pairs added as resource attributes on every metric (e.g. `env=prod,region=us-east-1`). Values may be percent-encoded. | +| `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` | Controls the aggregation temporality used for exported metrics. Accepts cumulative (values accumulate over the lifetime of the process), delta (values report the change since the last export). Use delta for backends like Datadog that expect delta temporality. Default: cumulative. | | | `PGDOG_OTEL_NAMESPACE` | Metric name prefix. Equivalent to `namespace`. | | `DD_API_KEY` | Datadog API key. Equivalent to `datadog_api_key`. | From dd4d69b41f924164195e07732d4222ddb44e0062 Mon Sep 17 00:00:00 2001 From: Kennan Hunter Date: Sun, 2 Aug 2026 07:54:50 -0400 Subject: [PATCH 2/4] improve documentation for temporality as a toml value --- docs/configuration/pgdog.toml/otel.md | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/docs/configuration/pgdog.toml/otel.md b/docs/configuration/pgdog.toml/otel.md index cba17214..a8767c12 100644 --- a/docs/configuration/pgdog.toml/otel.md +++ b/docs/configuration/pgdog.toml/otel.md @@ -67,6 +67,27 @@ How often, in milliseconds, to push metrics to the OTLP endpoint. _Default:_ `10000` +### `temporality_preference` + +How exported metric points describe their value over time. See the [OpenTelemetry metric points spec](https://opentelemetry.io/docs/specs/otel/metrics/data-model/#metric-points). + +| Value | Behavior | +| ------------ | --------------------------------------------------------------- | +| `Cumulative` | Points report the value accumulated since the exporter started. | +| `Delta` | Points report the change since the last export. | + + + + +_Default:_ `Cumulative`, or `Delta` when `datadog_api_key` is set. + +Values are case-insensitive. + +!!! warning "Datadog" +Sending `Cumulative` sums and histograms to Datadog is stateful and lossy: all points on a timeseries must reach the same Agent/exporter (constraining how you scale collectors), the first point of a new series may be dropped (causing gaps on restart), and histogram min/max may be missing or approximated. See [Datadog's OTLP delta temporality guide](https://docs.datadoghq.com/opentelemetry/guide/otlp_delta_temporality/?tab=python#implications-of-using-cumulative-aggregation-temporality). + + PgDog logs a warning when `datadog_api_key` is set alongside `Cumulative`. Set `IGNORE_DATADOG_CUMULATIVE_WARNING=1` to silence it. + ## Environment variables PgDog honors the standard OpenTelemetry environment variables. When set, they override the @@ -79,10 +100,11 @@ corresponding `pgdog.toml` values (or supply a value when the setting is omitted | `OTEL_METRIC_EXPORT_INTERVAL` | Push interval in milliseconds. Equivalent to `push_interval`. | | `OTEL_SERVICE_NAME` | Sets the `service.name` resource attribute. Takes precedence over any value set via `OTEL_RESOURCE_ATTRIBUTES`. _Default:_ `pgdog`. | | `OTEL_RESOURCE_ATTRIBUTES` | Comma-separated `key=value` pairs added as resource attributes on every metric (e.g. `env=prod,region=us-east-1`). Values may be percent-encoded. | -| `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` | Controls the aggregation temporality used for exported metrics. Accepts cumulative (values accumulate over the lifetime of the process), delta (values report the change since the last export). Use delta for backends like Datadog that expect delta temporality. Default: cumulative. | | +| `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` | Aggregation temporality for exported metrics. Equivalent to `temporality_preference`. | | `PGDOG_OTEL_NAMESPACE` | Metric name prefix. Equivalent to `namespace`. | | `DD_API_KEY` | Datadog API key. Equivalent to `datadog_api_key`. | + PgDog automatically sets the following resource attributes on every exported metric: `service.name` (defaults to `pgdog`), `service.instance.id`, and `host.name`. Use `OTEL_RESOURCE_ATTRIBUTES` to add or override any of these. From 2f59fabc9abf4ca1bb2246fac76f8444e6ab72f8 Mon Sep 17 00:00:00 2001 From: Kennan Hunter Date: Sun, 2 Aug 2026 09:11:14 -0400 Subject: [PATCH 3/4] describe locking metrics --- dev.sh | 0 docs/features/metrics.md | 2 ++ 2 files changed, 2 insertions(+) mode change 100644 => 100755 dev.sh diff --git a/dev.sh b/dev.sh old mode 100644 new mode 100755 diff --git a/docs/features/metrics.md b/docs/features/metrics.md index 16b200bc..44e2de8e 100644 --- a/docs/features/metrics.md +++ b/docs/features/metrics.md @@ -146,9 +146,11 @@ The following metrics are exported via the OpenMetrics endpoint: |------------|-------------|------| | `clients` | Total number of connected clients. | `gauge` | | `cl_waiting` | Clients waiting for a connection from a pool. | `gauge` | +| `cl_locked` | Clients currently locked to a specific connection. | `gauge` | | `sv_active` | Servers currently serving client requests. | `gauge` | | `sv_idle` | Servers available for clients to use. | `gauge` | | `sv_idle_xact` | Servers currently idle in transaction. | `gauge` | +| `sv_locked` | Servers currently locked. | `gauge` | | `maxwait` | How long the first (oldest) client in the queue has waited (in seconds). | `gauge` | | `errors` | Errors that connections in the pool have experienced. | `counter` | | `out_of_sync` | Connections that have been returned to the pool in a broken state. | `counter` | From 81193800fcdada0d0e4eced0b80814191039091d Mon Sep 17 00:00:00 2001 From: Kennan Hunter Date: Sun, 2 Aug 2026 09:17:47 -0400 Subject: [PATCH 4/4] include DataDog and Grafana example links in metrics --- docs/features/metrics.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/features/metrics.md b/docs/features/metrics.md index 44e2de8e..abccdf16 100644 --- a/docs/features/metrics.md +++ b/docs/features/metrics.md @@ -34,8 +34,9 @@ You can connect to the admin database using any PostgreSQL client. It supports c ## OpenMetrics -[OpenMetrics](https://openmetrics.io/) is a standard for displaying metrics that can be ingested by a multitude of agents, e.g., Datadog, Prometheus, etc. The metrics are shown in a human-readable text format -and available from an HTTP endpoint. +[OpenMetrics](https://openmetrics.io/) is a standard for displaying metrics that can be ingested by a multitude of agents, e.g., Datadog, Prometheus, etc. The metrics are shown in a human-readable text format and available from an HTTP endpoint. + +We maintain multiple examples of OpenMetrics integrations, including [DataDog](https://github.com/pgdogdev/pgdog/tree/main/examples/datadog) and [Grafana using Prometheus](https://github.com/pgdog/pgdog/tree/main/examples/grafana_prometheus). ### Configuration