Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ to the exact version in `go.mod`.
- Site: <https://exp.pgsty.com>
- Product source: <https://github.com/pgsty/pg_exporter>
- Theme source: <https://github.com/pgsty/oink>
- Latest stable release documented: `v1.5.0`
- Latest stable release documented: `v1.6.0`

## Content layout

Expand Down Expand Up @@ -199,7 +199,7 @@ python3 bin/build_architecture_diagram.py

This writes four homepage diagrams to `static/img/architecture*.svg`: English
and Chinese, each with light and dark variants. Edit the generator rather than
the SVGs. The diagram describes stable v1.5.0: one process targets a PostgreSQL
the SVGs. The diagram describes stable v1.6.0: one process targets a PostgreSQL
instance or PgBouncer and serves `/metrics`. Unreleased components belong in
the Design archive. Product counts and their source boundary are recorded in
`data/home/metrics.yaml`.
Expand Down
6 changes: 3 additions & 3 deletions bin/build_architecture_diagram.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
static/img/architecture-{light,dark}.svg
static/img/architecture-zh-{light,dark}.svg

Facts checked against pg_exporter v1.5.0: one process targets either a
Facts checked against pg_exporter v1.6.0: one process targets either a
PostgreSQL instance or PgBouncer. PostgreSQL uses lib/pq; PgBouncer uses its
administration console. The process serves metrics on :9630/metrics.
Development-only components are described in the Design archive.
Expand Down Expand Up @@ -128,7 +128,7 @@

EN = {
"alt": (
"PG Exporter v1.5.0: each process collects one PostgreSQL instance or "
"PG Exporter v1.6.0: each process collects one PostgreSQL instance or "
"PgBouncer target and serves metrics on :9630/metrics for "
"Prometheus or VictoriaMetrics."
),
Expand Down Expand Up @@ -159,7 +159,7 @@

ZH = {
"alt": (
"PG Exporter v1.5.0:每个进程采集一个 PostgreSQL 实例或 PgBouncer 目标,"
"PG Exporter v1.6.0:每个进程采集一个 PostgreSQL 实例或 PgBouncer 目标,"
"通过 :9630/metrics 供 Prometheus 或 VictoriaMetrics 抓取。"
),
"cap_left": "每个进程连接一个目标",
Expand Down
14 changes: 12 additions & 2 deletions content/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,21 @@ pg_in_recovery 0

A database failure can still produce HTTP 200 with `pg_up 0`. Check the metrics as well as the HTTP status. Non-fatal collector errors omit that collector’s results; other collectors continue. A fatal collector error stops further collection on that database. HTTP 500 indicates a gathering or encoding error, such as conflicting metric definitions.

Since v1.6.0, at most two metrics requests are admitted across collection, waiting, and response writing. Collection is serial; the second request waits within its remaining budget, and further requests receive HTTP 503. The budget is 30 seconds, shortened by a valid positive finite `X-Prometheus-Scrape-Timeout-Seconds` header. A canceled waiter executes no SQL.

Cancellation stops work before the next collector or database. A started collector with a positive timeout finishes its predicate chain and main SQL within their shared deadline and caches the result normally. A negative timeout removes that deadline: the query then follows request cancellation. An interrupted collection returns 503 if the connection still permits a response. Driver/network delays mean these deadlines are not hard wall-clock limits; configure the DSN `connect_timeout` for connection handshakes.

An interrupted pass does not increment `pg_exporter_scrape_error_count` or `pg_exporter_server_scrape_error_count`; its summary log is INFO `interrupted`. Query errors and fatal-query ERROR logs remain observable. A real failure coinciding with request cancellation can therefore be omitted from these two pass counters. Without request cancellation, connection/precheck failures and fatal query failures, including their own timeouts, retain their error accounting. Non-fatal failures remain query-level errors.

Failed refreshes retain the existing failure TTL. In particular, canceling a negative-timeout fatal collector caches a fatal failure: a later live scrape can count that failure and report `pg_up 0` until a successful refresh. Keep a positive timeout on fatal collectors. See [scrape lifetime](/design/scrape-lifetime/) for the rationale.

### Self-Monitoring Metrics

The default built-in prefix is `pg` for PostgreSQL and `pgbouncer` for PgBouncer. `--namespace` changes this prefix. It does not change names defined in YAML.

| Metric | Type | Labels | Meaning |
|:---|:---|:---|:---|
| `pg_up` | Gauge | — | 1 when the main target’s last scrape completed without a connection or fatal collector error |
| `pg_up` | Gauge | — | Main target availability from collection; see the interruption rule below |
| `pg_version` | Gauge | — | Main target’s version number |
| `pg_in_recovery` | Gauge | — | 1 for a replica, 0 for a primary; use when `pg_up = 1` |
| `pg_exporter_build_info` | Gauge | version, revision, branch, builddate, goversion, goos, goarch | Constant 1 with build metadata |
Expand All @@ -83,11 +91,13 @@ Configured constant labels are added to these series. Cumulative per-database an

`--disable-intro` hides all built-in metrics in this table, including `pg_up`. YAML metrics, Go/process metrics, and HTTP health checks remain available.

During interruption, `pg_up` preserves the previous main-target availability, or 1 if this pass completed its precheck. If the very first precheck is interrupted before success, it remains 0 until a later successful scrape. Child-database interruptions follow the same per-server rule and do not increment the main-target error counter.

## Health Checks

### GET /up

`/up` reports the main target’s cached health state. The exporter probes the target once per second and also updates this state after a scrape. HTTP health requests read the cache without issuing SQL.
After initialization, only the once-per-second background health probe updates the HTTP health snapshot. Slow collection does not hold the probe lock, and completing a scrape cannot overwrite a newer probe result. HTTP health requests read the snapshot without executing SQL. `pg_up` and `pg_in_recovery` remain scrape metrics: a fatal collector can make `pg_up` zero while `/up` still returns 200 for a reachable database. The main PostgreSQL pool allows three connections, PgBouncer two, and each discovered database one.

| State in response body | HTTP status | Meaning |
|:---|:---|:---|
Expand Down
14 changes: 12 additions & 2 deletions content/api.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,21 @@ pg_in_recovery 0

数据库故障时,端点仍可能返回 HTTP 200,正文中 `pg_up` 为 0。检查 HTTP 状态时也要检查指标。普通采集器出错后,本轮不输出该采集器的结果,其他采集器继续执行。致命采集器出错后,停止该数据库的后续采集。HTTP 500 表示指标汇总或编码失败,例如指标定义冲突。

自 v1.6.0 起,采集、等待与响应写入阶段合计最多接纳两个指标请求。采集仍串行;第二个请求在剩余预算内等待,更多请求返回 HTTP 503。预算默认为 30 秒,合法、有限且为正数的 `X-Prometheus-Scrape-Timeout-Seconds` 请求头可以缩短它。等待期间取消的请求不执行 SQL。

请求取消后,不再启动下一个采集器或数据库。已启动且 timeout 为正数的采集器,其谓词链与主 SQL 共用同一期限,继续执行并正常缓存结果。负数 timeout 关闭该期限,此时查询随请求取消。中断的采集在连接仍允许响应时返回 503。驱动与网络阻塞意味着这些期限不是硬性的墙钟上限;建连握手需另行设置 DSN 的 `connect_timeout`。

中断轮次不增加 `pg_exporter_scrape_error_count` 和 `pg_exporter_server_scrape_error_count`,轮次汇总日志为 INFO `interrupted`;查询错误计数与 fatal 查询的 ERROR 日志仍然保留。因此,恰好与请求取消重叠的真实失败可能不计入这两个轮次计数。请求没有取消时,连接/预检失败和 fatal 查询失败(包括自身超时)仍按错误计数;非 fatal 失败仍体现在查询级计数中。

刷新失败沿用原有失败 TTL。尤其是负数 timeout 的 fatal 采集器,被取消后会缓存 fatal 失败;后续有效抓取可能对该失败计数并返回 `pg_up 0`,直到成功刷新。建议 fatal 采集器保留正数 timeout。设计理由见 [采集生命周期](/zh/design/scrape-lifetime/)。

### 自监控指标

PostgreSQL 的内置指标前缀默认为 `pg`,PgBouncer 默认为 `pgbouncer`。`--namespace` 可以修改此类前缀,但不会改变 YAML 中定义的指标名。

| 指标 | 类型 | 标签 | 含义 |
|:---|:---|:---|:---|
| `pg_up` | Gauge | — | 主目标最近一次抓取没有连接错误或致命采集器错误时为 1 |
| `pg_up` | Gauge | — | 采集所见的主目标可用状态;中断规则见下文 |
| `pg_version` | Gauge | — | 主目标的版本号 |
| `pg_in_recovery` | Gauge | — | 从库为 1,主库为 0;在 `pg_up = 1` 时使用 |
| `pg_exporter_build_info` | Gauge | version、revision、branch、builddate、goversion、goos、goarch | 恒为 1,标签记录构建信息 |
Expand All @@ -83,11 +91,13 @@ PostgreSQL 的内置指标前缀默认为 `pg`,PgBouncer 默认为 `pgbouncer`

`--disable-intro` 会隐藏表中所有内置指标,包括 `pg_up`。YAML 指标、Go/进程指标和 HTTP 健康检查仍然可用。

中断时,`pg_up` 保留主目标此前的可用状态;若本轮预检已成功,则保留为 1。首次预检在成功前被中断时,保持为 0,后续成功抓取可恢复。子数据库中断采用相同的单库规则,不增加主目标的错误计数。

## 健康检查

### GET /up

`/up` 返回主连接目标的缓存健康状态。exporter 每秒探测一次目标,抓取完成后也会更新状态。HTTP 健康请求只读取缓存,不执行 SQL。
初始化后,HTTP 健康状态只由每秒一次的后台探测更新。慢采集不再占用探测所需的锁,采集完成也不会覆盖更新的探测结果。HTTP 健康请求只读取快照,不执行 SQL。`pg_up` 与 `pg_in_recovery` 仍属于采集指标:fatal 采集器失败可能使 `pg_up` 为 0,而数据库可达时 `/up` 仍返回 200。PostgreSQL 主连接池最多三个连接,PgBouncer 最多两个,每个发现数据库最多一个。

| 响应正文 | HTTP 状态 | 含义 |
|:---|:---|:---|
Expand Down
12 changes: 6 additions & 6 deletions content/blog/design/container-trust.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@
title: "Container Trust Roots and Release Checks"
linkTitle: "Container Trust"
date: "2026-11-08T00:00:00+08:00"
lastmod: "2026-11-08T00:00:00+08:00"
lastmod: "2026-11-09T00:00:00+08:00"
authors: [Vonng]
categories: [design]
tags: [Design, Security, Compatibility]
summary: "Unreleased: include CA roots and the license in both scratch images, and exercise TLS and DEB artifacts in release validation."
summary: "v1.6.0: include CA roots and the license in both scratch images, and exercise TLS and DEB artifacts in release validation."
---

> **Status:** Implemented but unreleased.<br>
> **Decision date / last verification:** 2026-11-08 / 2026-11-08.<br>
> **Status:** Implemented in v1.6.0.<br>
> **Decision date / last verification:** 2026-11-08 / 2026-11-09.<br>
> **Scope:** Container and release-validation changes based on v1.5.0 source `1891254`, incorporating the packaging correction from `68c95ba`. No exporter runtime or driver change.<br>
> **Delivery:** Source is committed locally as `370b9fa`; the companion record is committed locally. Both Docker build paths, amd64/arm64 GoReleaser images, three DEB architectures, Go tests, static checks, and the documentation build are validated. Push, tag, published packages/images, deployment, and production verification have not occurred for this change.
> **Delivery:** Source `370b9fa` is included in v1.6.0 at `6bee809` and pushed. Both Docker build paths, amd64/arm64 GoReleaser images, three DEB architectures, Go tests, static checks, and the documentation build are validated. CI and draft artifacts are tracked by the v1.6.0 release gates; public tag, published images, deployment, and production verification remain separate.

## Problem

Expand Down Expand Up @@ -44,4 +44,4 @@ The TLS cases use a local server and generated certificates, without depending o

Release artifact checks retain RPM compatibility assertions and add DEB metadata, payload, configuration modes, conffiles, and installation checks. All DEB architectures are inspected; the container's native architecture is installed and its binary runs as `prometheus`. Checksums must verify successfully.

These gates validate a build candidate. Stable documentation and release notes must continue to describe v1.5.0 until a new tag and its artifacts actually ship this correction.
These gates validate a build candidate. The v1.6.0 manuals and release copy describe this correction; deploy them with the public release, after its artifacts are available.
12 changes: 6 additions & 6 deletions content/blog/design/container-trust.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@
title: "容器信任根与发布检查"
linkTitle: "容器信任根"
date: "2026-11-08T00:00:00+08:00"
lastmod: "2026-11-08T00:00:00+08:00"
lastmod: "2026-11-09T00:00:00+08:00"
authors: [Vonng]
categories: [design]
tags: [Design, Security, Compatibility]
summary: "尚未发布:为两条 scratch 镜像路径补齐 CA 根证书与许可证,并在发布验证中检查 TLS 连接和 DEB 产物。"
summary: "v1.6.0:为两条 scratch 镜像路径补齐 CA 根证书与许可证,并在发布验证中检查 TLS 连接和 DEB 产物。"
---

> **状态:** 已实现,未发布。<br>
> **决策日期/最后核验:** 2026-11-08/2026-11-08。<br>
> **状态:** 已实现,纳入 v1.6.0。<br>
> **决策日期/最后核验:** 2026-11-08/2026-11-09。<br>
> **范围:** 基于 v1.5.0 源码 `1891254` 的容器与发布验证改动,吸收 `68c95ba` 的打包修正。不修改 exporter 运行时代码或数据库驱动。<br>
> **交付:** 源码已在本地提交为 `370b9fa`,配套设计记录也已在本地提交。两条 Docker 构建路径、amd64/arm64 GoReleaser 镜像、三个 DEB 架构、Go 测试、静态检查和文档构建均已验证。本次改动尚未推送、打标签、发布软件包或镜像、部署,也未完成生产验证。
> **交付:** 源码 `370b9fa` 已纳入 v1.6.0 的 `6bee809` 并推送。两条 Docker 构建路径、amd64/arm64 GoReleaser 镜像、三个 DEB 架构、Go 测试、静态检查和文档构建均已验证。CI 与草稿制品以 v1.6.0 发布关卡为准;公开标签、镜像发布、部署和生产验证仍属独立步骤。

## 问题

Expand Down Expand Up @@ -44,4 +44,4 @@ TLS 用例使用本地服务与临时生成的证书,不依赖公共 TLS 服

发布产物检查保留 RPM 兼容性断言,补充 DEB 元数据、文件内容、配置权限、conffiles 和安装检查。检查所有 DEB 架构的内容,在容器原生架构上安装软件包,并以 `prometheus` 身份运行二进制。校验和必须验证成功。

这些检查只验收构建候选。新的标签与产物实际发布前,稳定版手册与发行说明仍应描述 v1.5.0 的行为。
这些检查只验收构建候选。v1.6.0 手册与发布正文描述本次修正,在正式公开版本与制品可用时一同上线。
41 changes: 41 additions & 0 deletions content/blog/design/progress-database-scope.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
title: "Cluster Progress with Safe Relation Labels"
linkTitle: "Progress Database Scope"
date: "2026-11-09T00:00:00+08:00"
lastmod: "2026-11-09T00:00:00+08:00"
authors: [Vonng]
categories: [design]
tags: [Design, PostgreSQL, Compatibility]
summary: "v1.6.0: keep cluster-wide progress and use numeric OIDs for user relations in other databases."
---

> **Status:** Implemented in v1.6.0.<br>
> **Decision / last verification:** 2026-11-09.<br>
> **Scope:** v1.6.0 at source `6bee809` on main, [#115](https://github.com/pgsty/pg_exporter/issues/115); PostgreSQL 12+ vacuum, index creation, and cluster progress. Not shipped in v1.5.0.<br>
> **Delivery:** Source committed and pushed as `6bee809`; local validation and [Test Release](https://github.com/pgsty/pg_exporter/actions/runs/42549895870) passed. Versioned archives, RPM/DEB packages, and checksums are built for the draft. A public tag, public images, deployment, and production verification remain separate gates.

## Decision

The progress views report work across the instance, but `relid::regclass` resolves names only within the connected database. An OID from another database can resolve to an unrelated local relation.

Keep `pg_vacuuming`, `pg_indexing`, and `pg_clustering` tagged `cluster, primary`. Resolve `relname` as `relid::regclass::text` when the row belongs to the connected database or the relation is a system catalog: OIDs below 16384 (`FirstNormalObjectId`) are assigned by initdb and name the same catalog in every database. For other relations in another database, emit the numeric OID as text, retaining `datname` and `pid` to identify the task. Do not filter out cross-database rows.

Keep metric names, labels, types, version ranges, TTLs, and three cluster-level queries per refresh. Cross-database coverage continues when auto-discovery is disabled, a database is excluded, or the monitoring role cannot connect to it, subject to the progress view's existing visibility rules. No new connections or name lookups are required.

## Tradeoff and rejected alternatives

A numeric OID is less readable than a relation name, but accurately reports the identifier available from the main connection. This fixes false names without losing catalog names, such as a VACUUM of `pg_attribute` in another database; it deliberately does not promise cross-database name resolution for user relations.

Per-database collection was rejected because it increases refresh queries from three to three per monitored database, including when no progress exists. It also loses progress from excluded or inaccessible databases. Adding lookup connections for active tasks duplicates discovery and permissions handling. Renaming labels would require a separate metric migration. Base-backup progress is unchanged.

## Acceptance and upgrade

Execute the PostgreSQL 12–16, 17, and 18+ SQL variants with local and foreign rows sharing the same relation OID. Verify the local name, the foreign numeric OID, and the name of a foreign system catalog, with one sample per task. Check primary/replica planning, that discovered connections do not duplicate cluster collectors, and cross-database coverage with discovery disabled or excluded databases.

Update split YAML and generated configuration together. Existing installations must merge or replace these collector definitions; a binary-only upgrade does not apply this SQL correction. Local SQL tests do not establish production-load acceptance.

## Local verification

PostgreSQL 12, 17, and 18 passed monitoring-role SQL integration tests covering all three query-version branches. Temporary progress views include a local and foreign task with the same relation OID; the local task receives its name and the foreign task its numeric OID. A real PostgreSQL 18 index build remained visible once per metric with discovery disabled, with the database excluded, and when the monitoring role lacked CONNECT on that database. Planning tests cover PostgreSQL 12–19 and prevent duplicate collection on discovered connections.

A review follow-up added a foreign `pg_catalog.pg_attribute` row to the fixture. On PostgreSQL 18.6 it reports `pg_attribute`; the earlier condition reported `1249`. A live throttled VACUUM of `pg_attribute` in another database confirmed the same, while a user table there still reported its OID. PostgreSQL 12 and 17 were not rerun for this follow-up.
Loading
Loading