Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions pg-partman/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# SPDX-FileCopyrightText: Copyright © contributors to CloudNativePG, established as CloudNativePG a Series of LF Projects, LLC.
# SPDX-License-Identifier: Apache-2.0

ARG BASE=ghcr.io/cloudnative-pg/postgresql:18-minimal-trixie
FROM $BASE AS builder

ARG PG_MAJOR
ARG EXT_VERSION

USER 0

# Install extension via `apt-get`
RUN apt-get update && apt-get install -y --no-install-recommends \
"postgresql-${PG_MAJOR}-partman=${EXT_VERSION}"

FROM scratch
ARG PG_MAJOR

# Licenses
COPY --from=builder /usr/share/doc/postgresql-${PG_MAJOR}-partman/copyright /licenses/postgresql-${PG_MAJOR}-partman/

# Libraries
COPY --from=builder /usr/lib/postgresql/${PG_MAJOR}/lib/pg_partman_bgw.so /lib/
COPY --from=builder /usr/lib/postgresql/${PG_MAJOR}/lib/bitcode/ /lib/bitcode/

# Share
COPY --from=builder /usr/share/postgresql/${PG_MAJOR}/extension/pg_partman* /share/extension/

# Binaries (maintenance scripts)
COPY --from=builder /usr/lib/postgresql/${PG_MAJOR}/bin/check_unique_constraint.py /bin/
COPY --from=builder /usr/lib/postgresql/${PG_MAJOR}/bin/dump_partition.py /bin/
COPY --from=builder /usr/lib/postgresql/${PG_MAJOR}/bin/vacuum_maintenance.py /bin/
Comment on lines +29 to +32

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes!

Comment on lines +30 to +32

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This python scripts have hard dependencies on python and psycopg2 so adding them straight to the /bin directory is not enough to have them working properly. The dependencies should be packaged as well in the extension container in a way they can be resolved by those scripts at runtime. As far as I understand, they are there mainly for maintenance operations and are not referenced directly by the extension. I'd rather avoid including them here as I believe the extension container should be kept as light as possible. Those scripts seem more client-side tools as they connect to Postgres over the network; they don't need to run inside the Postgres container at all. We can explore and document alternatives to execute them externally in a way they can run in a K8s env and target a CNPG Cluster.

@GabriFedi97 GabriFedi97 Jul 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to execute the scripts externally via a Kubernetes Job created in the CNPG Cluster namespace. Leaving the Job definition as a POC:

apiVersion: batch/v1
kind: Job
metadata:
  name: partman-maint-poc
  namespace: <your-cluster-namespace>
  labels:
    app: partman-maint-poc
spec:
  backoffLimit: 0
  ttlSecondsAfterFinished: 3600
  template:
    metadata:
      labels:
        app: partman-maint-poc
    spec:
      restartPolicy: Never
      containers:
      - name: maint
        image: python:3-slim
        # --- Connection info shared from the CNPG-generated app secret ---
        env:
        - name: DB_URI
          valueFrom: { secretKeyRef: { name: pg-partman-app, key: uri } }
        - name: PGHOST
          value: pg-partman-rw
        - name: PGPORT
          valueFrom: { secretKeyRef: { name: pg-partman-app, key: port } }
        - name: PGUSER
          valueFrom: { secretKeyRef: { name: pg-partman-app, key: username } }
        - name: PGPASSWORD
          valueFrom: { secretKeyRef: { name: pg-partman-app, key: password } }
        - name: PGDATABASE
          valueFrom: { secretKeyRef: { name: pg-partman-app, key: dbname } }
        # Verify the CNPG server cert against the cluster CA
        - name: PGSSLMODE
          value: verify-full
        - name: PGSSLROOTCERT
          value: /etc/ca/ca.crt
        volumeMounts:
        - name: ca
          mountPath: /etc/ca
          readOnly: true
        command: ["bash", "-c"]
        args:
        - |
          set -euo pipefail

          echo "### 1. Install dependencies"
          apt-get update && apt-get install -y --no-install-recommends curl
          pip install --root-user-action=ignore --no-cache-dir psycopg2-binary

          echo "### 2. Fetch the pg_partman maintenance scripts (v5.4.3)"
          mkdir -p /opt/partman && cd /opt/partman
          for s in check_unique_constraint dump_partition vacuum_maintenance; do
            curl -sfLO "https://raw.githubusercontent.com/pgpartman/pg_partman/v5.4.3/bin/common/$s.py" \
            || curl -sfLO "https://raw.githubusercontent.com/pgpartman/pg_partman/v5.4.3/bin/$s.py"
          done
          chmod +x ./*.py
          ls -l ./*.py

          echo "### 3. Prove the scripts run (deps satisfied)"
          python vacuum_maintenance.py --help

          echo "### 4. Best-effort real run against the cluster"
          python vacuum_maintenance.py -c "$DB_URI" --all \
            && echo "vacuum_maintenance completed" \
            || echo "NOTE: script ran but exited non-zero (likely partman privileges/config) — mechanism is proven"

          echo "### DONE"
      volumes:
      - name: ca
        secret:
          secretName: pg-partman-ca

To make it work properly I had to grant the app user the permissions to the part_config and part_config_sub tables:

GRANT ALL PRIVILEGES ON TABLE part_config, part_config_sub TO app;

This is merely a POC to show the approach. We can revisit the details in case we agree on the proposal.


USER 65532:65532
123 changes: 123 additions & 0 deletions pg-partman/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
# pg_partman
<!--
SPDX-FileCopyrightText: Copyright © contributors to CloudNativePG, established as CloudNativePG a Series of LF Projects, LLC.
SPDX-License-Identifier: Apache-2.0
-->

[pg_partman](https://github.com/pgpartman/pg_partman) is a PostgreSQL extension
for automated table partition management. It supports both time-based and
ID-based partitioning with automatic creation and maintenance of child tables.

The extension includes a background worker (`pg_partman_bgw`) that can
automatically run partition maintenance at configured intervals, removing the
need for external cron jobs.

For more information, see the
[official documentation](https://github.com/pgpartman/pg_partman).

## Usage

The `pg_partman` extension must be loaded via `shared_preload_libraries` to
enable the background worker. The worker periodically runs
`partman.run_maintenance_proc()` to create new partitions and drop expired ones.

### 1. Add the pg_partman extension image to your Cluster

Define the `pg-partman` extension under the `postgresql.extensions` section of
your `Cluster` resource. For example:

```yaml
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: cluster-pg-partman
spec:
imageName: ghcr.io/cloudnative-pg/postgresql:18-minimal-trixie
instances: 1

storage:
size: 1Gi

postgresql:
shared_preload_libraries:
- pg_partman_bgw
extensions:
- name: pg-partman
image:
# renovate: suite=trixie-pgdg depName=postgresql-18-partman
reference: ghcr.io/cloudnative-pg/pg-partman:5.4.3-18-trixie
```

### 2. Enable the extension in a database

You can install `pg_partman` in a specific database by creating or updating a
`Database` resource. For example, to enable it in the `app` database:

```yaml
apiVersion: postgresql.cnpg.io/v1
kind: Database
metadata:
name: cluster-pg-partman-app
spec:
name: app
owner: app
cluster:
name: cluster-pg-partman
extensions:
- name: pg_partman
# renovate: suite=trixie-pgdg depName=postgresql-18-partman
version: '5.4.3'
```

### 3. Verify installation

Once the database is ready, connect to it with `psql` and run:

```sql
\dx
```

You should see `pg_partman` listed among the installed extensions.

## Included utilities

This image also bundles the following Python maintenance scripts in `/bin/`:

- `check_unique_constraint.py` — validates unique constraints across partitions
- `dump_partition.py` — exports individual partitions for archival
- `vacuum_maintenance.py` — targeted vacuum operations on partitioned tables

> [!NOTE]
> These scripts require a Python 3 runtime and the `psycopg2` library, which
> are not included in the minimal base image. They are provided for
> environments where Python is available on the host or in a sidecar container.

Comment on lines +84 to +94

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section would have to change to document the alternative way to execute the scripts in case we agree on https://github.com/cloudnative-pg/postgres-extensions-containers/pull/159/changes#r3601885000.

## Contributors

This extension is maintained by:

- Erling Kristiansen (@egkristi)

The maintainers are responsible for:

- Monitoring upstream releases and security vulnerabilities.
- Ensuring compatibility with supported PostgreSQL versions.
- Reviewing and merging contributions specific to this extension's container
image and lifecycle.

---

## Licenses and Copyright

This container image contains software that may be licensed under various
open-source licenses.

All relevant license and copyright information for the `pg_partman` extension
and its dependencies are bundled within the image at:

```text
/licenses/
```

By using this image, you agree to comply with the terms of the licenses
contained therein.
37 changes: 37 additions & 0 deletions pg-partman/metadata.hcl
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# SPDX-FileCopyrightText: Copyright © contributors to CloudNativePG, established as CloudNativePG a Series of LF Projects, LLC.
# SPDX-License-Identifier: Apache-2.0
metadata = {
name = "pg-partman"
sql_name = "pg_partman"
image_name = "pg-partman"
licenses = ["PostgreSQL"]
shared_preload_libraries = ["pg_partman_bgw"]
postgresql_parameters = {}
extension_control_path = []
dynamic_library_path = []
ld_library_path = []
bin_path = ["bin"]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for following this standard

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

env = {}
auto_update_os_libs = false
required_extensions = []
create_extension = true

versions = {
bookworm = {
"18" = {
// renovate: suite=bookworm-pgdg depName=postgresql-18-partman
package = "5.4.3-1.pgdg12+1"
// renovate: suite=bookworm-pgdg depName=postgresql-18-partman extractVersion=^(?<version>\d+\.\d+\.\d+)
sql = "5.4.3"
}
}
trixie = {
"18" = {
// renovate: suite=trixie-pgdg depName=postgresql-18-partman
package = "5.4.3-1.pgdg13+1"
// renovate: suite=trixie-pgdg depName=postgresql-18-partman extractVersion=^(?<version>\d+\.\d+\.\d+)
sql = "5.4.3"
}
}
}
}