Add standalone pgsql-exporter for external databases

charts/sourcegraph/CHANGELOG.md

@@ -8,6 +8,7 @@ Use `**BREAKING**:` to denote a breaking change
 
 ## Unreleased
 
+- Added `pgsql.postgresExporter.standalone`, `codeIntelDB.postgresExporter.standalone` and `codeInsightsDB.postgresExporter.standalone` (default `false`). When a database is external (`<database>.enabled: false`), setting this deploys `pgsql-exporter` as a standalone Deployment and Service, connecting with the database's `auth` credentials (including `sslmode`), so the Postgres panels in the Grafana dashboards work for external databases such as AWS RDS. The `sslmode` key is required in `auth.existingSecret`, as it already was for the application containers; the external-databases example README previously named it `pgsslmode`.
 - Added opt-in Redis config management through `redisCache.config.enabled` and `redisStore.config.enabled` (both default to `false`). Default deployments retain their image configs, custom mounts, authentication, and memory settings. When enabled, the chart mounts `/etc/redis/redis.conf` and supports `existingConfig`, `maxmemory`, `maxmemoryRatio`, `maxmemoryPolicy`, and `additionalConfig`. See [Enabling Redis config management](README.md#enabling-redis-config-management) before opting in, especially when using custom images or Secret-mounted configs.
 - With Redis config management enabled, the chart sizes `maxmemory` at 75% of `resources.limits.memory` to leave room for overhead; this reduces, but does not eliminate, OOM risk. Ratios must be greater than 0 and less than 1. Auto-sizing is skipped in local development mode or when the limit is absent or unrecognised. At the default 7Gi limit, opting in lowers the effective cap from 6GiB to 5.25GiB: `redis-cache` evicts earlier, but `redis-store` uses `noeviction`, so its write-error ceiling drops too. Set `redisStore.config.maxmemory: 6gb` to keep the old ceiling if the pod has sufficient memory.
 - `searcher.storageSize` now accepts any Kubernetes quantity (e.g. `150G`, `1Ti`, `1.5Ti`, `500000M`) when deriving `SEARCHER_CACHE_SIZE_MB` and `SYMBOLS_CACHE_SIZE_MB`. Previously anything other than `Gi` silently rendered both as `0`, which disables cache eviction and lets the cache volume fill up. Unparseable values, or values below `3M`, now fail the render unless `searcher.autoCacheSize` is enabled. The derived size now matches searcher's own auto-sizing arithmetic (45% of the volume in decimal megabytes), so existing `Gi` deployments see a slightly higher value on upgrade, e.g. the default `26Gi` renders `12562` instead of `11700`.

charts/sourcegraph/README.md

@@ -130,7 +130,7 @@ In addition to the documented values, all services also support the following va
 | codeInsightsDB.init.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":70,"runAsUser":70}` | Security context for the `alpine` initContainer, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
 | codeInsightsDB.name | string | `"codeinsights-db"` | Name used by resources. Does not affect service names or PVCs. |
 | codeInsightsDB.podSecurityContext | object | `{"fsGroup":70,"fsGroupChangePolicy":"OnRootMismatch","runAsUser":70}` | Security context for the `codeinsights-db` pod, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) |
-| codeInsightsDB.postgresExporter | object | `{}` | Configuration for the `pgsql-exporter` sidecar container |
+| codeInsightsDB.postgresExporter.standalone | bool | `false` | Deploy `pgsql-exporter` as a standalone Deployment to collect metrics from an external database (e.g. AWS RDS). Only applies when `codeInsightsDB.enabled` is `false`; the internal database always runs `pgsql-exporter` as a sidecar container |
 | codeInsightsDB.resources | object | `{"limits":{"cpu":"4","memory":"2Gi"},"requests":{"cpu":"4","memory":"2Gi"}}` | Resource requests & limits for the `codeinsights-db` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
 | codeInsightsDB.serviceAccount.create | bool | `false` | Enable creation of ServiceAccount for `codeinsights-db` |
 | codeInsightsDB.serviceAccount.name | string | `""` | Name of the ServiceAccount to be created or an existing ServiceAccount |
@@ -152,7 +152,7 @@ In addition to the documented values, all services also support the following va
 | codeIntelDB.image.name | string | `"postgresql-16"` | Docker image name for the `codeintel-db` image |
 | codeIntelDB.name | string | `"codeintel-db"` | Name used by resources. Does not affect service names or PVCs. |
 | codeIntelDB.podSecurityContext | object | `{"fsGroup":999,"fsGroupChangePolicy":"OnRootMismatch","runAsUser":999}` | Security context for the `codeintel-db` pod, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) |
-| codeIntelDB.postgresExporter | object | `{}` | Configuration for the `pgsql-exporter` sidecar container |
+| codeIntelDB.postgresExporter.standalone | bool | `false` | Deploy `pgsql-exporter` as a standalone Deployment to collect metrics from an external database (e.g. AWS RDS). Only applies when `codeIntelDB.enabled` is `false`; the internal database always runs `pgsql-exporter` as a sidecar container |
 | codeIntelDB.resources | object | `{"limits":{"cpu":"4","memory":"4Gi"},"requests":{"cpu":"4","memory":"4Gi"}}` | Resource requests & limits for the `codeintel-db` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
 | codeIntelDB.serviceAccount.create | bool | `false` | Enable creation of ServiceAccount for `codeintel-db` |
 | codeIntelDB.serviceAccount.name | string | `""` | Name of the ServiceAccount to be created or an existing ServiceAccount |
@@ -320,7 +320,7 @@ In addition to the documented values, all services also support the following va
 | pgsql.image.name | string | `"postgresql-16"` | Docker image name for the `pgsql` image |
 | pgsql.name | string | `"pgsql"` | Name used by resources. Does not affect service names or PVCs. |
 | pgsql.podSecurityContext | object | `{"fsGroup":999,"fsGroupChangePolicy":"OnRootMismatch","runAsGroup":999,"runAsUser":999}` | Security context for the `pgsql` pod, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) |
-| pgsql.postgresExporter | object | `{}` | Configuration for the `pgsql-exporter` sidecar container |
+| pgsql.postgresExporter.standalone | bool | `false` | Deploy `pgsql-exporter` as a standalone Deployment to collect metrics from an external database (e.g. AWS RDS). Only applies when `pgsql.enabled` is `false`; the internal database always runs `pgsql-exporter` as a sidecar container |
 | pgsql.resources | object | `{"limits":{"cpu":"4","memory":"4Gi"},"requests":{"cpu":"4","memory":"4Gi"}}` | Resource requests & limits for the `pgsql` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
 | pgsql.serviceAccount.create | bool | `false` | Enable creation of ServiceAccount for `pgsql` |
 | pgsql.serviceAccount.name | string | `""` | Name of the ServiceAccount to be created or an existing ServiceAccount |

charts/sourcegraph/examples/external-databases/README.md

@@ -12,6 +12,12 @@ For guidance on setup for the external instances, refer to the [Helm docs on usi
 
 The example [override.yaml] demonstrates various options for configuring access to the external databases.
 
+## Database metrics
+
+The internal databases run `pgsql-exporter` as a sidecar container, which feeds the Postgres panels in the Sourcegraph Grafana dashboards. External databases have no sidecar, so those panels stay empty.
+
+Set `<database>.postgresExporter.standalone: true` to deploy `pgsql-exporter` as its own Deployment, connecting to the external database with the same credentials the application uses. The example [override.yaml] enables it for `pgsql`.
+
 ## Providing credentials via Secret
 
 You can provide database credentials in a [Secret](https://kubernetes.io/docs/concepts/configuration/secret/) managed outside the helm chart in a secure manner.
@@ -30,7 +36,7 @@ data:
   password: ""
   port: ""
   user: ""
-  pgsslmode: "require" # optional, enable if using SSL
+  sslmode: "" # "require" if using SSL, otherwise "disable"
 ```
 
 The Secret name should be set in your override file in the `auth.existingSecret` key for each database. See the [override.yaml] for an example.

charts/sourcegraph/examples/external-databases/override.yaml

@@ -8,6 +8,9 @@ pgsql:
   enabled: false # disable internal pgsql database
   auth:
     host: pgsql.database.company.com # external pgsql host
+    sslmode: require
+  postgresExporter:
+    standalone: true # deploy pgsql-exporter against the external database, so the Grafana dashboards show its metrics
 
 
 # Connect to an external postgres instance with custom settings

charts/sourcegraph/templates/_helpers.tpl

@@ -252,8 +252,23 @@ app.kubernetes.io/name: jaeger
     secretKeyRef:
       key: user
       name: {{ $secretName }}
+{{- if (index $top.Values $service "enabled") }}
 - name: DATA_SOURCE_URI
   value: "localhost:$(DATA_SOURCE_PORT)/$(DATA_SOURCE_DB)?sslmode=disable"
+{{- else }}
+- name: DATA_SOURCE_HOST
+  valueFrom:
+    secretKeyRef:
+      key: host
+      name: {{ $secretName }}
+- name: DATA_SOURCE_SSLMODE
+  valueFrom:
+    secretKeyRef:
+      key: sslmode
+      name: {{ $secretName }}
+- name: DATA_SOURCE_URI
+  value: "$(DATA_SOURCE_HOST):$(DATA_SOURCE_PORT)/$(DATA_SOURCE_DB)?sslmode=$(DATA_SOURCE_SSLMODE)"
+{{- end }}
 {{- end }}
 
 {{/*

charts/sourcegraph/templates/postgres-exporter/postgres-exporter.Deployment.yaml

@@ -0,0 +1,88 @@
+{{- define "sourcegraph.postgresExporter.deployment" -}}
+{{- $top := index . 0 -}}
+{{- $service := index . 1 -}}
+{{- $database := index $top.Values $service -}}
+{{- $exporterName := printf "%s-exporter" $database.name -}}
+{{- $queriesFile := index (dict "pgsql" "queries.yaml" "codeIntelDB" "code_intel_queries.yaml" "codeInsightsDB" "code_insights_queries.yaml") $service -}}
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  annotations:
+    description: Prometheus exporter for the external {{ $database.name }} database.
+  labels:
+    {{- include "sourcegraph.labels" $top | nindent 4 }}
+    deploy: sourcegraph
+    app.kubernetes.io/component: {{ $exporterName }}
+  name: {{ $exporterName }}
+spec:
+  replicas: 1
+  revisionHistoryLimit: {{ $top.Values.sourcegraph.revisionHistoryLimit }}
+  selector:
+    matchLabels:
+      {{- include "sourcegraph.selectorLabels" $top | nindent 6 }}
+      app: {{ $database.name }}
+  template:
+    metadata:
+      annotations:
+        checksum/{{ $database.name }}.secret: {{ toJson $database.auth | sha256sum }}
+      {{- if $top.Values.sourcegraph.podAnnotations }}
+      {{- toYaml $top.Values.sourcegraph.podAnnotations | nindent 8 }}
+      {{- end }}
+      {{- if $database.podAnnotations }}
+      {{- toYaml $database.podAnnotations | nindent 8 }}
+      {{- end }}
+      labels:
+      {{- include "sourcegraph.selectorLabels" $top | nindent 8 }}
+      {{- if $top.Values.sourcegraph.podLabels }}
+      {{- toYaml $top.Values.sourcegraph.podLabels | nindent 8 }}
+      {{- end }}
+      {{- if $database.podLabels }}
+      {{- toYaml $database.podLabels | nindent 8 }}
+      {{- end }}
+        deploy: sourcegraph
+        app: {{ $database.name }}
+    spec:
+      containers:
+      - name: pgsql-exporter
+        env:
+        {{- if not $top.Values.sourcegraph.disableKubernetesSecrets }}
+        {{- include "sourcegraph.dataSource" (list $top $service) | nindent 8 }}
+        {{- end }}
+        {{- range $name, $item := $database.postgresExporter.env }}
+        - name: {{ $name }}
+          {{- $item | toYaml | nindent 10 }}
+        {{- end }}
+        - name: PG_EXPORTER_EXTEND_QUERY_PATH
+          value: /config/{{ $queriesFile }}
+        image: {{ include "sourcegraph.image" (list $top "postgresExporter") }}
+        imagePullPolicy: {{ $top.Values.sourcegraph.image.pullPolicy }}
+        ports:
+        - name: http-metrics
+          containerPort: 9187
+        {{- if not $top.Values.sourcegraph.localDevMode }}
+        resources:
+          {{- toYaml $top.Values.postgresExporter.resources | nindent 10 }}
+        {{- end }}
+        securityContext:
+          {{- toYaml $top.Values.postgresExporter.containerSecurityContext | nindent 10 }}
+        terminationMessagePolicy: FallbackToLogsOnError
+      securityContext:
+        {{- toYaml $database.podSecurityContext | nindent 8 }}
+      {{- include "sourcegraph.nodeSelector" (list $top $service) | trim | nindent 6 }}
+      {{- include "sourcegraph.affinity" (list $top $service) | trim | nindent 6 }}
+      {{- with include "sourcegraph.priorityClassName" (list $top $service) | trim }}{{ . | nindent 6 }}{{- end }}
+      {{- include "sourcegraph.tolerations" (list $top $service) | trim | nindent 6 }}
+      {{- with $top.Values.sourcegraph.imagePullSecrets }}
+      imagePullSecrets:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- include "sourcegraph.renderServiceAccountName" (list $top $service) | trim | nindent 6 }}
+{{- end -}}
+
+{{- range $service := list "pgsql" "codeIntelDB" "codeInsightsDB" }}
+{{- $database := index $.Values $service }}
+{{- if and (not $database.enabled) $database.postgresExporter.standalone }}
+---
+{{ include "sourcegraph.postgresExporter.deployment" (list $ $service) }}
+{{- end }}
+{{- end }}

charts/sourcegraph/templates/postgres-exporter/postgres-exporter.Service.yaml

@@ -0,0 +1,34 @@
+{{- define "sourcegraph.postgresExporter.service" -}}
+{{- $top := index . 0 -}}
+{{- $service := index . 1 -}}
+{{- $database := index $top.Values $service -}}
+apiVersion: v1
+kind: Service
+metadata:
+  annotations:
+    prometheus.io/port: "9187"
+    sourcegraph.prometheus/scrape: "true"
+  labels:
+    # Prometheus derives the `job` label, which the Grafana dashboards group by, from `app`
+    app: {{ $database.name }}
+    deploy: sourcegraph
+    app.kubernetes.io/component: {{ $database.name }}-exporter
+  name: {{ $database.name }}-exporter
+spec:
+  ports:
+  - name: http-metrics
+    port: 9187
+    targetPort: http-metrics
+  selector:
+    {{- include "sourcegraph.selectorLabels" $top | nindent 4 }}
+    app: {{ $database.name }}
+  type: ClusterIP
+{{- end -}}
+
+{{- range $service := list "pgsql" "codeIntelDB" "codeInsightsDB" }}
+{{- $database := index $.Values $service }}
+{{- if and (not $database.enabled) $database.postgresExporter.standalone }}
+---
+{{ include "sourcegraph.postgresExporter.service" (list $ $service) }}
+{{- end }}
+{{- end }}

charts/sourcegraph/tests/postgresExporterStandalone_test.yaml

@@ -0,0 +1,122 @@
+suite: postgresExporterStandalone
+templates:
+- postgres-exporter/postgres-exporter.Deployment.yaml
+- postgres-exporter/postgres-exporter.Service.yaml
+- pgsql/pgsql.StatefulSet.yaml
+tests:
+- it: should not render standalone exporters by default
+  templates:
+  - postgres-exporter/postgres-exporter.Deployment.yaml
+  - postgres-exporter/postgres-exporter.Service.yaml
+  asserts:
+  - hasDocuments:
+      count: 0
+- it: should not render a standalone exporter for an internal database
+  templates:
+  - postgres-exporter/postgres-exporter.Deployment.yaml
+  - postgres-exporter/postgres-exporter.Service.yaml
+  set:
+    pgsql:
+      postgresExporter:
+        standalone: true
+  asserts:
+  - hasDocuments:
+      count: 0
+- it: should not render a standalone exporter for an external database unless enabled
+  templates:
+  - postgres-exporter/postgres-exporter.Deployment.yaml
+  - postgres-exporter/postgres-exporter.Service.yaml
+  set:
+    pgsql:
+      enabled: false
+  asserts:
+  - hasDocuments:
+      count: 0
+- it: should render one Deployment and Service per external database with standalone enabled
+  templates:
+  - postgres-exporter/postgres-exporter.Deployment.yaml
+  - postgres-exporter/postgres-exporter.Service.yaml
+  set:
+    pgsql:
+      enabled: false
+      postgresExporter:
+        standalone: true
+    codeIntelDB:
+      enabled: false
+      postgresExporter:
+        standalone: true
+    codeInsightsDB:
+      enabled: false
+  asserts:
+  - hasDocuments:
+      count: 2
+  - isKind:
+      of: Deployment
+    template: postgres-exporter/postgres-exporter.Deployment.yaml
+  - isKind:
+      of: Service
+    template: postgres-exporter/postgres-exporter.Service.yaml
+  - equal:
+      path: metadata.name
+      value: pgsql-exporter
+    documentIndex: 0
+  - equal:
+      path: metadata.name
+      value: codeintel-db-exporter
+    documentIndex: 1
+- it: should connect to the external host with the sslmode from the secret
+  template: postgres-exporter/postgres-exporter.Deployment.yaml
+  set:
+    codeInsightsDB:
+      enabled: false
+      auth:
+        existingSecret: "my-secret"
+      postgresExporter:
+        standalone: true
+  asserts:
+  - equal:
+      path: spec.template.spec.containers[0].name
+      value: pgsql-exporter
+  - contains:
+      path: spec.template.spec.containers[0].env
+      content:
+        name: DATA_SOURCE_HOST
+        valueFrom:
+          secretKeyRef:
+            key: host
+            name: my-secret
+  - contains:
+      path: spec.template.spec.containers[0].env
+      content:
+        name: DATA_SOURCE_URI
+        value: "$(DATA_SOURCE_HOST):$(DATA_SOURCE_PORT)/$(DATA_SOURCE_DB)?sslmode=$(DATA_SOURCE_SSLMODE)"
+  - contains:
+      path: spec.template.spec.containers[0].env
+      content:
+        name: PG_EXPORTER_EXTEND_QUERY_PATH
+        value: /config/code_insights_queries.yaml
+- it: should label the Service so Prometheus sets job to the database name
+  template: postgres-exporter/postgres-exporter.Service.yaml
+  set:
+    codeIntelDB:
+      enabled: false
+      postgresExporter:
+        standalone: true
+  asserts:
+  - equal:
+      path: metadata.labels.app
+      value: codeintel-db
+  - equal:
+      path: metadata.annotations["sourcegraph.prometheus/scrape"]
+      value: "true"
+  - equal:
+      path: metadata.annotations["prometheus.io/port"]
+      value: "9187"
+- it: should keep the sidecar exporter connecting to localhost
+  template: pgsql/pgsql.StatefulSet.yaml
+  asserts:
+  - contains:
+      path: spec.template.spec.containers[1].env
+      content:
+        name: DATA_SOURCE_URI
+        value: "localhost:$(DATA_SOURCE_PORT)/$(DATA_SOURCE_DB)?sslmode=disable"