Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
329 changes: 328 additions & 1 deletion charts/aisix-cp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,27 @@

Helm chart for AISIX control plane (cp-api, dp-manager, dashboard)

AISIX is an AI gateway: it fronts LLM providers with routing, rate limiting,
budgets, caching, guardrails, and observability behind an OpenAI-compatible API.
This chart installs the **control plane** — the console and management services
that configure the gateways and collect what they report.

It installs three services and, by default, a PostgreSQL database:

* `cp-api` — the Admin API, and the externally reachable entry point. It also
reverse-proxies the dashboard, so the console and the API share one origin.
* `dp-manager` — issues gateway certificates over mutual TLS and delivers
configuration to the gateways.
* `dashboard` — the Next.js console, reached through `cp-api`.
* PostgreSQL — the shared datastore. The chart deploys a bundled single
instance by default, or connects to an [external database](#external-postgresql).

The gateways are **not** installed by this chart. Install them separately with
the [`aisix`](../aisix/README.md) chart, or any of the other options in the
[on-premises installation guide](https://docs.api7.ai/ai-gateway/on-premises/deployment);
they connect out to `dp-manager`, so the control plane needs no inbound access
to gateway hosts.

## Maintainers

| Name | Email | Url |
Expand All @@ -16,7 +37,221 @@ Helm chart for AISIX control plane (cp-api, dp-manager, dashboard)
|------------|------|---------|
| https://charts.bitnami.com/bitnami | postgresql | 12.12.10 |

## Private PostgreSQL CA trust for the dashboard
## Prerequisites

* Kubernetes v1.23+
* Helm v3+
* OpenSSL, to generate the secrets below
* Secrets, which have no defaults you can deploy with. **The chart fails the
render** while one is missing or still holds its placeholder, so an install
that supplies none of them stops before anything reaches the cluster. Two are
needed in every mode:

| Value | Generate with |
| --- | --- |
| `secrets.masterKey` | `openssl rand -base64 32` |
| `secrets.betterAuthSecret` | `openssl rand -base64 48` |

The database credentials depend on which database you use. With the bundled
PostgreSQL — `postgresql.builtin: true`, the default — set both of these,
unless `postgresql.auth.existingSecret` supplies them instead:

| Value | Generate with |
| --- | --- |
| `postgresql.auth.postgresPassword` | `openssl rand -hex 24` |
| `postgresql.auth.password` | `openssl rand -hex 24` |

With an [external database](#external-postgresql) neither is read; that mode
needs `externalDatabase.existingSecret` or `externalDatabase.password`
instead, and fails the render with neither.

Whichever mode, use a **URL-safe** database password — `openssl rand -hex 24`,
not `-base64`. The password of the role that serves application connections is
embedded in a `postgres://` DSN, and `+`, `/` and `=` corrupt it. On the
bundled database that role is `postgres` by default, because
`postgresql.auth.usePostgresUserForAppConnections` defaults to `true`; set
both passwords URL-safe and the question does not arise.

**Keep `secrets.masterKey`.** It encrypts stored provider credentials and the
private key of the certificate authority that issued your gateway
certificates. Neither is recoverable from a database backup without it, so an
upgrade or a reinstall that supplies a *new* key leaves the existing rows
unreadable. Store it, its `secrets.masterKeyID`, and `secrets.betterAuthSecret`
the way you store a database backup — see
[Backup and Recovery](https://docs.api7.ai/ai-gateway/on-premises/backup-and-recovery).

## Install

Generate the secrets into a values file and keep that file: it is what you
upgrade with, and it is the only copy of the master key.

```sh
cat > cp-values.yaml <<EOF
secrets:
masterKey: "$(openssl rand -base64 32)"
betterAuthSecret: "$(openssl rand -base64 48)"
postgresql:
auth:
postgresPassword: "$(openssl rand -hex 24)"
password: "$(openssl rand -hex 24)"
EOF
```

Then install the chart:

```sh
helm repo add api7 https://charts.api7.ai
helm repo update

helm install aisix-cp api7/aisix-cp --namespace aisix --create-namespace \
--version 1.3.0 \
-f cp-values.yaml
```

Or pass the four values on the command line instead of keeping a file — the
same install, as long as you record what you generated:

```sh
helm install aisix-cp api7/aisix-cp --namespace aisix --create-namespace \
--version 1.3.0 \
--set secrets.masterKey="$(openssl rand -base64 32)" \
--set secrets.betterAuthSecret="$(openssl rand -base64 48)" \
--set postgresql.auth.postgresPassword="$(openssl rand -hex 24)" \
--set postgresql.auth.password="$(openssl rand -hex 24)"
```

`cp-api` runs the schema migration before it binds its port, so its first pod
can take a while to turn ready. A `startupProbe` covers that; wait for the pods:

```sh
kubectl -n aisix rollout status deploy/aisix-cp-api
```

### Reach the console

All three services are `ClusterIP` by default. `cp-api` serves both the Admin
API and the dashboard, so one port-forward reaches everything:

```sh
kubectl -n aisix port-forward svc/aisix-cp-api 8080:8080
```

Open `http://localhost:8080` while it runs. Sign-in is checked against the
origin in `api.publicBaseURL`, which defaults to `http://localhost:8080` and
matches this port-forward (`127.0.0.1` works too). Reaching the console at any
other address — an Ingress, a LoadBalancer, a different port — is refused at
sign-in with an "address not allowed" error until you either point
`api.publicBaseURL` at that address and upgrade, or add it to
`AISIX_TRUSTED_ORIGINS` through `ui.extraEnvVars`, which is additive and so
covers a console reached through more than one hostname.

On first visit, select **Create an account**, register the first user, accept
the agreement, and create the first organization. The account you create this
way is the deployment's first administrator; there is no seeded password.

### Attach a gateway

Set `api.dpmgrBaseURL` to the address gateways will dial and upgrade the
release. It is both the endpoint the console writes into the install commands
it generates and the host `dp-manager` seeds into the TLS server certificate it
presents — left empty it does neither, and a gateway dialing an IP literal
fails the handshake with nothing logged to say why.

The console's generated commands carry a gateway image, taken from
`api.dpImage`. Left empty — the default — it follows the chart's `appVersion`,
so this release hands out `docker.io/api7/aisix:1.3.0`.
Set it only to pin a different image, and remember that a value set explicitly
carries forward across upgrades, so gateways added later come up on the old
image.

## Upgrade

The control-plane chart and the gateway chart ship with the same `version` and
`appVersion` every release, so both move to the same release tag. **Upgrade the
control plane first, then the gateways** — a gateway may lag the control plane,
not the other way round. Take a database backup before you start.

```sh
helm repo update

helm upgrade aisix-cp api7/aisix-cp --namespace aisix -f cp-values.yaml
```

Upgrade with your own values file rather than `--reuse-values`: the same
secrets must be supplied again, and `--reuse-values` replays the previous
release's fully resolved values, chart defaults included, so a default this
chart changed — the probe budgets among them — is not adopted. If you never
kept a file, `helm get values aisix-cp --namespace aisix` prints the overrides
the release was installed with.

Wait for the `cp-api` pods to become ready before upgrading the gateways: the
schema migration runs on first start under the new version.

`cp-api` refuses to start when the database was last run by a release older
than this one supports upgrading from, naming the version it found and the
oldest it accepts. Upgrade to a supported version first; to proceed anyway,
after taking a backup, pass `AISIX_ALLOW_UNSUPPORTED_UPGRADE=1` through
`api.extraEnvVars`. See
[Upgrade](https://docs.api7.ai/ai-gateway/on-premises/upgrade).

## Uninstall

```sh
helm uninstall aisix-cp --namespace aisix
```

Two things deliberately survive it. The bundled PostgreSQL PVC is not owned by
the release, so the database — and everything in it — stays until you delete it
by hand. The chart's own Secret carries `helm.sh/resource-policy: keep`, so the
master key survives too, which is what lets a reinstall read the data that is
still there. Remove them only when you mean to discard the deployment:

```sh
kubectl -n aisix get pvc # find the database volume
kubectl -n aisix delete pvc data-aisix-cp-postgresql-0
kubectl -n aisix delete secret aisix-cp-secrets
```

## Configuration examples

The full list of values is in the [Parameters](#parameters) table below. Put
these in your values file and pass it with `-f`.

### External PostgreSQL

The bundled PostgreSQL is a single instance and is not a production database.
Point the control plane at one you operate instead — provision the role and
privileges it needs first, per
[External Database](https://docs.api7.ai/ai-gateway/on-premises/external-database):

```yaml
postgresql:
builtin: false

externalDatabase:
host: postgres.example.com
port: 5432
username: aisix
database: aisix_cloud
existingSecret: aisix-cp-db # key: password
sslmode: require
```

```sh
kubectl -n aisix create secret generic aisix-cp-db \
--from-literal=password='<url-safe-password>'
```

`externalDatabase.existingSecret` reads the password from a Secret you manage,
under the key `password`; `externalDatabase.password` takes it inline instead.
One of the two is required — with neither, the render fails.
The external database must be reachable before the control plane starts — only
the bundled mode gets a wait-for-database init container. The two
`postgresql.auth.*` passwords are not read in this mode, and the placeholder
rejection is skipped with them, but `secrets.masterKey` and
`secrets.betterAuthSecret` are still required.

### Private PostgreSQL CA trust for the dashboard

When external PostgreSQL uses a private CA, the dashboard's Node.js PostgreSQL
client needs to trust that CA. Otherwise database-backed authentication can fail
Expand Down Expand Up @@ -59,6 +294,98 @@ the process starts, and this chart does not automatically restart Pods when an
externally managed certificate changes. Manage the CA resource and these values
in your deployment source so subsequent GitOps syncs preserve the configuration.

### A console served from another origin

`cp-api` proxies the dashboard, so a normal install has one origin and nothing
is ever cross-origin — which is why `api.corsAllowedOrigins` is empty by
default and writes no CORS headers at all. Populate it only to let a dashboard
served from somewhere else call this API directly, such as a frontend-only
preview deployment:

```yaml
api:
publicBaseURL: https://console.example.com
corsAllowedOrigins:
- https://console.example.com
- https://*-api7ai.vercel.app
```

An entry is a bare origin (`https://host[:port]`), or `https://*` plus a host
suffix starting with `-` or `.` and naming at least three labels. It must be
spelled the way a **browser** serializes an origin: no path, query, fragment,
userinfo or trailing dot, no bare `*`, https only (loopback may use http), a
punycode host, no default port written out. The chart applies cp-api's own
rules and fails the render on anything else, so a value that installs is one
cp-api will start with. Read a wildcard suffix as collision-resistance rather
than a boundary — it is a byte suffix, and a shared preview host allocates
names first-come-first-served — so a grant that needs a real boundary belongs
on a domain you control.

### OpenShift

The chart installs under the default `restricted-v2` security context
constraint. Since 1.3.0 it pins no UID: the pod security contexts for `api`,
`dpm` and `ui` carry only `runAsNonRoot` and `seccompProfile`, so the platform
assigns the UID and GID, and the only paths written at runtime are `emptyDir`
volumes. No custom SCC, `anyuid`, or service-account change is needed.

The bundled PostgreSQL is a dependency chart and still pins its own UID, which
a parent chart cannot make conditional. Turn its two security contexts off at
install time:

```sh
helm install aisix-cp api7/aisix-cp --namespace aisix --create-namespace \
--version 1.3.0 \
-f cp-values.yaml \
--set postgresql.primary.podSecurityContext.enabled=false \
--set postgresql.primary.containerSecurityContext.enabled=false
```

Nothing else needs disabling — in particular do not turn off
`postgresql.shmVolume`, or PostgreSQL falls back to the runtime's 64Mi
`/dev/shm` and parallel queries fail with `could not resize shared memory
segment`. Using an [external database](#external-postgresql) avoids the
question entirely, and is the better production choice on any platform.

To pin a fixed UID back — on a cluster that does not assign one, or to match
existing volume ownership — set the three keys the defaults leave out. These
are the values releases before 1.3.0 pinned:

```yaml
api:
podSecurityContext:
runAsUser: 10001
runAsGroup: 101
fsGroup: 101
dpm:
podSecurityContext:
runAsUser: 10001
runAsGroup: 101
fsGroup: 101
ui:
podSecurityContext:
runAsUser: 1001
runAsGroup: 65533
fsGroup: 65533
```

### Prometheus metrics

`cp-api` serves its own operational metrics — the `aisix_cp_` family — on a
separate listener and a separate `ClusterIP` Service, never on the API port.
It is on by default; create a ServiceMonitor for it with:

```yaml
api:
metrics:
serviceMonitor:
enabled: true
labels:
release: prometheus
```

## Parameters

## Values

| Key | Type | Default | Description |
Expand Down
Loading
Loading