diff --git a/docs/en/solutions/Configuring_per_node_DNS_on_ACP_clusters_with_NodeNetworkConfigurationPolicy.md b/docs/en/solutions/Configuring_per_node_DNS_on_ACP_clusters_with_NodeNetworkConfigurationPolicy.md new file mode 100644 index 000000000..55c7d35ca --- /dev/null +++ b/docs/en/solutions/Configuring_per_node_DNS_on_ACP_clusters_with_NodeNetworkConfigurationPolicy.md @@ -0,0 +1,84 @@ +--- +kind: + - How To +products: + - Alauda Container Platform +ProductsVersion: + - 4.1.0,4.2.x +--- + +# Configuring per-node DNS on ACP clusters with NodeNetworkConfigurationPolicy + +## Issue + +Cluster administrators sometimes need to change DNS settings on an individual node — upstream nameservers, search domains, or resolver options — without logging in to the host and hand-editing files. On Alauda Container Platform this can be done declaratively through the Kubernetes API with the kubernetes-nmstate operator: the operator serves the `NodeNetworkConfigurationPolicy` (NNCP) CRD at `nmstate.io/v1` (served and stored at `v1`), and a policy carrying a `dns-resolver` block rewrites the matched node's `/etc/resolv.conf` to the configured values. + +Prerequisite — NetworkManager-managed nodes only: this mechanism operates through NetworkManager, which owns and generates `/etc/resolv.conf` on the node (the file carries the header `# Generated by NetworkManager`, and NetworkManager renders the DNS values from its connection profiles, e.g. `DNS1=...` in an `ifcfg` profile). The end-to-end flow below was verified on CentOS 7.9.2009 nodes running NetworkManager 1.18.8 with nmstate 2.2.4x. Nodes whose network stack is not managed by NetworkManager (for example netplan- or systemd-networkd-based distributions) do not meet this prerequisite. No nmstate package is required on the host itself — the operator's handler image ships `nmstatectl`. + +## Resolution + +Step 1 — install the kubernetes-nmstate operator: + +The NNCP resource type is not served until the operator is installed; before installation, `kubectl explain nodenetworkconfigurationpolicy` fails with a hard error, and afterwards it resolves to `GROUP nmstate.io, VERSION v1`. The v4.1.4 operator bundle (`kubernetes-nmstate-operator`, channel `alpha`, handler image tag `kubernetes-nmstate-handler:v4.1.4`) is available from the platform package repository and ships the operator manifests together with the four `nmstate.io` CRDs; the operator, handler, and webhook run in the `nmstate` namespace. As shipped, the bundle's CSV may reference an internal build registry; repoint the images to the platform registry per the bundle's `relatedImages` list. After installation the `nmstate.io` API group serves `nodenetworkconfigurationpolicies` (short name `nncp`, `v1`), `nodenetworkconfigurationenactments` (`v1beta1`), `nodenetworkstates` (`v1beta1`), and `nmstates`, and the handler publishes a `NodeNetworkState` object for every node in the cluster. + +```bash +kubectl api-resources --api-group=nmstate.io +kubectl explain nodenetworkconfigurationpolicy +kubectl get nodenetworkstates +``` + +Step 2 — create the DNS policy: + +Set `spec.nodeSelector` to the target node's `kubernetes.io/hostname` label to scope the policy to that single node — on a three-node cluster this produced exactly one enactment object, on the matched node only. The `desiredState.dns-resolver.config` block carries three fields in the nmstate schema shipped with the handler: `server` (list of DNS server IPs), `search` (list of search domains), and `options` (resolver option strings). The option strings exercised end-to-end on this platform were `attempts:3` and `timeout:2`; other option strings are accepted at the schema level. + +```yaml +apiVersion: nmstate.io/v1 +kind: NodeNetworkConfigurationPolicy +metadata: + name: node1-dns +spec: + nodeSelector: + kubernetes.io/hostname: + desiredState: + dns-resolver: + config: + server: + - 192.168.16.19 + - 223.5.5.5 + search: + - internal.example + options: + - attempts:3 + - timeout:2 +``` + +Apply the manifest with the CLI; creation returns the standard confirmation and the policy's `spec.nodeSelector` reads back with the single matched hostname: + +```bash +kubectl create -f node1-dns.yaml +``` + +The policy reaches `Available=True` with reason `SuccessfullyConfigured` within seconds of creation, and one `NodeNetworkConfigurationEnactment` (NNCE) appears for the matched node while unmatched nodes get none. + +On the matched node, `/etc/resolv.conf` is rewritten: the `nameserver` entries become the policy's servers in policy order, the policy's search domains are written into the `search` line, and the option strings land on a single `options` line. The `# Generated by NetworkManager` header remains the first line of the rewritten file; the policy's hostname selector confined the change to the matched node — the control node checked before and after showed an unchanged `/etc/resolv.conf`, and no enactment was created for unmatched nodes — and name resolution from the node keeps working on the new configuration. + +Rollback: applying a follow-up policy that carries the node's original `dns-resolver` values rewrites `/etc/resolv.conf` back to its prior content exactly (the test configuration left no residue), and node name resolution was confirmed working after the restore. + +## Diagnostic Steps + +Check the policy and per-node enactment status; a healthy rollout shows the policy `Available=True` with reason `SuccessfullyConfigured` and one NNCE for the matched node: + +```bash +kubectl get nncp +kubectl get nnce +``` + +Confirm the result directly on the target node by reading `/etc/resolv.conf` — a read-only check performed with a node debug pod: + +```bash +kubectl debug node/ -it --image= -- chroot /host cat /etc/resolv.conf +``` + +The expected file starts with the `# Generated by NetworkManager` header; in the verified run the policy's `search` line came next, followed by the `nameserver` entries and then the `options` line. + +As a final sanity check, resolve an external hostname from the target node (for example with `getent hosts `); this succeeded on the rewritten configuration and again after the rollback.