feat(api): add spec.tolerations for member Pods

README.md

@@ -24,7 +24,7 @@ The full design rationale is in [docs/concepts.md](docs/concepts.md).
 - **PodDisruptionBudget**: per-cluster PDB selects voting members only (`role=voter`); `minAvailable` is the quorum of whichever is larger, the live voter count or the intended cluster size, so `kubectl drain` cannot voluntarily push the cluster below quorum — even while node churn shrinks live membership.
 - **TLS (BYO Secrets or cert-manager)**: `spec.tls.client` / `spec.tls.peer` enable TLS on each surface independently. Material comes from either user-provided Secrets (`serverSecretRef` / `operatorClientSecretRef` / `secretRef`) or operator-emitted `cert-manager.io/v1` Certificates (`certManager.{serverIssuerRef,operatorClientIssuerRef,issuerRef}`) — mutually exclusive per subtree, enforced by CEL. mTLS is the implicit mode when an operator-client source is supplied; server-TLS-only when it isn't. The whole `tls` subtree is CEL-locked immutable post-create. cert-manager-emitted certs auto-renew via cert-manager; Pod-side rotation is a manual one-at-a-time `kubectl delete pod` either way. See [docs/concepts.md](docs/concepts.md#tls).
 - **Resource sizing**: `spec.resources` (a `corev1.ResourceRequirements`) sets the etcd container's CPU/memory requests and limits. Unset uses a conservative 100m/128Mi-request default. Updates take effect on newly-created members; pair with a `VerticalPodAutoscaler` targeting the cluster for live recommendation/rollout.
-- **Scheduling & extra metadata**: `spec.affinity` and `spec.topologySpreadConstraints` pass through to every member Pod (anti-affinity is not defaulted — set it for production); `spec.additionalMetadata` merges user labels/annotations onto every object the operator creates (member Pods, data PVCs, Services, PDB, `EtcdMember` CRs), with operator-owned keys winning on collision. All three apply on object creation and are latched like the rest of the spec. See [docs/concepts.md](docs/concepts.md#pod-scheduling-and-additional-metadata).
+- **Scheduling & extra metadata**: `spec.affinity`, `spec.topologySpreadConstraints` and `spec.tolerations` pass through to every member Pod (anti-affinity is not defaulted — set it for production); `spec.additionalMetadata` merges user labels/annotations onto every object the operator creates (member Pods, data PVCs, Services, PDB, `EtcdMember` CRs), with operator-owned keys winning on collision. All of these apply on object creation and are latched like the rest of the spec. See [docs/concepts.md](docs/concepts.md#pod-scheduling-and-additional-metadata).
 - **Zone-local clients**: the `etcd-operator.cozystack.io/client-service-traffic-distribution` annotation on an `EtcdCluster` (`PreferClose` / `PreferSameZone` / `PreferSameNode`) sets `trafficDistribution` on the `<cluster>-client` Service, so in a cluster stretched across zones clients prefer a ready member in their own zone. Removing the annotation clears the field; an invalid value is reported as a Warning Event and ignored. See [docs/concepts.md](docs/concepts.md#client-service-traffic-distribution).
 - **Monitoring / autoscaling hooks**: every member Pod always exposes a plaintext `metrics` container port at `2381` (etcd's `/health` + Prometheus `/metrics`) for `VMPodScrape` / `PodMonitor`. The `EtcdCluster` CRD exposes the `/scale` subresource with a populated `status.selector`, making it a valid target for `kubectl scale` and `VerticalPodAutoscaler.targetRef`.
 - **Locking pattern**: `status.observed` snapshots the in-flight target so mid-flight spec edits don't corrupt consensus; `progressDeadline` bounds how long the operator will spend trying to reach a target.

api/v1alpha2/etcdcluster_types.go

@@ -548,6 +548,16 @@ type EtcdClusterSpec struct {
 	// +optional
 	TopologySpreadConstraints []corev1.TopologySpreadConstraint `json:"topologySpreadConstraints,omitempty"`
 
+	// Tolerations lets member Pods schedule onto tainted nodes (for example
+	// a dedicated platform node pool). Passed straight through to each
+	// member Pod's spec.tolerations. Tolerations only permit scheduling onto
+	// tainted nodes; pair them with affinity to steer members there.
+	//
+	// Updates take effect on newly-created members (scale-up, replacement);
+	// the operator does not roll existing Pods to apply a change in place.
+	// +optional
+	Tolerations []corev1.Toleration `json:"tolerations,omitempty"`
+
 	// Options carries etcd server tuning flags (backend quota,
 	// auto-compaction, raft snapshot count) passed to each member's
 	// command line. A closed typed set — see EtcdOptions for why there
@@ -622,6 +632,11 @@ type ObservedClusterSpec struct {
 	// +optional
 	TopologySpreadConstraints []corev1.TopologySpreadConstraint `json:"topologySpreadConstraints,omitempty"`
 
+	// Tolerations is the locked target tolerations for member Pods.
+	// Latched with the rest of the target spec.
+	// +optional
+	Tolerations []corev1.Toleration `json:"tolerations,omitempty"`
+
 	// AdditionalMetadata is the locked target extra labels/annotations
 	// stamped onto objects created for this cluster. Latched with the rest
 	// of the target spec so a mid-flight metadata edit only applies to

api/v1alpha2/etcdmember_types.go

@@ -115,6 +115,12 @@ type EtcdMemberSpec struct {
 	// +optional
 	TopologySpreadConstraints []corev1.TopologySpreadConstraint `json:"topologySpreadConstraints,omitempty"`
 
+	// Tolerations mirrors EtcdCluster.spec.tolerations at the time this
+	// member was created. Passed straight to the Pod's spec.tolerations at
+	// build time.
+	// +optional
+	Tolerations []corev1.Toleration `json:"tolerations,omitempty"`
+
 	// Options mirrors EtcdCluster.spec.options at the time this member
 	// was created. The member controller renders the set fields as etcd
 	// command-line flags at Pod-build time; existing members are not

charts/etcd-operator/crd-bases/etcd-operator.cozystack.io_etcdclusters.yaml

@@ -1602,6 +1602,52 @@ spec:
                         must be set
                       rule: has(self.secretRef) != has(self.certManager)
                 type: object
+              tolerations:
+                description: |-
+                  Tolerations lets member Pods schedule onto tainted nodes (for example
+                  a dedicated platform node pool). Passed straight through to each
+                  member Pod's spec.tolerations. Tolerations only permit scheduling onto
+                  tainted nodes; pair them with affinity to steer members there.
+
+                  Updates take effect on newly-created members (scale-up, replacement);
+                  the operator does not roll existing Pods to apply a change in place.
+                items:
+                  description: |-
+                    The pod this Toleration is attached to tolerates any taint that matches
+                    the triple <key,value,effect> using the matching operator <operator>.
+                  properties:
+                    effect:
+                      description: |-
+                        Effect indicates the taint effect to match. Empty means match all taint effects.
+                        When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
+                      type: string
+                    key:
+                      description: |-
+                        Key is the taint key that the toleration applies to. Empty means match all taint keys.
+                        If the key is empty, operator must be Exists; this combination means to match all values and all keys.
+                      type: string
+                    operator:
+                      description: |-
+                        Operator represents a key's relationship to the value.
+                        Valid operators are Exists and Equal. Defaults to Equal.
+                        Exists is equivalent to wildcard for value, so that a pod can
+                        tolerate all taints of a particular category.
+                      type: string
+                    tolerationSeconds:
+                      description: |-
+                        TolerationSeconds represents the period of time the toleration (which must be
+                        of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
+                        it is not set, which means tolerate the taint forever (do not evict). Zero and
+                        negative values will be treated as 0 (evict immediately) by the system.
+                      format: int64
+                      type: integer
+                    value:
+                      description: |-
+                        Value is the taint value the toleration matches to.
+                        If the operator is Exists, the value should be empty, otherwise just a regular string.
+                      type: string
+                  type: object
+                type: array
               topologySpreadConstraints:
                 description: |-
                   TopologySpreadConstraints controls how member Pods are spread across
@@ -3083,6 +3129,47 @@ spec:
                           the field is being added from nil.
                         type: string
                     type: object
+                  tolerations:
+                    description: |-
+                      Tolerations is the locked target tolerations for member Pods.
+                      Latched with the rest of the target spec.
+                    items:
+                      description: |-
+                        The pod this Toleration is attached to tolerates any taint that matches
+                        the triple <key,value,effect> using the matching operator <operator>.
+                      properties:
+                        effect:
+                          description: |-
+                            Effect indicates the taint effect to match. Empty means match all taint effects.
+                            When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
+                          type: string
+                        key:
+                          description: |-
+                            Key is the taint key that the toleration applies to. Empty means match all taint keys.
+                            If the key is empty, operator must be Exists; this combination means to match all values and all keys.
+                          type: string
+                        operator:
+                          description: |-
+                            Operator represents a key's relationship to the value.
+                            Valid operators are Exists and Equal. Defaults to Equal.
+                            Exists is equivalent to wildcard for value, so that a pod can
+                            tolerate all taints of a particular category.
+                          type: string
+                        tolerationSeconds:
+                          description: |-
+                            TolerationSeconds represents the period of time the toleration (which must be
+                            of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
+                            it is not set, which means tolerate the taint forever (do not evict). Zero and
+                            negative values will be treated as 0 (evict immediately) by the system.
+                          format: int64
+                          type: integer
+                        value:
+                          description: |-
+                            Value is the taint value the toleration matches to.
+                            If the operator is Exists, the value should be empty, otherwise just a regular string.
+                          type: string
+                      type: object
+                    type: array
                   topologySpreadConstraints:
                     description: |-
                       TopologySpreadConstraints is the locked target spread configuration

charts/etcd-operator/crd-bases/etcd-operator.cozystack.io_etcdmembers.yaml

@@ -1389,6 +1389,48 @@ spec:
                     type: object
                     x-kubernetes-map-type: atomic
                 type: object
+              tolerations:
+                description: |-
+                  Tolerations mirrors EtcdCluster.spec.tolerations at the time this
+                  member was created. Passed straight to the Pod's spec.tolerations at
+                  build time.
+                items:
+                  description: |-
+                    The pod this Toleration is attached to tolerates any taint that matches
+                    the triple <key,value,effect> using the matching operator <operator>.
+                  properties:
+                    effect:
+                      description: |-
+                        Effect indicates the taint effect to match. Empty means match all taint effects.
+                        When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
+                      type: string
+                    key:
+                      description: |-
+                        Key is the taint key that the toleration applies to. Empty means match all taint keys.
+                        If the key is empty, operator must be Exists; this combination means to match all values and all keys.
+                      type: string
+                    operator:
+                      description: |-
+                        Operator represents a key's relationship to the value.
+                        Valid operators are Exists and Equal. Defaults to Equal.
+                        Exists is equivalent to wildcard for value, so that a pod can
+                        tolerate all taints of a particular category.
+                      type: string
+                    tolerationSeconds:
+                      description: |-
+                        TolerationSeconds represents the period of time the toleration (which must be
+                        of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
+                        it is not set, which means tolerate the taint forever (do not evict). Zero and
+                        negative values will be treated as 0 (evict immediately) by the system.
+                      format: int64
+                      type: integer
+                    value:
+                      description: |-
+                        Value is the taint value the toleration matches to.
+                        If the operator is Exists, the value should be empty, otherwise just a regular string.
+                      type: string
+                  type: object
+                type: array
               topologySpreadConstraints:
                 description: |-
                   TopologySpreadConstraints mirrors EtcdCluster.spec.topologySpreadConstraints

controllers/etcdcluster_controller.go

@@ -449,6 +449,7 @@ func (r *EtcdClusterReconciler) bootstrap(
 				AdditionalMetadata:        cluster.Status.Observed.AdditionalMetadata,
 				Affinity:                  cluster.Status.Observed.Affinity,
 				TopologySpreadConstraints: cluster.Status.Observed.TopologySpreadConstraints,
+				Tolerations:               cluster.Status.Observed.Tolerations,
 				Options:                   cluster.Status.Observed.Options,
 				ImagePullSecrets:          cluster.Status.Observed.ImagePullSecrets,
 				Bootstrap:                 true,
@@ -854,6 +855,7 @@ func (r *EtcdClusterReconciler) scaleUp(
 			AdditionalMetadata:        cluster.Status.Observed.AdditionalMetadata,
 			Affinity:                  cluster.Status.Observed.Affinity,
 			TopologySpreadConstraints: cluster.Status.Observed.TopologySpreadConstraints,
+			Tolerations:               cluster.Status.Observed.Tolerations,
 			Options:                   cluster.Status.Observed.Options,
 			ImagePullSecrets:          cluster.Status.Observed.ImagePullSecrets,
 			Bootstrap:                 false,
@@ -2138,6 +2140,7 @@ func snapshotSpecIntoObserved(cluster *lll.EtcdCluster) {
 		Resources:                 cluster.Spec.Resources,
 		Affinity:                  cluster.Spec.Affinity,
 		TopologySpreadConstraints: cluster.Spec.TopologySpreadConstraints,
+		Tolerations:               cluster.Spec.Tolerations,
 		AdditionalMetadata:        cluster.Spec.AdditionalMetadata,
 		Options:                   cluster.Spec.Options,
 		ImagePullSecrets:          cluster.Spec.ImagePullSecrets,
@@ -2160,6 +2163,7 @@ func specEqualsObserved(cluster *lll.EtcdCluster) bool {
 		equality.Semantic.DeepEqual(o.Resources, cluster.Spec.Resources) &&
 		equality.Semantic.DeepEqual(o.Affinity, cluster.Spec.Affinity) &&
 		equality.Semantic.DeepEqual(o.TopologySpreadConstraints, cluster.Spec.TopologySpreadConstraints) &&
+		equality.Semantic.DeepEqual(o.Tolerations, cluster.Spec.Tolerations) &&
 		equality.Semantic.DeepEqual(o.AdditionalMetadata, cluster.Spec.AdditionalMetadata) &&
 		equality.Semantic.DeepEqual(o.Options, cluster.Spec.Options) &&
 		equality.Semantic.DeepEqual(o.ImagePullSecrets, cluster.Spec.ImagePullSecrets)