Add a networkService backend form to HTTPProxy - #443
Conversation
An HTTPProxy can point at an endpoint, a connector or an instance, but not at the group of members a NetworkService resolves, so a consumer still has to name each member's address in the proxy and keep the list current as capacity moves between locations. Add a fourth backend form that names a NetworkService and one of its ports. The proxy controller resolves the service's members into EndpointSlices that carry each member's location, so Envoy receives every member in the locality its interface names and prefers the nearest. A service with no members keeps an empty slice, because the rule's backendRef names it and a missing slice aborts the whole route loop for the Gateway. Backend TLS is rejected on this form, and a service is only reachable inside its own network. Key changes: - Add the networkService backend to the HTTPProxy API and validation - Resolve members into location-bearing EndpointSlices, sharded at 100 - Report partial programming when a service exceeds one shard - Pin that an empty slice still resolves into a downstream Service
|
Went through the backend union and the membership resolution. The CEL rewrite is backwards compatible (with no One question before approving. Slice count now varies with live membership, but the apply loop in Is that cleanup handled somewhere I have missed, or deferred to a later PR in the stack? If it needs doing here, listing the slices this proxy owns for the backend and deleting the ones outside the desired set would cover it, plus a test for a shard-count decrease. Two minor notes while I am here. The endpoint address family is pinned from whichever member sorts first, and members whose only address is of the other family are dropped with nothing but a log line, so a mixed-family namespace loses members silently. And |
A networkService backend's slice count follows live membership, but the apply loop only ever created or updated the slices it wanted, so a shard left behind when membership dropped back under the boundary kept its stale addresses and its controller reference and nothing removed it. The service's address family was pinned from whichever member sorted first, so a mixed-family membership lost members with nothing but a log line to show for it. And a shard overflow overwrote whatever the Programmed condition already said, hiding a gateway or connector policy that was not ready behind a less urgent reason. Delete every slice this proxy controls that is not in the desired set once the desired set has been applied, which also clears the slice a removed rule leaves behind. Pick IPv6 whenever any member can be reached over it, since the platform is IPv6-first, and name the members left behind in status rather than a log. Only let a partial result downgrade a condition that is otherwise True. Key changes: - Prune owned EndpointSlices outside the desired set after applying it - Choose the address family across all members, defaulting to IPv6 - Report members without the chosen family as partial programming - Withhold partial programming behind an earlier Programmed=False - Move the EndpointSlice apply loop into its own method
|
Thanks, all three held up. Addressed in 896a856. Stale shards. Nothing removed them. After the desired slices are applied, the controller now lists the slices in the namespace and deletes any it controls that are outside the desired set. Scoping by controller reference rather than the service label means it also catches a backend that switches away from the networkService kind, and clears the slice a removed rule leaves behind, which was the same gap. Shard zero is always desired, and shards beyond it are never named by a backend reference or given the gateway finalizer, so the prune can never delete something the route loop resolves. The reconcile-level test walks 101 members down to 50, back up past the boundary, and then off the kind entirely, asserting the shard count and that the route still names shard zero at each step. Address family. The family is now chosen across all members before any endpoint is built: IPv6 if any member can be reached over it, since the platform is IPv6-first, else IPv4. An empty service publishes IPv6 for the same reason, so a drained-and-refilled IPv6 service no longer flips the slice through the delete-and-recreate path. Members holding no address of the chosen family are named in the Programmed condition under a new reason rather than dropped into a log line. Shard overflow keeps precedence when both apply. I went with this over one slice set per family because the gateway builds one downstream service per backend reference, which would split traffic by reference weight instead of by member count. Programmed masking. Partial programming now only downgrades a condition that is currently True, and logs what it withheld. A gateway that is not yet programmed or a connector policy that is not ready keeps its own reason, and the partial reason resurfaces on the next reconcile once that clears. Merging reasons didn't fit because the reason is a single token the activity-event code keys on. One structural note: the endpoint slice apply loop moved into its own method unchanged, since adding the prune call pushed the reconcile over the complexity limit. That's the large block in the diff. |
2 participants
An HTTP proxy can point at an endpoint, a connector, or an instance, but not at the group of members a network service resolves, so a consumer still names each member's address in the proxy and keeps that list current as capacity moves between locations. This adds a fourth backend form naming a network service and one of its ports, and the proxy controller resolves the members into endpoint slices that carry each member's location so the edge receives every member in the locality its interface names. A service with no members keeps an empty slice on purpose, because the rule's backend reference names that slice and withholding it would abort the whole route loop for the gateway rather than degrade one rule. This is the fourth of six stacked pull requests splitting the network services prototype, and it builds on the network service type from the third.
Test plan