feat: allow L4 upstream schemes (tcp/tls/udp) for stream backends - #449
feat: allow L4 upstream schemes (tcp/tls/udp) for stream backends#449AlinsRan wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (10)
💤 Files with no reviewable changes (1)
🚧 Files skipped from review as they are similar to previous changes (9)
📝 WalkthroughWalkthroughThe change expands upstream scheme validation and documentation to support ChangesL4 upstream scheme support
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant Route as ApisixRoute or TCPRoute
participant Translator as Stream translator
participant APISIX as APISIX upstream
participant Backend as TLS backend
Route->>Translator: Provide stream backend with scheme tls
Translator->>APISIX: Configure scheme and endpoint host
APISIX->>Backend: Open TLS connection
Backend-->>APISIX: Return upstream response
🚥 Pre-merge checks | ✅ 5 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
conformance test report - apisix modeapiVersion: gateway.networking.k8s.io/v1
date: "2026-07-29T08:06:44Z"
gatewayAPIChannel: experimental
gatewayAPIVersion: v1.6.0
implementation:
contact:
- https://github.com/apache/apisix-ingress-controller/issues
organization: APISIX
project: apisix-ingress-controller
url: https://github.com/apache/apisix-ingress-controller.git
version: v2.0.0
kind: ConformanceReport
mode: default
profiles:
- core:
result: partial
skippedTests:
- HTTPRouteHTTPSListener
- HTTPRouteInvalidBackendRefUnknownKind
- HTTPRouteInvalidCrossNamespaceBackendRef
- HTTPRouteInvalidNonExistentBackendRef
- HTTPRouteListenerHostnameMatching
- HTTPRouteMultipleGateways
- HTTPRouteNoBackendRefs
statistics:
Failed: 0
Passed: 30
Skipped: 7
extended:
result: partial
skippedTests:
- HTTPRouteRedirectPortAndScheme
statistics:
Failed: 0
Passed: 12
Skipped: 1
supportedFeatures:
- GatewayAddressEmpty
- GatewayPort8080
- HTTPRouteBackendProtocolWebSocket
- HTTPRouteDestinationPortMatching
- HTTPRouteHostRewrite
- HTTPRouteMethodMatching
- HTTPRoutePathRewrite
- HTTPRoutePortRedirect
- HTTPRouteQueryParamMatching
- HTTPRouteRequestMirror
- HTTPRouteResponseHeaderModification
- HTTPRouteSchemeRedirect
unsupportedFeatures:
- BackendTLSPolicy
- BackendTLSPolicySANValidation
- GatewayBackendClientCertificate
- GatewayFrontendClientCertificateValidation
- GatewayFrontendClientCertificateValidationInsecureFallback
- GatewayHTTPListenerIsolation
- GatewayHTTPSListenerDetectMisdirectedRequests
- GatewayInfrastructurePropagation
- GatewayStaticAddresses
- HTTPRoute303RedirectStatusCode
- HTTPRoute307RedirectStatusCode
- HTTPRoute308RedirectStatusCode
- HTTPRouteBackendProtocolH2C
- HTTPRouteBackendRequestHeaderModification
- HTTPRouteBackendTimeout
- HTTPRouteCORS
- HTTPRouteNamedRouteRule
- HTTPRouteParentRefPort
- HTTPRoutePathRedirect
- HTTPRouteRequestMultipleMirrors
- HTTPRouteRequestPercentageMirror
- HTTPRouteRequestTimeout
- HTTPRouteRetry
- HTTPRouteRetryBackendTimeout
- HTTPRouteRetryConnectionError
- ListenerSet
name: GATEWAY-HTTP
summary: Core tests partially succeeded with 7 test skips. Extended tests partially
succeeded with 1 test skips.
- core:
result: partial
skippedTests:
- GRPCRouteListenerHostnameMatching
statistics:
Failed: 0
Passed: 14
Skipped: 1
extended:
result: success
statistics:
Failed: 0
Passed: 1
Skipped: 0
supportedFeatures:
- GatewayAddressEmpty
- GatewayPort8080
unsupportedFeatures:
- GatewayBackendClientCertificate
- GatewayFrontendClientCertificateValidation
- GatewayFrontendClientCertificateValidationInsecureFallback
- GatewayHTTPListenerIsolation
- GatewayHTTPSListenerDetectMisdirectedRequests
- GatewayInfrastructurePropagation
- GatewayStaticAddresses
- ListenerSet
name: GATEWAY-GRPC
summary: Core tests partially succeeded with 1 test skips. Extended tests succeeded.
- core:
result: partial
skippedTests:
- TLSRouteHostnameIntersection
- TLSRouteInvalidBackendRefNonexistent
- TLSRouteInvalidBackendRefUnknownKind
- TLSRouteSimpleSameNamespace
statistics:
Failed: 0
Passed: 16
Skipped: 4
extended:
result: partial
skippedTests:
- TLSRouteTerminateSimpleSameNamespace
statistics:
Failed: 0
Passed: 3
Skipped: 1
supportedFeatures:
- GatewayAddressEmpty
- GatewayPort8080
- TLSRouteModeTerminate
unsupportedFeatures:
- GatewayBackendClientCertificate
- GatewayFrontendClientCertificateValidation
- GatewayFrontendClientCertificateValidationInsecureFallback
- GatewayHTTPListenerIsolation
- GatewayHTTPSListenerDetectMisdirectedRequests
- GatewayInfrastructurePropagation
- GatewayStaticAddresses
- ListenerSet
- TLSRouteModeMixed
name: GATEWAY-TLS
summary: Core tests partially succeeded with 4 test skips. Extended tests partially
succeeded with 1 test skips.
succeededProvisionalTests:
- GatewayOptionalAddressValue |
conformance test report - apisix-standalone modeapiVersion: gateway.networking.k8s.io/v1
date: "2026-07-29T08:07:03Z"
gatewayAPIChannel: experimental
gatewayAPIVersion: v1.6.0
implementation:
contact:
- https://github.com/apache/apisix-ingress-controller/issues
organization: APISIX
project: apisix-ingress-controller
url: https://github.com/apache/apisix-ingress-controller.git
version: v2.0.0
kind: ConformanceReport
mode: default
profiles:
- core:
result: partial
skippedTests:
- HTTPRouteHTTPSListener
- HTTPRouteInvalidBackendRefUnknownKind
- HTTPRouteInvalidCrossNamespaceBackendRef
- HTTPRouteInvalidNonExistentBackendRef
- HTTPRouteListenerHostnameMatching
- HTTPRouteMultipleGateways
- HTTPRouteNoBackendRefs
statistics:
Failed: 0
Passed: 30
Skipped: 7
extended:
result: partial
skippedTests:
- HTTPRouteRedirectPortAndScheme
statistics:
Failed: 0
Passed: 12
Skipped: 1
supportedFeatures:
- GatewayAddressEmpty
- GatewayPort8080
- HTTPRouteBackendProtocolWebSocket
- HTTPRouteDestinationPortMatching
- HTTPRouteHostRewrite
- HTTPRouteMethodMatching
- HTTPRoutePathRewrite
- HTTPRoutePortRedirect
- HTTPRouteQueryParamMatching
- HTTPRouteRequestMirror
- HTTPRouteResponseHeaderModification
- HTTPRouteSchemeRedirect
unsupportedFeatures:
- BackendTLSPolicy
- BackendTLSPolicySANValidation
- GatewayBackendClientCertificate
- GatewayFrontendClientCertificateValidation
- GatewayFrontendClientCertificateValidationInsecureFallback
- GatewayHTTPListenerIsolation
- GatewayHTTPSListenerDetectMisdirectedRequests
- GatewayInfrastructurePropagation
- GatewayStaticAddresses
- HTTPRoute303RedirectStatusCode
- HTTPRoute307RedirectStatusCode
- HTTPRoute308RedirectStatusCode
- HTTPRouteBackendProtocolH2C
- HTTPRouteBackendRequestHeaderModification
- HTTPRouteBackendTimeout
- HTTPRouteCORS
- HTTPRouteNamedRouteRule
- HTTPRouteParentRefPort
- HTTPRoutePathRedirect
- HTTPRouteRequestMultipleMirrors
- HTTPRouteRequestPercentageMirror
- HTTPRouteRequestTimeout
- HTTPRouteRetry
- HTTPRouteRetryBackendTimeout
- HTTPRouteRetryConnectionError
- ListenerSet
name: GATEWAY-HTTP
summary: Core tests partially succeeded with 7 test skips. Extended tests partially
succeeded with 1 test skips.
- core:
result: partial
skippedTests:
- GRPCRouteListenerHostnameMatching
statistics:
Failed: 0
Passed: 14
Skipped: 1
extended:
result: success
statistics:
Failed: 0
Passed: 1
Skipped: 0
supportedFeatures:
- GatewayAddressEmpty
- GatewayPort8080
unsupportedFeatures:
- GatewayBackendClientCertificate
- GatewayFrontendClientCertificateValidation
- GatewayFrontendClientCertificateValidationInsecureFallback
- GatewayHTTPListenerIsolation
- GatewayHTTPSListenerDetectMisdirectedRequests
- GatewayInfrastructurePropagation
- GatewayStaticAddresses
- ListenerSet
name: GATEWAY-GRPC
summary: Core tests partially succeeded with 1 test skips. Extended tests succeeded.
- core:
result: partial
skippedTests:
- TLSRouteHostnameIntersection
- TLSRouteInvalidBackendRefNonexistent
- TLSRouteInvalidBackendRefUnknownKind
- TLSRouteSimpleSameNamespace
statistics:
Failed: 0
Passed: 16
Skipped: 4
extended:
result: partial
skippedTests:
- TLSRouteTerminateSimpleSameNamespace
statistics:
Failed: 0
Passed: 3
Skipped: 1
supportedFeatures:
- GatewayAddressEmpty
- GatewayPort8080
- TLSRouteModeTerminate
unsupportedFeatures:
- GatewayBackendClientCertificate
- GatewayFrontendClientCertificateValidation
- GatewayFrontendClientCertificateValidationInsecureFallback
- GatewayHTTPListenerIsolation
- GatewayHTTPSListenerDetectMisdirectedRequests
- GatewayInfrastructurePropagation
- GatewayStaticAddresses
- ListenerSet
- TLSRouteModeMixed
name: GATEWAY-TLS
summary: Core tests partially succeeded with 4 test skips. Extended tests partially
succeeded with 1 test skips.
succeededProvisionalTests:
- GatewayOptionalAddressValue |
conformance test reportapiVersion: gateway.networking.k8s.io/v1
date: "2026-07-29T08:25:12Z"
gatewayAPIChannel: experimental
gatewayAPIVersion: v1.6.0
implementation:
contact:
- https://github.com/apache/apisix-ingress-controller/issues
organization: APISIX
project: apisix-ingress-controller
url: https://github.com/apache/apisix-ingress-controller.git
version: v2.0.0
kind: ConformanceReport
mode: default
profiles:
- core:
failedTests:
- GatewayModifyListeners
result: failure
statistics:
Failed: 1
Passed: 14
Skipped: 0
extended:
result: success
statistics:
Failed: 0
Passed: 1
Skipped: 0
supportedFeatures:
- GatewayAddressEmpty
- GatewayPort8080
unsupportedFeatures:
- GatewayBackendClientCertificate
- GatewayFrontendClientCertificateValidation
- GatewayFrontendClientCertificateValidationInsecureFallback
- GatewayHTTPListenerIsolation
- GatewayHTTPSListenerDetectMisdirectedRequests
- GatewayInfrastructurePropagation
- GatewayStaticAddresses
- ListenerSet
name: GATEWAY-GRPC
summary: Core tests failed with 1 test failures. Extended tests succeeded.
- core:
failedTests:
- GatewayModifyListeners
- TLSRouteHostnameIntersection
- TLSRouteInvalidBackendRefNonexistent
- TLSRouteInvalidBackendRefUnknownKind
- TLSRouteSimpleSameNamespace
result: failure
statistics:
Failed: 5
Passed: 15
Skipped: 0
extended:
failedTests:
- TLSRouteTerminateSimpleSameNamespace
result: failure
statistics:
Failed: 1
Passed: 3
Skipped: 0
supportedFeatures:
- GatewayAddressEmpty
- GatewayPort8080
- TLSRouteModeTerminate
unsupportedFeatures:
- GatewayBackendClientCertificate
- GatewayFrontendClientCertificateValidation
- GatewayFrontendClientCertificateValidationInsecureFallback
- GatewayHTTPListenerIsolation
- GatewayHTTPSListenerDetectMisdirectedRequests
- GatewayInfrastructurePropagation
- GatewayStaticAddresses
- ListenerSet
- TLSRouteModeMixed
name: GATEWAY-TLS
summary: Core tests failed with 5 test failures. Extended tests failed with 1 test
failures.
- core:
failedTests:
- GatewayModifyListeners
- HTTPRouteMultipleGateways
- HTTPRouteNoBackendRefs
result: failure
skippedTests:
- HTTPRouteHTTPSListener
statistics:
Failed: 3
Passed: 33
Skipped: 1
extended:
result: partial
skippedTests:
- HTTPRouteRedirectPortAndScheme
statistics:
Failed: 0
Passed: 12
Skipped: 1
supportedFeatures:
- GatewayAddressEmpty
- GatewayPort8080
- HTTPRouteBackendProtocolWebSocket
- HTTPRouteDestinationPortMatching
- HTTPRouteHostRewrite
- HTTPRouteMethodMatching
- HTTPRoutePathRewrite
- HTTPRoutePortRedirect
- HTTPRouteQueryParamMatching
- HTTPRouteRequestMirror
- HTTPRouteResponseHeaderModification
- HTTPRouteSchemeRedirect
unsupportedFeatures:
- BackendTLSPolicy
- BackendTLSPolicySANValidation
- GatewayBackendClientCertificate
- GatewayFrontendClientCertificateValidation
- GatewayFrontendClientCertificateValidationInsecureFallback
- GatewayHTTPListenerIsolation
- GatewayHTTPSListenerDetectMisdirectedRequests
- GatewayInfrastructurePropagation
- GatewayStaticAddresses
- HTTPRoute303RedirectStatusCode
- HTTPRoute307RedirectStatusCode
- HTTPRoute308RedirectStatusCode
- HTTPRouteBackendProtocolH2C
- HTTPRouteBackendRequestHeaderModification
- HTTPRouteBackendTimeout
- HTTPRouteCORS
- HTTPRouteNamedRouteRule
- HTTPRouteParentRefPort
- HTTPRoutePathRedirect
- HTTPRouteRequestMultipleMirrors
- HTTPRouteRequestPercentageMirror
- HTTPRouteRequestTimeout
- HTTPRouteRetry
- HTTPRouteRetryBackendTimeout
- HTTPRouteRetryConnectionError
- ListenerSet
name: GATEWAY-HTTP
summary: Core tests failed with 3 test failures. Extended tests partially succeeded
with 1 test skips.
succeededProvisionalTests:
- GatewayOptionalAddressValue |
APISIX accepts `tcp`, `tls` and `udp` as upstream schemes for L4 proxying, but both `ApisixUpstream.spec.scheme` and `BackendTrafficPolicy.spec.scheme` restricted the value to the L7 set, so an upstream that requires TLS from APISIX (plain TCP in, TLS out) could only be configured through the Admin API. The translation path already carries the scheme to the generated upstream for both ApisixRoute stream backends and Gateway API TCPRoute backends, so only the CRD validation needed to be relaxed. Sync of apache/apisix-ingress-controller#2830 (fixes apache/apisix-ingress-controller#2803).
075d13c to
6e34fbf
Compare
| // For L4 proxy, it can be `tcp`, `tls`, or `udp`. | ||
| // The L4 values apply to stream routes only; using them for an HTTP route | ||
| // makes the upstream unreachable. | ||
| // +kubebuilder:validation:Enum=http;https;grpc;grpcs;tcp;tls;udp; |
There was a problem hiding this comment.
[P1] Update the Helm-bundled CRDs with this enum
The CRDs bundled in api7/api7-helm-chart still allow only http, https, grpc, and grpcs for both ApisixUpstream and BackendTrafficPolicy. A standard Helm installation will therefore have the API server reject tcp/tls/udp before this controller sees the resource. Please provide the paired Helm chart update and release dependency with this CRD change.
| if framework.ProviderType == framework.ProviderTypeAPI7EE { | ||
| // The API7 EE Admin API only accepts tcp/udp as the upstream scheme | ||
| // of a stream service. | ||
| Skip("scheme: tls is not supported by the API7 EE provider") |
There was a problem hiding this comment.
[P1] Keep the API7EE provider path working
The CRD now accepts scheme: tls, but the API7EE Control Plane currently permits only tcp and udp for stream-service upstreams. The controller therefore accepts this Kubernetes object and then fails to sync it through the API7EE provider; skipping the only API7EE end-to-end path hides that broken behavior. Please land the corresponding Control Plane support and run this case without the skip, or keep tls unavailable in this repository until that provider supports it.
There was a problem hiding this comment.
Landing the Control Plane support: api7/api7ee-3-control-plane#2869.
The restriction was not a data plane limitation — the APISIX schema bundled in the Dashboard already allows tcp/tls/udp. Two Control Plane validations narrowed the stream set to tcp/udp: the StreamScheme enum in the OpenAPI schema (upstream embedded in a stream service) and validateUpstreamScheme() (standalone upstream API, where the body is oneOf(HTTPUpstream, StreamUpstream) so the spec cannot tell the service types apart). Both accept tls now.
I will drop the Skip() here and run the API7EE case once #2869 is merged and a CP image with it is available.
Description
Sync of apache/apisix-ingress-controller#2830 (fixes apache/apisix-ingress-controller#2803).
APISIX accepts
tcp,tlsandudpas upstream schemes for L4 proxying, andscheme: tlsis what makes the gateway establish the TLS session with the upstream for a stream route (plain TCP in, TLS out).Both
ApisixUpstream.spec.schemeandBackendTrafficPolicy.spec.schemerestricted the value to the L7 set (http|https|grpc|grpcs), so this could only be configured through the Admin API.The translation path already carries the scheme into the generated upstream for both entry points:
ApisixRoutestream backends merge theApisixUpstreamnamed after the backend Service;TCPRoutebackends take the scheme fromBackendTrafficPolicy.So only CRD validation needed relaxing.
Tests
TestTranslateApisixRouteStreamUpstreamScheme,TestTranslateTCPRouteUpstreamSchemeTest ApisixRoute With StreamRoute / TCP Proxy with TLS upstreamandTCPRoute E2E Test / TCPRoute With BackendTrafficPolicy— both send plain HTTP over a TCP listener to nginx's TLS port and assert200+X-Port: 443, which only passes if the gateway performs the TLS handshake with the upstream.Known follow-ups (not in this PR)
BackendTrafficPolicy.spec.schemekeeps+kubebuilder:default=http, so a policy attached only for load balancing still writesscheme: httponto an L4 upstream.SchemeToPorthas no entry fortls, so anexternalNodes/ExternalName upstream without an explicit port defaults to80.portLevelSettingsresets the top-levelschemefor ports that do not set it.Summary by CodeRabbit
tcp,tls, andudp.schemevalues for relevant CRDs so L4 can be used with stream-based routing.schemefield descriptions to clearly distinguish L7 (http,https,grpc,grpcs) vs L4 (tcp,tls,udp) behavior and limitations for HTTP routes.