Skip to content

WIP: Application load balancer controller#3

Open
fischerman wants to merge 107 commits into
mainfrom
albc
Open

WIP: Application load balancer controller#3
fischerman wants to merge 107 commits into
mainfrom
albc

Conversation

@fischerman

@fischerman fischerman commented Jun 24, 2026

Copy link
Copy Markdown
Member

How to categorize this PR?

/kind enhancement

What this PR does / why we need it:

Implements the first alpha version of the STACKIT ALB controller.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Breaking changes:

@ske-prow ske-prow Bot added do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 24, 2026
@nschad nschad changed the title Draft: Application load balancer controller WIP: Application load balancer controller Jun 25, 2026
@ske-prow ske-prow Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 25, 2026
@nschad

nschad commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

@fischerman either mark the PR as "Draft" or use "WIP" as the title. The PR will then automatically marked as WIP/Draft

@ske-prow

ske-prow Bot commented Jun 25, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign dergeberl for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ske-prow ske-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 25, 2026
@ske-prow ske-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 25, 2026
Comment thread .golangci.yaml
Comment thread pkg/controller/ingress/setup.go Outdated

@dergeberl dergeberl left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my first batch of review comments. Only had a look into the /cmd/*, /pkg/controller/* (except the tests) and the *.md files, yet. I will continue with my review for the rest.

Maybe some comments are already resolved as you also pushed during my review (I did not crosscheck again, before submitting the comments now).

Comment thread docs/user.md Outdated
Comment thread docs/user.md Outdated
Comment thread docs/user.md Outdated
Comment thread docs/user.md
Comment thread cmd/application-load-balancer-controller/main.go Outdated
Comment thread docs/user.md
### Limits

The following limitations are imposed directly by the STACKIT ALB API (not the controller itself):
- Maximum targets per pool: An individual target pool can contain a maximum of 250 targets.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit should be 1000 or?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Limit is still 250. SKE will look into limiting the auto scaler to 250 for clusters the have the ALB extension enabled.

Comment thread docs/user.md Outdated
Comment thread docs/user.md Outdated
Comment thread docs/user.md
Comment thread README.md

@dergeberl dergeberl left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments for the stackit package and testutil

Comment thread pkg/stackit/config/config.go Outdated
Comment thread pkg/controller/ingress/ingressclass_controller.go Outdated
Comment thread pkg/stackit/applicationloadbalancer.go Outdated
Comment thread pkg/stackit/applicationloadbalancer.go Outdated
Comment thread pkg/stackit/error.go
Comment thread pkg/stackit/suite_test.go Outdated
Comment thread pkg/stackit/error.go
Comment thread pkg/testutil/service/service.go Outdated
Comment thread pkg/testutil/ingress/ingress.go Outdated
Comment thread pkg/controller/ingress/spec/worktree.go Outdated
Comment thread cmd/application-load-balancer-controller/main.go Outdated
Comment thread pkg/controller/ingress/ingressclass_controller.go Outdated
Comment thread pkg/controller/ingress/ingressclass_controller.go
Comment thread pkg/controller/ingress/spec/worktree.go Outdated
Comment thread pkg/controller/ingress/spec/worktree.go Outdated
Comment thread pkg/controller/ingress/spec/worktree.go Outdated
Comment thread pkg/controller/ingress/spec/worktree.go
Comment thread pkg/controller/ingress/spec/worktree.go Outdated
Comment thread pkg/controller/ingress/spec/worktree.go
Comment thread pkg/controller/ingress/update.go Outdated
Comment thread pkg/controller/ingress/update.go Outdated
Comment thread pkg/controller/ingress/update.go Outdated
Comment thread pkg/controller/ingress/update.go Outdated
Comment thread pkg/controller/ingress/ingressclass_controller.go
Comment thread pkg/controller/ingress/suite_test.go
Comment thread pkg/controller/ingress/spec/worktree.go
CertificateIds: []string{},
},
}
for fingerprint, cert := range t.certificates {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Map interactions are not deterministic. If I remember correctly the order in the ALB matters, as always the first match will be used. This could lead in a flapping which cert is used if multiple are fitting.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5c25864

The order is stable but cannot be influenced by the customer.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Document behavior when two certificates collide.

Comment thread pkg/controller/ingress/ingressclass_controller_test.go
Comment thread pkg/controller/ingress/update_test.go Outdated
Comment thread pkg/controller/ingress/update_test.go Outdated
Comment thread pkg/controller/ingress/update_test.go Outdated
@@ -0,0 +1,302 @@
package ingress

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the update.go are multiple functions which also should be tested, like getServicesForIngresses, getTLSSecretsFromIngresses, getCertificatesForIngressClass. But also the logic for duplicateCerts should get tested.

Comment thread pkg/controller/ingress/update_test.go Outdated
Comment thread pkg/controller/ingress/spec/worktree_test.go Outdated
Comment thread pkg/controller/ingress/spec/worktree_test.go
Comment thread pkg/controller/ingress/spec/worktree.go Outdated

tree := &WorkTreeALB{
ingressClass: ingressClass,
planID: GetAnnotation(AnnotationPlanID, "", ingressClass),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we validate that we only get working PlanIDs. For NLBs we check that in cloud-provider-stackit

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ties together with https://github.com/stackitcloud/application-load-balancer-controller/pull/3/changes#r3511950744

However, this does not only break an ingress but the entire ALB. Because we introduce the webhook I would just return an error here and not process any updates.

@@ -0,0 +1,658 @@
package spec

@dergeberl dergeberl Jun 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also add tests for:

  • the AnnotationHTTPSOnly and that it not serves HTTP.
  • the sorting based on AnnotationPriority and creation timestamp.
  • AnnotationPlanID
  • the handling when setting not allowed values in annotation (like no valid int/port, not valid cidr in source range)
  • GetUnusedCertificates

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added tests for 1, 2 and 5 in c9d5229

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated comment

Comment thread deploy/deployment.yaml
@@ -0,0 +1,85 @@
apiVersion: apps/v1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should align this with the deploy of CSI and CCM in cloud-provider-stackit (which are also not perfect, see stackitcloud/cloud-provider-stackit#1162).
For example:

  • we can use also the kube-system namespace as this is also a system component like CSI and CCM
  • the cloud config don't need to be a secret
  • the mount path of the service account and cloud config
  • the apiEndpoints don't need to be in the example (this is only internally needed)
  • lets also create a /deploy folder instead if the /samples/deploy/
  • we need to find a image tag which is working and don't need to be updated like latest (not sure if this will be tagged on release)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With #6 we get a latest tag for tagged versions. Lets use that in the deployment.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 5248047

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vote for latest tag here. We can leave it like it is for now and solve this directly before merging. After merge I will create a release which then also creates the latest tag

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will keep the thread unresolved as a reminder.

Comment thread pkg/controller/ingress/ingressclass_controller.go Outdated
Comment thread pkg/controller/ingress/spec/worktree.go
Comment thread pkg/controller/ingress/spec/worktree.go
Comment thread cmd/application-load-balancer-controller/main.go
Comment thread cmd/application-load-balancer-controller/main.go
Comment thread hack/tools.mk Outdated
Comment thread docs/user.md
Comment on lines +55 to +70
for {
req := cl.client.DefaultAPI.ListCertificates(ctx, projectID, region)
if nextPage != "" {
req = req.PageId(nextPage)
}
page, err := req.Execute()
if err != nil {
return nil, err
}
certs = append(certs, page.Items...)
if ptr.Deref(page.NextPageId, "") != "" {
nextPage = *page.NextPageId
} else {
break
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a context with timeout here and also check if the context is closed in the for loop. The SDK itself should have a timeout in the HTTP client which would error the call if something is stuck. If the SKD returns no errors but always a NextPageID we would stuck in that loop forever.

Nit:
I would prefer turn around the check and avoid the else. Like this:

        nextPage = ptr.Deref(page.NextPageId, "")
		if nextPage == "" {
            break
		} 
		}

@fischerman fischerman Jul 10, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a context with timeout here...

Why? As you said yourself: "The SDK itself should have a timeout in the HTTP client which would error the call if something is stuck"

and also check if the context is closed in the for loop.

I pass the context into every request. So that is covered.

If the SKD returns no errors but always a NextPageID we would stuck in that loop forever.

That would require a severe bug in the API or SDK. To protect against the worst I've added a hard limit on the number of pages in cfe62b2. But that should never happen.

I would prefer turn around the check and avoid the else. Like this:

Also done in cfe62b2

Comment thread pkg/controller/ingress/diff/diff.go Outdated
Comment thread pkg/controller/ingress/spec/worktree.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/enhancement Enhancement, improvement, extension size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants