Skip to content

Revert #476 "SPLAT-2713: Update AWS CCM e2e test module version" - #491

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
redhat-chai-bot:revert-pr-476
Jul 15, 2026
Merged

Revert #476 "SPLAT-2713: Update AWS CCM e2e test module version"#491
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
redhat-chai-bot:revert-pr-476

Conversation

@redhat-chai-bot

@redhat-chai-bot redhat-chai-bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

This reverts merge commit d9a0c00 (PR #476).

Reason

PR #476 bumped the k8s.io/cloud-provider-aws/tests/e2e module, introducing two NLB security group transition tests that consistently time out (0/9 pass rate), causing 5.0 CI payload 5.0.0-0.ci-2026-07-15-075942 to be rejected.

Failing tests

  • [cloud-provider-aws-e2e] loadbalancer NLB should transition from BYO SG to Managed Security Group — context deadline exceeded (loadbalancer.go:625)
  • [cloud-provider-aws-e2e] loadbalancer NLB should transition from Managed SG to BYO Security Group — Timed out after 120.001s (loadbalancer.go:523)

The tests expect AWS CCM to reconcile NLB security groups within 120s, but reconciliation does not complete in time. The failure appears related to the upgrade path from 4.22 → 5.0, where installer-created IAM permissions from 4.22 do not include the new permissions required by the 5.0 CCM feature.

References


@neisw requested in Slack thread

Summary by CodeRabbit

  • Bug Fixes

    • Improved AWS region detection across supported AWS partitions.
    • Added safer handling for missing or invalid region configuration.
    • Simplified region selection using configured environment values.
  • Tests

    • Updated AWS test tooling for more reliable regional configuration.
    • Improved validation of platform behavior when no ConfigMaps should be created.
  • Chores

    • Updated the AWS testing dependency to a newer pinned version.

…pdate-cccmo-tests"

This reverts commit d9a0c00, reversing
changes made to a1fd447.
@openshift-ci-robot

openshift-ci-robot commented Jul 15, 2026

Copy link
Copy Markdown

@redhat-chai-bot: This pull request references SPLAT-2713 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

This reverts merge commit d9a0c00 (PR #476).

Reason

PR #476 bumped the k8s.io/cloud-provider-aws/tests/e2e module, introducing two NLB security group transition tests that consistently time out (0/9 pass rate), causing 5.0 CI payload 5.0.0-0.ci-2026-07-15-075942 to be rejected.

Failing tests

  • [cloud-provider-aws-e2e] loadbalancer NLB should transition from BYO SG to Managed Security Group — context deadline exceeded (loadbalancer.go:625)
  • [cloud-provider-aws-e2e] loadbalancer NLB should transition from Managed SG to BYO Security Group — Timed out after 120.001s (loadbalancer.go:523)

The tests expect AWS CCM to reconcile NLB security groups within 120s, but reconciliation does not complete in time. The failure appears related to the upgrade path from 4.22 → 5.0, where installer-created IAM permissions from 4.22 do not include the new permissions required by the 5.0 CCM feature.

References


@neisw requested in Slack thread

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 15, 2026
@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Walkthrough

AWS e2e tests now resolve regions from environment variables or the cluster Infrastructure resource, with local partition-aware validation. The BareMetal controller test uses a direct ConfigMap assertion instead of polling, and the AWS e2e dependency is updated.

Changes

AWS region resolution

Layer / File(s) Summary
Environment-driven region setup
openshift-tests/ccm-aws-tests/main.go
Region discovery checks LEASED_RESOURCE, AWS_REGION, and AWS_DEFAULT_REGION in order, while kubeconfig assignment occurs later during initialization.
Infrastructure region loading and validation
openshift-tests/ccm-aws-tests/e2e/aws/helper.go, openshift-tests/ccm-aws-tests/e2e/common/helper.go, openshift-tests/ccm-aws-tests/go.mod
AWS configuration reads the Infrastructure region, validates fallback values with a partition-aware regex, removes shared region helpers, and updates the e2e dependency pin.

BareMetal ConfigMap assertion

Layer / File(s) Summary
Direct ConfigMap assertion
pkg/controllers/cloud_config_sync_controller_test.go
The test lists managed-namespace ConfigMaps directly and asserts that none exist without Eventually polling.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant TestEnvironment
  participant getRegionFromEnv
  participant loadAWSConfig
  participant getRegionFromInfrastructure
  participant InfrastructureAPI
  TestEnvironment->>getRegionFromEnv: Read region environment variables
  getRegionFromEnv->>loadAWSConfig: Set AWS_REGION from first non-empty value
  loadAWSConfig->>getRegionFromInfrastructure: Request cluster AWS region
  getRegionFromInfrastructure->>InfrastructureAPI: Fetch Infrastructure resource
  InfrastructureAPI-->>getRegionFromInfrastructure: Return AWS region
  getRegionFromInfrastructure-->>loadAWSConfig: Return region or error
  loadAWSConfig->>loadAWSConfig: Validate fallback region
Loading

Possibly related PRs

Suggested reviewers: mfbonfigli, mtulio

🚥 Pre-merge checks | ✅ 13 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Test Structure And Quality ⚠️ Warning BareMetal test now uses unbounded cl.List(ctx, ...) on context.Background(), replacing the prior timed Eventually poll. Restore a bounded timeout for the cluster read (e.g. context.WithTimeout or Eventually(..., timeout)) and prefer a message-bearing assertion.
Ipv6 And Disconnected Network Test Compatibility ⚠️ Warning FAIL: the vendored AWS loadbalancer e2e suite builds URLs with fmt.Sprintf("http://%s:%d/...", host, port) and calls external AWS APIs. Use net.JoinHostPort for IPv6-safe URLs, and skip or adapt tests that require external AWS connectivity in disconnected jobs.
✅ Passed checks (13 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes that this PR reverts the AWS CCM e2e test module update from #476.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed No changed Ginkgo titles contain dynamic values; the touched It/Describe/Context strings are static and descriptive.
Microshift Test Compatibility ✅ Passed Added loadbalancer cases use only Kubernetes Services/Deployments and AWS SDK; no MicroShift-unsupported OpenShift APIs, tags, or guards were implicated.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PASS: The PR removes the NLB SG-transition e2e cases; remaining LB tests target a single node or current worker-node count, with no required multi-node/HA assumption.
Topology-Aware Scheduling Compatibility ✅ Passed No new topology-sensitive scheduling logic was introduced; the touched files only change AWS config/test plumbing, and existing worker-node/hostname affinity in vendored tests predates this PR.
Ote Binary Stdout Contract ✅ Passed No process-level stdout writes were added: main.go only uses logrus (stderr by default) and framework.Logf writes to GinkgoWriter, with no fmt.Print/Stdout/klog setup added.
No-Weak-Crypto ✅ Passed Changed files only adjust AWS region/config and a controller test; searches found no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret compares.
Container-Privileges ✅ Passed PASS: The PR diff only changes Go test/helper code; no changed manifest or container spec contains privileged/hostPID/hostNetwork/hostIPC/SYS_ADMIN/allowPrivilegeEscalation settings.
No-Sensitive-Data-In-Logs ✅ Passed PASS: the new/changed logs only print AWS region and test env values; no secrets, PII, hostnames, or tokens are logged.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from nrb and theobarberbany July 15, 2026 15:15
@mdbooth

mdbooth commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 15, 2026
@openshift-ci

openshift-ci Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mdbooth

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 15, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openshift-tests/ccm-aws-tests/e2e/aws/helper.go`:
- Around line 45-50: Update the getRegionFromInfrastructure fallback handling so
a failure is not silently ignored when cfg.Region already matches
awsRegionPattern. Log the retrieval error before continuing with the valid
configured region, while preserving the existing invalid-region error return and
successful region assignment behavior.

In `@pkg/controllers/cloud_config_sync_controller_test.go`:
- Around line 607-609: Update the ConfigMap listing assertion in the test to use
a child context with an explicit timeout instead of the unbounded suite-level
context. Create the timeout context immediately before cl.List, pass it to that
call, and ensure the context is released after the operation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4bc38063-d675-4715-b21d-dbec075b02bc

📥 Commits

Reviewing files that changed from the base of the PR and between d9a0c00 and cfb95ec.

⛔ Files ignored due to path filters (4)
  • openshift-tests/ccm-aws-tests/go.sum is excluded by !**/*.sum
  • openshift-tests/ccm-aws-tests/vendor/k8s.io/cloud-provider-aws/tests/e2e/cloudconfig.go is excluded by !**/vendor/**
  • openshift-tests/ccm-aws-tests/vendor/k8s.io/cloud-provider-aws/tests/e2e/loadbalancer.go is excluded by !**/vendor/**
  • openshift-tests/ccm-aws-tests/vendor/modules.txt is excluded by !**/vendor/**
📒 Files selected for processing (5)
  • openshift-tests/ccm-aws-tests/e2e/aws/helper.go
  • openshift-tests/ccm-aws-tests/e2e/common/helper.go
  • openshift-tests/ccm-aws-tests/go.mod
  • openshift-tests/ccm-aws-tests/main.go
  • pkg/controllers/cloud_config_sync_controller_test.go
💤 Files with no reviewable changes (1)
  • openshift-tests/ccm-aws-tests/e2e/common/helper.go

Comment on lines +45 to 50
region, err := getRegionFromInfrastructure(ctx)
if err == nil {
cfg.Region = region
} else if !common.AWSRegionPattern.MatchString(cfg.Region) {
} else if !awsRegionPattern.MatchString(cfg.Region) {
return aws.Config{}, fmt.Errorf("AWS region %q is not valid and failed to get region from Infrastructure: %w", cfg.Region, err)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Do not ignore the error when falling back to a valid region.

If getRegionFromInfrastructure fails, but the fallback cfg.Region is valid according to the regex, the error returned from getRegionFromInfrastructure is silently swallowed. As per path instructions, "Never ignore error returns".

If the error is acceptable in this fallback scenario, it should at least be logged so that underlying API or RBAC failures are not completely hidden.

💻 Proposed fix
 	region, err := getRegionFromInfrastructure(ctx)
 	if err == nil {
 		cfg.Region = region
-	} else if !awsRegionPattern.MatchString(cfg.Region) {
-		return aws.Config{}, fmt.Errorf("AWS region %q is not valid and failed to get region from Infrastructure: %w", cfg.Region, err)
+	} else {
+		if !awsRegionPattern.MatchString(cfg.Region) {
+			return aws.Config{}, fmt.Errorf("AWS region %q is not valid and failed to get region from Infrastructure: %w", cfg.Region, err)
+		}
+		framework.Logf("Failed to get region from Infrastructure (falling back to %q): %v", cfg.Region, err)
 	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
region, err := getRegionFromInfrastructure(ctx)
if err == nil {
cfg.Region = region
} else if !common.AWSRegionPattern.MatchString(cfg.Region) {
} else if !awsRegionPattern.MatchString(cfg.Region) {
return aws.Config{}, fmt.Errorf("AWS region %q is not valid and failed to get region from Infrastructure: %w", cfg.Region, err)
}
region, err := getRegionFromInfrastructure(ctx)
if err == nil {
cfg.Region = region
} else {
if !awsRegionPattern.MatchString(cfg.Region) {
return aws.Config{}, fmt.Errorf("AWS region %q is not valid and failed to get region from Infrastructure: %w", cfg.Region, err)
}
framework.Logf("Failed to get region from Infrastructure (falling back to %q): %v", cfg.Region, err)
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openshift-tests/ccm-aws-tests/e2e/aws/helper.go` around lines 45 - 50, Update
the getRegionFromInfrastructure fallback handling so a failure is not silently
ignored when cfg.Region already matches awsRegionPattern. Log the retrieval
error before continuing with the valid configured region, while preserving the
existing invalid-region error return and successful region assignment behavior.

Source: Path instructions

Comment on lines +607 to +609
allCMs := &corev1.ConfigMapList{}
Expect(cl.List(ctx, allCMs, &client.ListOptions{Namespace: targetNamespaceName})).To(Succeed())
Expect(len(allCMs.Items)).To(BeZero())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Bound the ConfigMap list call.

cl.List uses the suite-level context.Background() without a deadline, so the test can hang indefinitely if the API server is unavailable. Create a child context with an explicit timeout for this cluster operation.

As per path instructions, operations interacting with clusters must include timeouts.

Proposed fix
 			allCMs := &corev1.ConfigMapList{}
-			Expect(cl.List(ctx, allCMs, &client.ListOptions{Namespace: targetNamespaceName})).To(BeNil())
+			listCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
+			defer cancel()
+			Expect(cl.List(listCtx, allCMs, &client.ListOptions{Namespace: targetNamespaceName})).To(Succeed())
 			Expect(len(allCMs.Items)).To(BeZero())
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
allCMs := &corev1.ConfigMapList{}
Expect(cl.List(ctx, allCMs, &client.ListOptions{Namespace: targetNamespaceName})).To(Succeed())
Expect(len(allCMs.Items)).To(BeZero())
allCMs := &corev1.ConfigMapList{}
listCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
Expect(cl.List(listCtx, allCMs, &client.ListOptions{Namespace: targetNamespaceName})).To(Succeed())
Expect(len(allCMs.Items)).To(BeZero())
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/controllers/cloud_config_sync_controller_test.go` around lines 607 - 609,
Update the ConfigMap listing assertion in the test to use a child context with
an explicit timeout instead of the unbounded suite-level context. Create the
timeout context immediately before cl.List, pass it to that call, and ensure the
context is released after the operation.

Source: Path instructions

@neisw

neisw commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

/override ci/prow/e2e-aws-ovn
/override ci/prow/e2e-aws-ovn-upgrade
/override ci/prow/e2e-azure-ovn-upgrade
/override ci/prow/level0-clusterinfra-azure-ipi-proxy-tests

@openshift-ci

openshift-ci Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

@neisw: Overrode contexts on behalf of neisw: ci/prow/e2e-aws-ovn, ci/prow/e2e-aws-ovn-upgrade, ci/prow/e2e-azure-ovn-upgrade, ci/prow/level0-clusterinfra-azure-ipi-proxy-tests

Details

In response to this:

/override ci/prow/e2e-aws-ovn
/override ci/prow/e2e-aws-ovn-upgrade
/override ci/prow/e2e-azure-ovn-upgrade
/override ci/prow/level0-clusterinfra-azure-ipi-proxy-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@neisw

neisw commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

/label acknowledge-critical-fixes-only
/verified by ci

@openshift-ci openshift-ci Bot added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label Jul 15, 2026
@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jul 15, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@neisw: This PR has been marked as verified by ci.

Details

In response to this:

/label acknowledge-critical-fixes-only
/verified by ci

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-merge-bot
openshift-merge-bot Bot merged commit 4aa859c into openshift:main Jul 15, 2026
14 checks passed
@openshift-ci

openshift-ci Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

@redhat-chai-bot: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants