Skip to content

TRT-2657: Revert "add remaining CNO NetworkPolicies" (#2959)#2999

Merged
stbenjam merged 1 commit into
openshift:masterfrom
redhat-chai-bot:ship-help/revert-2959-cno-networkpolicies
May 8, 2026
Merged

TRT-2657: Revert "add remaining CNO NetworkPolicies" (#2959)#2999
stbenjam merged 1 commit into
openshift:masterfrom
redhat-chai-bot:ship-help/revert-2959-cno-networkpolicies

Conversation

@redhat-chai-bot
Copy link
Copy Markdown

@redhat-chai-bot redhat-chai-bot commented May 7, 2026

This reverts commit 5928824 (merge commit for #2959).

Why

The new NetworkPolicies cause cloud-network-config-controller to CrashLoopBackOff during upgrade (API server i/o timeout). 0% pass rate across all platforms (AWS, Azure, GCP) in payload 5.0.0-0.ci-2026-05-07-142711, blocking all 5.0 CI upgrade jobs.

The failures were visible in every upgrade job on the original PR: #2959 (comment)

References

/label trt-incident

Summary by CodeRabbit

  • Chores
    • Updated network policies for cluster networking components, modifying traffic isolation rules and egress configurations in system namespaces.

…2959)

This reverts commit 5928824.

The new NetworkPolicies cause cloud-network-config-controller to
CrashLoopBackOff during upgrade (API server i/o timeout). 0% pass
rate across all platforms (AWS, Azure, GCP) in payload
5.0.0-0.ci-2026-05-07-142711, blocking all 5.0 CI upgrade jobs.

TRT incident: TRT-2657
Payload analysis: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/logs/periodic-ci-openshift-release-main-claude-payload-agent/2052395627291086848/artifacts/claude-payload-agent/openshift-claude-payload-agent/artifacts/payload-analysis-5.0.0-0.ci-2026-05-07-142711-summary.html
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 7, 2026

@redhat-ship-help: The label(s) /label trt-incident cannot be applied. These labels are supported: acknowledge-critical-fixes-only, platform/aws, platform/azure, platform/baremetal, platform/google, platform/libvirt, platform/openstack, ga, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, px-approved, docs-approved, qe-approved, ux-approved, no-qe, rebase/manual, cluster-config-api-changed, run-integration-tests, verified, ready-for-human-review, approved, backport-risk-assessed, bugzilla/valid-bug, cherry-pick-approved, jira/skip-dependent-bug-check, jira/valid-bug, ok-to-test, stability-fix-approved, staff-eng-approved. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

Details

In response to this:

This reverts commit 5928824 (merge commit for #2959).

Why

The new NetworkPolicies cause cloud-network-config-controller to CrashLoopBackOff during upgrade (API server i/o timeout). 0% pass rate across all platforms (AWS, Azure, GCP) in payload 5.0.0-0.ci-2026-05-07-142711, blocking all 5.0 CI upgrade jobs.

The failures were visible in every upgrade job on the original PR: #2959 (comment)

References

/label trt-incident

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels May 7, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@redhat-ship-help: This pull request references Jira Issue OCPBUGS-83800, which is invalid:

  • expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

This reverts commit 5928824 (merge commit for #2959).

Why

The new NetworkPolicies cause cloud-network-config-controller to CrashLoopBackOff during upgrade (API server i/o timeout). 0% pass rate across all platforms (AWS, Azure, GCP) in payload 5.0.0-0.ci-2026-05-07-142711, blocking all 5.0 CI upgrade jobs.

The failures were visible in every upgrade job on the original PR: #2959 (comment)

References

/label trt-incident

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 7, 2026

Walkthrough

Network policy and namespace security configuration is being restructured across the cluster-network-operator and cloud-network-config-controller namespaces. The openshift-network-operator namespace gains default-deny network policies and additional metadata, while the cloud-network-config-controller namespace has its network policy rules reorganized.

Changes

Network Policy Restructuring

Layer / File(s) Summary
Namespace Metadata Updates
manifests/0000_70_cluster-network-operator_00_namespace.yaml
The openshift-network-operator namespace receives new annotations (openshift.io/node-selector, workload.openshift.io/allowed), labels for run-level and pod security policies, and cluster monitoring.
Default-Deny Policy Consolidation
manifests/0000_70_cluster-network-operator_00_namespace.yaml, manifests/01-cncc-namespace.yaml
A default-deny NetworkPolicy with empty ingress/egress lists is added to the openshift-network-operator namespace; the same resource is removed from the openshift-cloud-network-config-controller namespace.
Self-Hosted Controller Policy
bindata/cloud-network-config-controller/self-hosted/networkpolicy.yaml
The cloud-network-config-controller's self-hosted network policy definition is updated (15 lines removed).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 12
✅ Passed checks (12 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed This PR is a revert that only modifies Kubernetes manifest YAML files (networkpolicy.yaml, namespace manifests). No Ginkgo test files or test names were modified. The check is not applicable.
Test Structure And Quality ✅ Passed This PR reverts NetworkPolicy YAML manifest changes. No test files or Ginkgo code are modified, making the test quality check not applicable to this PR.
Microshift Test Compatibility ✅ Passed This PR is a revert of previous NetworkPolicy changes and only modifies manifest YAML files. No new Ginkgo e2e tests are added. The check is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests are added in this PR. The changes are confined to YAML manifest files (NetworkPolicy and namespace definitions). The SNO compatibility check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed PR reverts NetworkPolicies only. No deployment manifests, operators, or workload resources with scheduling constraints are modified. NetworkPolicies define network rules, not pod scheduling.
Ote Binary Stdout Contract ✅ Passed PR reverts NetworkPolicy changes. This repository contains only operational binaries, not OTE test binaries. The OTE Binary Stdout Contract check is not applicable here.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR reverts NetworkPolicy changes in manifest YAML files only. No new Ginkgo e2e tests are added. The IPv6/disconnected network check applies only to new e2e tests, so it is not applicable.
Title check ✅ Passed The title accurately describes the main change: reverting a commit that added CNO NetworkPolicies, matching the PR's stated objective of reverting commit #2959 due to upgrade failures.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from arghosh93 and miheer May 7, 2026 23:49
@openshift-ci openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label May 7, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 7, 2026

Hi @redhat-ship-help. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@stbenjam stbenjam changed the title Revert "OCPBUGS-83800: add remaining CNO NetworkPolicies" (#2959) Revert "add remaining CNO NetworkPolicies" (#2959) May 7, 2026
@openshift-ci-robot openshift-ci-robot removed the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 7, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@redhat-ship-help: No Jira issue is referenced in the title of this pull request.
To reference a jira issue, add 'XYZ-NNN:' to the title of this pull request and request another refresh with /jira refresh.

Details

In response to this:

This reverts commit 5928824 (merge commit for #2959).

Why

The new NetworkPolicies cause cloud-network-config-controller to CrashLoopBackOff during upgrade (API server i/o timeout). 0% pass rate across all platforms (AWS, Azure, GCP) in payload 5.0.0-0.ci-2026-05-07-142711, blocking all 5.0 CI upgrade jobs.

The failures were visible in every upgrade job on the original PR: #2959 (comment)

References

/label trt-incident

Summary by CodeRabbit

  • Chores
  • Updated network policies for cluster networking components, modifying traffic isolation rules and egress configurations in system namespaces.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot removed the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label May 7, 2026
@stbenjam stbenjam changed the title Revert "add remaining CNO NetworkPolicies" (#2959) TRT-2657: Revert "add remaining CNO NetworkPolicies" (#2959) May 7, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 7, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

openshift-ci-robot commented May 7, 2026

@redhat-ship-help: This pull request references TRT-2657 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "5.0.0" version, but no target version was set.

Details

In response to this:

This reverts commit 5928824 (merge commit for #2959).

Why

The new NetworkPolicies cause cloud-network-config-controller to CrashLoopBackOff during upgrade (API server i/o timeout). 0% pass rate across all platforms (AWS, Azure, GCP) in payload 5.0.0-0.ci-2026-05-07-142711, blocking all 5.0 CI upgrade jobs.

The failures were visible in every upgrade job on the original PR: #2959 (comment)

References

/label trt-incident

Summary by CodeRabbit

  • Chores
  • Updated network policies for cluster networking components, modifying traffic isolation rules and egress configurations in system namespaces.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@stbenjam
Copy link
Copy Markdown
Member

stbenjam commented May 7, 2026

/ok-to-test

@openshift-ci openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 7, 2026
@stbenjam
Copy link
Copy Markdown
Member

stbenjam commented May 7, 2026

/payload-job periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 7, 2026

@stbenjam: trigger 2 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/04dbc3e0-4a70-11f1-8f36-f0abf143b526-0

@stbenjam
Copy link
Copy Markdown
Member

stbenjam commented May 7, 2026

Let's see what payload jobs say. AWS also had an outage earlier which might've caused the job failures

@openshift-merge-robot
Copy link
Copy Markdown
Contributor

Fix included in release 5.0.0-0.nightly-2026-05-07-185738

@petr-muller
Copy link
Copy Markdown
Member

petr-muller commented May 8, 2026

Stephens's failed with

  * could not run steps: step [release-inputs:latest] failed: failed to wait for importing imagestreamtags on ci-op-kqnxixmq/stable: failed to reimport the tag ci-op-kqnxixmq/stable:vcf-migration-operator: unable to import tag ci-op-kqnxixmq/stable:vcf-migration-operator with message Internal error occurred: [dockerimage.image.openshift.io "quay.io/openshift/ci:ocp_5.0_vcf-migration-operator" not found, dockerimage.image.openshift.io "quay-proxy.ci.openshift.org/openshift/ci:ocp_5.0_vcf-migration-operator" not found] on the image stream even after (6) imports: timed out waiting for the condition 

/payload-job periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 8, 2026

@petr-muller: trigger 2 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/47063b10-4ab6-11f1-9733-f59b456cc67a-0

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 8, 2026

@petr-muller: trigger 2 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/74fb5e10-4ab6-11f1-8709-04fccfb2554f-0

@petr-muller
Copy link
Copy Markdown
Member

huh pj-rehearse reacts on comment edit 🤔

@petr-muller
Copy link
Copy Markdown
Member

/retest

@danwinship
Copy link
Copy Markdown
Contributor

/lgtm

The failures were visible in every upgrade job on the original PR: #2959 (comment)

And this is why its bad when your CI has an overall pass rate of negative twelve percent.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 8, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 8, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danwinship, redhat-ship-help

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 8, 2026
@petr-muller
Copy link
Copy Markdown
Member

The /payload jobs are solid green. I am bit surprised to still see the failures in the native update presubmits:

But I guess these can be caused by updating from the version with the bug.

/verified by https://pr-payload-tests.ci.openshift.org/runs/ci/47063b10-4ab6-11f1-9733-f59b456cc67a-0

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label May 8, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@petr-muller: This PR has been marked as verified by https://pr-payload-tests.ci.openshift.org/runs/ci/47063b10-4ab6-11f1-9733-f59b456cc67a-0.

Details

In response to this:

The /payload jobs are solid green. I am bit surprised to still see the failures in the native update presubmits:

But I guess these can be caused by updating from the version with the bug.

/verified by https://pr-payload-tests.ci.openshift.org/runs/ci/47063b10-4ab6-11f1-9733-f59b456cc67a-0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@petr-muller
Copy link
Copy Markdown
Member

/override ci/prow/e2e-aws-ovn-upgrade-ipsec

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 8, 2026

@petr-muller: Overrode contexts on behalf of petr-muller: ci/prow/e2e-aws-ovn-upgrade-ipsec

Details

In response to this:

/override ci/prow/e2e-aws-ovn-upgrade-ipsec

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@petr-muller
Copy link
Copy Markdown
Member

/override ci/prow/e2e-aws-ovn-upgrade

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 8, 2026

@petr-muller: Overrode contexts on behalf of petr-muller: ci/prow/e2e-aws-ovn-upgrade

Details

In response to this:

/override ci/prow/e2e-aws-ovn-upgrade

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@stbenjam stbenjam merged commit 5794883 into openshift:master May 8, 2026
23 of 30 checks passed
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 8, 2026

@redhat-ship-help: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/hypershift-e2e-aks bf486f8 link true /test hypershift-e2e-aks
ci/prow/e2e-metal-ipi-ovn-dualstack-bgp-local-gw bf486f8 link true /test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
ci/prow/security bf486f8 link false /test security
ci/prow/e2e-azure-ovn-upgrade bf486f8 link true /test e2e-azure-ovn-upgrade
ci/prow/e2e-gcp-ovn-upgrade bf486f8 link true /test e2e-gcp-ovn-upgrade

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants