Add AI code review tools evaluation policy by cblecker · Pull Request #8910 · kubernetes/community

cblecker · 2026-03-22T14:47:40Z

Summary

Adds github-management/ai-code-review-tools.md establishing a policy for evaluating AI-powered code review tools on a per-repo opt-in basis
Covers the full lifecycle: requesting a new tool, privacy/security assessment, approval, 90-day pilot, evaluation, and removal
Privacy and security assessment is owned by the GitHub Administration Team

Context

There has been growing interest in AI code review tools across Kubernetes orgs (kubernetes/org#5930). This policy provides a consistent framework for evaluating these requests.

fixes kubernetes/org#6234

/sig contributor-experience
/area github-management

Priyankasaggu11929 · 2026-03-23T07:15:08Z

should we create a gh issue template in k/org specifically for these requests?

we can add a checkbox for folks requesting (sig leads / subproject leads) to acknowledge the Kubernetes AI usage policy - https://github.com/kubernetes/community/blob/main/contributors/guide/pull-requests.md#ai-guidance

Also, i am thinking should we ask the requesting folks as well to upfront provide a list of permissions that the integration would require. (gh admins will still do the verification)

aojea · 2026-03-27T06:23:22Z

+
+## Evaluation and Decision
+
+At the end of the pilot period, the sponsoring subproject provides a summary


how is this summary provided? is this going to be an artifact somewhere? is providing some structure to the summary a good idea so we can have consistency on the reports?

A github issue for tracking, perhaps?

Added mention of this

ameukam · 2026-04-03T08:52:04Z

+This policy covers AI tools that automatically review pull requests, such as
+CodeRabbit or GitHub Copilot code review. It does not cover other AI-powered
+tooling such as CI/CD, security scanning, or code generation assistants.
+


what about open source tools leveraging AI ? we could have something built in the open just for issue triage for example.

The CNCF provides https://dosu.dev/ for this purpose already. ref: https://contribute.cncf.io/resources/services/hosted-tools/#tools

But imo I'm not sure this policy should be in scope for your question.

I don't think this matters if it's open source or not -- it's more about the role in the software development lifecycle.. specifically code review.

jasonbraganza

/lgtm

Fairly comprehensive beginning. Cannot think of anything better.

jasonbraganza · 2026-04-06T06:12:22Z

/hold

kannon92 · 2026-04-06T15:20:24Z

This looks great to me!

Happy to help with evaluation for JobSet and Kueue (with help from @mimowo and @tenzen-y).

Hope we can unblock AI reviews soon.

jberkus · 2026-04-09T23:52:51Z

+
+- Identify the tool and link to its documentation
+- Describe the use cases and what the subproject is trying to accomplish
+- Explain why existing approved tools do not meet their needs


Is there a list of already-approved tools somewhere?

IIRC k8s repo admins can enable Copilot auto code reviews, but as @kannon92 mentioned in kubernetes/org#5930 (comment), it requires PR authors to have Copilot subscription first, or the maintainers need to manually request reviews from Copilot

OK. I'm just saying that if we have existing tools, we should link to a list somewhere, so that it'll save a lot of duplicate requests.

Speaking of this, if a tool is approved in one SIG repo/subproject, can we assume it's approved to use in other repos as well (may still need admin to configure it, but no additional approval is needed)?

@janetkuo assuming there is not extra $$$'s involved perhaps?

Added a blank section for approved tools

janetkuo

This looks great. Would love to try out different AI code review tools!

janetkuo · 2026-04-16T00:12:46Z

+
+- Identify the tool and link to its documentation
+- Describe the use cases and what the subproject is trying to accomplish
+- Explain why existing approved tools do not meet their needs


IIRC k8s repo admins can enable Copilot auto code reviews, but as @kannon92 mentioned in kubernetes/org#5930 (comment), it requires PR authors to have Copilot subscription first, or the maintainers need to manually request reviews from Copilot

k8s-ci-robot · 2026-04-16T00:16:01Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cblecker, janetkuo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~github-management/OWNERS~~ [cblecker]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

janetkuo · 2026-04-28T17:54:55Z

@jasonbraganza is this ready to go?

jasonbraganza · 2026-04-29T00:12:49Z

Apologies! Please unhold at will. I held it, because I wanted voices, wiser and more experienced than mine, to chime in.

…

On Tue, 28 Apr 2026, at 23:25, Janet Kuo wrote: *janetkuo* left a comment (kubernetes/community#8910) <#8910 (comment)> @jasonbraganza <https://github.com/jasonbraganza> is this ready to go? — Reply to this email directly, view it on GitHub <#8910 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFJRTIQ7XQ6M36VI6EAT2D4YDWAPAVCNFSM6AAAAACW2YA6OCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DGMZXHAZTSNZRGQ>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

Priyankasaggu11929 · 2026-04-29T01:04:12Z

I think there are a bunch of inline comments we need to address before we merge it.

Add a policy document for evaluating AI-powered code review tools (such as CodeRabbit) on a per-repo opt-in basis. The policy covers requesting new tools, privacy/security assessment, pilot structure, evaluation criteria, and removal process.

k8s-ci-robot · 2026-05-04T04:53:06Z

New changes are detected. LGTM label has been removed.

k8s-ci-robot added sig/contributor-experience Categorizes an issue or PR as relevant to SIG Contributor Experience. area/github-management Issues or PRs related to GitHub Management subproject labels Mar 22, 2026

k8s-ci-robot requested review from MadhavJivrajani and jasonbraganza March 22, 2026 14:47

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 22, 2026

cblecker mentioned this pull request Mar 22, 2026

REQUEST: Enable coderabbit github integration for jobset and kueue kubernetes/org#5930

Open

aojea reviewed Mar 27, 2026

View reviewed changes

ameukam reviewed Apr 3, 2026

View reviewed changes

jasonbraganza reviewed Apr 6, 2026

View reviewed changes

k8s-ci-robot assigned jasonbraganza Apr 6, 2026

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 6, 2026

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 6, 2026

jberkus reviewed Apr 9, 2026

View reviewed changes

janetkuo approved these changes Apr 16, 2026

View reviewed changes

janetkuo mentioned this pull request Apr 16, 2026

REQUEST: New AI review tool integration for agent-sandbox kubernetes/org#6296

Open

camilamacedo86 mentioned this pull request Apr 19, 2026

Please enable Copilot reviews kubernetes-sigs/controller-tools#1380

Open

Add AI code review tools evaluation policy

a635916

Add a policy document for evaluating AI-powered code review tools (such as CodeRabbit) on a per-repo opt-in basis. The policy covers requesting new tools, privacy/security assessment, pilot structure, evaluation criteria, and removal process.

cblecker force-pushed the ai-code-review-policy branch from cecd6cd to a635916 Compare May 4, 2026 04:53

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 4, 2026

k8s-ci-robot removed the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label May 4, 2026

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 4, 2026


		## Evaluation and Decision

		At the end of the pilot period, the sponsoring subproject provides a summary

Conversation

cblecker commented Mar 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Context

Uh oh!

Priyankasaggu11929 commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kannon92 Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jasonbraganza left a comment

Choose a reason for hiding this comment

Uh oh!

jasonbraganza commented Apr 6, 2026

Uh oh!

kannon92 commented Apr 6, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

janetkuo Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

janetkuo left a comment

Choose a reason for hiding this comment

Uh oh!

janetkuo Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Apr 16, 2026

Uh oh!

janetkuo commented Apr 28, 2026

Uh oh!

jasonbraganza commented Apr 29, 2026 via email • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Priyankasaggu11929 commented Apr 29, 2026

Uh oh!

k8s-ci-robot commented May 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

cblecker commented Mar 22, 2026 •

edited

Loading

Priyankasaggu11929 commented Mar 23, 2026 •

edited

Loading

kannon92 Apr 6, 2026 •

edited

Loading

janetkuo Apr 16, 2026 •

edited

Loading

janetkuo Apr 16, 2026 •

edited

Loading

jasonbraganza commented Apr 29, 2026 via email •

edited

Loading