fix: prevent private workflows from selecting hosted runners - #1325
Conversation
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a mandatory enforcement policy for runner selection in private repositories. By integrating a validation layer into the workflow generation process, it ensures that private projects exclusively utilize self-hosted runners, preventing accidental reliance on GitHub-hosted infrastructure. The changes include robust validation logic, updated agent guidance, and comprehensive tests to maintain configuration integrity. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Private runners must be found, In the code where secrets bound. Hosted options cast away, Self-hosted rules are here to stay. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements a policy requiring private repositories to use self-hosted CI runners. It updates the repository rules documentation, adds agent instructions, and introduces workflow validation to enforce self-hosted runners in private repositories while stripping GitHub-hosted runner configurations. The review feedback highlights critical issues in the new workflow validation logic, specifically noting that the 'strategy' block in GitHub Actions does not always contain a 'matrix' (which would cause a validation crash), and suggesting robustness improvements for handling YAML parsing errors, matrix keys with hyphens, and scalar matrix values.
| const workflowSchema = z.object({ | ||
| jobs: z.record( | ||
| z.string(), | ||
| z | ||
| .object({ | ||
| 'runs-on': runnerSchema.optional(), | ||
| strategy: z.object({ matrix: z.record(z.string(), z.unknown()) }).optional(), | ||
| }) | ||
| .nullable() | ||
| ), | ||
| }); |
There was a problem hiding this comment.
In GitHub Actions, a strategy block can be defined without a matrix (for example, to configure fail-fast or max-parallel options). Making matrix required in the Zod schema will cause workflowSchema.parse to throw a validation error and crash wbfy on such valid workflows. Making matrix optional prevents these unexpected crashes.
| const workflowSchema = z.object({ | |
| jobs: z.record( | |
| z.string(), | |
| z | |
| .object({ | |
| 'runs-on': runnerSchema.optional(), | |
| strategy: z.object({ matrix: z.record(z.string(), z.unknown()) }).optional(), | |
| }) | |
| .nullable() | |
| ), | |
| }); | |
| const workflowSchema = z.object({ | |
| jobs: z.record( | |
| z.string(), | |
| z | |
| .object({ | |
| 'runs-on': runnerSchema.optional(), | |
| strategy: z.object({ matrix: z.record(z.string(), z.unknown()).optional() }).optional(), | |
| }) | |
| .nullable() | |
| ), | |
| }); |
There was a problem hiding this comment.
The 52-repository private workflow audit contains 17 strategy blocks, all with matrix; none needs this alternative shape. The repository instructions require handling only observed organization cases and fixing unsupported custom workflows rather than adding compatibility branches. This validator intentionally fails before writes when a custom workflow is outside the supported shape.
| for (const { name: fileName } of entries.filter((entry) => entry.isFile() && /\.ya?ml$/u.test(entry.name))) { | ||
| const workflow = workflowSchema.parse(yaml.load(await fs.readFile(path.join(workflowsPath, fileName), 'utf8'))); | ||
| for (const [jobName, job] of Object.entries(workflow.jobs)) { | ||
| if (!job?.['runs-on']) continue; | ||
| const runner = job['runs-on']; | ||
| if (isSelfHosted(runner)) continue; | ||
| const match = typeof runner === 'string' && /^\$\{\{\s*matrix\.(\w+)\s*\}\}$/u.exec(runner); | ||
| const candidates: unknown[] = []; | ||
| if (match && job.strategy) { | ||
| const key = match[1]!; | ||
| const matrix = job.strategy.matrix; | ||
| const values = z.array(z.unknown()).safeParse(matrix[key]); | ||
| if (values.success) candidates.push(...values.data); | ||
| const include = z.array(z.record(z.string(), z.unknown())).safeParse(matrix.include); | ||
| if (include.success) | ||
| candidates.push(...include.data.map((row) => row[key]).filter((value) => value !== undefined)); | ||
| } else { | ||
| candidates.push(runner); | ||
| } | ||
| const allowed = | ||
| candidates.length > 0 && | ||
| candidates.every( | ||
| (candidate) => | ||
| isSelfHosted(candidate) || | ||
| (config.repository?.toLowerCase() === 'github:willbooster/cheerlings' && | ||
| fileName === 'build-desktop-apps.yml' && | ||
| jobName === 'build' && | ||
| candidate === 'windows-latest') | ||
| ); | ||
| if (!allowed) { | ||
| throw new Error( | ||
| `${fileName}: jobs.${jobName}.runs-on must select self-hosted runners in a private repository. Fix the workflow before running wbfy.` | ||
| ); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
There are several robustness and correctness improvements that can be made here:
- Error Handling: If a workflow file contains invalid YAML syntax,
yaml.loadwill throw an error, crashing the entirewbfyexecution. Wrapping the file reading and parsing in atry-catchblock and usingsafeParseallowswbfyto gracefully skip invalid files or log a warning instead of crashing. - Matrix Key Regex: The current regex
/^\$\{\{\s*matrix\.(\w+)\s*\}\}$/uuses\w+which does not match hyphens. Matrix keys in GitHub Actions frequently contain hyphens (e.g.,runner-typeoros-version). Changing this to[\w-]+ensures correct matching. - Scalar Matrix Values: A matrix key can sometimes map to a scalar value (e.g., a single string) instead of an array. Checking if the value is an array and wrapping it in an array if not makes the candidate resolution more robust.
for (const { name: fileName } of entries.filter((entry) => entry.isFile() && /\.ya?ml$/u.test(entry.name))) {
let parsedYaml: unknown;
try {
parsedYaml = yaml.load(await fs.readFile(path.join(workflowsPath, fileName), 'utf8'));
} catch {
continue;
}
const parsed = workflowSchema.safeParse(parsedYaml);
if (!parsed.success) continue;
const workflow = parsed.data;
for (const [jobName, job] of Object.entries(workflow.jobs)) {
if (!job?.['runs-on']) continue;
const runner = job['runs-on'];
if (isSelfHosted(runner)) continue;
const match = typeof runner === 'string' && /^\$\{\{\s*matrix\.([\w-]+)\s*\}\}$/u.exec(runner);
const candidates: unknown[] = [];
if (match && job.strategy?.matrix) {
const key = match[1]!;
const matrix = job.strategy.matrix;
const matrixValue = matrix[key];
if (matrixValue !== undefined) {
const values = Array.isArray(matrixValue) ? matrixValue : [matrixValue];
candidates.push(...values);
}
const include = z.array(z.record(z.string(), z.unknown())).safeParse(matrix.include);
if (include.success) {
candidates.push(...include.data.map((row) => row[key]).filter((value) => value !== undefined));
}
} else {
candidates.push(runner);
}
const allowed =
candidates.length > 0 &&
candidates.every(
(candidate) =>
isSelfHosted(candidate) ||
(config.repository?.toLowerCase() === 'github:willbooster/cheerlings' &&
fileName === 'build-desktop-apps.yml' &&
jobName === 'build' &&
candidate === 'windows-latest')
);
if (!allowed) {
throw new Error(
`${fileName}: jobs.${jobName}.runs-on must select self-hosted runners in a private repository. Fix the workflow before running wbfy.`
);
}
}
}There was a problem hiding this comment.
Skipping invalid YAML or schema failures would bypass the requested fail-fast runner check, so those errors must propagate. The 52-repository audit has no hyphenated runner matrix keys; unsupported custom selections must be fixed explicitly under the repository's canonical-format rule. GitHub documents matrix axes as arrays, not scalar values: https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations . No observed caller requires the proposed scalar fallback.
1 participant
Requirements
Why
Private caller workflows can retain github_hosted_runner and runs_on overrides across regeneration. Custom jobs can also select hosted runners outside the reusable workflows.
Customer Summary
Private reusable callers lose hosted overrides while valid self-hosted OS/size/host constraints survive. Custom jobs must explicitly require self-hosted runners; the approved cheerlings Windows desktop job remains allowed. Unknown repository visibility skips workflow generation.
Technical Summary
Normalize runner inputs by the actual reusable callee, including custom filenames. Validate direct runners and observed static matrix selections before any workflow writes, rejecting unsupported selections. Update generated agent guidance and expected repository rules.
Testing
bun run verify-full and PR CI passed. Added generator boundary tests for override cleanup, preserved custom labels, idempotence, and rejection before writes. Existing Rust workflow fixtures explicitly declare known public visibility. Audited 52 active private WillBooster repositories against the validator with the self-host-utils label migration.
Notes
Companion enforcement WillBooster/reusable-workflows#515 and explicit host migrations WillBooster/self-host-utils#62 and WillBooster/judge#2217 are merged. Custom workflows must follow the supported shape; unsupported selections and malformed YAML stop generation. This is a generation-time check, not an organization-wide GitHub execution policy.