Skip to content

fix(ci): retry GitHub OIDC request timeouts before Azure login - #13287

Merged
nicolejms merged 5 commits into
mainfrom
nicolejms-fix-azure-login-failure
Oct 14, 2026
Merged

nicolejms merged 5 commits into
mainfrom
nicolejms-fix-azure-login-failure

Conversation

@nicolejms

Copy link
Copy Markdown
Contributor

Summary

Add bounded retries to GitHub OIDC token requests in the cloud functional workflow. Keep Azure login, subscription selection, and permanent errors outside the retry path. Use the same identity, audience, cloud, subscription, and permissions as before. Clear the Azure CLI account cache after each job.

Reason for change

Fixes #13206

In run 37251883349, the core-resource cloud job failed before tests started. The full login log shows Failed to get ID Token, Error Code: undefined, and Error Message: Request timeout. The GitHub request began at 01:40:16.604Z and failed at 01:40:21.620Z. The job already had id-token: write; it had not reached Azure token exchange.

The pinned Azure login action calls the Actions toolkit OIDC client. Its HTTP response retries do not cover a rejected request. It also has no input for a prefetched token or request timeout retries. This PR uses a small shared script for the workflow's two Azure login steps so retries apply only at token acquisition, not to the whole login or test step.

The script makes at most three token requests, with a 30-second limit per request and two- and four-second delays. Only timeouts, selected connection errors, and HTTP 408, 429, 500, 502, 503, and 504 are retried. Permission, certificate, invalid-response, Azure login, and subscription errors fail without retry. Tokens are masked before use, are not written to files or step outputs, and are excluded from command failure messages. Trust settings are unchanged.

How to test

  • node --test .github/scripts/azure-oidc-login_test.mjs — passed, 43 tests. Coverage includes a real local HTTP request timeout followed by recovery, retry limits, permanent errors, token handling, Azure command failures, and workflow cleanup.

  • Repeated the full suite with the following PowerShell command — passed, 100/100 runs; 4,300 passing checks:

    for ($i = 1; $i -le 100; $i++) {
        node --test .github\scripts\azure-oidc-login_test.mjs *> $null
        if ($LASTEXITCODE -ne 0) { throw "OIDC regression suite failed on iteration $i" }
    }
  • actionlint -shellcheck= .github/workflows/functional-test-cloud.yaml — passed.

  • shellcheck --shell=bash added-commands.sh — passed. The generated check file contained every added shell command: node ./.github/scripts/azure-oidc-login.mjs, node --test ./.github/scripts/azure-oidc-login_test.mjs, and az account clear.

  • actionlint -shellcheck shellcheck.exe .github/workflows/functional-test-cloud.yaml — reported 17 existing ShellCheck findings. The same check on the unchanged base file reported the same 17 messages; no new findings were added. Unrelated shell code was not changed.

  • npm exec --yes --package=pnpm@11.24.0 -- pnpm exec markdownlint-cli2 docs/contributing/contributing-code/contributing-code-tests/running-functional-tests.md --config .github/linters/.markdownlint-cli2.yaml — passed.

  • npm exec --yes --package=pnpm@11.24.0 -- pnpm exec markdown-table-formatter docs/contributing/contributing-code/contributing-code-tests/running-functional-tests.md --check — passed.

  • npm exec --yes --package=cspell -- cspell lint --config .github/linters/.cspell.yml --no-progress docs/contributing/contributing-code/contributing-code-tests/running-functional-tests.md — passed.

  • git diff --check and git diff --cached --check — passed.

  • git -c gpg.format=openpgp verify-commit HEAD — passed. The commit has a GPG signature, the required DCO sign-off, and the Copilot co-author trailer.

make test-azure-oidc-login is wired into make test; GNU Make is not available on this host, so its exact Node command was run directly. The full unit suite, live Azure login, and cloud functional tests were not run locally. This Windows session has no runner OIDC context.

File change summary

File Summary of change
.github/scripts/azure-oidc-login.mjs Request the GitHub token with bounded transient-error retries, then run Azure login once.
.github/scripts/azure-oidc-login_test.mjs Add 43 regression tests for retries, errors, token handling, and workflow cleanup.
.github/workflows/functional-test-cloud.yaml Use the shared login script in both jobs and clear Azure sessions after cleanup.
build/test.mk Add the login regression test target to the unit-test entry point.
docs/contributing/contributing-code/contributing-code-tests/running-functional-tests.md Explain the request retry limits and local test command.

Bound retries to GitHub token acquisition in the cloud functional workflow.
Keep Azure login and subscription failures outside the retry path. Preserve
OIDC trust and clear the Azure CLI account cache after each job.

Add local regression coverage for timeouts, HTTP and network errors,
permanent failures, token handling, and workflow cleanup.

Fixes: #13206

Signed-off-by: nicolejms <nicolej@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: nicolejms <nicolej@microsoft.com>
Copilot AI balanced review requested due to automatic review settings October 14, 2026 17:05
@nicolejms
nicolejms requested review from a team as code owners October 14, 2026 17:05
@github-actions github-actions Bot added pr:needs-rebase The pull request has merge conflicts pr:waiting-for-review A reviewer owns the next action; required approval is pending or unverifiable labels Oct 14, 2026
@nicolejms nicolejms added the pr:standard Ongoing maintenance, minor improvements, documentation updates, and routine development work label Oct 14, 2026
Signed-off-by: nicolejms <nicolej@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot removed the pr:needs-rebase The pull request has merge conflicts label Oct 14, 2026
@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

  • .github/workflows/functional-test-cloud.yaml

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The authentication-path changes are well tested locally but still require human review or live Azure validation.

0 open findings

What changed in this PR

Adds bounded retries for transient GitHub OIDC token failures in cloud functional tests, addressing #13206 without retrying Azure login itself.

Changes:

  • Adds and tests a shared OIDC/Azure login script.
  • Integrates it into both cloud workflow jobs with cleanup.
  • Documents and wires the regression tests into make test.
File Description
.github/​scripts/​azure-oidc-login.mjs Implements bounded OIDC retries and Azure CLI login.
.github/​scripts/​azure-oidc-login_test.mjs Tests retries, failures, token handling, and workflow integration.
.github/​workflows/​functional-test-cloud.yaml Uses the script and clears Azure sessions.
build/​test.mk Adds the regression suite to make test.
docs/​contributing/​contributing-code/​contributing-code-tests/​running-functional-tests.md Documents retry and cleanup behavior.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@nicolejms
nicolejms enabled auto-merge October 14, 2026 17:18
Signed-off-by: nicolejms <nicolej@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Unit Tests

    2 files  ±0    448 suites  ±0   15m 43s ⏱️ + 3m 1s
7 319 tests ±0  7 317 ✅ ±0  2 💤 ±0  0 ❌ ±0 
8 752 runs  ±0  8 750 ✅ ±0  2 💤 ±0  0 ❌ ±0 

Results for commit 2d00d27. ± Comparison against base commit ffd051c.

♻️ This comment has been updated with latest results.

@codecov

codecov Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.56%. Comparing base (ffd051c) to head (2d00d27).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #13287      +/-   ##
==========================================
- Coverage   60.57%   60.56%   -0.01%     
==========================================
  Files         771      771              
  Lines       45608    45608              
==========================================
- Hits        27625    27624       -1     
- Misses      17983    17984       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread .github/workflows/functional-test-cloud.yaml Outdated
@github-actions github-actions Bot added pr:waiting-for-author The author needs to address human review feedback or human-endorsed inline feedback and removed pr:waiting-for-review A reviewer owns the next action; required approval is pending or unverifiable labels Oct 14, 2026
Signed-off-by: nicolejms <nicolej@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added pr:waiting-for-review A reviewer owns the next action; required approval is pending or unverifiable and removed pr:waiting-for-author The author needs to address human review feedback or human-endorsed inline feedback labels Oct 14, 2026
@radius-functional-tests

radius-functional-tests Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Radius functional test overview

🔍 Go to test action run

Click here to see the test run details
Name Value
Repository radius-project/radius
Commit ref 2d00d27
Unique ID func13286428ca
Image tag pr-func13286428ca
  • Dapr: 1.14.4
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func13286428ca
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func13286428ca
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func13286428ca
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func13286428ca
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func13286428ca
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

@github-actions github-actions Bot added pr:review-approved Required reviews are approved; checks may still be pending and removed pr:waiting-for-review A reviewer owns the next action; required approval is pending or unverifiable labels Oct 14, 2026
@nicolejms
nicolejms added this pull request to the merge queue Oct 14, 2026
Merged via the queue into main with commit 394095f Oct 14, 2026
91 checks passed
@nicolejms
nicolejms deleted the nicolejms-fix-azure-login-failure branch October 14, 2026 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr:review-approved Required reviews are approved; checks may still be pending pr:standard Ongoing maintenance, minor improvements, documentation updates, and routine development work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduled functional test failed - Run ID: 37251883349

3 participants