Skip to content

MAINT: Fail CoPyRIT deployment pipeline when initialization fails - #3068

Open
Justin Song (jsong468) wants to merge 1 commit into
microsoft:mainfrom
jsong468:copyrit_pipeline_fail_on_initialization_fail
Open

Justin Song (jsong468) wants to merge 1 commit into
microsoft:mainfrom
jsong468:copyrit_pipeline_fail_on_initialization_fail

Conversation

@jsong468

@jsong468 Justin Song (jsong468) commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Description

Previously, deployment could pass even when PyRIT initialization failed: /api/health checks web-server liveness, not whether the runtime can accept runs and sends.

  • Add an uncached /api/ready endpoint that returns HTTP 200 when PyRIT is ready and HTTP 503 otherwise, with runtime state and the serving ACA revision.
  • Exempt readiness from authentication, compatibility, and runtime-admission checks so deployment can inspect failed initialization without exposing configuration or exception details. Existing business-route protections remain unchanged.
  • Gate app deployment on readiness from the expected revision through the same public/private entry point as the existing health check. Revision matching prevents an older healthy container from producing a false success.
  • Fail immediately on failed, restart-required, or stopping; otherwise poll within a five-minute budget. Connection failures, malformed responses, and timeouts cannot pass deployment.
  • Print revision-specific CLI log instructions and Azure Portal Monitoring → Log stream guidance, with Monitoring → Logs for historical errors, so engineers can find the startup traceback.
  • Preserve /api/health, configuration recovery, and infrastructure-only deployment behavior. A failed deployment does not deliberately terminate the backend or automatically roll back the image/database.

Tests and Documentation

  • Added backend tests for readiness states, revision reporting, cache headers, middleware exemptions, and preserved liveness/configuration recovery after startup failure.
  • Added mocked pipeline tests for revision matching, fail-fast behavior, invalid responses, bounded retries, diagnostics, and deployment gating.
  • Ran the focused suite below: 450 tests and 79 subtests passed
python -m pytest tests\unit\backend\test_runtime_lifecycle.py tests\unit\backend\test_compatibility.py tests\unit\backend\test_auth_middleware.py tests\unit\infra\test_code_deployment.py -q
  • Updated the deployment README with readiness behavior, timeout budgets, and troubleshooting instructions.

@richlundeen

Copy link
Copy Markdown
Contributor

The rest of this change looks good to me. I think we should reuse /api/health instead of adding /api/ready.

We can add ready, state, and revision to the existing response and let the pipeline check those fields. Please keep HTTP 200 and the existing response fields: Front Door, the frontend, and the CLI use this endpoint to check backend availability, so it must remain available when runtime initialization fails. This would also avoid adding another exception in the auth, compatibility, and runtime middleware.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants