Skip to content

TEST: harden async lifecycle test observation - #3122

Merged
Richard Lundeen (richlundeen) merged 2 commits into
microsoft:mainfrom
romanlutz:romanlutz-async-lifecycle-reliability
Oct 11, 2026
Merged

Richard Lundeen (richlundeen) merged 2 commits into
microsoft:mainfrom
romanlutz:romanlutz-async-lifecycle-reliability

Conversation

@romanlutz

Copy link
Copy Markdown
Contributor

Description

The Copilot session-retirement and FIFO scenario-launch tests have failed in the merge queue with short coordination deadlines. Their asyncio.wait_for watchdogs could cancel the operation being observed, including adding a second cancellation while session deletion was pending or cancelling the final scenario execution instead of simply reporting a timeout.

Reuse the existing wait_for_completion_async helper from #3011 to bound observation without injecting cancellation:

  • Observe mocked SDK deletion before cancelling the Copilot send, and reuse the existing mocked startup-I/O fixture. Preserve the original cancellation message, failed-release cause, deletion counts, and client-stop ordering.
  • Gate mocked FIFO handoff reads with events. Preserve execution order, maximum concurrency of one, durable queue positions, transient failure recovery, cancellation persistence, and an empty final queue.
  • Add blocked-watchdog cases that deliberately expire an observer with timeout=0 while its dependency remains blocked. The operation must stay pending and finish correctly after release. Release and drain all owned tasks in finally before mocks are removed, including shielded target cleanup and scheduler handoff tasks.

Only the two test files change. No production deadlines, APIs, cancellation semantics, shared helpers or fixtures, dependency manifests, coverage thresholds, or CI gates change.

Tests and Documentation

Validation on Windows with CPython 3.14.4 and github-copilot-sdk 1.0.16. SDK I/O remained mocked and memory isolated.

Both complete files: 265 passed in each of two runs; the final run reported 20 existing deprecation warnings.

uv run --no-sync python -m pytest -n 4 --dist=loadfile tests\unit\prompt_target\target\test_github_copilot_target.py tests\unit\backend\test_scenario_run_service.py -q

Repeated xdist controls: five independent runs, six passed per run, with no failure reruns.

uv run --no-sync python -m pytest -n 4 --dist=load tests\unit\prompt_target\target\test_github_copilot_target.py::test_cancelled_send_keeps_retirement_owned_until_cleanup_async tests\unit\backend\test_scenario_run_service.py::TestScenarioRunServiceStartRun::test_concurrent_launches_run_one_at_a_time_in_fifo_order_async -q

Lint, formatting, and applicable hooks: passed, including hooks during commit.

uv run --no-sync ruff check tests\unit\prompt_target\target\test_github_copilot_target.py tests\unit\backend\test_scenario_run_service.py
uv run --no-sync ruff format --check tests\unit\prompt_target\target\test_github_copilot_target.py tests\unit\backend\test_scenario_run_service.py
uv run --no-sync pre-commit run --files tests\unit\prompt_target\target\test_github_copilot_target.py tests\unit\backend\test_scenario_run_service.py
git diff --check

Deterministic negative controls: in separate Python processes, substituting the legacy observer produced three expected cancellation-assertion failures. A selective legacy observer on FIFO final execution produced one expected failure because the executor swallowed the injected cancellation and wait_for returned without raising the watchdog timeout. The finalizers drained owned work in these failure paths.

Type-check limitation: the following command reports one unrelated, pre-existing redundant-condition-strict diagnostic at Copilot test line 898 (if reset_task is not None; upstream line 897). The unchanged baseline had two diagnostics; this changeset removes the other and introduces none. The production-only type-check hook does not apply to these test files.

uv run --no-sync ty check --extra-search-path tests tests\unit\prompt_target\target\test_github_copilot_target.py tests\unit\backend\test_scenario_run_service.py

Linux/CI and Python 3.11 were not run locally. Documentation changes are not applicable to this test-only fix.

JupyText: not run; no notebooks or documentation code changed.

Use event-controlled Copilot session retirement and FIFO handoff gates with deterministic blocked-watchdog regressions. Preserve original cancellation and durable queue behavior, and release and drain owned tasks before mock teardown.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@richlundeen
Richard Lundeen (richlundeen) added this pull request to the merge queue Oct 11, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 11, 2026
@richlundeen
Richard Lundeen (richlundeen) added this pull request to the merge queue Oct 11, 2026
Merged via the queue into microsoft:main with commit 72e6138 Oct 11, 2026
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants