Skip to content

[test-qa-hourly] test: guard attribution interval endpoint - #441

Open
i-xtsu-sixyou-ken-mei wants to merge 2 commits into
mainfrom
test-qa/ongoing-coverage-20260911
Open

[test-qa-hourly] test: guard attribution interval endpoint#441
i-xtsu-sixyou-ken-mei wants to merge 2 commits into
mainfrom
test-qa/ongoing-coverage-20260911

Conversation

@i-xtsu-sixyou-ken-mei

Copy link
Copy Markdown
Collaborator

QA scope

Continuous Test QA run from current main (5271fbc07aaafd62a88cb8675ead4171f4e45f96).

Inspection

  • Recent commits, including merged QA PR [test-qa-hourly] test: cover impressions reach fallback #440 and integrated Control Center changes.
  • Open PRs: none at run start, so a new ongoing QA branch was required.
  • Latest main CI run 34491406884: quick-gates, code-quality, tests-related gates, security, coverage and deploy gating completed successfully.
  • .github/workflows/ci.yml gate composition.
  • scripts/coverage-summary.ts coverage aggregation path.
  • social-attribution.ts interval endpoint selection and existing tests.

Change

Added one focused regression test proving computePostActivity() never leaks an observation captured after the snapshot interval into the current interval. With a 100-view baseline, 140-view endpoint at the interval boundary, and a later 1,000-view observation, the expected delta remains 40.

No production code, CI gates, docs, mocks, ignores, or thresholds changed.

Validation

  • Focused local validation attempted via DevSpace, but the connector returned the transient account-connection error We couldn't connect your account. Please try again. No local files were modified through that environment.
  • Authoritative PR CI will validate the committed test.

Commands / results

  • GitHub main commit inspection: PASS
  • GitHub open PR inspection: PASS (none open)
  • Main CI run 34491406884 jobs inspection: PASS
  • CI workflow inspection: PASS
  • Coverage script inspection: PASS
  • Focused local test: BLOCKED by transient DevSpace connector account error

Next QA target

After this PR is green, validate that standardized 24h growth metrics with collection_status='unavailable' cannot contribute to experiment samples24h, reach, or engagement. Current metrics24hForPostIds() filters only post ID and measurement_window === '24h', while the standardized query intentionally retains collection status.

Copy link
Copy Markdown
Collaborator Author

Test QA run — CI-first repair

Inspection

  • Reused this open Test QA PR and confirmed branch test-qa/ongoing-coverage-20260911 was already synced with main before the fix (ahead 1 / behind 0).
  • Inspected recent main commits, open PR state, active PR patch, CI workflow, failed run #2512 (34496330787), quick-gates artifact, coverage artifact availability, and the new attribution-boundary test.
  • CI #2512: code-quality, coverage, e2e, check-dead-env and other non-quick gates passed; quick-gates failed.
  • Downloaded quick-gates-verify-1 artifact and inspected .ai-verify/logs/{format,repo,contracts}.log plus result.json.

Root cause

format was the only failed quick gate. Prettier required the helper signature in social-attribution-boundary.test.ts to be multiline. repo and contracts both passed.

Action

Commit 293c362a9a0df88ff8a8cbe8227cb0f000ac6f42test(control-center): format attribution boundary helper

Only the test helper formatting changed; no production code, CI gates, ignores, thresholds, docs, or mocks were modified.

Commands / results

  • GitHub compare main...test-qa/ongoing-coverage-20260911: PASS (ahead 1 / behind 0 before fix)
  • PR patch inspection: PASS (single added test file)
  • CI run #2512 jobs inspection: FAIL isolated to quick-gates
  • Download/inspect quick-gates-verify-1: PASS
    • format: FAIL — exact Prettier diff identified
    • repo: PASS
    • contracts: PASS
  • CI workflow inspection: PASS (format repo contracts / type-check lint deadcode dup / test analytics unchanged)
  • Local focused validation: BLOCKED by transient DevSpace connector account error (We couldn't connect your account. Please try again.)
  • New-head GitHub CI: pending discovery/run creation at time of this update

Auto-merge

Not enabled while the repaired head is not yet proven green.

Next QA target

Once CI is green, validate that standardized 24h metrics with collection_status='unavailable' cannot contribute to experiment samples24h, reach, or engagement.

Copy link
Copy Markdown
Collaborator Author

Hourly QA run — CI-first investigation

CI #2513 (34502732237) is still failing, so this run did not add another regression test.

Inspection

Root cause

This is repository schedule-registry drift introduced on main, not a failure caused by the QA test. Commit 368116dd7e7655e63bd1078f919dfad30e3e7500 added .github/workflows/ops-operator.yml with workflow name Ops Operator and cron */5 * * * *. GitHub code search shows 9 scheduled workflow files, while .github/schedules.json still has only the previous 8 workflow-sourced cron entries; cron-failure-alert.yml also subscribes only to those previous 8 workflow names and omits Ops Operator.

check-schedules-registry.sh intentionally requires both invariants:

  1. every workflow cron must exactly match a workflow-sourced registry row; and
  2. every scheduled workflow name must exactly match the cron failure-alert subscriptions.

Therefore fixing #2513 correctly requires updating schedule operational configuration (.github/schedules.json and cron-failure-alert.yml, or otherwise changing the new Ops Operator schedule). Those files are outside this automation's allowed QA-change scope (tests / fixtures / minimal test utilities / minimal implementation fixes), so I am not weakening the gate or smuggling an unrelated config change into this test PR.

Commands / results

  • PR metadata / active diff inspection: PASS
  • Recent main commits inspection: PASS
  • Open PR search: PASS
  • CI #2513 workflow-run + job inspection: FAIL isolated to deploy-gates schedule-registry assertion
  • Scheduled workflow code search: PASS — 9 scheduled workflows found
  • .github/schedules.json inspection: PASS — Ops Operator missing
  • cron-failure-alert.yml inspection: PASS — Ops Operator missing from subscriptions
  • scripts/check-schedules-registry.sh inspection: PASS — confirms both exact-match checks
  • Coverage aggregator inspection: PASS — Istanbul/Vitest + Cobertura/pytest inputs remain supported
  • Focused local validation: BLOCKED by transient DevSpace account connection error (We couldn't connect your account. Please try again.)

Action

No code change this run: a safe CI repair is outside the permitted QA diff surface. PR #441 remains unchanged and should not be auto-merged while deploy-gates is red.

Next QA target

Once the repository schedule drift is repaired and CI is green, resume the queued data-quality target: verify that standardized 24h metrics with collection_status='unavailable' cannot contribute to experiment samples24h, reach, or engagement.

Copy link
Copy Markdown
Collaborator Author

QA run update

Rechecked the current QA PR, latest main, CI jobs, the active diff, the scheduled workflow registry, and the newly-added Ops Operator workflow.

Result

No new QA code change was made in this run because CI is still blocked by the same repository-level schedule registry drift, not by the QA diff.

  • PR head: 293c362a9a0df88ff8a8cbe8227cb0f000ac6f42
  • CI run 34502732237 / #2513: tests, coverage, code-quality, quick-gates, e2e, security, and check-dead-env all PASS.
  • Only deploy-gates fails at Assert schedules match .github/schedules.json.
  • main has not advanced past 5271fbc07aaafd62a88cb8675ead4171f4e45f96, so there is no upstream fix to sync yet.
  • .github/workflows/ops-operator.yml defines Ops Operator with cron */5 * * * *.
  • .github/schedules.json still has no corresponding ops-operator workflow entry.
  • PR diff remains exactly one QA test file (social-attribution-boundary.test.ts, +59/-0).

Commands / results

  • Open PR search: PASS — reused [test-qa-hourly] test: guard attribution interval endpoint #441.
  • PR metadata/diff inspection: PASS — open, non-draft, mergeable, QA-only diff.
  • Recent main commit inspection: PASS — no newer main commit/upstream fix.
  • PR CI run/job inspection: FAIL only in deploy schedule consistency gate; all test/quality/coverage/security gates PASS.
  • .github/schedules.json inspection: confirms missing Ops Operator registration.
  • .github/workflows/ops-operator.yml inspection: confirms scheduled cron exists.
  • No code/config mutation performed because fixing the schedule registry/workflow alert configuration is outside this QA task's allowed change scope.

Auto-merge

Not enabled: required CI is red (deploy-gates). No protections were bypassed.

Next QA target

Once the repository-level schedule drift is fixed and CI is green, validate whether standardized 24h metrics with collection_status='unavailable' can incorrectly contribute to experiment samples24h, reach, or engagement; add the smallest regression test/minimal consumer guard only if reproduced.

Copy link
Copy Markdown
Collaborator Author

Hourly QA follow-up

Rechecked the repository source of truth before making any new QA change.

  • PR [test-qa-hourly] test: guard attribution interval endpoint #441 is still open, non-draft, mergeable, and remains QA-only: one test file (social-attribution-boundary.test.ts), +59/-0.
  • Head 293c362a9a0df88ff8a8cbe8227cb0f000ac6f42 CI #2513 is still red only because deploy-gates fails at Assert schedules match .github/schedules.json; code-quality, E2E, security, and dead-env gates pass.
  • main is still 5271fbc07aaafd62a88cb8675ead4171f4e45f96; there is no newer upstream commit to sync and no repository-side fix for the schedule-registry drift yet.
  • The current PR diff was reviewed again and does not touch workflow/schedule config, so this failure remains unrelated to the QA test.

No code was changed this run. Fixing the current CI failure requires repository workflow/schedule-registry changes outside this QA automation's allowed scope, and weakening/bypassing the gate would be unsafe.

Commands / results

  • Recent PR / main inspection — PASS
  • PR diff review — PASS, QA-only
  • CI #2513 jobs inspection — PASS for diagnosis; blocker unchanged (deploy-gates schedule registry assertion)
  • Safe sync with main — no-op; branch base is still current main
  • New focused test / broader validation — intentionally not run because CI-first blocker is unresolved

Auto-merge: not enabled; required CI is red.

Next QA target after CI is green: verify that standardized 24h rows with collection_status='unavailable' cannot contribute to experiment samples24h, reach, or engagement; add a regression test/minimal consumer guard only if repository evidence reproduces the issue.

Copy link
Copy Markdown
Collaborator Author

Test QA run update

Inspection

CI status

CI is still blocked exclusively by deploy-gatesAssert schedules match .github/schedules.json. code-quality, E2E, security, and check-dead-env are green on this head. This remains the previously identified repository schedule-registry drift introduced by the Ops Operator scheduled workflow, not a failure caused by this QA test diff.

Action

No code change this run. The correct repository fix would require workflow/schedule-registry configuration changes outside this QA automation's allowed test/fixture/minimal-implementation scope. I did not weaken the gate, add an ignore, or create a noise test while CI is red.

Commands / results

  • PR metadata inspection: PASS
  • Head workflow run inspection: FAIL (deploy-gates schedule registry assertion only)
  • CI jobs inspection: PASS for code-quality / E2E / security / check-dead-env; deploy-gates FAIL
  • Open PR inspection: PASS ([test-qa-hourly] test: guard attribution interval endpoint #441 only)
  • Recent main commits inspection: PASS; no new upstream fix to sync
  • Branch sync: NOT NEEDED (main unchanged from PR base)
  • Focused/broad validation: NOT RERUN because authoritative CI is blocked before a new QA behavior change is appropriate

Auto-merge

Not enabled: required CI remains red.

Next QA target

Once schedule-registry drift is fixed and CI returns green, validate that standardized 24h metrics with collection_status='unavailable' cannot contribute to experiment samples24h, reach, or engagement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant