Implementation Plan: Accuracy-v2 and evidence-grounded resume authoring

Status: AA0 APPROVED 2026-10-04; AA010/AA011 NATIVE ACCEPTED; AA1 ENGINEERING CLOSED / OWNER ACCEPTED; LOCKED FIELD GATE FAILED; INDEPENDENT VALIDATION DEFERRED; AA2 ENGINEERING CLOSED / VERIFIED LOCALLY; FORMAL GATE OPEN Parent plan: plan.md AA3 status: LOCAL IMPLEMENTATION DELIVERED; DOMAIN/DATA ACCEPTANCE OPEN AA4 status: LOCAL ENGINEERING COMPLETE AND VERIFIED; FORMAL GATE OPEN Executable tasks: todo.md, task IDs AA000–AA092 Spec: ../docs/ATS_SCORING_AND_AUTHORING_SPEC.md Research: ../docs/ATS_SCORING_AND_AUTHORING_RESEARCH.md Decisions: ../docs/decisions/0006-evidence-gated-multidimensional-scoring.md, ../docs/decisions/0007-canonical-resume-and-verified-rendering.md

1. Outcome

Evolve Resumide from a transparent synthetic-regression analyzer into a candidate-first, evidence-grounded workspace that:

• reads native, scanned, and mixed resumes with measured confidence and candidate correction;
• reports deterministic ATS Compatibility and job-specific Job Alignment without pretending to know an employer's hidden decision;
• lets candidates fix exact lines on the result page using verified facts and reversible AI suggestions;
• produces versioned PDF/DOCX and later LaTeX artifacts;
• parses every export again before calling it verified;
• earns domain/vendor claims one gate at a time through representative evaluation.

2. Delivery constraints

• The workspace owner approved the complete AA000–AA005 design and Gate AA0 on 2026-10-04 through ADR-0009 candidate 1. The earlier limited approval and sequencing correction remain historical evidence. AA010/AA011 native conformity is accepted as recorded in the todo. ADR-0010 allows disclosed AI-assisted synthetic engineering labels; ADR-0008 records the later bounded local image/runtime authorization. Actual OCR isolation/resource/cleanup and adapter acceptance must still be measured. No external provider, real-data collection, numeric calibration or production release is approved.
• Existing results remain readable with explicit legacy score semantics.
• Each implementation task is a vertical slice limited to approximately five primary files; migrations/fixtures generated by a slice are counted separately but reviewed together.
• Every slice includes unit/contract/integration/browser evidence appropriate to its risk.
• Provider selection, real-data collection, scoring weights, product claims, and employer mode require explicit human approval.
• Full verification remains npm run verify:ci; future Accuracy-v2 commands cannot be treated as present until implemented and passing.

3. Dependency graph

AA0 truth, schema, corpus and provider decisions
 ├── AA1 document evidence and candidate verification
 │    └── AA2 canonical resume and immutable versions
 ├── AA3 job intelligence and occupation/domain packs
 │    └── AA4 deterministic scoring v2 and confidence
 │          └── AA5 same-page editor and grounded writing
 │                └── AA6 verified PDF/DOCX rendering
 │                      └── AA7 sandboxed LaTeX rendering
 └── AA8 representative evaluation and claim promotion
       └── AA9 recruiter/vendor calibration pilot

AA1 and AA3 may proceed in parallel after AA0 contracts are frozen. AA6 template design may prototype against fixtures after AA2, but verified labels depend on AA4 and parse-back evidence. AA9 is deferred until candidate-mode validity, privacy, fairness, and human-oversight gates pass.

4. Phase AA0: Truth contract and evaluation design

Outcome

Product semantics, data permissions, score dimensions, evidence states, legacy behavior, corpus design, and provider bake-off rules are approved before code or data collection.

Tasks

• AA000: Approve capability map, non-goals, and claim ladder.
• AA001: Approve score names, dimensions, unknown/skipped behavior, and legacy-score treatment.
• AA002: Freeze evidence, canonical resume, job-requirement, score, suggestion, and artifact contracts.
• AA003: Approve corpus governance, annotation handbook, statistical plan, and data retention.
• AA004: Approve OCR/document-provider bake-off and cost/privacy ceiling.
• AA005: Approve first occupation/domain, locale, and artifact-format release scope.

Gate AA0

Architecture, scoring, privacy, evaluation, and product owners sign the decisions. No scoring weight is approved from intuition alone; initial weights are hypotheses to be evaluated.

5. Phase AA1: Document evidence

2026-10-11 final closeout: Engineering AA010–AA015 is closed and owner-accepted; the final report records the task matrix and acceptance obligations AA1-R1–R5. Further AA2 work was paused during the owner-requested AA1 wrap-up. Engineering work AA012–AA015 is implemented under ADR-0011. The owner deferred unavailable independent reviewers; engineering does not wait for their assignment. Development fields pass 96/96; the actual frozen held-out synthetic run fails field precision/recall (92.31% / 75%). Gate AA1 is open, no calibration or representative acceptance is claimed, and unsafe evidence remains unscored. Closeout evidence.

Outcome

One uploaded resume produces a page/word/field evidence graph with native/OCR provenance, geometry, reading order, quality, disagreements, and candidate-verification states. Scoring is withheld when critical evidence is unsafe.

Vertical slices

• AA010: Persist page-level evidence and critical-field assertions for one native-text PDF.
• AA011: Add rendered-page geometry and reading-order evidence to Parse View.
• AA012: Implement selective OCR adapter slice for one scanned and one mixed fixture.
• AA013: Add native/OCR reconciliation and calibrated review-required policy.
• AA014: Add candidate correction/confirmation of uncertain critical fields.
• AA015: Build extraction-v2 benchmark and provider comparison report.

Gate AA1

Approved extraction gates pass on locked native, scanned, mixed, layout, language, and adverse-quality segments. A failure reaches a visible review/unsupported state rather than numeric scoring.

6. Phase AA2: Canonical resume and immutable versions

2026-10-11 engineering authorization: The owner approved progression under ADR-0012 / E-003. AA1's failed field gate remains visible; AA2 cannot promote uncertain evidence. AA020–AA024 are implemented with immutable local persistence, source-preserving import, strict confirmation and concurrency controls, owner-only history/compare, and privacy cleanup. Completed local verification is tracked in the AA2 closeout. Formal Gate AA2 acceptance remains open under E-003 while the failed AA1 dependency is unresolved.

Outcome

The candidate has a template-independent structured resume derived from verified evidence. The original remains immutable, edits create child versions, and every factual value has provenance and confirmation state.

Vertical slices

• AA020: Add canonical resume/version/fact persistence and migration recovery notes.
• AA021: Map verified extraction into contact, section, entry, role, date, bullet, and skill objects.
• AA022: Add version lineage, optimistic concurrency, and append-only change events.
• AA023: Add candidate fact verification and unsupported-fact rejection.
• AA024: Add server-loaded structured resume read model and authorized version comparison.

Gate AA2

Import, correction, concurrent edit, rollback-as-new-version, deletion/export, ownership, and audit tests pass. Historical facts retain their source and confirmation state. A corrected value starts unconfirmed without the former value's citation; the prior version preserves that citation and confirmation.

7. Phase AA3: Job intelligence and domain packs

2026-10-12 local implementation: The owner requested AA3. AA030–AA036 engineering paths and the /analysis/:id/requirements review workspace are implemented and locally tested. Status and receipts separate the two-family synthetic regression from representative acceptance. The first domain pack is a review candidate; no taxonomy release or market dataset is adopted. Formal Gate AA3 remains open for pack review, source/mapping validity, representative annotation/sample evidence and calibration. Live model adoption/variance/cost are not claimed from synthetic adapter tests.

Outcome

A supplied job description becomes a versioned, evidence-linked rubric with typed requirements, required/preferred/contextual importance, and occupation/skill mappings.

Vertical slices

• AA030: Persist structured JD requirements with exact source spans.
• AA031: Add deterministic parsing for explicit years, education, certifications, location, authorization, and requirement cues.
• AA032: Add bounded model-proposed requirement structure with evidence validation and uncertainty.
• AA033: Add versioned O*NET/ESCO ingestion and concept mapping boundary.
• AA034: Build the first domain pack and explicit unsupported-domain behavior.
• AA035: Add optional dated market-corpus contract without collection or scraping.
• AA036: Build job-intelligence benchmark and annotation report.

Gate AA3

Typed-requirement, importance, mapping, and evidence gates pass per released domain/locale. Market frequency remains disabled unless a permissioned dated dataset exists.

8. Phase AA4: Deterministic scoring v2

Outcome

ATS Compatibility and Job Alignment are deterministic, versioned, evidence-backed, confidence-gated, and separated from qualitative model output.

Vertical slices

• AA040: Introduce score-policy and dimension contracts with legacy adapters.
• AA041: Implement Evidence Confidence and no-score gating end to end.
• AA042: Implement continuous ATS Compatibility dimensions and evidence trace.
• AA043: Implement typed Job Alignment requirement states and no-double-count semantics.
• AA044: Remove LLM category scores from authoritative numeric scoring and relabel historical results.
• AA045: Add score explanation/read models and optional Application Readiness experiment behind a flag.
• AA046: Build scoring-validity, stability, metamorphic, and fairness benchmarks.

Gate AA4

All deterministic, evidence, confidence, per-domain expert-agreement, counterfactual, and legacy-compatibility gates pass. Product/UX review confirms the score is not understood as an employer decision probability.

9. Phase AA5: Same-page authoring and grounded writing

Outcome

The result page becomes an editable resume workspace. Every proposed change is evidence-linked, reversible, candidate-controlled, and rescored after acceptance.

Vertical slices

• AA050: Add result-page structured outline and focused line/section editing.
• AA051: Add finding-to-line navigation and ranked impact/effort queue.
• AA052: Add structured suggestion contract with source fact IDs and affected rule/requirement IDs.
• AA053: Add clarification questions for missing result, measurement, scope, and method.
• AA054: Add XYZ and alternative bullet patterns without mandatory fabricated metrics.
• AA055: Add accept/edit/reject/undo and immutable change ledger.
• AA056: Add projected-versus-actual deterministic score delta.
• AA057: Add writer-grounding benchmark, adversarial prompts, and manual candidate review flow.

Gate AA5

Unsupported facts cannot enter an accepted version through the normal workflow; browser tests cover keyboard/screen-reader use, concurrent edits, provider failure, undo, and candidate correction. The small model is retained only if it passes the same locked writer benchmark and cost/latency report as alternatives.

10. Phase AA6: Verified PDF and DOCX rendering

Outcome

The canonical resume renders to approved, profile-appropriate PDF and DOCX templates. Artifacts are parsed back and receive a verified label only when critical parity passes.

Vertical slices

• AA060: Define renderer interface, template manifest, escaping, and artifact states.
• AA061: Implement semantic preview and one ATS-oriented PDF template.
• AA062: Implement one DOCX template using the same canonical content.
• AA063: Implement parse-back comparison and export verification report.
• AA064: Add A4/US Letter, overflow, page-count, link, and font/encoding policies.
• AA065: Build render-parity corpus and accessible-artifact evaluation.

Gate AA6

Critical-field and normalized-text parity pass for every released template/format/locale combination. Failure removes the verified label and blocks the artifact from the recommended download action.

11. Phase AA7: Sandboxed LaTeX rendering

Outcome

Candidates can generate, preview, download, and optionally inspect LaTeX source without exposing the worker or user data to arbitrary TeX execution.

Vertical slices

• AA070: Approve LaTeX compiler/runtime threat model and package allowlist.
• AA071: Implement canonical-to-LaTeX renderer with complete escaping tests.
• AA072: Implement isolated compilation with no shell escape/network/host access and hard resource limits.
• AA073: Add profile templates rather than one fixed student/technology layout.
• AA074: Add LaTeX diagnostics, preview, source download, and safe failure UX.
• AA075: Add compiled-PDF parse-back and accessibility/template benchmark.

Gate AA7

Security-owner sandbox review, malicious-input tests, resource exhaustion tests, render parity, accessibility evidence, and rollback plan pass before production enablement.

12. Phase AA8: Representative validity and staged claims

Outcome

Resumide earns one domain-level accuracy claim using a locked representative corpus, documented uncertainty, and independent expert review.

Vertical slices

• AA080: Acquire permitted data for the first domain under approved governance.
• AA081: Run independent annotation, adjudication, and agreement analysis.
• AA082: Freeze development/calibration/test partitions and near-duplicate controls.
• AA083: Calibrate scoring dimensions without using the locked test partition.
• AA084: Run locked extraction, requirement, scoring, writer, render, fairness, latency, cost, and failure evaluations.
• AA085: Publish a versioned model/ruleset/dataset card and limitations report.
• AA086: Promote only the achieved claim level and stage rollout with monitoring/rollback.

Gate AA8

The lower confidence bound for every required segment meets the approved gate. An aggregate pass cannot hide a failed segment. Independent reviewers approve construct validity and claim language.

13. Phase AA9: Recruiter and vendor calibration pilot

Outcome

An allowlisted pilot can define a transparent job rubric and compare Resumide extraction/matching with a named ATS environment without enabling autonomous rejection.

Vertical slices

• AA090: Design recruiter calibration, notice, override, correction, and appeal contract.
• AA091: Run one controlled named-vendor/version ingestion comparison with synthetic/consented data.
• AA092: Run one career-center or recruiter pilot with fairness, security, legal, and human-oversight review.

Gate AA9

Pilot evidence supports only the named domain/vendor/configuration. No automatic rejection, hidden score, or general vendor-equivalence claim is enabled.

14. Verification checkpoints

At every phase gate:

npm run format:check
npm run lint
npm run typecheck
npm run test
npm run test:integration
npm run build

Add the phase-specific benchmark command only after its task implements it. Before candidate rollout, run npm run verify:ci, the Accuracy-v2 aggregate command, production-build browser state matrix, privacy/security checks, hosted worker flow, and the required manual/domain-expert review.

15. Risks and mitigations

Risk	Impact	Mitigation
A polished number exceeds evidence	Critical	Separate dimensions, confidence gate, claim ladder, locked evaluation, human approval
OCR/provider confidence is miscalibrated	High	Provider bake-off, corpus calibration, disagreement preservation, candidate correction
LLM inserts plausible false experience	Critical	Fact IDs, candidate confirmation, deterministic verifier, accepted-change ledger
Keyword optimization becomes stuffing	High	Typed requirements, evidence strength, no duplicate credit, human-readable checks
Domain aggregate hides weak segments	High	Per-segment gates and lower confidence bounds
Real resume data creates privacy risk	Critical	Permissioned acquisition, minimization, anonymization, isolation, retention/deletion
PDF/DOCX/LaTeX looks correct but parses incorrectly	High	Parse-back comparison and verified label policy
LaTeX compilation becomes an execution boundary	Critical	Isolated compiler, allowlist, no network/shell escape, hard resource caps
Free-product AI cost grows	High	Deterministic-first pipeline, bounded small-model path, escalation by uncertainty, cost benchmark
Employer mode creates discrimination/selection risk	Critical	Separate gate, protected-feature exclusion, adverse-impact review, notice, appeal, human decision
Legacy scores are silently reinterpreted	High	Versioned legacy adapter, non-comparable label, migration/read-model tests

16. Human checkpoints

The workspace owner must explicitly approve AA0 before implementation. Later approval must name the evidence reviewed; “tests pass” is insufficient when the gate requires a real corpus, live provider, recruiter panel, ATS environment, security review, or accessibility review.

AA4 local engineering (2026-10-12)

Owner-accepted E-004 authorizes AA040–AA046 synthetic localhost engineering while upstream formal gates remain open. The versioned diagnostics engine/read model, source-bound readiness, technical traces, typed alignment, cited prose without numeric model authority, legacy presenter and validity/error/bootstrap harness are implemented. The default-off SCORING_V2_ENABLED route has no numeric scoring activation, no new storage migration and no readiness aggregate. See AA4 closeout for actual verification and the remaining formal calibration/domain/expert/fairness/comprehension obligations.

AA5 local authoring (2026-10-12)

Owner-approved ADR-0014/E-005 permits AA050–AA057 local engineering. The default-off analysis editor, finding navigation, strict suggestion/clarification contracts, grounded patterns, transactional authoring ledger, candidate undo/restore and synthetic adapter benchmark are implemented. Numeric deltas remain withheld and actual model/candidate/domain evaluation remains open. See AA5 closeout and authoring design.

• decisions/0014-aa5-local-engineering-sequencing.md: owner-approved local AA5 engineering under E-005; formal acceptance is separate.
• design/CANDIDATE_AUTHORING_WORKSPACE.md: candidate editor, grounded patches/clarification, ledger and private recovery/export behavior.
• evaluation/AA5_ENGINEERING_CLOSEOUT.md: implementation, final local verification and remaining formal AA5 gates.

Local candidate authoring (AA5)

Apply the existing migration chain including 20261012160703_candidate_authoring_transactions.sql to the isolated local Supabase stack. Enable AUTHORING_ENABLED=true alongside the AA4/canonical/evidence/JD flags. Open /analysis/:id/score to edit sections/facts, navigate findings, provide truthful clarification and review grounded bullet patterns. Accept/reject/undo/restore are explicit; native forms work without JavaScript and failed/stale saves retain corrections.

npm run test:aa5:local verifies actual local transactions, privacy and concurrent writes. npm run benchmark:writer-grounding compares known-synthetic injected adapters; -- --release emits a blocked receipt and exits 1. No actual model is selected or called. Numeric deltas remain unavailable pending AA4 calibration. See AA5 closeout.