feat(evolution): FKST Evolution first proof and control-plane decision logic - #5892
Open
chronoai-shining wants to merge 27 commits into
Open
feat(evolution): FKST Evolution first proof and control-plane decision logic#5892chronoai-shining wants to merge 27 commits into
chronoai-shining wants to merge 27 commits into
Conversation
FKST Evolution decides whether a repository's generated product artifacts still describe its current source. That decision rests entirely on the six fingerprints of the specification's section 17, so the fingerprints — not the generators — are the component whose being wrong invalidates everything downstream. They ship first, and alone. The specification fixes the SHAPE of a hash leaf but calls it "conceptual", deferring the exact serialization to implementations while requiring that it be documented and covered by test vectors. `src/hash.ts` is that documentation: fixed-width big-endian length prefixes, raw UTF-8 paths with no normalization, six-byte octal modes, and a distinct domain tag per leaf family so repository content cannot forge a generator input. `test/hash.test.ts` pins the resulting digests as vectors a second implementation must reproduce. Two decisions are worth calling out because the obvious alternative is wrong: - Duplicate manifest keys are rejected by a scanning pass rather than by JSON.parse, which resolves them last-wins before any reviver runs. Since the manifest projection enters the output fingerprint, a document carrying two `outputFingerprint` values would otherwise hash one and display the other. - A convergence condition that cannot be evaluated reports `not-evaluable` and downgrades the verdict, never passes. The specification says a repository is converged only when all six conditions hold, and an unevaluated condition does not hold — it is unknown. GitHub access sits behind a `GitHubPort` interface so the decision logic is testable without a network, and so the control plane can later supply an installation-token client without touching the decision at all. Part of #5866
Enrollment requires an explicit `source.productRelevant` set: the specification ships no default, and open question 40.16 defers the question of whether a defensible one exists to Phase 1 measurement. The asymmetry is why a guess is not acceptable here — a too-broad set is merely expensive and visibly so, while a too-narrow one fails silently, because the artifact that was never regenerated produces no signal and nobody files a bug for a thing that did not happen. The set errs wide within `backend/src`, `frontend/src` and `skills`, and narrow outside them. Tests are excluded from cycle admission but remain covered for provenance and remain evidence: excluding them stops a test-only commit from launching a video render, which is not the same as ignoring them. `intent/` is the human half of the model and is deliberately separate from `observed/`. Evolution reads both and writes only the second; the confinement check rejects `config.yaml` and `intent/**` in a sync pull request unconditionally. Keeping them apart is what stops an inference from silently becoming product strategy — the protected facts in `overrides.yaml` are exactly the claims that cannot be derived from source, and a generated artifact that contradicts one must block rather than resolve in favour of the inference. `source.branch` records the specification's `@default` sentinel even though the control plane cannot express it yet: `validate_branch_name` permits only `[A-Za-z0-9._/-]`. Recording the correct value keeps the gap visible instead of encoding today's limitation as the design. Part of #5866
The observed model is the machine half: one capability, "Queue work for a running session", and the journey that verifies it. Identifiers are opaque and allocated once, deliberately not derived from the title or path, so that a rename cannot produce a different identity by construction rather than by instruction. Every limitation recorded on the capability is a documented response of `backend/src/routes/canvas/work_item.rs` — the 400, 403, 409 and 422 branches — rather than an inference from reading the happy path. The work-authority limitation is the one most worth stating precisely: repository admin or maintain permission is NOT work authority, which is a distinction the endpoint enforces and a reader would otherwise get wrong. The change record is sharded by year and commit prefix so its location is computable from its identity alone. It uses the relation lists rather than `removed` for identity transitions: encoding a merge as two removals plus an addition would make generated release notes publish "features removed" about a product that lost nothing. Part of #5866
A journey is both product evidence and capture source. This one asserts that the work composer offers the session's complete effective label set and accepts a Markdown body verbatim, and the same passing run produces the three screenshots every other artifact derives from. That coupling is the point: a demo cannot drift away from the behaviour it claims to show, because the video cannot be produced by a run whose assertions failed. The journey lives under the Evolution root and is reached by pointing the test runner's `testDir` at it — the specification's "consumption" pattern. Copying generated files out to a conventional location would create a second maintained copy, so the runner is configured to look inward instead. Reusing `frontend/e2e/fixtures.ts` rather than copying it is safe and deliberate: those fixtures are the repository's single description of the API's wire shapes, and their only `@playwright/test` import is type-only, so importing them cannot pull a second runner instance into the journey. Three settings are pinned rather than defaulted, because the specification requires each capture to record its viewport, scale factor, locale and theme: a Playwright upgrade that changed a default would otherwise silently alter every screenshot while the fingerprints still reported them current. Retries are zero for the same reason — a result that only holds on the second attempt is a flaky demo, not evidence. `"type": "module"` has to be declared in the root manifest because that is the nearest one to `.fkst/evolution/journeys/`. The repository has no root-level CommonJS for it to affect. `@playwright/test` is pinned exactly to the version `frontend/` resolves to, so the runner cannot drift ahead and pull a second browser revision. Part of #5866
Both artifacts derive from capability cap_9d41 and journey jny_2f7c, and both embed the screenshots that journey captured, so the prose, the images and the assertions describe one state of the product rather than three. The documentation leads with the prerequisites table because every entry in it maps to a specific refusal. A reader whose button is missing or whose request failed can identify which requirement they are missing instead of guessing, which is what the intent file's "name the failure and its remedy" voice asks for. The skill is not reformatted documentation. It carries the source commit it was verified against, the exact request and response shapes, the side effects, a deterministic success check, and the failure table — and it names journey jny_2f7c as its conformance test, so a failing journey invalidates the skill rather than leaving an agent to follow stale instructions confidently. One claim in both is worth stating plainly because it is easy to get wrong and expensive to get wrong: a 201 alone does not mean the session is working the item. The durable pick-up acknowledgement is the claim. Part of #5866
The deck's editable Marp source is committed; the rendered PDF and the MP4 are not. The specification keeps large videos, PDFs and presentation binaries out of the Evolution root and stores exactly those formats as content-addressed GitHub Release assets, so the renderer writes them to build output and the manifest references them by hash. Two couplings keep the artifacts honest. The video's title card is the deck's own first slide, rendered from the same source, so the deck and the demo cannot disagree about what the product is called or what it claims. The caption track is generated from the journey's checkpoints rather than written separately, so a caption cannot describe a step the journey no longer performs. Caption alignment corrects for the gap between Playwright starting the recording at context creation and the journey's first line running. The lead is measured rather than assumed — the final checkpoint is followed by a known hold, so the leftover duration is the lead — and clamped, so a slow teardown cannot push every caption late. Captions are muxed as a subtitle track instead of burned into the picture: burning them in would make them impossible to turn off and would force a re-encode of the picture to substitute a localized track. Marp is driven against the Chromium that Playwright already installed for the journeys. Letting it fetch its own browser would change machine state outside this project, and rendering the deck with a different engine than the screenshots were captured with invites subtle disagreement between them. Part of #5866
The plan states what the generator produces and which capability and journey each artifact represents. It lives outside `.fkst/evolution/` on purpose: the specification classifies everything under that root as either human intent or Evolution output, and the plan is neither. Placing it under the root would also fold the generator's own configuration into the output fingerprint it computes. The Release entries carry no tag. The tag is derived from the input fingerprint and the asset name from the content hash, both by rule, so generator input cannot point an artifact at an unrelated Release or claim a name whose bytes do not match. The rendered deck PDF is recorded `required: false` while its Markdown source is required. It is reproducible from that source, so a missing render is a render problem — not a reason to report the repository as non-converged. Part of #5866
`detectToolchain` recorded the checkout's directory name alongside the tool versions. That is not a fact about the generator: it makes `generatorEnvFingerprint` differ between two clones of the same commit, and it publishes a local path in a file the repository ships. Only tool versions belong in the provenance record. Part of #5866
The manifest is the compact convergence record that replaces a database cursor or a hosted artifact registry: which trusted source state was used, which generator revision produced the outputs, all six fingerprints, which artifacts exist and where, and whether verification passed. `outputFingerprint` covers a canonical projection of the manifest itself, serialized as RFC 8785. Naming the scheme matters because these bytes are hashed — "sorted keys and no insignificant whitespace" leaves number formatting, Unicode escaping and duplicate-key handling unspecified, so two conforming implementations would disagree about the fingerprint of one document. Including the projection is what stops `verification` and `artifacts` from sitting outside the hash while the convergence conditions read presence and status out of them. Every verification entry records a null check-run id. That is the honest state rather than an omission: no control plane exists yet to publish one, and the corroboration rule treats a null id as a failure to corroborate, not as a pass. The verifier consequently reports CONVERGED_PENDING_CONTROL_PLANE, which is the truthful verdict for this deployment. Part of #5866
Records the ownership boundaries, the command sequence, and why artifacts are committed before the manifest — the manifest hashes the Git tree rather than the working directory, so an uncommitted edit must not be able to produce a manifest describing bytes no reviewer will ever see. Names the exit codes and, more usefully, states plainly why this deployment reports CONVERGED_PENDING_CONTROL_PLANE rather than CONVERGED: condition 4 cannot be evaluated without a control plane to publish the check runs it corroborates, and an unevaluated condition is not a pass. Part of #5866
The verifier computed the source tree at the current revision but the generator tree at the manifest's own `observedHead`. Condition 1 therefore compared the manifest's generator fingerprint against itself: a changed generator could never fail convergence, and the "generatorPinnedFingerprint moved" full-rebuild trigger could never fire. Editing a prompt, a template, or this toolchain would leave every enrolled repository reporting converged against artifacts its current generator would no longer produce. The bug was in wiring rather than in the decision, so the fix is structural: manifest assembly and condition 1 now share one `resolveInputs`, and cannot diverge by having two call sites drift apart. Caught by the toolchain itself — adding a README under `tools/evolution/` moved the pinned tree, and the verifier reported converged when it should not have. Part of #5866
The generator tree moved when the README and the condition-1 wiring fix landed, so the input fingerprint — and with it the derived Release tag — changed. Rebuilt rather than hand-edited: the tag is derived from the input fingerprint by rule, and the asset names from their content hashes. Part of #5866
`{commit}` was substituted with the branch head, so the resolved package
reference — and through it `generatorPinnedFingerprint` — moved on every
commit to the repository. Including the commit that writes the manifest. The
input fingerprint could therefore never match the one just recorded, and the
post-merge no-op that the entire self-trigger design rests on was unreachable:
each cycle's own manifest commit immediately invalidated it.
The distinction is invisible when packages live in their own repository, where
the branch head IS the package's commit. It only bites when a package lives
inside the source repository, as this proof's generator does. Resolving to the
last commit that touched the referenced path is correct in both cases.
Found by running the verifier against its own output rather than by reading
the code.
Part of #5866
Re-running the journey against an unchanged product produces different PNG bytes: measured maximum per-pixel luma difference 2/255, mean 0.006. The images are visually identical — it is headless Chromium's rasterization, not a product change. It matters because the output fingerprint hashes bytes, so a capture rewritten for no reason reads as managed-output drift and, under the bootstrap `block` policy, stops the lane. Until change-impact analysis decides when a capture is adopted — the rule the specification already states for video rendering, not yet applied to screenshots — the committed PNGs are canonical and a verification run's captures should be discarded. Recorded rather than worked around: a proof that quietly re-committed its own captures every run would report convergence it had not earned. Part of #5866
Headless Chromium's rasterization is not bit-reproducible: re-running a journey against an unchanged product yields PNGs differing by up to 2/255 per pixel. The output fingerprint hashes bytes, so re-committing those visually identical captures moved `outputFingerprint`, which convergence reads as managed-output drift and which the bootstrap `block` policy turns into a stopped lane. Every media-bearing repository would have sat in permanent false drift — the failure mode that made the media artifact classes unusable at scale. Journeys now write captures to build output. A separate adoption step decides which of them replace committed bytes, using the rule the specification already states for video rendering and leaves implicit for screenshots: regenerate only when a dependency moved. A capture depends on the product surface (the input fingerprint) and on the journey that drives it (the journey artifact's recorded content hash). The journey lives under the Evolution root, so editing it does NOT move the input fingerprint — without that second term a rewritten journey would keep publishing screenshots of a flow it no longer performs. Unknown hashes on either side count as changed. Failing towards regeneration is recoverable; failing towards "keep" lets a stale capture outlive the journey that justified it. When a refresh is wanted but no fresh bytes exist, the last known good artifact is preserved and the gap is logged rather than hidden. Part of #5866
…itory The spec ships no `source.productRelevant` default and defers the question to Phase 1 measurement, because the failure modes are asymmetric: a too-broad set is merely expensive and visibly so, while a too-narrow one fails silently. This replaces the guess with a number. The oracle replays first-parent history and scores several candidate selectors over the same commits, so they are directly comparable. Over 300 merged pull requests on this repository: narrow (source trees) 70.0% admission shipped (config.yaml) 71.3% broad (+ deploy, schemas, manifests) 74.7% everything (`**`) 97.7% Two findings. The curated sets span 4.7 points — the choice among them is close to a rounding error. `**` sits 27 points away, confirming the failure the fingerprint split exists to remove: admission and merge staleness are never both satisfiable under it on an active repository. So the expensive mistake is not curating at all, not mis-curating. `backend/src` alone drives 183 of 300 admissions, which is the shape a default should follow: the source trees, minus tests. Admission is decided by path intersection, the same method condition 5 prescribes, deliberately not a per-commit tree hash — the spec forbids hashing every commit in a range on every reconcile. A change plus its exact revert is therefore counted twice where a fingerprint comparison would count zero, so the oracle over-reports slightly. That is the safe direction for sizing. Part of #5866
Evolution names its trusted source branch `@default` so that a repository which renames or switches its default branch stops reconciling against the old one. `validate_branch_name` rejects `@`, so the sentinel was unexpressible. Added as a separate `parse_branch_ref` rather than by relaxing the validator. That validator is shared by trigger parsing, delivery grants and audit-argument bounds, where a `@`-bearing value is meaningless; admitting one there would widen a security-relevant input check for every caller in order to serve one. Branch NAMES stay exactly as strict as before, and a test pins that. An unknown sentinel is reported by name rather than falling through to the generic character-class rule, which would send the reader looking for the wrong bug. Part of #5866
…ents The router dispatched `installation`, `installation_repositories` and `issues` and dropped everything else, so a change reaching the trusted branch — the event class Evolution is built around — was only ever noticed by the periodic full resync, roughly ten minutes later. Each new class is a thin classifier in the shape of the existing issue nudge: decide relevance from the payload, enqueue the same `(installation_id, repo)` hint, return. The reconciler re-reads GitHub and decides what to do, so correctness still comes from level-triggered reconciliation rather than from event delivery. Relevance is decided without an API call. GitHub sends `repository.default_branch` on every event, so "is this the trusted branch" is answerable from the payload; resolving it per event would add a round trip to the hot path of every push in every installed repository. Two rules carry real consequence: - A published Release whose tag begins `fkst-evolution/` is ignored. Publishing a release otherwise means "full rebuild", and Evolution publishes its own artifact Releases — without the exclusion the two-phase publication protocol drives the loop rebuild -> publish -> rebuild. - A merged pull request produces both a `pull_request.closed` and a `push`. Both converge on the identical repository hint, so the pair yields one reconcile pass rather than two work items. A push whose payload carries no default branch nudges rather than drops: a spurious reconcile is cheap and idempotent, while a dropped one leaves the trusted head uncovered until the next resync. Part of #5866
The backend edits are product-relevant, so the input fingerprint and its derived Release tag moved. Part of #5866
The control plane needs the configuration for its own decisions — the trusted branch, the product-relevant selector that decides cycle admission and merge staleness, and the safety policy — so it cannot take the generator's word for any of them. Every rule fails closed, and two are worth naming because the tempting alternative is wrong: - An absent `source.productRelevant` is an error, not a default. No default set exists, and an empty one silently disables all cycle admission — the failure mode that produces no signal at all, because the artifact that was never regenerated tells nobody. - `deny_unknown_fields` throughout, so `allowProductionDat: false` is a parse error rather than a policy that reads as active while the default silently applies. It is also what implements "a managed output may not carry a destination": the subtree is fixed by schema, so any extra key is a failure. The rules mirror `tools/evolution/src/config.ts` case for case, and the tests mirror its tests, because two implementations of one schema drift silently otherwise. `serde_yaml` is declared directly rather than added: it is already compiled into the binary as a transitive dependency of `kube-client`, so this records an existing edge and introduces no new code — the lock gains one line and zero package entries. Part of #5866
…ge hook The repo-level auto-merge hook merges any mergeable bot pull request once ANY session on the repository has opted in. Evolution's merge is the opposite: path-scoped to `.fkst/evolution/`, pinned to the current source head, and gated on an Evolution-owned required check. Enrolling Evolution on a repository that already had an auto-merge session would hand the artifact pull requests to the generic path and bypass every gate that makes autonomous artifact merging safe. This is a prerequisite for the first phase that writes anything, not a later cleanup, so it lands before any sync pull request can exist. Identification is by head branch — the lane's branch name is Evolution-owned by construction — paired with a head-repository check. That pairing is not optional diligence: a fork's head ref is a bare branch name the contributor chooses, so without it anyone could name a branch `fkst/evolution/<16 hex>` and opt their own pull request out of the repository's configured behaviour. Both cases are pinned by regression tests. Part of #5866
Evolution may write under `.fkst/evolution/` and nowhere else — never `config.yaml`, never `intent/**`, never `.fkst/packages/`. This lives in its own module, apart from configuration parsing, and that separation is the point rather than filing preference. `config.yaml` is repository content, so a boundary derived from it would be a boundary the thing it bounds gets to move. Keeping the comparison fixed and configuration-free is what makes "repository content cannot expand the agent's write boundary" true instead of aspirational. A `..` segment is refused outright. Git trees never produce one, so it can only arrive from generator-supplied input — and `.fkst/evolution/../../backend/src/main.rs` passes a naive prefix test while naming a file outside the root. Comparison is by prefix rather than substring for the same reason, so `src/.fkst/evolution/x.md` does not pass either. Violations are returned as a list rather than reduced to a boolean: an operator told only "confinement failed" has to go looking for which path. Note this is a merge-time veto, not a write prevention. An installation token has no ref or path scope, so the credential cannot stop the write; a branch ruleset stops the direct-push bypass, and this detects the rest. Part of #5866
Autonomous merge must compare the recomputed input fingerprint against the manifest immediately before merging, and no native GitHub mechanism can do that: once armed, native auto-merge merges when required checks turn green regardless of what happened to the base branch in the interim. So the gate is a check run the control plane owns and re-evaluates on every reconcile. Base advancement does not change a pull request's head, so nothing in GitHub re-evaluates it on its own — the level-triggered reconcile is what flips it. The `neutral` conclusion on non-sync pull requests is the part that must not be skipped. A ruleset's required-status-check rule conditions on the protected REF; it has no head-branch, author or App condition. Once this check is required on a branch, every pull request targeting that branch must report it — and with the default `artifactRepository: "."` that branch is the product repository's own default branch, the base of every ordinary human pull request. Publish only on the sync pull request and every human pull request sits forever at "Expected — waiting for status to be reported", unmergeable by anyone, in the repository Evolution exists to document. Neutral is also the truthful answer: Evolution asserts nothing about those pull requests. Checks are ordered by how actionable their report is, not by convenience: an unconfined path set is a different problem from a stale fingerprint, and reporting whichever was tested first would send the reader after the wrong one. Two tests pin that ordering. Part of #5866
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked Issue
Closes #5866
Closes #5891
Summary
Builds FKST Evolution's first working slice from the draft specification on the
evolution-temp-specbranch: a vertical proof that carries one product capability all the wayto auto-generated documentation, an agent skill, screenshots, a captioned demo video and a slide
deck — plus the control-plane decision logic those artifacts will eventually be gated by.
Nothing here is enabled for any repository. Evolution has no reconcile loop yet, so the new
evolutionmodule is decision logic the reconciler will call, not behaviour that runs today. Theonly live change to existing behaviour is that four webhook event classes now reach the reconciler
instead of being dropped, and that Evolution's own sync pull requests are excluded from the generic
auto-merge hook.
What it produces
Capability
cap_9d41("Queue work for a running session") → journeyjny_2f7c→ six artifactclasses, all derived from one Playwright run against the real SPA using the existing
frontend/e2esynthetic fixtures. The same passing run that verifies the capability produces thescreenshots and the video frames, so a demo cannot drift from the behaviour it claims to show.
Rendered binaries (PDF, MP4) are content-addressed GitHub Release assets, not committed files.
Three defects the toolchain caught in itself
revision but the generator tree at the manifest's head, so condition 1 compared the manifest
against itself. Any prompt or template edit would have left every repository reporting converged
against artifacts its generator would no longer produce. Fixed structurally — build and verify
now share one
resolveInputs.{commit}to the branch head, so thegenerator fingerprint moved on every commit — including the commit that writes the manifest,
which instantly invalidated it. Now resolves to the last commit touching the package path.
Chromium rasterization). Since the output fingerprint hashes bytes, re-committing visually
identical captures read as managed-output drift and, under the bootstrap
blockpolicy, stoppedthe lane. Captures are now adopted only when their inputs moved.
Two specification open questions, settled with evidence
fkst-substratedependency. Thepackagesbranch has noqueue_mode,max_in_flightor any concurrency limit; the lane lock is atomic ref creation instead. This wasthe item that could have made the rollout depend on a repository CLAUDE.md forbids us changing.
70.0 / 71.3 / 74.7 % while
include: ["**"]admits 97.7 %. Curated sets span under five points;**sits 27 away.Both are recorded with their working on #5866.
Type of Change
Affected Area
No product code changes. The journey exercises the SPA through its existing E2E fixtures; nothing
under
frontend/srcis touched.How to Test
verifyreportsCONVERGED_PENDING_CONTROL_PLANEand is byte-identical on a second run.Condition 4 is
not-evaluablerather than passing, because no control plane exists yet to publishthe check run it corroborates — the honest verdict, not an omission.
Reviewer notes
Three changes reach outside the new module and are worth a deliberate look:
package.jsonat the repository root gains"type": "module"and a pinned@playwright/testdevDependency. The generated journeys live under.fkst/evolution/journeys/,whose nearest manifest is the root one, so the flag has to be declared there. The repository has
no root-level CommonJS for it to affect —
backend/is Rust andfrontend/declares its own.Playwright is pinned exactly to the version
frontend/resolves to, so the runner cannotdrift ahead and pull a second browser revision.
serde_yamlbecomes a direct dependency. It is already compiled into the binary as atransitive dependency of
kube-client, so this records an existing edge: the lock gains one lineand zero package entries.
tools/tree. The generator is human-owned source, deliberately outside.fkst/evolution/— everything under that root is either human intent or Evolution output, andfolding the generator's own configuration into the output fingerprint it computes would be
circular. Per §28.1 these producer roles ultimately belong in
fkst-packages; moving them isfollow-up work.
Checklist
Closes #N).develop(notmaindirectly).Evolution) built in 27 self-contained commits, each reviewable on its own and each leaving the
tree green. Splitting it further would separate the proof from the toolchain that verifies it.
fmt,clippy -D warnings, full workspace suite, 100 toolchaintests, 74 Evolution module tests, gitleaks clean.
Known gaps
Deliberate, and recorded in
tools/evolution/README.md:GitHub I/O is not.
verifyrun twice stands in for the restart-and-reconcile no-op proof.checks: writeon the GitHub App. Adding a permissionplaces every existing installation in a pending state until an account owner approves it, so
nothing downstream of the gate can be exercised against a real repository until that happens.