Skip to content

Latest commit

 

History

History
355 lines (310 loc) · 17.7 KB

ARCHITECTURE.md

File metadata and controls

355 lines (310 loc) · 17.7 KB

Architecture

Status: the system as implemented. Where this page and a numbered decision disagree, the decision wins. The blueprints it grew from are kept, superseded, in docs/history/.

Brokkr is a deterministic process manager wrapped around stochastic, fallible effects. You hand it a feature and a recipe: a delivery strategy as reviewable data, identified by content digest. The engine drives agent sessions through the recipe's phases and rules on each typed result with a pinned first-match-wins table. Sessions are leaves; only the table selects a transition. Unknowns never advance: schema violations, unmatched results, exhausted retries and security findings park or stop the run with the raw evidence attached. Every claim — done, verified, parked, stopped, paid — is a journaled fact that replays byte for byte, and the operator's judgment enters only as journal events. They are hash-chained and unsigned (decision 0008 defers signing): a verified chain detects an edit that left the hashes stale, not a rewrite that recomputed them, nor who wrote it.

The shape

flowchart TB
  operator([operator]) -- "run · resume · retry · stop" --> cli
  subgraph binary["the brokkr binary — how far a delivery advances"]
    cli["brokkr-cli<br/>commands · embedded UI · driver entry"]
    runtime["brokkr-runtime<br/>engine loop · bundles · recovery · the boundary"]
    core["brokkr-core — PURE<br/>envelope · hashing · fold · policy"]
    store[("brokkr-store<br/>SQLite journal, hash-chained")]
    protocol["brokkr-protocol<br/>forge-driver/v1 over stdio"]
    view["brokkr-view — PURE<br/>one display derivation"]
    bridge["brokkr-bridge<br/>Looper dispatch over verified journals"]
    cli --> runtime & view & bridge & core & store & protocol
    runtime --> core & store & protocol
    view --> core
    bridge --> core & store & runtime
    store --> core
  end
  protocol -- "NDJSON" --> harness([Claude Code · Codex · dsh · LaneTally · exec<br/>capability, as leaf effects])
  seatbelt-probe["brokkr-seatbelt-probe<br/>test-support"]
Loading

Every edge is a declared dependency, every [dependencies] entry between workspace crates is an edge, and every crate is drawn (decision 0037).

brokkr-core performs no I/O, clock reads, randomness or process execution, so state and ruling replay exactly; effects sit above, journaled. brokkr-view, equally pure, renders one answer, HTML or terminal text, from brokkr-core, serde, serde_json and unicode-width alone (decision 0013). Gates hold both against mistakes, not adversaries (decision 0071 ruling 1): clippy.toml, deny.toml bans, brokkr-cli/tests/layering/.

Brokkr decides how far delivery advances. Product priorities are decided above it; costs are measured beside it from seat ids and checkpoints; harnesses below supply capability. None overrides another layer's authority.

The journal is the run

flowchart LR
  e1["seq 1<br/>run/started"] --> e2["seq 2<br/>phase/entered"] --> e3["seq 3<br/>effect/requested"] --> en["seq n<br/>…"]
  e2 -. "previous_hash" .-> e1
  e3 -. "previous_hash" .-> e2
  en -. "previous_hash" .-> e3
  en ==> fold{{"fold(events)"}} ==> state["RunState<br/>phase · control status · protocol cursor · counters"]
Loading

Every event is a hash-chained envelope (contracts/event-envelope.v1.schema.json): seq contiguous from 1, previous_hash chaining, event_hash over the canonical bytes, causation_id naming the event that caused it. The fold derives everything, and an event impossible at the current cursor fails it closed: a journal that violates the protocol is corrupt, not reinterpretable. Replay is byte-deterministic. Resume is replay.

flowchart LR
  db[("forge.db<br/>append-only triggers · chain built inside the append transaction")]
  db -- "export" --> ndjson["canonical NDJSON + pinned manifest"]
  ndjson -- "verify-run" --> offline["chain · envelopes · fold — offline"]
  ndjson -- "import" --> other[("another journal<br/>byte-identical · run-id collision refused · never merged")]
  db -- "anchor" --> ref["refs/forge/‹run›<br/>tamper evidence, unsigned"]
Loading

A concurrent writer conflicts instead of forking history. Import relocates one run; journals never merge (decision 0027). The anchor is evidence, not proof: the ref is unsigned, and decision 0008 defers the signing service.

Every effect, in order

stateDiagram-v2
  direction LR
  [*] --> requested: effect/requested committed
  requested --> started: effect/started durable, then the driver spawns
  started --> started: effect/checkpointed
  started --> succeeded: effect/succeeded
  started --> failed: effect/failed
  started --> indeterminate: effect/indeterminate · crash · driver vanished · in flight at restart
  succeeded --> [*]: typed result to the table
  failed --> requested: attempts < max_attempts
  failed --> parked: run/parked · attempts exhausted
  indeterminate --> parked: run/parked · never auto-retried
  parked --> [*]: awaiting_operator, raw evidence attached
Loading

The order is the durable outbox: the request is committed before execution, the start is durable before the driver spawns, checkpoints land as they arrive, and each attempt ends in exactly one terminal fact. A crash at any boundary recovers without losing a committed fact and without turning an uncertain effect into a success. Seat input is a pure function of the journal and the pinned bundle; recovery rebuilds it and refuses to execute anything whose digest differs from what was requested.

Autonomy is bounded (decision 0006). Determinate failures retry up to max_attempts, deadline kills by the watchdog included. indeterminate never retries, because a retry could duplicate or re-pay for finished work. Exhaustion, schema violations, unmatched results and unknown anything park the run with the raw evidence attached — never repaired, coerced, or handed to a model to fix (decision 0001). Operator commands are journal events, not prose. operator supersede records the named residual findings an operator closes, with the actor and optional closing-run citation (decision 0047); it does not rewrite a past verdict or resume a completed run.

Policy is data

flowchart LR
  result["typed result<br/>from the seat"] --> inputs
  engine["engine-owned inputs<br/>consecutive_failures · drift_detected · dirty_worktrees · reviewed_heads<br/>strategy · fixes_docs_only · realm_facts · visits_‹phase›"] --> inputs
  declared["inputs the seat declared"] --> inputs
  other["anything else"] -- "dropped before the table or the record" --> bin["nothing"]
  inputs["evaluation inputs"] --> table{{"policy.json<br/>first match wins · closed vocabulary"}}
  table --> next["next phase"]
  table --> park["park → awaiting_operator"]
  table --> stop["stop"]
  table -- "nothing matched" --> park
Loading

The strict core evaluates the table first-match-wins (decision 0004). The condition vocabulary is closed and checked at load, so a misspelt key refuses to load rather than silently never firing. An absent input never satisfies a condition; an unreadable one parks. Provenance is compile-time (decision 0007): engine-owned inputs overlay anything a seat claims, declared inputs pass, everything else is dropped before it reaches the table or the record.

The outer machine is a linear state machine by constitution (decision 0002): one active phase, a totally ordered journal. This is the fast recipe's table, the one the front page runs:

stateDiagram-v2
  direction LR
  [*] --> implement
  implement --> implement: broken, once
  implement --> verify: complete
  implement --> review: docs-only returned fix
  verify --> review: pass
  verify --> implement: fail, bounded by visits_implement
  review --> ship: clean · low/info residual as named debt
  review --> implement: residual above low, bounded by visits_implement
  implement --> parked: blocked by its environment
  review --> parked: reforging exhausted
  ship --> ship: ready
  ship --> review: HEAD drifted
  ship --> done: shipped
  implement --> stop: blocked by the work · broken twice
  verify --> stop: fail at exhaustion
  review --> stop: security-hold · above-medium residual at exhaustion
  ship --> stop: dirty tree
  parked --> [*]: awaiting_operator
  done --> [*]
  stop --> [*]
Loading

brokkr compile rejects any table where the protected review phase can be skipped on a path to a non-stop terminal. The way back is bounded in the machine's own vocabulary (decision 0022): a rule may read visits_<phase>_gte, the same count the graph renders as ×N, and a rule may rule a park instead of a phase. The seat a run returns to receives the result that sent it back as context.returned_from.

The world is chosen at invocation (decision 0023): realms.json names the repositories a run may see and the journal they share, is pinned by content hash into the run manifest, and resume rehydrates it from that pin rather than from the disk. A crossing is a file one realm publishes and another pins by its bytes, verified at world load (decision 0057).

A bundle, resolved

flowchart LR
  base["recipe: base"] -- "extends" --> leaf["recipe: derived<br/>differences only · override / remove markers"]
  leaf --> compose{{"compose — pure, leaf-first"}} --> flat["ONE flat bundle<br/>policy · seats · charters"]
  flat --> resolve{{"resolve"}}
  agents["agents/‹name›.json<br/>charter · model chain · limits · inputs · tools"] --> resolve
  adapters["adapters/‹provider›.json<br/>model ids · permissions · trust_tier · egress class per route"] --> resolve
  resolve --> checks{"gate site → trusted tier?<br/>secret bindings → route class ≥ minimum?<br/>every declaration expressible?"}
  checks -- "no" --> refuse["compile refuses, naming agent · provider · capability"]
  checks -- "yes" --> manifest["run manifest<br/>digest pins recipe, chain and adapters<br/>resume uses exactly this or refuses"]
Loading

A bundle is reviewable text: policy, one seat per phase, pinned by content digest. A seat is a single session or a panel: members fan out inside one effect, join as a barrier in declared order, and a closed-vocabulary aggregate produces the one typed result the machine sees. On a parked panel, operator retry --member <tag> re-runs the named members and carries settled verdicts from their original attempts only when the reviewed head, clean tree, pinned bundle and inputs still match, at both command and execution time (decision 0093). A sequence member uses <step>:<member>; only a first-step panel can carry. The same aggregate and policy route the joined result. The journal and view name each carried verdict's source attempt; plain retry runs every member. A run driven with --secrets-file uses the same store on the member retry.

Composition (decision 0017) is a pre-pass over recipe sources that resolves the chain into one flat bundle before anything is parsed: no inheritance at run time, no dynamic lookup. Named things merge by name, redefining needs the marker, removal fails if its target is absent, and the constitutional lint runs on the resolved table.

Agents and adapters are data (decision 0016). Unsupported capabilities are explicit refusals. Each driver-bearing site declares work or gate; gates require a trusted adapter. Secret bindings require the resolved route to meet the bundle's egress_minimum (decisions 0021 and 0036).

Egress classes are local, contracted and uncontracted, declared for the adapter's destination and separately for each route it serves. Unknown tiers are untrusted; unknown classes and routes are uncontracted. A local endpoint gets no gate authority merely by being local. Fallback is bounded to failure before acceptance; sessions cannot switch models midway. Authorising adapters are pinned, so a changed trust declaration changes bundle identity.

Triage, the council and the SDD route

recipes/triage opens with a chief-grade triage gate that rules a closed delivery class: chore, feature, design, engine or escalate. The fold keeps it as the engine-owned strategy, and select bodies read it to seat the later offices (decision 0041). design and engine take the spec-driven route of decision 0042: specify, clarify, design, tasks and analyze, then implement. The council is Brokkr's: position seats argue as a panel and a chief architect drafts. The dialect is the realm's: realms.json names one, such as OpenSpec, whose validate and check tools Brokkr pins and runs as boxed exec gates; the dialect's own workflow never drives the machine. Returns are bounded by visits_<phase>: a spec defect goes back to specify, an oversized implementation back to triage.

A dialect's artifacts track one slice of work and are never the realm's living truth (decision 0081): the decisions rule, this page maps what is built, and the code and tests are the executable contract. This repository speaks OpenSpec. A change under openspec/changes/ is a slice in flight; openspec/specs/ is the tool's record of the requirements archived slices left, not a contract, and where it disagrees with a decision or the code, the decision and the code win. A change is archived by the dialect's archive step when every task in its tasks.md is ticked, and a test refuses a finished change left in place. An evidence ledger belongs in docs/evidence/ or a run's refs/forge anchor, not in a change directory.

Drivers

sequenceDiagram
  participant E as engine
  participant D as brokkr driver ‹kind›
  participant H as harness
  E->>D: hello
  D-->>E: capabilities
  opt the driver supports resume
    E->>D: resume — the session it opened
  end
  E->>D: start — seat prompt, result path, deadline
  D->>H: spawn on the host
  D-->>E: accepted
  loop each turn
    H-->>D: session stream
    D-->>E: checkpoint — bounded: turns · tools · usage · cost
  end
  H-->>D: exit, typed result file
  D-->>E: result
  E->>D: shutdown
  opt answered by the adapters, never sent · a deadline kills the process
    E->>D: cancel
    D-->>E: cancelled
  end
  Note over E,D: unknown message types fail closed · a driver that vanishes after accepted leaves the attempt indeterminate
Loading

Drivers speak forge-driver/v1 (contracts/driver-protocol.v1.schema.json): NDJSON over stdio, stdout protocol-only, stderr captured as an artifact. The adapters for Claude Code, Codex, dsh, LaneTally's Claude Code wrapper and any prompt-in/result-file-out harness (exec) are built in as {brokkr} driver <kind> (decision 0009), while the protocol stays language-neutral for third-party drivers. What stands around a seat's hands is the realm's boundary (decision 0046): namespace, seatbelt, container, harness or open, pinned per site, rendered unboxed under harness or open; hands-less seats get none. Decision 0008's driver.confine is refused (0046 ruling 5) until slice (iii) builds container.

Built today: namespace (Linux/WSL2, bubblewrap), harness and open; seatbelt and container refuse at start. Harness gates need a measured adapter fragment, so not every shipped recipe runs under harness (status). The manifest pins the selected boundary, and CLI, TUI and web readouts retain that fact.

Verification, in layers

Layer What it pins
Differential corpus A frozen 97-case corpus in fixtures/ pins the evaluator: contract data, never regenerated.
Machine proof End-to-end scenarios drive the real binary and real subprocess protocol through success, retries, stops, parks, crash recovery at every durable boundary, panels, boxed hands and bundle pinning.
Self-delivery recipes/self lets the engine deliver changes to this repository; shipped is the sole entry into done, and the operator keeps push and merge.
Brokkr verification recipes/verify examines a delivered change with a verify seat namespace boxes and an unboxed review seat under the operator's Claude Code permissions (security model). It has hard-stopped its author's work on a real security finding.

The operating surface

Every verb, flag and exit code is in the CLI reference, rendered from the binary's own definitions. brokkr ui, brokkr tui and brokkr inspect are three renderers over the same brokkr-view models (decision 0014): read-only, no operator command, nothing written to the journal. brokkr costs reports per-seat attempts, turns and USD from journal checkpoints, keyed by the stable seat ids a cost ledger can join on.

Release preparation

release-manager prepares versions, commit-derived notes, documentation and organization-profile patches. Its charter is portable; the realm's house file supplies extensible configuration, covered by the existing house digest and prompt assembly. Configuration describes required work and evidence; it does not execute commands or add checks to a gate.

recipes/release combines the manager and library reviewer with fast's policy and namespace-boxed exec gates. The shipped verifier is Rust-specific; another stack replaces it through recipe composition. External patches and base commits are reviewed in the handoff. Publication and cross-repository application are verified separately from local run completion.

Run-level gate entry

Proposed decision 0094 keeps entry out of the bundle manifest. One typed admission module checks the compiled table and whole-gate assessments; the engine reads the journaled entry at its start cursor and rechecks it on resume. Core owns the direction predicate, reachability walk and entry reader. Runtime observes continued runs; view derives the header sentence and rail order. The CLI, TUI and browser only paint those facts (decision 0071 rulings 1, 3, 5 and 7).