Repository navigation
design(ast): fix the role inventory and prove the role-table macro under the gates #1691
Description
Objective
Fix the complete role inventory and prove, with compiled code, that one role_table! declaration can express every row shape the design needs and pass the project's gates — before any consumer is migrated.
Umbrella: R00: #1690 · Planning ID: R01: #1691
Completion prerequisites: None; this is the first work package.
Status: implemented on branch refactor/1691-role-table-proof, rebased onto origin/main e1137739, as two commits: 0f49f387 (the role tables) and b9ea546f (a pre-existing Elixir quadratic the performance review found while sizing the rows' gates — see the comments). Not yet pushed or merged.
Why this exists
The umbrella's row shape (owners, payers, projection, self-paying owners, SpaceKind mapping, text gate) was designed on paper and reviewed on paper. Four things can only be settled by compiling: that a gated kind expands to its own arm (K => gate(..)) rather than a guard on a shared pattern; that opens_space can be emitted without repeating the kinds FuncOpener/Container already name; that text_gated_by emits only the (node, code, ancestors) spelling; and that the expansion passes clippy::pedantic -D warnings (match_same_arms in particular) and the rustfmt-bail gate — a metavariable in a pattern position makes rustfmt skip the whole match silently (.claude/rules/formatting.md, "Two causes, one measurement"). The design review named this the first thing to settle. "DSL" below means the row grammar the macro accepts.
The role inventory is fixed here too, because R02: #1692's census iterates it and a role added later would reopen every table. The set must include everything R03: #1693–R09: #1699 consume.
Implementation steps
-
big-code-analysis-ast/src/roles/mod.rs: therole_table!macro and the fixed role set —Call,StringLit,Comparison,ShortCircuit,EagerLogical,Ternary,NullSafe,CaseArm,Guard,Branch,Switch,Loop,Catch,ElseIf,Exit,Modifier,DeclKeyword,Primitive,FuncOpener,Closure,Container,Member,Comment,DocComment,Docstring,DocTarget,DocRunOpener,DocRunDirective,HoldsDocInBody— each with a one-line meaning in the module doc. Generated names come from a fixed mapping (the@namedarms), not identifier concatenation. - Two membership sets per row: raw (
OWNERS/PAYERS/EXCLUDED/WRAPSconsts, every kind the row names, for the coverage ratchet) and effective (owns,pays_at_leaf,pays_at_wrapper; a self-paying owner is in both effective sets and is the one permitted overlap). The disjointness check is on the declared lists. - Two structurally named payer projections,
pays_at_leafandpays_at_wrapper. Amended from the proposal: a row has three paying modes rather than "each defaulting to the single payer list" — apays [..]list splits the projections (leaf = payers +pays_itselfowners, wrapper = owners); no list means the owners are the paying nodes under both;pays nevermeans neither. Which projection a metric reads is decided in the root crate. The Python comparison is the fixture. -
wraps [kinds]onStringLitrows, emitted asstring_lit::WRAPSforGetter::string_operand_type. Amended: no stand-in was needed — tree-sitter-java 0.23.5 emits a realstring_interpolationchild ofstring_literalforSTR."a \{x}", verified withbca dumpand pinned by a parsed fixture. -
roles/java.rswith one row of each shape: plain (Exit), gated (LT/GTunderbinary_expression), projected (BinaryExpression→project "operator"), self-paying (InstanceofExpression(pays_itself)), mapping (Container), gated mapping (ObjectCreationExpression(opens Class gated_by has_class_body)),FuncOpenerandContainersharing oneopens_spacewithout restating kinds,pays never(onGuard, a stand-in, said so on the row),excludes [QMARK "…"],absent,pending. -
roles/elixir.rswithtext_gated_byrows (FuncOpenerondef/defp/defmacro/defmacroptarget bytes outside aquoteblock;Containerondefmodule). The macro emits no byte-less spelling at all, so the text path cannot emit one by accident. - Reject at expansion time, with a readable error: a role both declared and
absent(module-name collision); a kind in both owners and payers (constassertion over the slices viaroles::kinds_overlap); a kind in two rows of the same role (same collision). Added on review:pays_itselfon apays neverrow (compile_error!: contradicts) and on a row with nopayslist (compile_error!: redundant). - Nothing wired into a consumer;
#![allow(dead_code)]on the module with a comment saying R03: #1693 removes it.
Design decisions (the DSL is fixed by the decision note in roles/mod.rs)
| Proposed | Shipped | Why |
|---|---|---|
Role => … rows |
Role { … } blocks |
check-rustfmt-bail.py reads an indented … => line as a match arm; probe reported 2 permanent bails |
gated kind as its own match arm |
its own if node.kind_id() == K { return gate(..) } clause |
a match whose arms sit inside $( … )* is a permanent rustfmt bail (probed: 2 stuck arms); same-body arms trip match_same_arms |
bare owner suffixes (K pays_itself) |
parenthesised (K(pays_itself)) |
bare suffixes after $owner:ident hit local ambiguity with tt repetition |
Excluded(role, reason) standalone entry |
row-level excludes [K "why"] → <role>::EXCLUDED |
a standalone entry needs a second per-role name table for nothing it adds; the exclusion sits beside the row whose author rejected the kind (maintainer decision) |
| projections default to the single payer list | three paying modes (split / owners / never) | a plain row answering wrapper=true, leaf=false would have been a contradiction the first review caught; the mode makes a split visible only where declared |
text_gated_by as a distinct path |
same expansion as gated_by, distinct keyword |
the signature (node, code, ancestors) is the only one emitted, so the distinction is documentation at the row |
FuncOpener row lists Java's lambda (get_space_kind maps it to Function) |
lambda is Closure only |
the walk promotes on promotes_to_func_space_with_code, which excludes lambdas; the getter's answer is never read for one. The live is_func_space / get_space_kind disagreement is recorded on the row (grammar-dispatch §7) |
each kind's clause reads node.kind_id() |
read once per emitted fn, carried in the macro's context bundle | the read is a non-inlinable FFI call; one per clause blocked the bitmap codegen matches! gets (12 calls → 1 in java::opens_space) |
Not expressible, by design: a kind claimed by the wrong role, an incomplete entry, whether a consumer reads the right projection (#1690, "What this does and does not claim").
Tests and edge cases
- Derived unit test per table: declared
owners ∩ payers = ∅per role (declared_owner_and_payer_lists_are_disjoint_per_role, ungated — it parses nothing); effective overlap only at self-paying owners (asserted per occurrence in the Java comparison test). - A parsed fixture per row shape, locating every occurrence of each claimed kind and asserting
owns/pays_at_leaf/pays_at_wrapperper occurrence: Java comparison (owner vs<token vs generic<,instanceofat both projections,&&chain owned by neither), projection on theoperatorfield, string templatewraps,opens_spacefold with lambda closure-vs-space,Guardpays-never, plain-vs-never both projections; Python wrapper-vs-leaf (a < b <= c,not in,is not,for … in), barein/isonly under the wrapper,lambdaclosure opens nothing,"a" f"{x}"concatenation; Elixirdefmodule/def/defmacro/quote'ddef/IO.putsper call, same bytes outsidequote,fn … endclosure and opener. - Six
compile_faildoctests + one compiling-shape doctest on the macro. - Perturbations (14, each restored and the tree verified):
under_binary_expression→ always-true fails exactlycomparison_row_separates_the_owner_from_its_tokenandangle_tokens_pay_only_under_a_binary_expression(two, not one as the issue predicted — the second fixture exists precisely to isolate the gate). Full table in the completion comment.
Acceptance criteria
-
cargo clippy -p big-code-analysis-ast --all-targets --all-features -- -D warningsclean. -
./utils/check-rustfmt-bail.py --showreports zero stuck arms for all fourroles/*.rs;cargo fmt --checkclean. - A decision note in the module doc: which shapes needed a separate arm, which coalesced, what could not be expressed.
Verification and completion evidence
- Clippy, rustfmt and rustfmt-bail commands with output; the perturbation result;
make pre-commitBCA_GATE:line — in the completion comment.
Activity
dekobon commentedon Oct 23, 2026
Tracking is ready
The body now links the master issue and the work packages it references. Native parent and blocked-by relationships match the listed completion prerequisites. Implementation and acceptance checkboxes remain open; prerequisite closure and the R04: #1694 go decision govern execution order.
dekobon commentedon Oct 23, 2026
Implemented
Commit 4d86542d on 02666d8, branch refactor/1691-role-table-proof. Not pushed; no PR. Files: the four roles modules (new), lib.rs (+1 line), CHANGELOG.md. The issue body's checkboxes and its Design decisions table are the record of what shipped and what deviated from the proposal.
dekobon commentedon Oct 23, 2026
Gates
dekobon commentedon Oct 23, 2026
Perturbations
Fourteen, each an exact edit, run with --no-fail-fast, restored and the tree verified. Every one failed only the named test(s): the Java angle-bracket gate fails two (the occurrence test and the fixture built to isolate that gate); Guard changed from pays never to plain fails three; every other case fails exactly one. One perturbation that passed was a finding: a first-draft Python not-inside-compound gate was redundant with the parent check and was removed, with the row comment saying why.
dekobon commentedon Oct 23, 2026
Review
astra-review, three rounds. Round 1, four blocking: a plain row answered its two projections differently and pays never still paid at the wrapper (fixed with three paying modes and a parsed both-projections test); Python StringLit dropped ConcatenatedString (added, fixtured); Java's Comparison owner gate admitted the short-circuit tokens (narrowed, asserted); metric-policy wording in row comments (scrubbed, rule written into the module doc). Round 2: pays_itself bypassed pays never (now a compile_error, with doctests). Round 3: no blocking findings; one missing doctest added.
Unresolved
None in scope. Tables are dead code by design until R03.
dekobon commentedon Oct 23, 2026
Simplify pass
Commit amended to a2bacb77 (same tree content plus a simplify-rust pass; no behaviour change). The three attribute scans in the macro now share one accumulator shape, and the paying mode is two const booleans read directly in pays_at_leaf and pays_at_wrapper instead of six dispatch arms. All fifteen perturbations re-run and still fail only their named tests. make pre-commit: BCA_GATE: pass (gate=pre-commit).
dekobon commentedon Oct 23, 2026
Test audit pass
Commit amended to 17385422. An audit-tests pass found the membership rows (Exit, Call, StringLit, Closure, FuncOpener, Container) were restated from the live classifiers with nothing pinning the restatement: eight mutations dropping a kind from a row failed zero tests. Added one parity test per table, membership_rows_agree_with_the_live_classifiers, asserting row::owns against is_call / is_string / is_closure / promotes_to_func_space_with_code / get_space_kind_with_code per node on a fixture that emits every declared kind, with a fixture guard. All eight mutations now fail exactly that test; the earlier fifteen perturbations still discriminate. The disjointness test now covers all three tables. 18 lib tests + 6 doctests; make pre-commit: BCA_GATE: pass (gate=pre-commit).
dekobon commentedon Oct 23, 2026
Four-reviewer pass and the fixes
Branch rebased onto main e113773; now two commits: 1e6fcd93 (role tables) and 3b07b5d9 (Elixir fix, below). Reviewers: the review skill, the rust-optimize skill, a performance agent, and astra-review on the least-reviewed surfaces, then astra-review again on every fix (two rounds, no blockers remaining).
Correctness (astra): Java's lambda is no longer a FuncOpener. The walk promotes on promotes_to_func_space_with_code, which excludes lambdas, so the earlier parity test's
|| is_closurewas hiding a live is_func_space / get_space_kind disagreement; the row now records that disagreement and the test uses the walk's unaugmented predicate. Java EagerLogical is pending R05, not absent (JLS 15.22.2 makes boolean & | ^ eager; the grammar cannot tell them from the integral forms). Ternary and Elixir's Comparison/ShortCircuit/EagerLogical are pending R05 per the roadmap. The role meanings for DocRunOpener, DocRunDirective, HoldsDocInBody and Exit now name the Checker contract each restates, with an expansion under the table; Branch and Loop say statement or expression; Guard and Member are disambiguated. kinds_overlap is doc(hidden).dekobon commentedon Oct 23, 2026
Performance
The macro read node.kind_id() once per clause; that FFI call is not inlinable, so LLVM could not fold the clause chain into the bitmap test a matches! gets (12 calls in java::opens_space). It is now read once per emitted function. opens_space is documented as the single gated scan for promotion plus label.
Separately, sizing the Elixir text gate found a pre-existing quadratic in checker/elixir.rs: a def with no quote ancestor climbs to the root, and one def per nesting level measured 0.12 / 0.47 / 1.97 s at depths 1000 / 2000 / 4000 (exponent 2.0) on main. The existing nom/nested-quote probe could not see it because every def there sits one step under a quote. Commit 3b07b5d9 folds the answer down the walker's chain (a new FoldLane, the Python bare-string pattern), falling back to the climb without a lent cache: 0.05 s at depth 4000. A new probe nom/elixir-unquoted-def reads exponent 2.10 OVER BOUND on the old code and 0.99 after; a parity test asserts fold == climb on every call of a fixture with quoted, unquoted, nested and after-quote defs; make chain-audit passes; a full-workspace INSTA_FORCE_PASS run leaves zero .snap.new or .pending-snap and a clean output submodule, so the output is unchanged. Own CHANGELOG entry.
dekobon commentedon Oct 23, 2026
Reduction and coverage
rust-optimize: the five attribute scanners are one scan with three accumulator slots and three named terminals; the pass-through payer rule is gone; the row-shape rules forward the owner list as one token tree. A duplicate gate, pays_itself or opens now fails to expand in every scan. Declined: dropping the mode helper (less readable), deleting the disjointness test (its remaining assertions and its role when the const assert is removed are both measured), and deriving the Java comparison gate from the payer list (a semantic coupling). The merge exposed one terminal no table reaches, a gated pays_itself; a synthetic table in the module's tests covers it.
Evidence on the final tree: 19 role lib tests, 6 macro doctests, the Elixir fold parity test; 22 perturbations each failing only named tests, including the re-recorded gated self-payer case; clippy -D warnings, fmt, rustfmt-bail (zero stuck arms), cargo doc -D warnings, check-test-lang-gates and the membership comparison against the base all clean; make pre-commit: BCA_GATE: pass (gate=pre-commit) on the squashed tree, re-running on the rebased tip.
dekobon commentedon Oct 23, 2026
make pre-commit on the rebased tip 3b07b5d9 (base e113773): BCA_GATE: pass (gate=pre-commit). Working tree clean, no snapshot drift.
dekobon commentedon Oct 23, 2026
Final tree
Branch refactor/1691-role-table-proof is 0f49f38 (role tables) + b9ea546 (Elixir quote fold) on main e113773. make pre-commit: BCA_GATE: pass (gate=pre-commit) on b9ea546; working tree clean.
A late review-skill pass (run against an earlier tree) added eight findings. Six fixed: the Java StringLit row comment claimed the #420 interpolation skip is live for Java and it is not (getter/java.rs never calls string_operand_type; the template wrapper and its interpolated name are both billed) — filed as #1703, row keeps wraps [StringInterpolation] with a FIXME(#1703) since R08 wiring it is the fix; a kind declared twice in owns or pays, and an excluded kind also owned or paid, now fail const assertions (kinds_repeat, excluded_kinds), each pinned by a compile_fail doctest whose failure reason was confirmed standalone; duplicate gate / opens attributes are pinned by two more compile_fail doctests (no rules expected); the three parity tests now call the _with_code spellings they claimed to; row-attribute order is documented. One declined with evidence (the disjointness loop is the check that fires if the const assert is removed, measured). One no-change (macro visibility; consistent with kind_sets.rs and STABILITY.md's doc(hidden) carve-out). Lambda-in-FuncOpener was the same finding astra-review raised and had already been fixed. astra-review confirmed the delta: no blockers; one wording nit folded in.
Doctests: 9 compile_fail + 1 valid shape; 19 role lib tests; Elixir fold parity test; 22 perturbations each failing only named tests.
refactor(ast): prove the role-table macro and fix the role inventory
refactor(ast): prove the role-table macro and fix the role inventory
docs(skills): bring add-lang up to the live AST crate and role tables