clvm_cov: source-level coverage for Chialisp/CLVM - #555
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
e4b791c to
c795997
Compare
A clvm_cov binary, alongside run/brun/cldb, reports which source expressions
of a puzzle a run (or a corpus of runs) executed.
It reuses the compiler and cldb: every source location survives compilation
(sha256tree symbols), and cldb resolves each executed step to a Srcloc.
clvm_cov compiles each .clsp, builds a sha256tree->srcloc table, decorates the
corpus program and args hex with it, steps the program under CldbRun, and
unions the stepped locations. covered = reachable ∩ executed, at (file, line)
granularity.
The corpus is JSONL, one {program, args, source?, layer?} per line. When source
is omitted (a runtime capture sees only program+args), clvm_cov attributes the
program against a global sha256tree->source table built from every .clsp under
--source/--include, uncurrying a curried capture to recover and identify an
inner puzzle. Output is LCOV (a union plus one file per layer) and a --summary
table: per-file union%, per-layer totals, and the cold worklist.
Recompiling honors the source's dialect sigil, so a *standard-cl-25* source
compiles optimized and byte-matches an optimized producer's hex; a treehash
mismatch warns per source. Non-inline defun branches are line-accurate; inline
arms collapse to a shared location.
Adds a [[bin]] clvm_cov (thin main + driver) and compile_modern_programs in
comp_input (returns the compiled programs, not the module export summary).
c795997 to
8ca0d19
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 8ca0d19. Configure here.
| if parse_srcloc_string(loc_str).is_some() { | ||
| reachable.insert(loc_str.clone()); | ||
| } | ||
| } |
There was a problem hiding this comment.
Multi-export inflates reachable lines
Medium Severity
For an explicit source pointing at a multi-export module, compile_source unions every export’s symbol table into reachable, and run_corpus adds all of those lines to the denominator. A corpus record whose program hex is only one export still steps just that tree, so sibling exports’ lines appear reachable but never execute and show up as false cold lines and lower union coverage.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 8ca0d19. Configure here.
| steps += 1; | ||
| } | ||
|
|
||
| Ok((executed, matched)) |
There was a problem hiding this comment.
Failed runs report coverage silently
Medium Severity
run_record stops stepping when CldbRun hits RunErr or RunExn, but still returns Ok with a partial executed set. run_corpus merges that into coverage and emits no warning, unlike treehash mismatches, so failed corpus invocations can look like successful partial runs.
Reviewed by Cursor Bugbot for commit 8ca0d19. Configure here.
|
will read |
2 participants
Adds
clvm_cov, a source-level coverage tool for Chialisp/CLVM, as a new binary alongsiderun/brun/cldb. Given a corpus of CLVM invocations(program, args), it reports which source lines of each puzzle ran and which stayed cold.How
It reuses the compiler and
cldb. Each source location survives compilation (sha256tree symbols), andcldbresolves every executed step to aSrcloc.clvm_covcompiles each.clsp, builds asha256tree -> srcloctable, decorates the corpus program and args hex with it, steps the program underCldbRun, and unions the stepped locations.covered = reachable ∩ executed, at (file, line) granularity.Input / output
The corpus is JSONL, one
{program, args, source?, layer?}per line. Whensourceis omitted (a runtime capture sees onlyprogram+args), the program is attributed against a globalsha256tree -> sourcetable built from every.clspunder--source/--include, uncurrying a curried capture to identify an inner puzzle. Output is LCOV (a union plus one file perlayer) and a--summarytable: per-file union%, per-layer totals, and the cold worklist.Granularity
Recompiling honors the source's dialect sigil, so a
*standard-cl-25*source compiles optimized and byte-matches an optimized producer's hex; a treehash mismatch warns per source. Non-inlinedefunbranches are line-accurate; inline arms collapse to a shared location.Files
Cargo.toml— declares[[bin]] clvm_covsrc/classic/bins/clvm_cov.rs— thin mainsrc/classic/clvm_tools/clvm_cov.rs— driver + unit testssrc/classic/clvm_tools/comp_input.rs— addscompile_modern_programs(returns the compiled programs, not the module export summary)src/classic/clvm_tools/mod.rs— registers the modulecargo build --release --bin clvm_covand the module's 9 unit tests pass.Note
Medium Risk
Large new tooling surface (~1700 LOC) and a new
compile_modern_programsAPI on the shared compile input path; behavior of existingcompile_modernis unchanged but the compiler integration is non-trivial.Overview
Adds a new
clvm_covbinary that reports which.clspsource lines ran vs stayed cold from a JSONL corpus of(program, args)hex invocations.Coverage reuses the same compile pipeline as
run/cldb: buildsha256tree → srcloctables, decorate corpus hex, step withCldbRun, and aggregate reachable ∩ executed at (file, line). Output is LCOV (union plus per-layerfiles) and an optional--summarywith cold-line worklists.Source-less records (runtime captures without
source) compile every.clspunder--source/--includeinto a global attribution table (parallel compiles viaCLVM_COV_JOBS), with curry uncurrying to re-identify inner puzzles and overlay per-source maps when sibling templates share subtrees.RunAndCompileInputData::compile_modern_programsreturns actual compiled program SExps (not module export summaries) so module and multi-export attribution matches on-chain hex.Reviewed by Cursor Bugbot for commit 8ca0d19. Bugbot is set up for automated code reviews on this repo. Configure here.