Skip to content

README.md

Latest commit

 

History

History
105 lines (82 loc) · 4.26 KB

README.md

File metadata and controls

105 lines (82 loc) · 4.26 KB
Raw

Contract-claim validation

This folder retains evidence and reproducible derivations used to assess the correspondence between CVE claims and source artifacts in data/index.csv. Its main output is validation.csv; its evidence text corpus also feeds labeling and localization.

Validation asks whether retained evidence supports the dataset's mapping from a CVE description to a source artifact and, where given, a deployment. A match does not independently confirm that the code is vulnerable; a non_match rejects the indexed correspondence rather than the CVE claim itself.

Validation only applies to index rows with kind=contract. Library rows express source-level claims without a specific deployment and are outside this contract-claim classification.

Pipeline

Install and activate the shared environment once:

bash scripts/install_dependencies.sh
source .venv/bin/activate

Rebuild all derivations from retained inputs with:

python validation/scripts/build_validation.py

The entry point runs these scripts in order:

  1. extract_etherscan_interfaces.py compiles retained verified sources into contract function interfaces.
  2. extract_runtime_selectors.py extracts selectors from deployed runtime.
  3. build_cve_evidence_index.py indexes CVE relationships in retained reference evidence.
  4. collect_cve_texts.py builds a provenance-preserving evidence corpus.
  5. match_index_claim_texts.py matches indexed identities, addresses, functions, and code against that corpus.
  6. check_index_contracts.py checks source, deployment, function, and runtime invariants.
  7. reduce_index_claim_text_matches.py reduces observations to one assessment per contract row.
  8. build_validation_csv.py exports the compact summary.
  9. classify_validation.py applies reviewed decisions and classifications to validation.csv.

Missing compiler releases are installed automatically with solc-select. The build fails if a required release cannot be installed or selected, or if it cannot compile the retained source.

Important files

  • validation.csv is the compact per-CVE result; validation.md specifies its checks and columns.
  • evidence/ contains acquired or manually captured inputs, including call results, manually saved sources, and supporting documents referenced by CVE records.
  • decisions/CVE-*.json contains current manual classifications. Each file has a decision of match or non_match and a note copied into validation.csv.
  • decision.schema.json describes those reviewed decision files.
  • generated/ contains reproducible interfaces, runtime selectors, the text corpus, match observations, contract checks, and reduced assessments.
  • scripts/ contains the pipeline entry point, its stages, and shared helpers.

Evidence and decisions persist; generated files can be rebuilt. Decisions are edited in place and represent current state. Stored paths are repository-relative, while external URLs remain absolute identifiers.

Refreshing reference evidence

Archive acquisition is extra work and must be requested explicitly:

python validation/scripts/build_validation.py --update-archive

This first runs build_reference_archive.py, which calls the fetch, normalization, evidence-following, and image-archiving stages, and then runs the nine derivation stages above. Network refreshes can change retained evidence. Use --refresh-documents or --refresh-images with --update-archive to replace already retained material. Manually saved pages belong in evidence/manual_sources/.

OCR is also explicit and requires macOS:

python validation/scripts/build_validation.py --update-archive --perform-ocr

Use --refresh-ocr together with --perform-ocr and --update-archive to replace retained OCR. OCR is stored beside its archived image because it is platform-specific evidence that may not regenerate identically.

Focused diagnostics

The derivation scripts may be run individually in the order listed above. Interface and selector extraction accept selected input files for targeted updates. check_index_contracts.py --fail-on-mismatch gives a nonzero exit for CI-style checking; without it, mismatches are recorded in generated output and carried into validation.csv.