validation.md

Index claim validation

Each kind=contract row in data/index.csv encodes two principal claims:

1. The CVE identified by cve_id pertains to contractname in the verified source obtained from src_addr.
2. The vulnerable contract is deployed at addr, when a deployment address is recorded.

The validation pipeline primarily measures documentary and textual support for these claims. Its generated assessments do not, by themselves, prove that verified source and deployed runtime bytecode are identical.

In particular, match means that the evidence supports the indexed artifact correspondence. It does not independently establish exploitability or validate all technical assertions in the CVE. non_match rejects the proposed artifact correspondence, not the CVE itself.

Evidence corpus

The script collect_cve_texts.py constructs the CVE-related corpus from:

• CVE titles and descriptions in data/cves/;
• CVE reference fields and tags;
• retained reference-document metadata and document blocks in validation/evidence/reference_archive/;
• explicit code blocks in archived documents;
• optical character recognition (OCR) from archived images, marked as code only when it looks code-like; and
• curated manual documents listed in validation/evidence/manual_sources/index_claim_evidence.jsonl.

For a document shared by several CVEs, the collector tries to restrict the corpus to the relevant section. It uses, in order, explicit section hints, URL fragments, CVE-specific headings, and explicit CVE blocks in the prose. If no reliable boundary can be established, it retains the complete document rather than guessing.

Every corpus record retains provenance such as its source kind, document, block index, headings, and scoping method. This corpus is the independent textual material searched by most of the elementary checks below.

Elementary checks and their data dependencies

Check	Claim supported or challenged	Data used
Source-address mention	Links the CVE material to the source at src_addr	data/index.csv.src_addr; collected CVE/reference/manual/OCR corpus
Deployment-address mention	Identifies the claimed deployment addr	data/index.csv.addr; collected corpus
Contract-name mention	Links the CVE material to the indexed or explorer-reported contract identity	data/index.csv.contractname; explorer ContractName at src_addr and, if available, addr; collected corpus
Token-name mention	Identifies the source or deployment through its returned token name	Retained name() call result for src_addr or addr; collected corpus
Token-symbol mention	Identifies the source or deployment through its returned token symbol	Retained symbol() call result for src_addr or addr; collected corpus
Chain mention	Establishes chain context	data/index.csv.chain and configured aliases; collected corpus
Source-code snippet match	Links CVE code evidence to verified source at src_addr	Code-marked corpus blocks; explorer source for src_addr
Deployment-local code match	Links CVE code evidence to source associated with a distinct addr	Code-marked corpus blocks; explorer source for addr
CVE-named function in source	Tests whether a function explicitly named by the CVE exists in the proposed source	CVE description; ABI and Solidity source at src_addr
CVE-named function in runtime	Tests whether a CVE-named externally callable function is exposed by the deployment runtime	CVE description; compiled function signatures/selectors for addr; selectors extracted from runtime at addr
Same-address relation	Allows supported source identity to contribute to deployment identity	data/index.csv.src_addr, data/index.csv.addr, and the CVE-to-source assessment
Source-contract membership	Checks that contractname is uniquely defined by compiled verified source at src_addr	data/index.csv; generated Etherscan interfaces
Source evidence-function membership	Checks externally callable functions against the selected compiled contract interface and approximates private or internal membership by source-bundle declaration	CVE function observations; generated interface; verified source at src_addr
Source/deployment relationship	Applies the indexed/explorer contract-name and address truth table	contractname, src_addr, addr; explorer ContractName at src_addr
Distinct deployment identity	Checks a distinct deployment's explorer name and compiled contract entry when source is available	Explorer source and generated interface at addr
Deployment interface consistency	Compares the indexed source interface with optional deployment source and retained runtime selectors	Generated interfaces at src_addr and addr; runtime selectors at addr

Identifier checks

Source and deployment addresses

The matcher searches separately for src_addr and addr. Address matching is case-insensitive and boundary-aware. Each hit records the originating corpus block, character spans, context windows, and occurrence count.

• A src_addr occurrence supports the CVE-to-source claim.
• A distinct addr occurrence supports the deployment claim.
• Absence is recorded in a search summary but is not automatically treated as a contradiction.

Contract identity

The searched contract-name targets are:

• contractname from data/index.csv;
• ContractName from data/etherscan/<chain>-<src_addr>.json; and
• when present, ContractName from data/etherscan/<chain>-<addr>.json.

The check asks whether any of these identities occur in CVE-related material. Generic names such as token or contract receive less evidential weight than specific names.

Token identity

For both src_addr and addr, when retained call results exist, the matcher uses:

• name() as a case-insensitive token-name target; and
• symbol() as a case-sensitive token-symbol target.

The inputs are stored under validation/evidence/call_results/. Token identity is particularly useful when a reference names the affected token but does not print its address. Short or generic identities receive weaker weight.

Chain context

The matcher searches for the indexed chain and configured aliases. Chain evidence is retained as context but is not sufficient by itself to identify a specific source or deployment.

Code-identity checks

For every corpus block marked as code, the matcher compares the block with every verified source file associated with the checked address. A block is eligible only when its whitespace-compacted representation is at least 40 characters long.

The elementary comparison methods are:

1. Normalized-lines exact match: strip lines, remove empty lines, and find the resulting block verbatim in a source file.
2. Whitespace-insensitive exact match: remove all whitespace and perform an exact substring comparison.
3. Executable-code exact match with comment variation: tolerate apostrophe differences inside comments while preserving program text and string literals.
4. Fuzzy OCR match: for OCR-derived code only, use conservative n-gram containment followed by order-preserving similarity, including bounded contiguous OCR line windows.

The comparison runs against explorer source at src_addr. If addr is nonempty and differs from src_addr, it also runs independently against explorer source at addr.

An exact code match is the strongest textual evidence linking the CVE to the source. Fuzzy OCR evidence is weaker and generally needs independent address, contract, token, or function evidence to produce strong support. Conversely, an eligible code block that fails to match reduces confidence.

CVE-named function checks

Function presence in verified source

The matcher conservatively extracts function or method names explicitly stated in the CVE description. For every extracted name, it checks for an exact, case-sensitive occurrence:

• as a function in the explorer ABI; or
• as a Solidity function declaration in any verified source file at src_addr.

Source declarations retain their file and visibility: public, external, private, internal, or default public.

The outcomes are interpreted as follows:

• every explicitly named function is present: strong source support;
• any explicitly named function is absent from both ABI and all source files: strong mismatch and counterevidence; or
• no function is explicitly named: the check is unavailable, not failed.

A CVE-named function absent from both ABI and source is the principal automatic non_match criterion.

Function presence in deployed runtime

For a row with addr, the matcher:

1. finds compiled function signatures whose name equals the CVE-named function;
2. where possible, restricts candidates to the explorer-reported deployed ContractName; and
3. tests whether a candidate four-byte selector occurs in the selector set extracted from runtime at addr.

This check relies on:

• validation/generated/etherscan_functions/<chain>-<addr>.json;
• validation/generated/runtime_selectors/<chain>-<addr>.json; and
• optionally data/etherscan/<chain>-<addr>.json for the deployed contract name.

Selector presence shows that the deployed bytecode exposes a dispatcher entry compatible with the named externally callable function. It does not show that the deployed function body is identical to the source or that the function is necessarily the vulnerable implementation.

Private and internal functions do not have runtime dispatch selectors. The reducer therefore has a special same-address criterion for cases where strong address and contract identity evidence identifies the deployment and every CVE-named function is private or internal in that source.

Reduction into claim assessments

CVE-to-source assessment

The reducer combines:

• source-address evidence;
• contract identity;
• token identity;
• code identity against source at src_addr; and
• CVE-named function presence in the source or ABI.

Its main rules are:

• an exact code match gives strong support;
• fuzzy OCR code combined with independent address, identity, or named-function evidence can give strong support;
• contract or token identity and named-function presence can support the link when no eligible code comparison exists;
• an address occurrence alone is weak support;
• unmatched eligible CVE code reduces confidence; and
• a CVE-named function absent from both ABI and source is strong counterevidence.

Deployment assessment

The reducer combines:

• direct textual occurrence of addr;
• token identity returned by calls to addr;
• code matched against explorer source associated with addr;
• selectors for CVE-named externally callable functions observed in runtime at addr;
• whether addr equals src_addr; and
• the previously computed CVE-to-source assessment.

Its main rules are:

• an empty addr makes the deployment claim not applicable;
• if addr == src_addr, supported source identity also supports deployment identity;
• an explicit occurrence of a distinct deployment address is strong support;
• deployment-local source-code or runtime-function evidence supports a distinct deployment;
• address or token-identity evidence can provide support; and
• no deployment address, token, code, or function evidence produces no deployment support.

Automatic and manual classification

The automatic classifier is deliberately conservative:

• non_match is assigned when a function explicitly named by the CVE is absent from both the verified application binary interface (ABI) and every verified source file;
• match is assigned for high-confidence support of both the CVE-to-source and deployment claims;
• match can also be assigned for narrowly defined medium-confidence cases, including strong identity plus deployed-function confirmation or the same-address private/internal-function case; and
• all other cases remain unclassified, which does not assert that the row is false.

The reducer's categorical assessment is diagnostic only. Final classifications and review notes are applied after CSV construction from the current per-CVE decision files under validation/decisions/CVE-*.json; the filename identifies the CVE and the reducer has no manual decision input.

validation.csv summary

The normal way to regenerate this file is the complete validation command:

python validation/scripts/build_validation.py

The CSV export stage reads current contract rows from data/index.csv and assessments from validation/generated/index_claim_text_assessments.jsonl, then writes validation/validation.csv. Library rows are omitted because the contract matcher and reducer do not assess them.

Before writing anything, the script requires exactly one assessment for every contract CVE and rejects unknown or duplicate assessments. Current claim fields are read directly from data/index.csv rather than copied into generated records.

Common check values

All evidence-check columns use lowercase tri-state values:

• true: the check was applicable and its success condition was met;
• false: the check was executable but its success condition was not met; and
• empty: the check was skipped, not applicable, or lacked enough data to run.

For textual checks, false means that no supporting match was found. It does not by itself mean that the index claim is disproved. Likewise, true records some support without preserving whether the reducer rated that support weak, moderate, or strong. The generated JSON assessment remains the authoritative detailed result.

Columns

Column	Value semantics
cve_id	CVE identifier from the contract row in data/index.csv
source_address	true if at least one src_addr target matched; false if searched without a match
deployment_address	true if at least one addr target matched; false if searched without a match; empty when no deployment is recorded
contract_name	true if any indexed or explorer-derived contract-name target matched; false if available targets were searched without a match
token_name	true if any retained name() target matched; false if name targets existed but none matched; empty if no name target was available
token_symbol	true if any retained symbol() target matched; false if symbol targets existed but none matched; empty if no symbol target was available
chain	true if any chain alias matched; false if aliases were searched without a match
source_code	true if at least one eligible code block matched source at src_addr; false if eligible blocks were compared but none matched; empty if no eligible comparison was available
deployment_code	The same code rule applied to source at a distinct addr; empty when that comparison was not applicable or unavailable
source_function	true if every function explicitly named by the CVE was found in the source or ABI; false if any was absent; empty if no function check was available
deployment_function	true if every CVE-named function had available data and a selector present in deployment runtime; false if runtime/function data was available but that condition failed; empty if the check could not run
same_address	true if src_addr == addr; false if both are present and distinct; empty if no addr is recorded
source_contract	true if contractname uniquely identifies a compiled entry from verified source at src_addr; false if absent or ambiguous
source_function_interface	true if every CVE-named function is in that contract's compiled externally callable interface; false if at least one is not; empty if no function is named or the check is unavailable
source_function_nonpublic	For named functions outside the externally callable interface, true if every one has an exact private or internal declaration somewhere in the verified source bundle; false if any is absent; empty when not applicable
source_deployment_relation	Result of the contract-name/address truth table: an explorer-selected indexed contract must use addr == src_addr; a different explorer contract permits an empty or distinct addr, but not addr == src_addr
deployment_contract	For a distinct claimed deployment with retained explorer source, true if its ContractName equals the indexed name and that name uniquely identifies a generated compiled entry; false on mismatch; otherwise empty
deployment_function_interface	For a distinct deployment with verified source, true if every CVE-named function is in the deployed contract's compiled externally callable interface; false if at least one is not; otherwise empty
deployment_function_nonpublic	For named functions outside that deployment interface, true if every one has an exact private/internal declaration somewhere in the deployment's verified source bundle; false if any is absent; otherwise empty
deployment_source_interface	For a distinct deployment with verified source, true if its indexed-contract selector set equals the set compiled from src_addr; false on a difference; otherwise empty
address_runtime_interface	true if selectors extracted from runtime at addr equal the indexed-contract selector set compiled from src_addr; false on a difference; empty if unavailable or not applicable
classification	Final artifact-correspondence classification: match, non_match, or empty when no automatic rule or reviewed decision classifies the row
classification_reason	manual for a current reviewed classification; otherwise the triggered automatic criterion name; empty when the row is unclassified
note	Manual judgment note for a manual classification; otherwise empty

same_address is a factual relationship between index fields rather than an evidence search. Reviewed classifications are stored as current per-CVE decisions rather than as a separate CSV check column.