VeriCommit

Takes a patch from any model (or a human), applies it in a throwaway git worktree, runs the repo's own build and tests plus a stack of static checks, and writes an accept/reject verdict and a report. It never touches your working tree or current branch.

task + repo ──► context ──► candidates (DeepSeek / Claude / OpenRouter / mock)
                                ▼
              accepted patch + commit message + verification report

Status: alpha (0.1.0). Interfaces may change before 0.2. The gates are necessary conditions, not proof of correctness; see the calibration numbers below.

Install

pip install git+https://github.com/A-G0D/vericommit.git   # or: pip install -e ".[dev]"
vericommit --help

Requires Python 3.10+. The only runtime dependency is PyYAML (used to read repo CI workflows).

Quickstart

# 1. list provider specs and whether keys are visible
vericommit providers

# 2. generate + verify a patch for a task (default provider: deepseek)
vericommit run --repo /path/to/repo \
  --task "fix the bug in app/calc.py: add() returns a-b" \
  --providers deepseek --candidates 2 --budget 0.5

# 3. verify an existing (human or agent) patch with the same gates
git -C /path/to/repo diff > change.diff
vericommit check --repo /path/to/repo --patch change.diff

# 4. deterministic demo, no keys, no network
bash demo/run_mock_demo.sh

# 5. false-accept / false-alarm rates on the bundled corpus
vericommit calibrate

# 6. tests
python3 -m pytest -q            # or: python3 -m unittest discover -s tests -v

Outputs land in runs/<run-id>/: change.diff, commit_message.txt, report.json (per-candidate layers, tokens, cost), and REJECTED.md when nothing passes. --write-branch commits the accepted patch on a new branch and never moves HEAD.

Providers

spec	what	credential
deepseek / deepseek:reasoner	DeepSeek chat / reasoner	DEEPSEEK_API_KEY
claude / claude:sonnet / claude:opus	Claude Code CLI, headless, --max-turns 1	existing CLI login
openrouter:<model>	any OpenRouter model	OPENROUTER_API_KEY
mock	deterministic stub for tests	–

Keys are read from the environment, then from an optional ~/.config/vericommit/keys.env (one NAME=value per line). Values are never printed or logged. Use whatever model you trust to propose; the gates decide what lands.

Policy (policy.json, every key optional)

Allowed/forbidden globs, max diff lines/files, build/test/lint commands (auto-detected from the repo otherwise), hard vs soft check lists, candidate count, repairs, providers, per-run USD budget, test/build timeouts, branch/commit prefix, and whether autonomous-agent run is permitted. Defaults in vericommit/policy.py.

Verification layers (ordered; hard gates stop at the first failed gate)

#	layer	kind	catches
1	repo_policy	hard	repos whose contribution docs forbid autonomous agents
2	git_hygiene	hard	forbidden paths (tests/CI/deps/env), path allow-list, diff/file caps, binary & oversized files
3	no_test_tamper	hard	test/CI edits even if the policy is loosened (defense in depth)
4	safety	hard	secrets/tokens/private keys, shell=True, os.system, eval/exec, pickle, rm -rf /, chmod 777, curl | sh
5	build	hard	compile/build in a throwaway worktree, scrubbed env, timeout
6	tests	hard	the repo's own test command; flags a repo with no test command, or a run where no tests ran
7	lint	soft	repo linter/formatter if configured
8	contract	soft	public-API breakage via Python AST diff of changed modules
9	style	soft	repo line-length/indent profile, trailing whitespace, debug prints, TODO/FIXME, commit-message convention
10	changelog	soft	contribution docs require a changelog entry that is missing
11	injection	soft	prompt-injection signals in the context the model was given

Hard failures trigger a bounded repair loop (failure text is fed back to the model), then rejection. Every run writes a report, so an agent's work is auditable after the fact.

Working with real repositories (profile → plan → PR)

Every run starts by reading the repo's own contract: profile walks README/CONTRIBUTING/AGENTS/ AI-policy docs, .github/workflows/*.yml, PR/issue templates, changelogs, style configs, and the last 80 commit subjects. It writes CONVENTIONS.md, conventions.json, and a prompt fragment that the engine injects into every generation.

vericommit profile --repo /path/to/any/repo        # -> repo-profiles/<name>/
vericommit pr --repo /path/to/repo --plan plan.json \
    --providers deepseek --candidates 2 --budget 0.5   # multi-commit branch + runs/PR.md
vericommit track --repo owner/repo --pr 1234       # poll reviews -> runs/pr_tracker.jsonl

The AI-policy gate is hard and non-negotiable: repo_policy refuses to run on repos whose contribution docs forbid autonomous agents, and pr/run exit non-zero instead of producing a patch. The human author still writes the PR narrative, discloses AI assistance, and answers review comments in their own words.

Batch mode

batch runs a JSONL manifest of {repo, task} pairs concurrently (--workers N), each with its own candidate fan-out; --candidates K × --workers N provider calls are in flight at once. Costs and acceptance are written to runs/batch_report.json.

Calibration

vericommit calibrate replays a labeled corpus through the gates with no model and no network, and writes calibration.json and calibration.md to runs/calibration/. It takes about 5 s (measured 2026-11-07, Apple Silicon Mac; three runs: 4.5, 4.7, 4.7 s).

Corpus: 22 synthetic patches against the demo repo (vericommit/corpus/cases.json), 6 that should be accepted and 16 that should be rejected, each reject labeled with the layer expected to catch it. Run 2026-11-07:

	result
false-accept rate	2/16 = 12.5%
false-alarm rate	0/6 = 0.0%

layer	expected to catch	caught
git_hygiene	5	5
safety	6	6
build	1	1
tests	4	2

The two false accepts are deliberate: a fix hard-coded to the test inputs, and a correct fix bundled with an unrelated is_admin() -> True. No gate models either. The corpus is small, synthetic, written by the gate's author and Python-only, so the rates describe regression coverage, not behavior on real changes. Soft layers (lint, contract, style, changelog, injection) are not scored; they do not reject.

Independent check (2026-11-07): 18 realistic bad commits and 5 good ones, applied to 3 throwaway Python packages with unittest suites and run through vericommit check. Caught 15/18 (83.3%); false alarms 1/5 (20%). The 3 misses: a fix hard-coded to the test input, a behavior change no test covers, and a removed public function (contract flagged it, but it is soft). The false alarm is a good commit that adds its own test, which git_hygiene rejects by default. Seeds are small and written by the gate's author, so treat the rate as a rough upper bound.

Other measurements (2026-10-27, one machine, not a benchmark)

• Tests: python3 -m pytest -q passed 26/26 on Python 3.10-3.13 (29 with the calibration tests).
• Cross-provider run (deepseek + claude, 2 candidates): both passed the hard gates in 6.6 s (measured 2026-11-07, Apple Silicon Mac) for $0.1374; the chosen patch was a 2-line diff.
• --write-branch committed on a new branch and left main untouched.
• Batch of 3 repos with DeepSeek, 3 workers: 3 of 3 accepted.

What is deliberately not done yet

• Sandbox is opt-in and narrow. --sandbox docker (policy sandbox) runs the build, lint and test commands in docker run --rm --network none --read-only with a tmpfs /tmp, all capabilities dropped, 2 CPUs, 2 GB, 256 pids, and the throwaway worktree mounted at /work. If docker is missing or the daemon is down the check fails; it never falls back to a local run. Limits: the image (sandbox_image, default python:3.12-slim) must already contain your toolchain and dependencies, since there is no network; git steps still run locally; the worktree is mounted read-only (/tmp is the only writable path); git commands inside the container won't work because the git dir isn't mounted; docker is not a hard security boundary against kernel escapes. The docker argv is unit-tested; the integration test (VERICOMMIT_DOCKER_TESTS=1) has not been run yet because no docker daemon was available.
• No review-comment prediction. The style layer uses rules + repo statistics.
• Calibration is synthetic. See Calibration above; there is no corpus of real changes yet.
• Python-first contract checks. The API-diff layer understands Python ASTs; other languages get build/test/lint only.

Contributing

Read CONTRIBUTING.md first. AI-assisted contributions are allowed with disclosure; see AGENTS.md for the rules agents must follow here. Security reports: SECURITY.md. Community expectations: CODE_OF_CONDUCT.md.

License

MIT © 2026 A-G0D.