-
Notifications
You must be signed in to change notification settings - Fork 6
Property portfolio: requireInvariant discipline, 6-lens coverage checklist, multi-call techniques #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Property portfolio: requireInvariant discipline, 6-lens coverage checklist, multi-call techniques #43
Changes from 5 commits
54676e6
e989a5e
8ec7d12
146663f
af8c618
e585b7d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -35,11 +35,19 @@ | |
| 19. CVL automatically promotes operands of comparison/arithmetic operations to `mathint`. Explicit `to_mathint` casts are | ||
| rarely need. | ||
| 20. `persistent` ghosts should be used sparingly, and should *never* be used on ghosts which are intended to model/mirror | ||
| contract state. | ||
| contract state. However, ghosts tracking information never stored in contract state (e.g., call counters, cumulative transfer | ||
| amounts) are a legitimate use of `persistent`, and of expression summaries on calls to external dependencies. | ||
| 21. When referring to meaningful numerical constants in a specification (e.g., max fee percentage, asset expiry period, etc.) prefer using the `definition` feature of CVL | ||
| to provide meaningful names to the constants. | ||
| 22. Instead of `forall $type i` in a invariant, invariant parameters should be used instead. That is, instead of `invariant foo() forall uint i.logical_predicate(i)` simply write | ||
| `invariant foo(uint i) logical_predicate(i)`. The parameters of an invariant are implicitly universally quantified. | ||
| 23. Direct storage access should be used instead of mirroring contract state in ghosts via hooks. The only reason to use hooks and ghosts to reason about storage state is | ||
| if the ghost state is used for anything other than simply reading the values. | ||
| if the ghost state is used for anything other than simply reading the values. Hooks and ghosts are also appropriate when the information of interest is not stored in | ||
| contract state at all (e.g., counting how many times a function is called, or summing the amounts transferred across a sequence of calls). | ||
| 24. A bare `require` that constrains contract state inside a rule or `preserved` block is a spec smell: the assumption may not actually hold in reachable states, silently | ||
| weakening the proof. The default discipline is to state the assumption as an `invariant`, prove it, and assume it via `requireInvariant`. Bare requires over contract | ||
| state are acceptable only when accompanied by a comment referencing a justification or a documented trust assumption. Exception: capacity/overflow-avoidance bounds on | ||
| supplies, balances, or timestamps (e.g. `require totalSupply() <= 2^128`) exist to keep the solver tractable, not to model contract behavior — they are a recognized | ||
| documented-trust-assumption class; a comment naming the bound's purpose suffices, and no invariant attempt is expected because such bounds are typically not preserved | ||
| by the contract. In `preserved` blocks the stricter standard applies — `requireInvariant` or verifiable outside evidence; a comment alone is insufficient. | ||
|
Comment on lines
+47
to
+52
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we discussed this on zoom, but we already have guidance for both the judge and author to not use |
||
| </cvl_guidelines> | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. remove this test. does nothing |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| """Render smoke tests for prompt templates. | ||
|
|
||
| These templates are plain-jinja prompt fragments consumed by the property analysis / | ||
| generation / judge agents. The tests render them with minimal stand-in objects (jinja | ||
| only does attribute lookups, so `SimpleNamespace` suffices) and assert the load-bearing | ||
| content is present — a missing template variable or a broken include fails fast here | ||
| instead of mid-agent-run. | ||
| """ | ||
| from types import SimpleNamespace | ||
|
|
||
| from composer.templates.loader import load_jinja_template | ||
|
|
||
|
|
||
| def test_cvl_guidelines_render(): | ||
| out = load_jinja_template("cvl_guidelines.j2") | ||
| # Guideline 24: requireInvariant discipline over bare state requires. | ||
| assert "requireInvariant" in out | ||
| assert "spec smell" in out | ||
| # Guideline 24 carve-out: solver-capacity bounds need no invariant attempt. | ||
| assert "keep the solver tractable" in out | ||
| # Guideline 24 defers preserved blocks to the stricter (judge Criteria 4) standard. | ||
| assert "stricter standard" in out | ||
| # Guideline 20 carve-out: persistent counter ghosts are legitimate (judge-visible | ||
| # counterpart of the ghost-counter advice in cvl_additions.j2). | ||
| assert "never stored in contract state" in out | ||
| # Guideline 23 carve-out: hooks/ghosts for information never stored in contract state. | ||
| assert "not stored in" in out | ||
| assert out.strip().endswith("</cvl_guidelines>") | ||
|
|
||
|
|
||
| def test_cvl_additions_render(): | ||
| out = load_jinja_template("cvl_additions.j2") | ||
| # Storage snapshot / additivity idiom. | ||
| assert "lastStorage" in out | ||
| assert "at init" in out | ||
| # satisfy-witness companion rules, mapped under their parent property. | ||
| assert "satisfy" in out | ||
| assert "property_rules" in out | ||
| # Ghost counters via expression summaries. | ||
| assert "countDeposit() expect void" in out | ||
| assert out.strip().endswith("</cvl_advice>") | ||
|
|
||
|
|
||
| def _fake_component_context() -> SimpleNamespace: | ||
| """Minimal stand-in for ContractComponentInstance as accessed by the template.""" | ||
| contract = SimpleNamespace(name="Vault", solidity_identifier="Vault") | ||
| component = SimpleNamespace( | ||
| name="Deposits", | ||
| description="Handles user deposits", | ||
| requirements=["Users receive shares proportional to deposits"], | ||
| interactions=[], | ||
| ) | ||
| app = SimpleNamespace(application_type="an ERC4626 vault") | ||
| return SimpleNamespace(component=component, contract=contract, app=app, | ||
| ommer_contracts=[]) | ||
|
|
||
|
|
||
| def test_property_analysis_prompt_render(): | ||
| out = load_jinja_template( | ||
| "property_analysis_prompt.j2", | ||
| context=_fake_component_context(), | ||
| backend_guidance="BACKEND_GUIDANCE_SENTINEL", | ||
| # A valid Sort value (see composer/spec/service_host.py) exercising the | ||
| # non-greenfield template branch. | ||
| sort="existing", | ||
| prior_properties=[], | ||
| ) | ||
| # The context and backend guidance are threaded through. | ||
| assert "Deposits" in out | ||
| assert "BACKEND_GUIDANCE_SENTINEL" in out | ||
| # 6-lens coverage checklist, framed as brainstorming discipline, not a quota. | ||
| assert "Unit behavior" in out | ||
| assert "Variable transition" in out | ||
| assert "Multi-call / high-level" in out | ||
| assert "NOT a quota" in out | ||
| # Task steps ask for a record of swept lenses. | ||
| assert "which of the six coverage" in out | ||
|
|
||
|
|
||
| def test_property_analysis_prompt_render_with_prior_rounds(): | ||
| prior = [SimpleNamespace( | ||
| items=[SimpleNamespace(sort="invariant", title="solvency", | ||
| description="assets cover shares")], | ||
| reasoning="looked at deposit accounting", | ||
| )] | ||
| out = load_jinja_template( | ||
| "property_analysis_prompt.j2", | ||
| context=_fake_component_context(), | ||
| backend_guidance="", | ||
| sort="existing", | ||
| prior_properties=prior, | ||
| ) | ||
| assert "solvency" in out | ||
| assert "looked at deposit accounting" in out |
Uh oh!
There was an error while loading. Please reload this page.