Skip to content

Commit 4fdbf79

Browse files
committed
docs: align README with research note
1 parent 64f0547 commit 4fdbf79

1 file changed

Lines changed: 13 additions & 9 deletions

File tree

‎README.md‎

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -119,8 +119,10 @@ inspectable.
119119
| Incorrect / incomplete fix | **3** |
120120
| Provider failure | **1** |
121121

122-
Budget-bound is an observed terminal category; it does not mean that raising a
123-
budget would solve the task.
122+
Budget-bound termination is the dominant observed endpoint pattern among the
123+
15 non-resolved runs (**11/15**), indicating substantial execution-budget
124+
pressure under the frozen policy. This does not establish that increasing the
125+
budget would necessarily resolve those tasks.
124126

125127
#### Termination and correctness
126128

@@ -131,17 +133,19 @@ budget would solve the task.
131133
| `model-failed` → officially resolved | **1** |
132134
| `finished` → officially unresolved | **3** |
133135

134-
> **Termination status is not a correctness proxy.** Official grading remains
135-
> the correctness authority for this benchmark.
136+
> **Termination status should not be used as a substitute for evaluator-derived
137+
> resolution.** Termination reasons remain useful operational and diagnostic
138+
> signals.
136139
137140
#### Research Note
138141

139-
**[Auditing Runtime Termination as a Correctness Proxy for Coding Agents: A Frozen SWE-bench Verified Mini Case Study](docs/research/runtime-termination-correctness-proxy.md)**
142+
**[Auditing Runtime Termination as a Resolution Proxy: A Frozen Coding-Agent Case Study](docs/research/runtime-termination-correctness-proxy.md)**
140143

141-
An artifact-grounded measurement audit of typed runtime termination as a
142-
correctness proxy under one frozen configuration. The mechanical
143-
`finished → resolved / non-finished → unresolved` proxy misclassified **12/45
144-
graded runs (26.7%)**.
144+
A configuration-specific empirical audit of typed runtime termination as a
145+
resolution proxy under one frozen APC configuration. The mechanical
146+
`finished → resolved / non-finished → unresolved` rule disagreed with the
147+
official evaluator on **12/45 binary-outcome runs (≈26.7%)**, with errors in
148+
both directions.
145149

146150
## 🚀 Quick Start
147151

0 commit comments

Comments
 (0)