|
1 | | -## npx kanban (Research Preview) |
2 | | - |
3 | | -<p align="center"> |
4 | | - <img src="https://github.com/user-attachments/assets/2aa3dcc7-94e3-4076-bcfe-6d0272007cfe" width="100%" /> |
5 | | -</p> |
6 | | - |
7 | | -A replacement for your IDE better suited for running many agents in parallel and reviewing diffs. Each task card gets its own terminal and worktree, all handled for you automatically. Enable auto-commit and link cards together to create dependency chains that complete large amounts of work autonomously. |
8 | | - |
9 | | -> [!WARNING] |
10 | | -> Kanban is a research preview and uses experimental features of CLI agents like bypassing permissions and runtime hooks for more autonomy. We'd love your feedback in #kanban on our [discord](https://discord.gg/cline). |
11 | | -
|
12 | | -<div align="left"> |
13 | | -<table> |
14 | | -<tbody> |
15 | | -<td align="center"> |
16 | | -<a href="https://www.npmjs.com/package/kanban" target="_blank">NPM</a> |
17 | | -</td> |
18 | | -<td align="center"> |
19 | | -<a href="https://github.com/cline/kanban" target="_blank">GitHub</a> |
20 | | -</td> |
21 | | -<td align="center"> |
22 | | -<a href="https://github.com/cline/kanban/issues" target="_blank">Issues</a> |
23 | | -</td> |
24 | | -<td align="center"> |
25 | | -<a href="https://github.com/cline/kanban/discussions/categories/feature-requests?discussions_q=is%3Aopen+category%3A%22Feature+Requests%22+sort%3Atop" target="_blank">Feature Requests</a> |
26 | | -</td> |
27 | | -<td align="center"> |
28 | | -<a href="https://discord.gg/cline" target="_blank">Discord</a> |
29 | | -</td> |
30 | | -<td align="center"> |
31 | | -<a href="https://x.com/cline" target="_blank">@cline</a> |
32 | | -</td> |
33 | | -</tbody> |
34 | | -</table> |
35 | | -</div> |
36 | | - |
37 | | -### 1. Open kanban |
38 | | -```bash |
39 | | -# Run directly (no install required) |
40 | | -npx kanban |
| 1 | +# AgentPatchCheck |
| 2 | + |
| 3 | +AgentPatchCheck (APC) is an evidence-oriented runtime and evaluation toolkit |
| 4 | +for bounded software-repair agents. It separates agent execution, public |
| 5 | +verification, hidden evaluation, risk review, patch application, and official |
| 6 | +benchmark grading so that a completed agent turn is not mistaken for a correct |
| 7 | +repair. |
| 8 | + |
| 9 | +The project is a research preview. It is intended for controlled local and CI |
| 10 | +experiments, not unattended modification of production repositories. |
| 11 | + |
| 12 | +## Overview |
| 13 | + |
| 14 | +APC runs a repair task in an isolated Git worktree, exposes a bounded tool |
| 15 | +surface to the selected agent, records structured execution evidence, verifies |
| 16 | +the resulting patch, and keeps application to the target repository behind a |
| 17 | +separate approval step. Benchmark results preserve execution validity and |
| 18 | +grading validity independently from termination status. |
| 19 | + |
| 20 | +## Key Capabilities |
| 21 | + |
| 22 | +- Isolated, per-run Git worktrees and deterministic run identities. |
| 23 | +- Bounded agent iterations, tool calls, output, deadlines, and cancellation. |
| 24 | +- Harness-native, Codex, Cline, and script adapter boundaries already present |
| 25 | + in the repository. |
| 26 | +- Structured trajectories, evidence bundles, assessment reports, and risk |
| 27 | + policy checks with credential and sensitive-text redaction. |
| 28 | +- Public verification, hidden Oracle evaluation, patch capture, guarded apply, |
| 29 | + and evidence retention workflows. |
| 30 | +- Sequential benchmark orchestration, report comparison, SWE-bench prediction |
| 31 | + generation, and official evaluator integration. |
| 32 | + |
| 33 | +## Architecture |
| 34 | + |
| 35 | +The ordinary headless repair path is: |
| 36 | + |
| 37 | +```text |
| 38 | +TaskSpec |
| 39 | + -> policy validation |
| 40 | + -> isolated Git worktree |
| 41 | + -> selected agent adapter and bounded runtime |
| 42 | + -> public verification |
| 43 | + -> hidden Oracle / assessment |
| 44 | + -> EvidenceBundle and patch |
| 45 | + -> explicit approval and guarded apply |
| 46 | +``` |
| 47 | + |
| 48 | +The frozen SWE-bench path is: |
41 | 49 |
|
42 | | -# Or install globally |
43 | | -npm i -g kanban |
44 | | -kanban |
| 50 | +```text |
| 51 | +tracked manifest + operator environment |
| 52 | + -> src/agentpatchcheck/swebench-cli.ts |
| 53 | + -> APC Harness-native runtime and tooling |
| 54 | + -> prediction JSONL |
| 55 | + -> external official SWE-bench evaluator |
| 56 | + -> resolved / unresolved grading evidence |
45 | 57 | ``` |
46 | | -Run this from the root of any git repo. Kanban will detect your installed CLI agent and launch a local running webserver in your browser. No account or setup required, it works right out of the box. |
47 | 58 |
|
48 | | -### 2. Create tasks |
49 | | -Create a task card manually, or open the sidebar chat and ask your agent to break work down into tasks for you. Kanban injects board-management instructions into that session so you can simply ask it to add tasks, link tasks, or start work on your board. |
| 59 | +APC does not introduce a second evaluator or alternate runtime for published |
| 60 | +benchmark results. |
50 | 61 |
|
51 | | -### 3. Link and automate |
52 | | -<kbd>⌘</kbd> + click a card to link it to another task. When a card is completed and moved to trash, linked tasks auto-start. Combine with auto-commit for fully autonomous dependency chains: one task completes → commits → kicks off the next → repeat. It’s a pretty magical experience asking your agent to decompose a big task into subtasks that auto-commit - he’ll cleverly do it in a way that parallelizes for maximum efficiency and links tasks together for end-to-end autonomy. |
| 62 | +## What APC Implements |
53 | 63 |
|
54 | | -### 4. Start tasks |
55 | | -Hit the play button on a card. Kanban creates an ephemeral worktree just for that task so agents work in parallel without merge conflicts. Under the hood, it also symlinks gitignored files like `node_modules` so you don't have to worry about slow `npm install`s for each copy of your project. |
| 64 | +APC implements the bounded repair lifecycle, tool brokerage, execution and |
| 65 | +cancellation controls, evidence model, verification and assessment gates, |
| 66 | +risk-aware apply workflow, benchmark runner, SWE-bench adapter, dataset |
| 67 | +preparation boundary, prediction generation, and evaluator bridge contained |
| 68 | +under `src/agentpatchcheck/`, `scripts/`, and their tests. |
56 | 69 |
|
57 | | -> [!NOTE] |
58 | | -> [Symlinks (symbolic links)](https://en.wikipedia.org/wiki/Symbolic_link) are special "shortcuts" pointing to another file or directory, allowing access to the target from a new location without duplicating data. They work great in this case since you typically don't modify gitignored files in day-to-day work, but for when you do then don't use Kanban. |
| 70 | +## What APC Reuses |
59 | 71 |
|
60 | | -As agents work, Kanban uses hooks to display the latest message or tool call on each card, so you can monitor hundreds of agents at a glance without opening each one. |
| 72 | +- Cline Kanban's Apache-2.0 CLI, desktop, web UI, worktree, and task-management |
| 73 | + foundations. The original copyright and license are retained. |
| 74 | +- Selected DeepSeek Harness Code Mode mechanisms adapted under the MIT License. |
| 75 | + Details and the required notice are in `THIRD_PARTY_NOTICES.md`. |
| 76 | +- Installed provider SDKs and CLIs through explicit adapter boundaries. |
| 77 | +- The external official SWE-bench evaluator and upstream benchmark datasets. |
| 78 | + APC does not claim authorship of the evaluator and does not redistribute the |
| 79 | + complete benchmark datasets. |
61 | 80 |
|
62 | | -### 5. Review changes |
63 | | -Click a card to view the agent's TUI and a diff of all the changes in that worktree. Kanban includes its own checkpointing system so you can also see a diff from the last messages you've sent. Click on lines to leave comments and send them back to the agent. |
| 81 | +## Quick Start |
64 | 82 |
|
65 | | -To easily test and debug your app, create a Script Shortcut in settings. Use a command like `npm run dev` so that all you have to do is hit a play button in the navbar instead of remembering commands or asking your agent to do it. |
| 83 | +### Prerequisites |
66 | 84 |
|
67 | | -### 6. Ship it |
68 | | -When the work looks good, hit **Commit** or **Open PR**. Kanban sends a dynamic prompt to the agent to convert the worktree into a commit on your base ref or a new PR branch, and work through any merge conflicts intelligently. Or skip review by enabling auto-commit / auto-PR and the agent ships as soon as it's done. Move the card to trash to clean up the worktree (you can always resume later since Kanban tracks the resume ID). |
| 85 | +- Node.js 22 or newer. |
| 86 | +- npm 10 or newer. |
| 87 | +- Git. |
| 88 | +- An installed agent/provider required by the TaskSpec you choose to run. |
69 | 89 |
|
70 | | -### 7. Keep track with git interface |
71 | | -Click the branch name in the navbar to open a full git interface to browse commit history, switch branches, fetch, pull, push, and visualize your git all without leaving Kanban. Keep track of everything your agents are doing across branches as work is completed. |
| 90 | +Install the repository dependencies and build the existing CLIs: |
72 | 91 |
|
73 | | ---- |
| 92 | +```bash |
| 93 | +npm run install:all |
| 94 | +npm run build |
| 95 | +node dist/agentpatchcheck.js --help |
| 96 | +``` |
| 97 | + |
| 98 | +Run the smallest headless flow with a TaskSpec whose paths refer to your own |
| 99 | +checkout and target repository: |
| 100 | + |
| 101 | +```bash |
| 102 | +node dist/agentpatchcheck.js run --task-spec ./path/to/task-spec.json |
| 103 | +``` |
74 | 104 |
|
75 | | -[Apache 2.0 © 2026 Cline Bot Inc.](./LICENSE) |
| 105 | +See `docs/agentpatchcheck-headless-core.md` for the TaskSpec, evidence, |
| 106 | +assessment, approval, cleanup-preview, and guarded-apply contracts. |
| 107 | + |
| 108 | +### Provider environment variables |
| 109 | + |
| 110 | +Credentials are resolved at execution time from fixed logical references. Do |
| 111 | +not store credential values in TaskSpecs, manifests, evidence, or source files. |
| 112 | + |
| 113 | +| Logical credential reference | Environment variable | |
| 114 | +| --- | --- | |
| 115 | +| `openai-primary` | `OPENAI_API_KEY` | |
| 116 | +| `openai-secondary` | `OPENAI_API_KEY_SECONDARY` | |
| 117 | +| `deepseek-primary` | `DEEPSEEK_API_KEY` | |
| 118 | +| `gemini-primary` | `GEMINI_API_KEY` | |
| 119 | +| `provider-a-primary` | `AGENTPATCHCHECK_KEY_PROVIDER_A` | |
| 120 | +| `provider-b-primary` | `AGENTPATCHCHECK_KEY_PROVIDER_B` | |
| 121 | + |
| 122 | +Only configure the credential required by the selected TaskSpec or frozen |
| 123 | +provider profile. |
| 124 | + |
| 125 | +## Benchmark |
| 126 | + |
| 127 | +HAL SWE-bench Verified Mini fixed-50: |
| 128 | + |
| 129 | +- 35/50 official resolved (70.0%). |
| 130 | +- 50/50 valid executions. |
| 131 | +- 0 Harness-invalid. |
| 132 | +- 0 grading-invalid. |
| 133 | + |
| 134 | +This is the end-to-end result of `deepseek-v4-flash + APC Runtime/tooling + |
| 135 | +frozen execution policy`, as judged by the official SWE-bench evaluator. It is |
| 136 | +not a full SWE-bench Verified score, a DeepSeek model-only score, or evidence of |
| 137 | +an independently quantified APC uplift. |
| 138 | + |
| 139 | +### Reproducing the frozen SWE-bench path |
| 140 | + |
| 141 | +The repository tracks portable manifests, task identities, source revisions, |
| 142 | +and expected SHA-256 values. It intentionally does not track complete benchmark |
| 143 | +JSONL data, evaluator source, evaluator environments, repositories, images, |
| 144 | +predictions, logs, or scored artifacts. |
| 145 | + |
| 146 | +1. Obtain the HAL fixed-50 source data from the upstream |
| 147 | + `MariusHobbhahn/swe-bench-verified-mini` test split at revision |
| 148 | + `b316c349947c29963fce3f4a65967c9807a4b673`. Materialize the exact 50-row |
| 149 | + JSONL selected by the tracked manifest at |
| 150 | + `.agentpatchcheck/swebench/datasets/HAL-Verified-Mini-v1.full.jsonl` and |
| 151 | + verify its SHA-256 against the manifest before running. |
| 152 | +2. Clone or otherwise prepare the official SWE-bench evaluator outside this |
| 153 | + repository and check out frozen revision |
| 154 | + `7d92bde324b9b96d41fb3e5e1023c8476f17b0bf`. Use a Python environment that |
| 155 | + can import its Docker dependency and provide the evaluator's required Docker |
| 156 | + images separately. |
| 157 | +3. In the same shell, configure the existing bootstrap variables using paths |
| 158 | + on your machine: |
| 159 | + |
| 160 | + ```powershell |
| 161 | + $env:DEEPSEEK_API_KEY = "<secret>" |
| 162 | + $env:AGENTPATCHCHECK_SWEBENCH_MANIFEST = ".agentpatchcheck/swebench/datasets/HAL-Verified-Mini-v1.manifest.json" |
| 163 | + $env:AGENTPATCHCHECK_SWEBENCH_EVALUATOR_ROOT = "<path-to-swe-bench-checkout>" |
| 164 | + $env:AGENTPATCHCHECK_SWEBENCH_EVALUATOR_PYTHON = "<path-to-evaluator-python>" |
| 165 | + ``` |
| 166 | + |
| 167 | +4. From the AgentPatchCheck repository root, invoke the existing source |
| 168 | + entrypoint for exactly one instance and a fresh run identity: |
| 169 | + |
| 170 | + ```powershell |
| 171 | + node_modules/.bin/tsx src/agentpatchcheck/swebench-cli.ts ` |
| 172 | + --instance <instance-id-from-the-manifest> ` |
| 173 | + --run-id <fresh-run-id> |
| 174 | + ``` |
| 175 | + |
| 176 | +The manifest owns the dataset, evaluator revision, model, timeout, and frozen |
| 177 | +classification. Operator arguments cannot override those fields. Formal runs |
| 178 | +must follow the predeclared ordering and retry policy; the command above is not |
| 179 | +an instruction to rerun the published fixed-50 result. |
| 180 | + |
| 181 | +## Limitations |
| 182 | + |
| 183 | +- APC does not prove a patch correct merely because an agent finishes or emits |
| 184 | + a diff; correctness depends on verification and, for SWE-bench, official |
| 185 | + grading. |
| 186 | +- Provider access, evaluator setup, Docker images, upstream repositories, and |
| 187 | + benchmark data are external prerequisites. |
| 188 | +- The fixed-50 result measures one frozen end-to-end configuration and does not |
| 189 | + establish general performance, causal uplift, or state of the art. |
| 190 | +- The desktop and web UI foundations remain inherited from Cline Kanban while |
| 191 | + APC's release-facing workflow is centered on the headless repair runtime. |
| 192 | + |
| 193 | +## Project Status |
| 194 | + |
| 195 | +AgentPatchCheck is an open-source research preview. The current release target |
| 196 | +focuses on auditable, bounded software-repair execution and truthful benchmark |
| 197 | +reporting. Interfaces and operational requirements may change before a stable |
| 198 | +release. |
| 199 | + |
| 200 | +AgentPatchCheck is distributed under the Apache License 2.0. See `LICENSE` and |
| 201 | +`THIRD_PARTY_NOTICES.md` for retained upstream copyright and attribution. |
0 commit comments