feat(0.16.0): yank durable substrate, add AgentExecutionHandle contract

README.md

@@ -2,10 +2,11 @@
 
 Production runtime substrate for domain agents. Owns the task lifecycle
 (knowledge readiness, control loop, session resume, sanitized telemetry,
-durable runs across worker / DO crashes, canonical `RuntimeRunRow`
-persistence + cost ledger), the chat-model catalog + admission, and the
-declarative `defineAgent` manifest — so domain repos stop inventing their
-own.
+canonical `RuntimeRunRow` persistence + cost ledger), the chat-turn
+engine (NDJSON envelope + product hooks), the chat-model catalog +
+admission, and the declarative `defineAgent` manifest — so domain
+repos stop inventing their own. Long-running execution durability
+(reconnect, replay, dedup) lives in `@tangle-network/sandbox`.
 
 ```bash
 pnpm add @tangle-network/agent-runtime @tangle-network/agent-eval
@@ -17,12 +18,9 @@ pnpm add @tangle-network/agent-runtime @tangle-network/agent-eval
 |---|---|
 | `runAgentTask` | Single-shot adapter-driven task with eval/verification |
 | `runAgentTaskStream` | Streaming product loop with session resume + backends |
-| `runDurableTurn` | Checkpoint+replay chat turn — survives a worker crash *after* completion |
-| `runSupervisedTurn` | Always-attached durable turn — re-attaches an in-flight sandbox run *during* a crash |
-| `SessionSupervisorDO` | Cloudflare Durable Object host for `runSupervisedTurn` (with alarm-driven orphan re-attach) |
-| `DurableChatTurnEngine` | Framework-neutral chat-turn orchestrator (durable turn + NDJSON + session lifecycle + product hooks) |
+| `handleChatTurn` | Framework-neutral chat-turn orchestrator (NDJSON + `session.run.*` envelope + product hooks) |
+| `deriveExecutionId` | Stable substrate executionId for `X-Execution-ID` cross-process reconnect |
 | `startRuntimeRun` | Canonical production-run row + cost ledger |
-| `runDurable` + `*DurableRunStore` | General durable-step substrate (in-memory / file-system / D1) |
 | `defineAgent` | Declarative per-vertical agent manifest — surfaces, knowledge, rubric, run fn |
 | `resolveChatModel` / `validateChatModelId` / `getModels` | Router catalog fetch + fail-closed admission + precedence resolver |
 | `createTraceBridge` | Map `RuntimeStreamEvent` → `agent-eval` `TraceEvent` |
@@ -53,65 +51,54 @@ const result = await runAgentTask({
 console.log(result.status, result.runRecords)
 ```
 
-## Durable chat turns
+## Chat turns
 
-A 15-minute agentic turn must survive a Cloudflare worker isolate dying.
-`runDurableTurn` replays a *completed* turn from cache (worker died after
-the turn finished). `runSupervisedTurn` closes the harder gap — a turn
-interrupted *mid-stream* — by relocating the durability boundary off the
-ephemeral worker:
-
-- The supervisor drains every event into the substrate's own ordered log
-  (`appendStreamEvent`, idempotent on `eventId`).
-- It persists the substrate `RunHandle` the instant the sandbox yields it.
-- A fresh supervisor reads the log for its cursor and resumes via
-  `adapter.attach(handle, cursor)` — no event lost, none delivered twice.
-
-The reconnect glue is one typed contract — `SandboxReconnectAdapter` —
-implemented once per substrate, not per product.
+`handleChatTurn` wraps a product `produce()` hook with the `session.run.*`
+lifecycle envelope, drains the producer stream through the NDJSON line
+protocol, and calls the persist / post-process hooks after drain.
+Framework-neutral: takes already-resolved values, never a `Request` or
+`Context`.
 
 ```ts
-import { runSupervisedTurn, InMemoryDurableRunStore } from '@tangle-network/agent-runtime'
-
-const store = new InMemoryDurableRunStore()
-const supervised = runSupervisedTurn({
-  store, runId: `chat:${threadId}:${turnIndex}`, manifest, workerId,
-  adapter: mySandboxAdapter,
+import { handleChatTurn } from '@tangle-network/agent-runtime'
+
+const result = handleChatTurn({
+  identity: { tenantId: workspaceId, sessionId: threadId, userId, turnIndex },
+  hooks: {
+    produce: () => ({
+      stream: box.streamPrompt(prompt, sandboxOptions),
+      finalText: () => assembled,
+    }),
+    persistAssistantMessage: async ({ identity, finalText }) => db.insert(messages).values(...),
+    onTurnComplete: async ({ identity, finalText }) => extractProposals(finalText),
+    traceFlush: () => traceSink.flush(),
+  },
+  waitUntil: ctx.waitUntil,
 })
-for await (const event of supervised.stream) sendToClient(event)
-// supervised.mode() === 'fresh' | 'resumed' | 'replayed'
+return new Response(result.body, { headers: { 'content-type': result.contentType } })
 ```
 
-Full runnable: [`examples/durable-supervisor/`](./examples/durable-supervisor/).
+## Execution continuity
 
-### Cloudflare Durable Object host
+Long-running execution durability — reconnect, replay, dedup — lives in
+the substrate. `@tangle-network/sandbox`'s `box.streamPrompt`
+auto-reconnects in-call (extracts `executionId` from the response and
+replays via the runtime endpoint on drop). Cross-process reconnect —
+worker dies, a fresh worker resumes the same execution — requires
+either bypassing the SDK and POSTing directly with `X-Execution-ID`
+(see `tax-agent/sessions.ts`) or a future SDK release that surfaces the
+field on `PromptOptions`.
 
-`SessionSupervisorDO` hosts the supervisor on a real DO — `fetch` streams the
-turn, `alarm()` re-attaches a run a dropped response stream abandoned.
+`deriveExecutionId` is the convention helper for the stable id the
+product persists alongside its session row:
 
 ```ts
-import { createSessionSupervisorDO } from '@tangle-network/agent-runtime'
+import { deriveExecutionId } from '@tangle-network/agent-runtime'
 
-export const SessionSupervisor = createSessionSupervisorDO({
-  resolveRun(request, env, state)   { /* return RunSupervisorOptions */ },
-  resolveOrphan(runId, env, state)  { /* same, for the alarm path */ },
-  encodeEvent(event) { return `data: ${JSON.stringify(event)}\n\n` },
-})
-```
-
-```toml
-# wrangler.toml
-[[durable_objects.bindings]]
-name = "SESSION_SUPERVISOR"
-class_name = "SessionSupervisor"
-[[migrations]]
-tag = "v1"
-new_classes = ["SessionSupervisor"]
+const executionId = deriveExecutionId({ projectId, sessionId, turnIndex })
+// pass as `X-Execution-ID` header when calling the orchestrator directly
 ```
 
-CF types are structural (`DurableObjectStateLike`) — no
-`@cloudflare/workers-types` runtime dep.
-
 ## Chat-model resolution
 
 One primitive every chat handler needs and was hand-rolling per repo:
@@ -157,7 +144,7 @@ export const myAgent = defineAgent({
   knowledge: { /* requirements + provider */ },
   rubric: { /* dimensions + weights */ },
   run: async (ctx) => {
-    /* product-specific run — typically wraps runSupervisedTurn or runAgentTaskStream */
+    /* product-specific run — typically wraps handleChatTurn or runAgentTaskStream */
   },
 })
 ```
@@ -213,9 +200,6 @@ for await (const event of runAgentTaskStream({ task, backend, input })) {
 | `BackendTransportError` | Backend HTTP / IPC call returned non-success |
 | `SessionMismatchError` | Resume requested against a different backend |
 | `RuntimeRunStateError` | `RuntimeRunHandle` lifecycle methods called out of order |
-| `DurableRunLeaseHeldError` | Another worker holds a live lease on the run |
-| `DurableRunInputMismatchError` | A `runId` exists with a different manifest hash |
-| `DurableRunDivergenceError` | A step's intent changed across replays |
 
 All extend `AgentEvalError` (re-exported from `@tangle-network/agent-eval`)
 and carry a stable `code` so cross-package handlers pattern-match
@@ -240,7 +224,7 @@ console.log(telemetry.events, telemetry.summary())
 
 | Package | Owns |
 |---|---|
-| `agent-runtime` | Lifecycle, adapters, backends, durable substrate, supervisor + DO, model resolution, trace bridge, `defineAgent` |
+| `agent-runtime` | Task lifecycle, adapters, backends, chat-turn engine, execution-handle contract, model resolution, trace bridge, `defineAgent`. **Does not** own long-running execution state — that lives in `@tangle-network/sandbox` + orchestrator. |
 | `agent-runtime/platform` | Cross-site SSO (`PlatformAuthClient`) + integrations hub (`PlatformHubClient`) |
 | `agent-runtime/agent` | `defineAgent` + surfaces / outcome adapters |
 | `agent-runtime/analyst-loop` | `runAnalystLoop` — analyst registry driver |
@@ -263,16 +247,14 @@ Runnable in [`examples/`](./examples/). Every example imports from
 - [`openai-stream-backend/`](./examples/openai-stream-backend/) — `createOpenAICompatibleBackend`
 - [`runtime-run/`](./examples/runtime-run/) — production-run row + cost ledger
 - [`model-resolution/`](./examples/model-resolution/) — router catalog + fail-closed admission
-- [`durable-supervisor/`](./examples/durable-supervisor/) — cross-worker resume keystone
 - [`agent-into-reviewer/`](./examples/agent-into-reviewer/) — pipe one runtime's stream into a reviewer agent
-- [`chat-handler/`](./examples/chat-handler/) — `DurableChatTurnEngine.runTurn` (the centerpiece production pattern)
+- [`chat-handler/`](./examples/chat-handler/) — `handleChatTurn` (the centerpiece production pattern)
 - [`production-trace-sink/`](./examples/production-trace-sink/) — `createProductionTraceSink` data capture
 
 ## Tests
 
 ```bash
-pnpm test          # full Node suite (251 tests)
-pnpm test:workers  # real workerd DO integration test
+pnpm test
 pnpm typecheck
 pnpm lint
 pnpm build

docs/concepts.md

@@ -21,8 +21,15 @@ rest. Read this file once and the rest of the API falls into place.
    └───────────────────────────────────────┬─────────────────┘
                                            │
    ┌───────────────────────────────────────┴─────────────────┐
-   │  Durability     ─  runDurableTurn / runSupervisedTurn   │
-   │  + DurableRunStore + stream-event log + RunHandle        │
+   │  Chat-turn lifecycle ─  handleChatTurn(...)                 │
+   │  NDJSON + session.run.* envelope + persist/trace hooks   │
+   └───────────────────────────────────────┬─────────────────┘
+                                           │
+   ┌───────────────────────────────────────┴─────────────────┐
+   │  Execution continuity (substrate-owned)                  │
+   │  box.streamPrompt — auto-reconnect in-call; X-Execution-ID
+   │  header for cross-process. deriveExecutionId is the
+   │  convention helper.                                       │
    └───────────────────────────────────────┬─────────────────┘
                                            │
    ┌───────────────────────────────────────┴─────────────────┐
@@ -34,7 +41,7 @@ rest. Read this file once and the rest of the API falls into place.
 
 Each layer composes the one below it. You can use the bottom layers
 alone (a raw backend + the model catalog), or the whole stack
-(`defineAgent` → `runSupervisedTurn`) — they're the same primitives
+(`defineAgent` → `handleChatTurn`) — they're the same primitives
 nested.
 
 ## The task lifecycle
@@ -51,52 +58,33 @@ The adapter is *yours*. The lifecycle, the eval lift, the stop semantics,
 the cost ledger — all substrate. Streaming is the same shape:
 `runAgentTaskStream` yields `RuntimeStreamEvent`s as the loop progresses.
 
-## Durability — three levels
-
-A turn that "completes" means the response reached the client AND the
-side effects landed. A worker isolate can die anywhere in between. Pick
-the level that matches your turn length and substrate:
-
-| Level | Survives | When to reach for it |
-|---|---|---|
-| `runAgentTask` / `runAgentTaskStream` | nothing — a worker crash re-runs from the top | sub-second turns, no sandbox |
-| `runDurableTurn` | a worker crash *after* the turn finished (cached replay) | medium turns, no sandbox-reconnect |
-| `runSupervisedTurn` + `SessionSupervisorDO` | a worker crash *during* the turn — a fresh supervisor re-attaches to the in-flight sandbox run | long sandbox-backed turns |
-
-`runSupervisedTurn` works because the sandbox container is
-orchestrator-managed and **outlives** the worker. The supervisor:
-
-1. Drains every event into the substrate's own ordered log
-   (`appendStreamEvent`, idempotent on `eventId`).
-2. Persists a `RunHandle` (`setRunHandle`) the moment the substrate
-   yields a run id.
-3. Heartbeats the lease while attached.
-
-A fresh supervisor reads the log for its cursor and calls
-`adapter.attach(handle, cursor)` to resume past it — events through
-`cursor` are not re-delivered (the log's idempotency dedups the seam).
-
-## The reconnect adapter contract
-
-`SandboxReconnectAdapter` is one typed interface. Implement it **once
-per substrate** (the Tangle sandbox SDK, an OpenAI Assistants thread,
-whatever), never per product.
-
-```ts
-interface SandboxReconnectAdapter<TEvent> {
-  start(): AsyncIterable<SupervisedEvent<TEvent>>
-  attach(handle: RunHandle, afterEventId: string | undefined):
-    AsyncIterable<SupervisedEvent<TEvent>>
-}
-```
-
-`SupervisedEvent` carries an `eventId` (cursor + dedup key), a `payload`
-(your event type), and an optional `handle` (carried on the first frame
-once the substrate yields the run id).
-
-Conformance assertions live in `src/durable/tests/supervisor.test.ts` —
-copy them into your adapter's tests so substrate quirks surface there,
-not in a 15-minute production turn.
+## Execution continuity — substrate-owned
+
+Long-running execution durability — reconnect, replay, dedup — is the
+substrate's job, not agent-runtime's. The `@tangle-network/sandbox`
+SDK + orchestrator already handle it:
+
+- **In-call reconnect**: `box.streamPrompt` extracts `executionId` from
+  the response's `execution.started` event and replays via the runtime
+  endpoint if the stream drops. Transparent — callers do nothing.
+- **Cross-process reconnect**: a fresh Worker can resume a prior
+  Worker's execution by POSTing to the orchestrator's
+  `/agents/run/stream` with the `X-Execution-ID` header. The SDK's
+  public `PromptOptions` does not yet surface this; products bypass the
+  SDK and call the orchestrator directly when they need it (see
+  tax-agent's `sessions.ts`).
+- The orchestrator's buffer is 10k events / 2-min post-completion. A
+  retry past that window gets `execution_not_found` and re-runs.
+
+agent-runtime owns one helper, `deriveExecutionId({ projectId,
+sessionId, turnIndex })`, that produces the stable id the product
+persists on its session row.
+
+What lives in the Worker: auth, access control, product DB writes,
+prompt composition, routing. What lives in the substrate: the
+long-running execution, event buffering, replay-on-reconnect, dedup.
+The Worker stays a routing + persistence layer — it does not host
+execution state.
 
 ## The agent manifest
 
@@ -150,8 +138,8 @@ agents because nothing in this list is baked into it.
 
 1. `examples/basic-task/` — the smallest end-to-end.
 2. `examples/sandbox-stream-backend/` — what streaming looks like.
-3. `examples/runtime-run/` — the production-run row + cost ledger.
-4. `examples/model-resolution/` — pick + validate a model.
-5. `examples/durable-supervisor/` — the cross-worker resume keystone.
+3. `examples/chat-handler/` — `handleChatTurn` — the centerpiece chat handler.
+4. `examples/runtime-run/` — the production-run row + cost ledger.
+5. `examples/model-resolution/` — pick + validate a model.
 6. `examples/agent-into-reviewer/` — pipe one runtime's stream into a reviewer agent.
 7. The `README.md` entry-point table — every other primitive, one row each.

examples/README.md

@@ -15,9 +15,8 @@ which needs an `OPENAI_API_KEY`.
 | [`openai-stream-backend/`](./openai-stream-backend/) | `runAgentTaskStream` with `createOpenAICompatibleBackend` (real endpoint required) |
 | [`runtime-run/`](./runtime-run/) | `startRuntimeRun` + cost ledger + persistence adapter |
 | [`model-resolution/`](./model-resolution/) | `resolveChatModel` + `validateChatModelId` (fail-closed) + `getModels` |
-| [`durable-supervisor/`](./durable-supervisor/) | `runSupervisedTurn` — cross-worker resume keystone (fresh / resumed / replayed) |
 | [`agent-into-reviewer/`](./agent-into-reviewer/) | Pipe one runtime's stream into a reviewer agent (the "2-runtime" pattern) |
-| [`chat-handler/`](./chat-handler/) | `DurableChatTurnEngine.runTurn` — the centerpiece production chat handler (fresh / replay paths) |
+| [`chat-handler/`](./chat-handler/) | `handleChatTurn` — the centerpiece production chat handler |
 | [`production-trace-sink/`](./production-trace-sink/) | `createProductionTraceSink` — production data capture (RunRecord + OTLP + feedback) |
 
 ## Conventions
@@ -45,7 +44,6 @@ pnpm tsx examples/sse-stream/sse-stream.ts
 pnpm tsx examples/sandbox-stream-backend/sandbox-stream-backend.ts
 pnpm tsx examples/runtime-run/runtime-run.ts
 pnpm tsx examples/model-resolution/model-resolution.ts
-pnpm tsx examples/durable-supervisor/durable-supervisor.ts
 pnpm tsx examples/agent-into-reviewer/agent-into-reviewer.ts
 pnpm tsx examples/chat-handler/chat-handler.ts
 pnpm tsx examples/production-trace-sink/production-trace-sink.ts

examples/chat-handler/README.md

@@ -1,28 +1,12 @@
-# Durable chat handler
+# Chat handler
 
-The centerpiece production pattern every product chat handler
-implements. `DurableChatTurnEngine.runTurn` composes the substrate
-stack:
-
-- Builds the durable manifest from the chat identity (`tenantId` /
-  `sessionId` / `turnIndex`).
-- Drives `runDurableTurn` — checkpoint + replay.
-- Emits `session.run.*` lifecycle events around the producer stream.
-- Returns a `ReadableStream` of NDJSON-encoded `ChatStreamEvent`s — the
-  shape your HTTP/SSE route forwards verbatim.
-
-The example shows:
-
-- A fresh turn streaming events (the dots are `message.part.updated`).
-- A second, different turn — a fresh `runId`, the producer runs again.
-- A retry of turn 0 — same identity → same `runId` → the **replay path**
-  emits the cached final text without re-running the producer.
+`handleChatTurn` wraps a product `produce()` hook with the `session.run.*`
+lifecycle envelope, drains the producer stream through the NDJSON line
+protocol, and calls the persist / post-process hooks after drain.
 
 In production, `produce()` is a thin wrapper over `runAgentTaskStream(...)`
 against a real backend (`createOpenAICompatibleBackend` /
-`createSandboxPromptBackend`). For the cross-worker-during-turn case —
-worker dies *while* streaming — use `runSupervisedTurn` (see
-`examples/durable-supervisor/`).
+`createSandboxPromptBackend`).
 
 ```bash
 pnpm tsx examples/chat-handler/chat-handler.ts