Skip to content

feat(eve): replace Workflow with durable code mode - #3002

Open
ruiconti wants to merge 21 commits into
mainfrom
rui/code-mode-workflow-tool
Open

feat(eve): replace Workflow with durable code mode#3002
ruiconti wants to merge 21 commits into
mainfrom
rui/code-mode-workflow-tool

Conversation

@ruiconti

@ruiconti ruiconti commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Tool orchestration needs to await subagents and resume completed work after a crash. Previously, the experimental Workflow wrapper only orchestrated agents; this replaces it with a root-only code_mode workflow tool that discovers the full advertised catalog and invokes eligible tools and subagents. Enable it with experimental.codeMode: { mode: "eager" | "lazy", maxSubagents?: number }. Related to #2508.

Programs pin their catalog and subagent budget when dispatched. Ordinary calls execute in separate durable steps; subagent calls use the owning session's agent-invoke channel and await the child result. Calls issued together settle concurrently. Authorization can park and resume a call, and individual failures remain catchable by the program.

  • Discovery includes every advertised tool in both modes. Approval-gated tools, provider tools, framework controls, ordinary background tools, and authored workflow tools remain direct and are marked requiresDirectCall.
  • Connection discovery updates the next model step's catalog. Dynamic tools retain the captured values from the dispatching step.
  • Each program permits 100 subagent invocations by default. Retries and continuations count; excess calls reject before dispatch. maxSubagents changes the limit.
  • Breaking migration: remove the file exporting experimental_workflow() from eve/tools/workflow, then enable code mode in agent.ts. The old helper and harness-specific Workflow continuation machinery are removed. Authored "use workflow" tools remain supported.
  • Workflow compilation preserves availability metadata, so child sessions keep their ordinary tools and never receive the root-only code_mode wrapper.

Programs previously ran guest catch/finally handlers while parking a tool call, and deterministic program failures consumed workflow retries. The pinned Run patch now treats suspension as host control flow and marks guest-source errors separately from infrastructure errors. eve returns known program failures from the step as data, then reports the tool error to the model so it can revise the source. The one-program orchestration sentence is restored; eager still exposes eligible direct tools. The vendoring cache now fingerprints the installed worker so a same-version patch rebuilds the shipped runtime. Readable patch and reproduction.

Scripted Code Mode E2Es cover execution contracts. The real-model eager/lazy evals check complete pagination, overlapping independent reads, handling an unavailable source, and correct saved reports. They log program grouping and repeated page reads for diagnosis; they do not require a single program or benchmark savings against direct tools.

Optional tool arguments now retain their omission semantics at the provider boundary. eve explicitly disables implicit strict-mode normalization and still validates inputs before execution.

Validation

  • Rebased onto 4b5fad41b at ed2894a46: all 47 workspace build/typecheck tasks, 141 focused unit tests, 13 compiled-SDK integration tests, and the installed-extension scenario pass. Frozen installation, formatting/lint, docs and invariant checks pass. Published compatibility reports are preserved: new tool epoch 31 and dynamic-tool epoch 30 replace the removed Workflow contracts; subagent epoch 10 retains main's epoch 9. The new steering fixture now imports the cancellation tool from its own fixture after the sleeper fixture moved. GitHub reports no merge conflicts; required CI checks are pending.

  • Suspension/recovery repair at 0c77a2645: 63 focused eve unit tests, 13 compiled-SDK integration tests, and the packaged Code Mode worker-assets scenario pass. The underlying Run repair passes all 336 tests and its typechecks. eve runtime build, package and scripted-fixture typechecks, changed-file formatting/lint, docs checks and invariant checks pass. Independent source review found no blocking lifecycle/classification defect. The same compiled-SDK catch probe that timed out at 30s on the baseline now completes in 16.7ms; catch/finally and the exact 120,418-byte payload with eight distinct leaf calls pass. This is an in-process interruption/resolution probe, not a full workflow-crash or model-performance benchmark. Two scripted E2Es were added; CI is running on the rebased head.

  • Optional-argument fix: 41 tool unit tests and three input-validation integration tests pass; the new provider-boundary regression failed before the fix. Package and fixture typechecks, lint, docs, invariant checks, and the runtime build pass. The four-call Terra probe produced 0/2 valid inputs with strict omitted and 2/2 with strict: false. A real-model fixture eval was added; no eve test workflow was recorded for the benchmark baseline 143b0e5a7.

  • pnpm --filter eve exec vitest run --config vitest.unit.config.ts --maxWorkers=4: 7,985 passed, one skipped. The final child-visibility test adjustment also passed its focused harness run.

  • Targeted integration runs: 40 passed across code-mode sandbox/connection execution, turn and session-limit cancellation, cancelled-turn settlement, and authored workflow tools.

  • Targeted packaged scenarios: seven passed, covering local/remote child continuation, cancellation, code-mode compilation and sandbox assets, and installed extension compatibility.

  • pnpm --filter eve build:js, package tsc --noEmit, changed-file formatting/lint, pnpm guard:invariants, and pnpm docs:check passed.

  • pnpm --filter agent-code-mode --filter agent-cancellation --filter agent-subagents typecheck passed, including fixture builds. E2E evals were added/migrated and are left to CI.

  • New planning fixtures: both builds and typechecks passed with EVE_E2E_MODEL=mock; five assertion unit tests and two audit scenarios passed. Matrix discovery includes both fixtures on both registered models. Live-model eval results are pending CI; no E2E suite was run locally.

  • After rebasing onto eb5949a62: frozen installation, all 46 workspace build/typecheck tasks, invariant checks, changed-file formatting/lint, and docs checks passed. Focused coverage passed: 68 compiler/Code Mode unit tests, 16 Code Mode integration tests, the installed-extension scenario, and seven planning-fixture tests. Published extension history is preserved; the tool and dynamic-tool contracts now use epoch 28.

Benchmark diagnosis: GPT-5.6 Terra, 2026-09-06

At baseline 143b0e5a7, optional eager never selected Code Mode in 20 controlled attempts. Restoring the one-program instruction reduced lazy root-turn time by 33%, with 8/8 correct answers in both conditions. Forcing Code Mode was not uniformly beneficial.

The tables below are measurements of that baseline and its explicitly described experimental variants. The repair at 0c77a2645 has deterministic regression validation above; it has not been rerun through these model benchmarks.

The old Fable pilot and the newer Terra run are not a controlled comparison. The task files are unchanged (594 manifest files verified), but the model, eager tool availability, orchestration wording and runtime artifact changed. The pilot used an uncommitted experimental package with a local suspension fix; the benchmark baseline lacked that repair. This does not identify an upstream commit that introduced the bug.

Controlled selection experiment: 80 attempts. Same Terra model with high reasoning, runtime engine, tools and ten frozen cases; two repetitions across five task shapes and two seeds. Each column has 20 attempts. These are attempt success counts, not best-of-two scores. Every attempt and recovery stays in the timing totals.

Direct disables Code Mode. Optional eager is the implementation at baseline 143b0e5a7. Forced eager hides eligible direct tools. Guidance retains both tool surfaces and repeats the current selection/orchestration text in the main instructions; that intervention changes repetition and placement together. The latter two are experimental variants, not changes included in this PR.

Measurement Direct Optional eager Forced eager Guidance
Agent lifecycle, mean seconds 20.72 22.97 19.86 18.83
Root turn, mean seconds 12.80 14.96 11.91 10.75
Input tokens, total 529,059 816,375 404,617 533,075
Cached input, included above 405,467 665,433 341,995 436,711
Output tokens, total 23,583 26,456 18,646 20,397
Model steps 78 76 61 52
Code Mode programs 0 0 43 12
Failed dispatched actions 0 0 2 0
Root-model cost, USD 0.6730 0.8278 0.4486 0.5729
Attempts using Code Mode 0/20 0/20 20/20 12/20
Original passes 19/20 20/20 19/20 19/20
Passes after separate grading audit 20/20 20/20 19/20 19/20

Root-turn duration by shape; four attempts per condition in each row:

Shape Direct Optional eager Forced eager Guidance
Single account 3.44s 4.20s 4.12s 3.71s
One-project summary 6.46s 7.15s 7.04s 7.28s
Eight-project summary 8.05s 10.92s 8.54s 7.42s
32-project summary 16.88s 19.71s 13.94s 12.42s
32-project inventory 29.15s 32.84s 25.90s 22.92s

Optional eager used Code Mode zero times and added 3,673 input tokens to the first request in every first-repetition pair. The tool construction exposes both direct schemas and the Code Mode description. Forced Code Mode won three of four inventory comparisons; the fourth recovered from two syntax errors and took 50.63s versus direct’s 38.93s.

The direct large-task traces emit 32 domain calls within one model step, as separate streamed requests. This already avoids 32 separate model round trips; it does not prove simultaneous network execution. Code Mode can cross dependency boundaries and reduce intermediate data. These local synthetic fixtures do not establish production API latency or a universal fanout threshold.

Orchestration experiment: 24 attempts. All conditions use lazy mode, the same engine and discovery tools, and one/32-project summaries with two seeds and two repetitions. One intervention restores only the first orchestration sentence; the other restores the complete old three-sentence block from 287c074.

Measurement Current wording First sentence restored Full old block restored
Agent lifecycle, mean seconds 20.46 16.13 19.74
Root turn, mean seconds 12.83 8.62 12.23
Input tokens 113,697 71,838 99,470
Cached input, included above 92,015 49,582 71,204
Output tokens 4,265 4,322 5,621
Model steps 41 26 33
Discovery-only programs 10 8 10
Work programs 23 10 15
Failed dispatched actions 0 0 1
Root-model cost, USD 0.1237 0.1174 0.1523
Original passes 8/8 8/8 8/8

Restoring only the first sentence reduced work programs from 23 to 10 and model steps from 41 to 26, with nearly unchanged fixture work (148 versus 150 calls). Root-turn duration fell from 12.83s to 8.62s; seven of eight matched pairs were faster and all answers passed. The full old block averaged 12.23s and recovered from one invalid program after four workflow attempts (3.177s of failed-action time, retained). This supports a small orchestration change; it does not solve eager selection. The first-sentence intervention says to complete the task in one execution program and retains the current reuse instruction. Restoring the full block also adds discovery/retry wording and removes that reuse sentence; its individual clauses are not isolated. Two repetitions on fixed cases establish observed behavior, not a universal policy or statistical significance.

Confirmed failures and remaining defects. One direct answer was correct but the grader extracted an array instead of an object; its correction is reported separately. Forced and guidance each failed one summary by aggregating only the first 100 of 725 global domains. Native replay of those programs reproduces the wrong result, so these are agent pagination mistakes.

The two invalid programs in the recovered inventory trial each received four workflow attempts. Their failed-action intervals totaled 6.482s, including the first attempts and retry delays. The repair at 0c77a2645 returns known deterministic program failures to the model without repeating unchanged source. The old experimental error classifier did not cover the RUN_ERROR returned by these syntax errors, so restoring that classifier alone is insufficient.

A separate suspension defect reproduced in the baseline packaged SDK: certain try/catch or finally programs time out before a leaf tool executes. The catch reproducer timed out at 30s in the baseline package and completes in 17ms with the pilot’s repaired package; ordinary execution controls pass. None of the 55 Code Mode programs in the 80-trial selection run uses that pattern, and no timeout occurs there. The repair above addresses this defect; it was not the cause of those observed timings.

Measurement and provenance. Agent lifecycle includes startup and cleanup; root turn excludes both. Cleanup averages roughly 5.4s. Environment setup, building and grading are excluded from both durations. Input totals include cached context; output totals include reasoning. Provider caching stays enabled. Trials run sequentially with shuffled case order and rotated condition order; there are no full-trial retries. Internal workflow retries remain measured.

Recorded model + grader spend for these 104 attempts: $3.4635 ($3.0398 selection; $0.4237 orchestration). Combined recorded-spend stop $5; per-session stop $0.25. Stops are checked after charges, not hard billing caps.

Runtime commit: 143b0e5a7dc8a7335063b6d9dc2c3b54845cb1f9. Base archive SHA-256: 93a537f044ea2a45808b5e539e812c376b21bebf2db44a72a11045711f327bc8. Dataset SHA-256: 107296bae88bfad6b6e26b4aa8f4047f479702e2fe88ce8b54d711d93eafecd5. Harbor: 2fe1615503fed39ad82b7ce09b22497996b30f1f. Saved runtime archives, dependency locks, instructions, agent definitions and adapters match each experiment’s manifest. Raw grades remain unchanged.

Earlier full 18-case Terra snapshot at the same commit

This snapshot has one attempt per case/mode; its aggregate covers a different case mix from the controlled ten-case diagnostic above.

Measurement Direct Eager Lazy
Agent lifecycle, mean seconds 21.0 21.9 24.2
Root turn, mean seconds 13.4 14.3 16.5
Input tokens 377,864 762,582 269,641
Output tokens 18,812 19,405 17,990
Code Mode programs 0 1 71
Root-model cost, USD 0.5154 0.7033 0.4365
Original passes 18/18 18/18 17/18
Passes after separate audit 18/18 18/18 18/18

The lazy false negative was an extracted status label. Total recorded model + grader spend was $1.9670, separate from the 104-attempt diagnostic. The earlier Mini run had an invalid direct control and cannot establish a Code Mode speedup. The optional-argument schema fix at this commit was separately reproduced and validated; no real-model CI E2E result is claimed here.

Checklist

  • This change was requested or approved by a maintainer
  • I ran the relevant checks from CONTRIBUTING.md
  • I added tests and documentation where relevant
  • I added a changeset if this touches the published eve package
  • DCO sign-off passes for every commit (git commit --signoff)

Diff size

Docs — 11 files · +179 / -25

Documents Code Mode configuration, migration, selection guidance, and optional tool arguments.

Implementation — 87 files · +2033 / -1299

Replaces the Workflow wrapper with durable Code Mode execution and its tool/catalog integration; preserves optional arguments at the provider boundary. Includes the generated Run worker patch and its readable upstream source/test patch.

Tests — 124 files · +3987 / -1358

Covers runtime contracts, cancellation, tool visibility, real-model planning fixtures, optional-argument validation, suspension and program-error recovery. Counts fixture files with tests; the upstream regression tests embedded in the readable dependency patch are counted with that patch.

@vercel

vercel Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
eve-docs Ready Ready Preview, v0 Sep 7, 2026 3:53am UTC
eve-docs-4759 Ready Ready Preview, v0 Sep 7, 2026 3:53am UTC
eve-pkg Ready Ready Preview, v0 Sep 7, 2026 3:53am UTC

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

  • Runtime delta: function payloads 20.72 MB -> 21.10 MB (+380.3 kB ⚠️).

Delta vs main (4b5fad4)

Area Metric Baseline Current Delta
Package Packed tarball 8.65 MB 8.65 MB +2.6 kB ⚠️
Package Unpacked publish size 32.59 MB 32.60 MB +8.5 kB ⚠️
Package Installed footprint 77.67 MB 77.68 MB +8.5 kB ⚠️
Package Published files 3847 3845 -2
Package Installed files 7745 7743 -2
Package Installed package instances 33 33 0
Package Distinct installed package names 32 32 0
Package Installed dependency edges 51 51 0
Package Installed optional peer edges 7 7 0
Runtime Unique function payloads 2 2 0
Runtime Total function bytes 20.72 MB 21.10 MB +380.3 kB ⚠️
Runtime Public routes 18 18 0
Changed function payloads vs main (4b5fad4) (2)
Function Status Baseline Current Delta Route changes
functions/__server.func changed 10.36 MB 10.55 MB +190.1 kB ⚠️ none
functions/.well-known/workflow/v1/flow.func changed 10.36 MB 10.55 MB +190.1 kB ⚠️ none

eve init install

Metric Baseline Current Delta
Installed footprint 116.11 MB 116.12 MB +8.5 kB ⚠️
Installed packages 113 113 0
dependencies 4 4 0
devDependencies 2 2 0
Dependency package bytes 48.72 MB 48.73 MB +8.5 kB ⚠️
devDependency package bytes 5.04 MB 5.04 MB 0 B ➖
Build Metadata
  • Preset: vercel
  • Nitro: nitro@3.0.260903-beta
  • Output directory: apps/fixtures/weather-agent/.vercel/output
  • Build metadata timestamp: 2026-09-07T03:54:09.218Z
  • Route aliases: 18 public, 1 internal (19 total aliases)
  • Vercel routes in config: 21
  • Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down

Package Details

  • Package: eve@0.52.2
  • Package directory: packages/eve
  • Tarball: 8.65 MB (eve-0.52.2.tgz)
  • Unpacked payload: 32.60 MB across 3845 published files
  • Installed footprint: 77.68 MB across 7743 installed files
  • Installed root package: 31.17 MB
  • Installed dependencies: 46.51 MB
  • Installed package instances: 33
  • Distinct installed package names: 32
  • Installed dependency edges: 51
  • Installed optional peer edges: 7
  • Runtime dependencies: 2
  • Peer dependencies: 5 (4 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.
Graph metrics read only package.json files in package directories directly beneath a node_modules boundary, including nested boundaries. Each directory is one package instance; distinct names come from those manifests. Dependency edges count each unique name in dependencies or optionalDependencies per instance; optional peer edges count peerDependencies marked optional.

Heavy installed dependencies

  • eve: 31.17 MB (40.1%)
  • @rolldown/binding-linux-x64-gnu: 19.31 MB (24.9%)
  • ai: 7.01 MB (9.0%)
  • zod: 6.41 MB (8.3%)
  • undici: 3.51 MB (4.5%)
Publish payload breakdown
Published file size
🔴 dist/src/compiled/shadcn-registry/index.js       [#############...........] 9.76 MB 29.9%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [###.....................] 2.27 MB 7.0%
🟠 dist/src/compiled/@ai-sdk/code-mode/index.js     [#.......................] 1.03 MB 3.1%
🟡 dist/src/compiled/@vercel/blob/index.js          [#.......................] 901.4 kB 2.8%
🟡 dist/src/compiled/_chunks/workflow/signal-exi... [#.......................] 514.6 kB 1.6%
🔴 Other published files                            [########################] 18.14 MB 55.6%
Installed footprint breakdown
Installed package size
🔴 eve                             [########################] 31.17 MB 40.1%
🔴 @rolldown/binding-linux-x64-gnu [###############.........] 19.31 MB 24.9%
🔴 ai                              [#####...................] 7.01 MB 9.0%
🔴 zod                             [#####...................] 6.41 MB 8.3%
🟠 undici                          [###.....................] 3.51 MB 4.5%
🟠 nitro                           [#.......................] 1.89 MB 2.4%
🔴 Other installed packages        [######..................] 8.36 MB 10.8%
Runtime dependencies (2)
Package Range Notes
nitro 3.0.260903-beta
undici 8.9.0
Peer dependencies (5)
Package Range Notes
@opentelemetry/api ^1.0.0 optional peer
ai catalog:
braintrust ^3.0.0 optional peer
just-bash ^3.1.0 optional peer
microsandbox ^0.5.0 optional peer
eve init install drill-down

eve init install details

  • Command: eve init my-agent
  • Package manager: npm
  • Installed footprint: 116.12 MB across 9639 installed files
  • Installed packages: 113 total (107 transitive-only)
  • dependencies: 4 direct packages totaling 48.73 MB
  • devDependencies: 2 direct packages totaling 5.04 MB
  • Other transitive package files: 62.35 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

  • eve: 31.17 MB (26.8%)
  • @typescript/typescript-linux-x64: 27.95 MB (24.1%)
  • @rolldown/binding-linux-x64-gnu: 19.31 MB (16.6%)
  • zod: 10.37 MB (8.9%)
  • ai: 7.01 MB (6.0%)
Installed footprint breakdown
Installed package size
🔴 eve                              [########################] 31.17 MB 26.8%
🔴 @typescript/typescript-linux-x64 [######################..] 27.95 MB 24.1%
🔴 @rolldown/binding-linux-x64-gnu  [###############.........] 19.31 MB 16.6%
🔴 zod                              [########................] 10.37 MB 8.9%
🔴 ai                               [#####...................] 7.01 MB 6.0%
🟠 undici                           [###.....................] 3.51 MB 3.0%
🔴 Other installed packages         [#############...........] 16.79 MB 14.5%
dependencies (4)
Package Range Installed size Share
@vercel/connect 1.0.0 167.9 kB 0.1%
ai ^7.0.82 7.01 MB 6.0%
eve file:eve-0.52.2.tgz 31.17 MB 26.8%
zod 4.5.4 10.37 MB 8.9%
devDependencies (2)
Package Range Installed size Share
@types/node 24.x 2.54 MB 2.2%
typescript 7.0.2 2.50 MB 2.2%
Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 10.55 MB 50.0%
🔴 functions/__server.func                         [########################] 10.55 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 10.55 MB
Metric Value
Public routes /.well-known/workflow/v1/flow
Runtime nodejs24.x
Handler index.mjs
Payload 10.55 MB
Function files 10.55 MB across 116 files
Traced dependencies 0 B
Signal 🟠 Bundled file index.mjs is 2.02 MB (19.2%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                        [#########...............] 2.02 MB 19.2%
🟡 _libs/undici.mjs                 [####....................] 980.8 kB 9.3%
🟡 _chunks/esm-Fqlolk7e.mjs         [###.....................] 723.3 kB 6.9%
🟡 _chunks/chatgpt-model.mjs        [###.....................] 693.8 kB 6.6%
🟡 _chunks/signal-exit-Dsy-TT0V.mjs [###.....................] 616.2 kB 5.8%
🔴 Other bundled files              [########################] 5.51 MB 52.3%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 17 public routes, 1 internal alias • 10.55 MB
Metric Value
Public routes /
/.well-known/workflow/v1/webhook/[token]
/eve/v1/activity/[token]
/eve/v1/callback/[token]
/eve/v1/connections/[name]/callback/[attemptId]/[token]
/eve/v1/connections/[name]/callback/[token]
/eve/v1/health
/eve/v1/info
/eve/v1/session
/eve/v1/session/[parentSessionId]/subagents/[callId]/[childSessionId]/stream
/eve/v1/session/[sessionId]
/eve/v1/session/[sessionId]/cancel
/eve/v1/session/[sessionId]/clear
/eve/v1/session/[sessionId]/compact
/eve/v1/session/[sessionId]/reset
/eve/v1/session/[sessionId]/stream
/eve/v1/task-input/[token]
Internal aliases /__server
Runtime nodejs24.x
Handler index.mjs
Payload 10.55 MB
Function files 10.55 MB across 116 files
Traced dependencies 0 B
Signal 🟠 Bundled file index.mjs is 2.02 MB (19.2%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                        [#########...............] 2.02 MB 19.2%
🟡 _libs/undici.mjs                 [####....................] 980.8 kB 9.3%
🟡 _chunks/esm-Fqlolk7e.mjs         [###.....................] 723.3 kB 6.9%
🟡 _chunks/chatgpt-model.mjs        [###.....................] 693.8 kB 6.6%
🟡 _chunks/signal-exit-Dsy-TT0V.mjs [###.....................] 616.2 kB 5.8%
🔴 Other bundled files              [########################] 5.51 MB 52.2%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

  • Build pipeline: 6.19 s -> 6.13 s (-59.0 ms) vs main (4b5fad4).
  • Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `main (4b5fad4)`
Phase Baseline Current Delta
extension.check 12.4 ms 0.9 ms -11.5 ms
project.resolve 0.4 ms 0.4 ms 0.0 ms
workspace.create 0.6 ms 0.6 ms 0.0 ms
host.prepare 855.0 ms 671.9 ms -183.1 ms
vercel.service-prefix.resolve 2.6 ms 2.7 ms +0.1 ms
nitro.create 572.7 ms 618.8 ms +46.1 ms
sandbox.prewarm 250.6 ms 293.7 ms +43.1 ms
nitro.cache.prepare 0.3 ms 0.3 ms 0.0 ms
nitro.prepare 0.9 ms 0.8 ms -0.1 ms
nitro.public-assets 0.8 ms 0.8 ms 0.0 ms
nitro.prerender 0.5 ms 0.5 ms 0.0 ms
nitro.bundle 4.44 s 4.48 s +41.5 ms
nitro.cache.write 0.3 ms 0.4 ms +0.1 ms
vercel.workflow-function.materialize 47.3 ms 51.7 ms +4.4 ms
agent-summary.emit 0.7 ms 0.7 ms 0.0 ms
nitro.close 0.2 ms 0.1 ms -0.1 ms
output.publish 3.4 ms 3.7 ms +0.3 ms
workspace.remove 2.0 ms 2.3 ms +0.3 ms

@ruiconti ruiconti added the acknowledge-bundle-warning Acknowledge an intentional bundle footprint increase label Sep 3, 2026
@ruiconti
ruiconti force-pushed the rui/code-mode-workflow-tool branch from 5a2b9cd to 1e4f5a9 Compare September 3, 2026 21:31
@github-actions github-actions Bot removed the acknowledge-bundle-warning Acknowledge an intentional bundle footprint increase label Sep 3, 2026
@ruiconti ruiconti added the acknowledge-bundle-warning Acknowledge an intentional bundle footprint increase label Sep 3, 2026
@github-actions github-actions Bot removed the acknowledge-bundle-warning Acknowledge an intentional bundle footprint increase label Sep 4, 2026
@ruiconti ruiconti changed the title feat(eve): run code mode as a durable workflow tool feat(eve): replace Workflow with durable code mode Sep 4, 2026
`experimental.codeMode` moves eligible tools behind one framework `code_mode`
tool whose execute body is a workflow. The generated program runs in the
code-mode sandbox with every claimed tool replaced by an interrupt stub, so
the body parks at each nested call and resumes it on its own step boundary:
ordinary tools execute in a child step over the turn's serialized context;
subagent calls go through the owner's `agent-invoke` channel like any
workflow tool, so a program can launch and await subagents while the parent
keeps sole ownership of agent handles and session state.

The parent materializes the sandbox before dispatching `code_mode` so the
child reconnects to the recorded session instead of creating a second one.
The claimed catalog is pinned into the durable input at advertisement time.

Adding `codeMode` to `AgentExperimentalDefinition` bumps the subagent
extension epoch to 9; epoch 8 stays supported.

Signed-off-by: Rui Conti <ruiconti@gmail.com>
A `Promise.all` in a generated program parks every call in one sandbox
continuation. The body previously took the first pending call, settled it,
and resumed — so three subagents the model meant to run in parallel ran
one after another. `runCodeModeProgramStep` now returns the whole pending
batch; the body settles them together (owner `agent-invoke` requests and
child tool steps in flight at once) and hands all resolutions back in one
resume.

Resuming feeds each resolution the interrupt produced by the previous
`continue`, since the signed ledger advances per resolution; the sandbox
only re-runs the program after the last one. Replayed stubs return the
settled value from `codeModeInterrupt.resolution` instead of raising again,
which is what lets the program advance past the park point.

An integration test pins the sandbox contract this relies on: a batch parks
together, intermediate continues do not run the program, resolutions are
rejected for the wrong interrupt or a drifted tool catalog, and the program
observes every value exactly once.

Signed-off-by: Rui Conti <ruiconti@gmail.com>
Add the `agent-code-mode` fixture: a deterministic mock script drives one
`code_mode` call per directive and echoes its result. Evals cover the model
surface (claimed tools leave it, approval-gated ones stay), a single nested
call, a chain that threads one call's output into the next, and a
`Promise.all` over two subagents and a tool that asserts both children start
before `code_mode` resolves and every value comes back.

Running it exposed two wiring gaps. Coordination read the pre-advertisement
harness map, so the workflow task never carried code mode's pinned catalog
and the body rejected its input; the tool loop now hands coordination the
map `getAdvertisedTools` returned. Declared subagents are background tools
on the direct surface, so the "blocking only" rule excluded every real
subagent from programs; any `resultKind: "subagent"` tool is now claimable,
since the body awaits it through the owner channel regardless.

Child tool steps now receive the turn's message history in their execute
options, matching what a direct call sees.

Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants