test(e2e-agent-eval): cover local durable functions

tests/e2e-agent-eval/README.md

@@ -33,16 +33,18 @@ automatically on first run.
 | `relational-db` | blog (posts + comments) → foreign-key relationship + query-by-FK round-trip |
 | `variables` | function that consumes a project variable → variable set correctly + read at runtime |
 | `scheduled-function` | cron-scheduled function that writes to the DB → schedule registers + fires |
+| `durable-function` | local durable function → checkpointed steps + fresh execution result |
 | `function-generator` | QR-code generator function (npm-bundled + binary) that stores to a bucket |
 | `nextjs-local` | Next.js auth + todo web app → frontend compiles with the SDK + wired to the local API |
 | `oauth-login` | Google OAuth sign-in web app → provider configured + app builds + authorize redirect |
 | `realtime-local` | live chat app → realtime enablement + postgres-changes delivery round-trip |
 | `deploy-auth` | unauth "deploy to cloud" → detect auth need → **background** `volcano login` + float the device code |
 | `cloud-deploy` | authenticated + authorized "deploy to cloud" → project context → confirm → deploy → verify |
 
-Local scenarios use the local Docker stack (`volcano start`). The two cloud
-scenarios point at a local hosting server by default (`http://localhost:8000`,
-e.g. a `make dev` instance); set the env vars below to target a real cloud. See
+Local scenarios use the local Docker stack (`volcano start`), including
+`durable-function`. Cloud scenarios (`deploy-auth`, `cloud-deploy`) point at a
+hosting server by default (`http://localhost:8000`). Set the env vars below to
+target a cloud server. See
 [`RUNNING.md`](./RUNNING.md) for choosing the server image and API URL.
 
 ### Output

tests/e2e-agent-eval/RUNNING.md

@@ -92,8 +92,7 @@ Precedence: `VOLCANO_API_URL` env > runtime override > compiled default.
 
 **Cloud-scenario exception:** `deploy-auth` and `cloud-deploy` overwrite
 `VOLCANO_API_URL` in their setup from `CLAUDE_EVAL_CLOUD_API_URL` (default
-`localhost:8000`), so exporting it here does **not** control those two runs — set
-`CLAUDE_EVAL_CLOUD_API_URL` for them instead.
+`localhost:8000`). Set `CLAUDE_EVAL_CLOUD_API_URL` for these runs.
 
 ## Test against an already-running hosting server
 
@@ -111,7 +110,7 @@ Why `VOLCANO_IMAGE` is still needed: each local scenario's setup runs
 local scenarios rebuild a **clean** stack from `VOLCANO_IMAGE` each time. Point it
 at the tag the server is running (check with
 `docker inspect volcano-server --format '{{.Config.Image}}'`) to keep testing that
-build. The two cloud scenarios already self-set `VOLCANO_API_URL` from
+build. Cloud scenarios self-set `VOLCANO_API_URL` from
 `CLAUDE_EVAL_CLOUD_API_URL` (default `localhost:8000`).
 
 ## Cloud scenarios (`deploy-auth`, `cloud-deploy`)

tests/e2e-agent-eval/scenarios/durable-function/scenario.sh

@@ -0,0 +1,50 @@
+# shellcheck shell=bash
+# desc: build a checkpointed local durable function -> start it and check the execution result
+SCENARIO_PROMPT="Build a JavaScript Volcano durable function named 'double-number' locally. It takes an input object with an integer 'number'. Use two durable checkpointed steps: first double the number, then add one. Return an object with a 'value' field holding the result. Deploy it locally, start a test execution and check its result."
+SCENARIO_TIMEOUT=900
+
+scenario_setup() { eval_reset_local_stack; }
+
+scenario_verify() {
+  local artifact="$RESULTS_DIR/durable-result.txt"
+  : >"$artifact"
+  local deployed="false" checkpointed="false" succeeded="false"
+  local details="" start="" execution="" id="" number=$(( 10000 + RANDOM ))
+  local i
+  for i in $(seq 1 10); do
+    details=$(volcano durable get double-number 2>&1) || details=""
+    if echo "$details" | grep -qE '^Status:[[:space:]]+active'; then deployed="true"; break; fi
+    sleep 6
+  done
+  printf 'Function:\n%s\n' "$details" >>"$artifact"
+  local source_path="$SANDBOX_DIR/volcano/functions/double-number"
+  if [ -d "$source_path" ] || [ -f "$source_path.js" ] || [ -f "$source_path.ts" ]; then
+    [ "$(grep -RohE 'ctx[.]step[[:space:]]*\(' "$source_path" "$source_path.js" "$source_path.ts" --include='*.js' --include='*.ts' 2>/dev/null | wc -l | tr -d ' ')" -ge 2 ] && checkpointed="true"
+  fi
+  if [ "$deployed" = "true" ]; then
+    start=$(volcano durable start double-number --input "{\"number\":$number}" 2>&1) || start=""
+    printf '\nStart:\n%s\n' "$start" >>"$artifact"
+    id=$(printf '%s\n' "$start" | sed -nE 's/^ID:[[:space:]]*([0-9a-f-]{36}).*/\1/p' | head -1)
+    if [ -n "$id" ]; then
+      local i
+      for i in $(seq 1 20); do
+        execution=$(volcano durable executions get double-number "$id" 2>&1) || execution=""
+        if echo "$execution" | grep -qE '^Status:[[:space:]]+(succeeded|failed|timed_out|stopped|unknown)'; then break; fi
+        sleep 6
+      done
+      printf '\nExecution:\n%s\n' "$execution" >>"$artifact"
+      if echo "$execution" | grep -qE '^Status:[[:space:]]+succeeded' &&
+         echo "$execution" | grep -qE "^[[:space:]]*\"value\":[[:space:]]*$(( number * 2 + 1 ))[,}]?"; then
+        succeeded="true"
+      fi
+    fi
+  fi
+  [ "$deployed" = "true" ] && [ "$checkpointed" = "true" ] && [ "$succeeded" = "true" ] && PASS="true"
+  {
+    echo "# Result: $SCENARIO ($RUN_ID)"; echo
+    echo "**Pass:** $PASS (durable function active, two checkpointed steps in source, fresh execution returned the expected value)"
+    echo "**Agent exit code:** $AGENT_EXIT"; echo "**Agent wall time:** ${AGENT_WALL_S}s"
+    echo "**Function active:** $deployed | **Checkpointed:** $checkpointed | **Execution result correct:** $succeeded"
+    echo; echo 'See `durable-result.txt` and `metrics.json`.'
+  } >"$RESULTS_DIR/report.md"
+}

tests/e2e-agent-eval/scenarios/durable-function/verify-test.sh

@@ -0,0 +1,31 @@
+#!/usr/bin/env bash
+set -euo pipefail
+SCRIPT_DIR="$(cd "$(dirname "$0")/../.." && pwd)"
+source "$(dirname "$0")/scenario.sh"
+SANDBOX_DIR=$(mktemp -d)
+RESULTS_DIR=$(mktemp -d)
+trap 'rm -rf "$SANDBOX_DIR" "$RESULTS_DIR"' EXIT
+SCENARIO=durable-function RUN_ID=test AGENT_EXIT=0 AGENT_WALL_S=1
+mkdir -p "$SANDBOX_DIR/volcano/functions/double-number"
+printf '%s\n' 'await ctx.step("double", fn); await ctx.step("add", fn);' >"$SANDBOX_DIR/volcano/functions/double-number/index.js"
+volcano() {
+  case "$*" in
+    'durable get double-number') printf 'Status: active\n' ;;
+    'durable start double-number --input '* ) printf 'ID: 11111111-1111-4111-8111-111111111111\n' ;;
+    'durable executions get double-number '* ) printf 'Status: succeeded\nResult: {\n  "value": %s\n}\n' "$EXPECTED" ;;
+    *) return 1 ;;
+  esac
+}
+# Fix the random input so the mock can return the matching value.
+RANDOM=1234
+number=$RANDOM
+EXPECTED=$(( (10000 + number) * 2 + 1 ))
+RANDOM=1234
+PASS=false
+scenario_verify
+[ "$PASS" = true ]
+EXPECTED=0
+RANDOM=1234
+PASS=false
+scenario_verify
+[ "$PASS" = false ]