Harden execution with durable self-correction

CHANGELOG.md

@@ -6,6 +6,15 @@ This changelog is generated directly from git commit history (non-merge commits)
 
 ## [Unreleased]
 
+- Make correction execution narrower and convergence-aware: CSV verification now emits machine-readable row-width diagnostics and safely repairs unambiguous trailing-empty overflow in place before re-verification; mixed verifier checks are classified together; structured contract gaps and output-path violations use dedicated repair strategies instead of generic reruns; a failed verification-only retry is reclassified without falling through to broad execution; canonical deliverables lock to text-only completion after repair writes; correction fingerprints include canonical reason codes; and subtask-wide attempt/no-progress budgets prevent changing blocker labels from resetting recovery indefinitely
+- Reserve red `FAILED` outcomes for catastrophic safety/integrity blockers: verifier labels now normalize into a closed blocker taxonomy; targeted correction has an independent retry reserve; evidence-heavy and checkpoint passes receive contextual tool budgets plus an early checkpoint instruction; exhausted recoverable critical-path work becomes an amber `partial` dependency that downstream synthesis can use; unresolved ordinary gaps complete with an explicit degraded grade; global-budget and interrupted runs preserve resumable state; and desktop/TUI status projections no longer turn recoverable interruption into failure
+- Recover mechanically repairable CSV schema failures even when earlier semantic retries consumed the ordinary subtask budget: schema mismatches now receive a dedicated automatic correction handler, exact verifier-named file targets, an LLM-generated in-place repair plan, constrained retry context, deterministic re-verification, and at most one additional schema-repair attempt
+- Prevent login-walled, access-denied, or missing web pages from poisoning long research runs: repeated fetches of the same terminally unavailable URL are short-circuited toward search/alternate public sources; equivalent tool-budget reason codes share a checkpoint-continuation cycle; measurable progress can earn one bounded completion pass; and run-detail failure analysis now reads the newest event window so stale early 403/404 failures cannot mask the actual terminal cause
+- Add a unified durable self-correction controller above retry, verification-only retry, confirm-or-prune remediation, iteration gates, and replan: failures now become typed blockers with separate blocking/repairability policy, deterministic handler selection, restart-stable cycle IDs, structured progress/no-progress detection, idempotent action plans, lifecycle telemetry, and SQLite-backed cycle/attempt/action history; hard safety, path, artifact-integrity, canonical-write, authentication, and destructive-approval blockers remain fail-closed
+- Add DB migration `20260728_008_correction_lifecycle_v1` for durable correction cycles, attempts, progress snapshots, and typed idempotent actions
+- Make evidence-heavy process runs resilient to isolated source/tool failures: `method_resilient` now treats recoverable web, fact-checker, unavailable conversation-recall, and read-only discovery failures as warnings while preserving hard safety/integrity failures; fact checking bounds/concurrently executes semantic claim checks plus parallelizes source fetching to stay within its tool deadline, and required fact checks can suppress optional report writes that conflict with canonical-output policy; economic-data provider names are constrained in the tool schema before execution; sealed-artifact refresh failures now carry a distinct targeted-retry reason with exact files to reread
+- Replace repeated hot-loop semantic compaction with a hybrid default: deterministic structural reduction, a single bounded and cached semantic checkpoint near context pressure, exact anchor validation/repair, atomic tool exchanges, and deterministic excerpts only for emergency fallback; retain `semantic`, `tiered`, `legacy`, and `off` as explicit compatibility choices
+- Recover runner execution from oversized initial prompts and malformed post-compaction tool transcripts: emergency context fitting now runs even when compaction is configured off, tool exchanges remain atomic across critical merges, malformed histories are rebuilt as provider-safe context before retry, and failure analysis reports the actual runner preflight cause instead of downstream missing-output symptoms
 - Add DB migration `20260416_007_conversation_turn_metadata_v1` so thread user turns can durably persist explicit attachment/context metadata across resumes
 - Add explicit conversation-turn attachment metadata plumbing for desktop thread file mentions and pasted-image attachments
 - Refactor MCP + auth usability around a shared account authority: new MCP OAuth writes now use Loom secret refs with fingerprinted server identity, workspace-defined remote servers expose approval/trust state, and the desktop app now has a dedicated Integrations tab with management-grade MCP/account state plus guided account connect/routing actions

apps/desktop/src/api.ts

@@ -584,6 +584,8 @@ export interface RunDetail extends RunSummary {
   events_count: number;
   workspace: WorkspaceSummary;
   plan_subtasks: PlanSubtask[];
+  completion_grade?: string;
+  degraded_completion?: Record<string, unknown>;
 }
 
 export interface RunArtifact {

apps/desktop/src/components/RunsTab.test.tsx

@@ -98,7 +98,7 @@ describe("RunsTab", () => {
   });
 
   it("wraps long run goals in the selected run header instead of truncating them", () => {
-    const goal = "We are a film and TV production house that will be attending Banff World Media Festival in 2026 and need the entire prompt visible in the run header";
+    const goal = "We are preparing a detailed synthetic research brief and need the entire prompt visible in the run header";
     mockApp.selectedRunId = "run-abc";
     mockApp.runDetail = {
       id: "run-abc",
@@ -157,6 +157,44 @@ describe("RunsTab", () => {
     expect(screen.getByText(/did not queue follow-up remediation/i)).toBeInTheDocument();
   });
 
+  it("shows recoverable partial work as completed with gaps", () => {
+    mockApp.selectedRunId = "run-partial";
+    mockApp.runDetail = {
+      id: "run-partial",
+      goal: "Build a synthetic market brief",
+      status: "completed",
+      process_name: "market-research",
+      plan_subtasks: [
+        {
+          id: "collect-evidence",
+          description: "Collect public evidence",
+          status: "partial",
+          summary: "One source remained unavailable; alternate evidence was preserved.",
+          depends_on: [],
+          phase_id: "",
+          is_critical_path: true,
+          is_synthesis: false,
+        },
+        {
+          id: "synthesize",
+          description: "Synthesize the brief",
+          status: "completed",
+          summary: "Completed with caveats.",
+          depends_on: ["collect-evidence"],
+          phase_id: "",
+          is_critical_path: true,
+          is_synthesis: true,
+        },
+      ],
+    };
+
+    render(<RunsTab />);
+
+    expect(screen.getByText("Completed with recoverable gaps")).toBeInTheDocument();
+    expect(screen.getByText("completed with gaps")).toBeInTheDocument();
+    expect(screen.queryByText("Why This Failed")).not.toBeInTheDocument();
+  });
+
   it("does not mount tool-call payloads until the row is expanded", async () => {
     const user = userEvent.setup();
     mockApp.selectedRunId = "run-abc";

apps/desktop/src/components/RunsTab.tsx

@@ -196,15 +196,20 @@ function eventTypeBadgeLabel(eventType: string): string {
   }
 }
 
-function subtaskStatus(events: Array<{ event_type: string; data: Record<string, unknown> }>, subtaskId: string): "pending" | "running" | "completed" | "failed" {
+type RunSubtaskStatus = "pending" | "running" | "completed" | "partial" | "failed";
+
+function subtaskStatus(events: Array<{ event_type: string; data: Record<string, unknown> }>, subtaskId: string): RunSubtaskStatus {
   const matching = events.filter(
     (e) =>
       (e.event_type.startsWith("subtask_") || e.event_type.startsWith("phase_")) &&
       (e.data.subtask_id === subtaskId || e.data.phase_id === subtaskId),
   );
-  if (matching.some((e) => e.event_type === "subtask_completed" || e.data.status === "completed")) return "completed";
-  if (matching.some((e) => e.event_type === "subtask_failed" || e.data.status === "failed")) return "failed";
-  if (matching.some((e) => e.event_type === "subtask_started" || e.data.status === "running")) return "running";
+  const latest = matching[matching.length - 1];
+  if (!latest) return "pending";
+  if (latest.data.status === "partial") return "partial";
+  if (latest.event_type === "subtask_completed" || latest.data.status === "completed") return "completed";
+  if (latest.event_type === "subtask_failed" || latest.data.status === "failed") return "failed";
+  if (latest.event_type === "subtask_started" || latest.data.status === "running") return "running";
   return "pending";
 }
 
@@ -387,14 +392,16 @@ function SubtaskStatusIcon({
   status,
   animated = true,
 }: {
-  status: "pending" | "running" | "completed" | "failed";
+  status: RunSubtaskStatus;
   animated?: boolean;
 }) {
   switch (status) {
     case "completed":
       return <CheckCircle2 size={14} className="text-emerald-400" />;
     case "failed":
       return <AlertTriangle size={14} className="text-red-400" />;
+    case "partial":
+      return <AlertTriangle size={14} className="text-amber-400" />;
     case "running":
       return <Loader2 size={14} className={cn("text-sky-400", animated && "animate-spin")} />;
     default:
@@ -1467,7 +1474,8 @@ function RunDetailView({
       return apiPlan.map((s, i) => ({
         id: s.id,
         goal: s.description || s.id,
-        status: s.status as "pending" | "running" | "completed" | "failed",
+        status: s.status as RunSubtaskStatus,
+        summary: s.summary || "",
         depends_on: s.depends_on ?? [],
         phase_id: s.phase_id || "",
         is_critical_path: s.is_critical_path,
@@ -1488,9 +1496,18 @@ function RunDetailView({
     }
     return Array.from(ids.values()).sort((a, b) => a.order - b.order).map((n) => ({
       ...n,
+      summary: "",
       status: subtaskStatus(runTimeline, n.id),
     }));
   }, [runDetail, runTimeline]);
+  const recoverableGapNodes = planNodes.filter(
+    (node) => node.status === "partial" || node.status === "failed",
+  );
+  const hasRecoverableGaps = runStatus === "completed" && (
+    runDetail.completion_grade === "degraded" || recoverableGapNodes.length > 0
+  );
+  const hasVerificationWarnings = runStatus === "completed"
+    && runDetail.completion_grade === "verified_with_warnings";
 
   // --- Activity category filters ---
   type ActivityCategory = "tool" | "subtask" | "verify" | "model" | "task" | "other";
@@ -1621,6 +1638,16 @@ function RunDetailView({
               <span className="text-[11px] text-zinc-600 italic">ad-hoc</span>
             )}
             <StatusBadge status={runDetail.status} />
+            {hasRecoverableGaps && (
+              <span className="rounded-full border border-amber-400/20 bg-amber-500/10 px-2 py-0.5 text-[10px] font-semibold uppercase tracking-wide text-amber-200">
+                completed with gaps
+              </span>
+            )}
+            {hasVerificationWarnings && (
+              <span className="rounded-full border border-amber-400/20 bg-amber-500/10 px-2 py-0.5 text-[10px] font-semibold uppercase tracking-wide text-amber-200">
+                verified with warnings
+              </span>
+            )}
           </div>
         </div>
 
@@ -1699,6 +1726,58 @@ function RunDetailView({
 
       {/* --- Scrollable body --- */}
       <div className="flex-1 overflow-y-auto px-6 py-5 space-y-8">
+        {hasRecoverableGaps && (
+          <section className="rounded-2xl border border-amber-500/20 bg-amber-500/[0.06] p-4">
+            <div className="flex items-start gap-3">
+              <div className="mt-0.5 rounded-xl bg-amber-500/10 p-2 text-amber-300">
+                <AlertTriangle size={16} />
+              </div>
+              <div className="min-w-0 flex-1">
+                <h3 className="text-sm font-semibold text-amber-100">
+                  Completed with recoverable gaps
+                </h3>
+                <p className="mt-2 text-xs leading-5 text-zinc-300">
+                  Loom preserved usable checkpointed work and continued the run.
+                  {recoverableGapNodes.length > 0
+                    ? " Review these stages before relying on unsupported details:"
+                    : " Review the run activity before relying on unsupported details."}
+                </p>
+                {recoverableGapNodes.length > 0 && (
+                  <ul className="mt-2 space-y-1.5 text-xs leading-5 text-zinc-300">
+                    {recoverableGapNodes.map((node) => (
+                      <li key={node.id} className="flex items-start gap-2">
+                        <span className="mt-[7px] h-1 w-1 rounded-full bg-amber-400" />
+                        <span>
+                          <span className="font-medium text-zinc-200">{node.id}</span>
+                          {node.summary ? ` — ${node.summary}` : ""}
+                        </span>
+                      </li>
+                    ))}
+                  </ul>
+                )}
+              </div>
+            </div>
+          </section>
+        )}
+        {hasVerificationWarnings && (
+          <section className="rounded-2xl border border-amber-500/20 bg-amber-500/[0.06] p-4">
+            <div className="flex items-start gap-3">
+              <div className="mt-0.5 rounded-xl bg-amber-500/10 p-2 text-amber-300">
+                <AlertTriangle size={16} />
+              </div>
+              <div>
+                <h3 className="text-sm font-semibold text-amber-100">
+                  Verified with warnings
+                </h3>
+                <p className="mt-2 text-xs leading-5 text-zinc-300">
+                  Deliverables passed verification, but the run retained partial
+                  evidence, verifier warnings, or an explicitly recovered executor
+                  failure. Review the validity scorecard before relying on material claims.
+                </p>
+              </div>
+            </div>
+          </section>
+        )}
         {runStatus === "failed" && failureAnalysis && (
           <section className="rounded-2xl border border-red-500/20 bg-red-500/[0.06] p-4">
             <div className="flex items-start gap-3">
@@ -1787,6 +1866,8 @@ function RunDetailView({
                 const statusColor =
                   status === "completed"
                     ? "border-emerald-500/40 bg-emerald-500/5"
+                    : status === "partial"
+                      ? "border-amber-500/40 bg-amber-500/5"
                     : status === "failed"
                       ? "border-red-500/40 bg-red-500/5"
                       : status === "running"
@@ -1795,6 +1876,8 @@ function RunDetailView({
                 const lineColor =
                   status === "completed"
                     ? "bg-emerald-500/40"
+                    : status === "partial"
+                      ? "bg-amber-500/40"
                     : status === "failed"
                       ? "bg-red-500/30"
                       : "bg-zinc-700/50";
@@ -1809,7 +1892,7 @@ function RunDetailView({
                       {/* Node dot */}
                       <div className="shrink-0 my-1">
                         <SubtaskStatusIcon
-                          status={status as "pending" | "running" | "completed" | "failed"}
+                          status={status as RunSubtaskStatus}
                           animated={liveAnimationsEnabled}
                         />
                       </div>

apps/desktop/src/components/ThreadsTab.test.tsx

@@ -536,7 +536,7 @@ describe("ThreadsTab", () => {
     mockApp.visibleConversationEvents = [
       makeEvent(1, "user_message", { text: "tell me again" }),
       makeEvent(2, "user_message", { text: "tell me again" }),
-      makeEvent(3, "assistant_text", { text: "Based on my research, here's the easiest" }),
+      makeEvent(3, "assistant_text", { text: "Based on synthetic research, the next step" }),
       makeEvent(4, "turn_separator", { tokens: 42, tool_count: 0 }),
     ];
     mockApp.visibleConversationMessages = [
@@ -581,7 +581,7 @@ describe("ThreadsTab", () => {
         session_id: "conversation-1",
         turn_number: 4,
         role: "assistant",
-        content: "Based on my research, here's the easiest way to get a meeting with Blink49 at Banff.",
+        content: "Based on synthetic research, the next step is to contact Example Studios about Summit Festival.",
         tool_calls: [],
         tool_call_id: null,
         tool_name: null,
@@ -594,7 +594,7 @@ describe("ThreadsTab", () => {
 
     expect(screen.queryByText("tell me againtell me again")).not.toBeInTheDocument();
     expect(screen.getAllByText("tell me again")).toHaveLength(2);
-    expect(screen.getByText(/Based on my research, here's the easiest way/)).toBeInTheDocument();
+    expect(screen.getByText(/Based on synthetic research, the next step/)).toBeInTheDocument();
     expect(screen.getByText("Earlier answer")).toBeInTheDocument();
   });
 
@@ -786,7 +786,7 @@ describe("ThreadsTab", () => {
       makeEvent(1, "user_message", { text: "hello" }),
     ];
     mockApp.streamingThinking = [
-      "Let me search for blink49 and the Banff festival specifically:",
+      "Let me search for Example Studios and Summit Festival specifically:",
       "",
       "Let me try a broader search.",
     ].join("\n");
@@ -795,7 +795,7 @@ describe("ThreadsTab", () => {
     const { container } = render(<ThreadsTab />);
 
     expect(screen.getByText("Live")).toBeInTheDocument();
-    expect(screen.getByText("Let me search for blink49 and the Banff festival specifically:")).toBeInTheDocument();
+    expect(screen.getByText("Let me search for Example Studios and Summit Festival specifically:")).toBeInTheDocument();
     expect(screen.getByText("Let me try a broader search.")).toBeInTheDocument();
     expect(screen.getByText("Working through the numbers now")).toBeInTheDocument();
     expect(screen.queryByText("Thinking...")).not.toBeInTheDocument();

apps/desktop/src/conversationTimeline.test.ts

@@ -454,7 +454,7 @@ describe("conversationTimeline", () => {
     const liveItems = buildConversationTimelineItems([
       makeEvent(1, "user_message", { text: "tell me again" }),
       makeEvent(2, "user_message", { text: "tell me again" }),
-      makeEvent(3, "assistant_text", { text: "Based on my research, here's the easiest" }),
+      makeEvent(3, "assistant_text", { text: "Based on synthetic research, the next step" }),
       makeEvent(4, "turn_separator", { tokens: 42, tool_count: 0 }),
     ]);
     const historicalItems = buildConversationMessageTimelineItems([
@@ -464,7 +464,7 @@ describe("conversationTimeline", () => {
       makeMessage(
         4,
         "assistant",
-        "Based on my research, here's the easiest way to get a meeting with Blink49 at Banff.",
+        "Based on synthetic research, the next step is to contact Example Studios about Summit Festival.",
         "2026-03-29T12:00:03Z",
       ),
     ]);

apps/desktop/src/hooks/conversationReplay.test.ts

@@ -137,11 +137,11 @@ describe("conversation replay helpers", () => {
   it("preserves readable paragraph breaks between distinct live feedback chunks", () => {
     expect(
       appendStreamingThinkingChunk(
-        "Let me search for blink49 and the Banff festival specifically:",
+        "Let me search for Example Studios and Summit Festival specifically:",
         "Let me try a broader search.",
       ),
     ).toBe(
-      "Let me search for blink49 and the Banff festival specifically:\n\nLet me try a broader search.",
+      "Let me search for Example Studios and Summit Festival specifically:\n\nLet me try a broader search.",
     );
     expect(
       appendStreamingThinkingChunk("step one", " and step two"),