feat(improvement): converge on one measured proposal-to-activation path

[drewstone]

Decision

Ship one improvement path:

observe -> choose a mutable surface -> generate candidates -> measure the exact candidate -> propose -> review -> activate -> measure live -> return to prior state

The path is public and product-neutral through measurement and authorization.
Products own tenant storage, billing, approval policy, deployment, and delivery.

There will be no parallel Intelligence-only optimizer, no automatic write from candidate search, no greenfield schema generations, and no separate loop per surface.
Normal package semver remains; the product and wire model have one current shape.

Customer Outcome

An operator can open one investigation and see:

• what happened, with missing data shown rather than inferred;
• which surface is most likely responsible and why;
• which candidate generators were selected or skipped and why;
• the baseline and exact candidate diff;
• task quality, regressions, confidence interval, cost, tokens, and latency;
• an approve, reject, request-changes, or feedback decision;
• the exact target that would change;
• the measured result after activation;
• a one-action return to the prior state.

The same path must work for prompt, skills, tools, MCP servers, hooks, subagents, full agent profiles, memory, code, and knowledge systems.
External tool, skill, MCP, and extension search is a candidate source, not a separate product.

Audiences

Primary

• A customer operator who needs a useful recommendation and a safe decision, not package terminology.

Secondary

• An agent author who wants a prompt, skill, tool, or profile change with measured evidence.
• A product engineer who embeds the public packages without depending on Tangle's paid product or sandbox.
• A reviewer who must understand the diff, evidence, authority, and return path.
• A data operator who must distinguish complete, degraded, corrupt, and uncaptured telemetry.
• A provider or sandbox adapter author who supplies execution without owning improvement policy.

Fresh Evidence - July 17, 2026

Published package set

Checked with npm view and unpacked package tarballs:

Package	Latest
@tangle-network/agent-interface	0.30.0
@tangle-network/agent-eval	0.122.2
@tangle-network/agent-knowledge	3.0.1
@tangle-network/agent-profile-materialize	0.5.1
@tangle-network/agent-runtime	0.95.0
@tangle-network/traces	0.9.0

Intelligence currently resolves an older and incompatible set: eval 0.114.0, interface 0.27.2, runtime 0.93.0, and traces 0.8.27, while its workspace materializer uses interface 0.30.0.
This is the direct cause of the current type split tracked in agent-dev-container#3726.

What already exists and must be reused

• agent-interface already defines the exact candidate experiment, paired measurement, proposal, review, and activation authorization records.
• agent-eval already provides held-back-task comparison, paired confidence intervals, shared run spending, proposer composition, proposer comparison, GEPA, FAPO, SkillOpt, policy-edit, HALO, trace-analyst, and other candidate generators.
• agent-runtime already provides candidate execution, proposeAgentImprovement, proposal/review/activation record creation, and the knowledge improvement job.
• agent-knowledge already owns source ingestion, retrieval evaluation, memory adapters, knowledge candidates, quality checks, and promotion-safe candidate workspaces.
• Intelligence already has tenant-scoped artifact storage, admin approval, transactional active-pointer changes, an outbox, explicit external-delivery consent, and return-to-prior-state behavior.

The missing work is adoption and semantic cleanup, not another set of nouns or another engine.

Before implementation branches diverge, freeze the existing interface records and prove one packed blank install.
That packed artifact is the dependency for eval, knowledge, runtime, and Intelligence work; parallel development must not recreate the current multi-version type split.

Current proof is not a customer proof

• Fresh run: replay-e2e, loop-closeout-e2e, and observe-loop-e2e passed 8 of 8 executed tests in 2.20 seconds, with one LLM case skipped.
• The closeout receipt identifies its target as pglite+otlp-http+coder-stub and its positive activation case uses an auto-merge bypass policy.
• The observe receipt explicitly skips the agentic analyst unless a live LLM is enabled.
• products/intelligence/api/scripts/project-dossier-proof.ts is 3,929 lines and labels itself synthetic_local_executor_path with transportMode: "stubbed".
• The browser test stubs analysis, candidate, and approval routes.
• proof:hosted-readiness checked /ready and /v1/openapi.json in 1,014 ms, but did not authenticate, analyze traces, produce a candidate, approve it, activate it, or measure the result.

These are useful component checks.
None may be used as evidence that a customer journey works.

Trace truth is a release blocker with a measured fix in review

Command:

npx --yes @tangle-network/traces@latest analyze --harness claude-code --last 1

Published traces@0.9.0 observed on one real Claude Code session:

• 25,850 spans;
• reported duration 1,784,315,006,051 ms because the root span started at 1970-01-01;
• 17,537 reported model-call events;
• 6.23 billion summed token fields;
• 0% cost coverage;
• six corruption receipts.

Autopsy:

• the first JSONL record had no timestamp, and the Claude adapter fixed the root start to epoch instead of the first timestamped record;
• one provider message ID appeared as three assistant fragments with identical usage, and the adapter counted all three as separate model calls;
• the report presented those impossible values as measurements rather than rejecting or degrading them.

PR traces#37 fixes the parser and was checked read-only against a frozen Drew-owned session:

• 47,781 source rows across 182 files: one main file, 75 direct subagents, and 106 nested workflow agents;
• 11,141 source provider messages and 11,141 emitted model events;
• 12,261 source tool calls and 12,261 emitted tool calls;
• exact input, output, cache-read, and cache-write totals;
• 75 of 75 invoked agents attached to their tool call;
• 22 of 22 depth-two agents attached to the declared parent agent;
• zero duplicate span IDs and zero unresolved parent references;
• root time 2026-07-12T02:30:16.568Z through 2026-07-17T19:27:27.971Z, or 493,031,403 ms rather than 56.6 years;
• six malformed rows remain explicit corruption receipts;
• cost remains unavailable rather than estimated or reported as zero.

The package proof passed 203 tests, typecheck, build, and packed-binary installation, and an independent Opus review reported no blockers.
This check is not complete until #37 merges, a release is published, and the same command succeeds through @latest.
Uri's read-only report remains blocked on that release and account-scoped rerun.

PR #3727 is transitional product protection, not the shared improvement path

PR agent-dev-container#3727 merged into develop as c8f6eafc while this audit was running.
Its head adds 1,131 lines across five files.
It adapts scoreRedTeamOutput, which evaluates responses to hostile scenarios, into a static scanner over ordinary candidate artifacts.
Eleven commits were needed to repair masking, false exclusions, incomplete scanning, custom-case behavior, invalid costs, reason leakage, size limits, null artifacts, and upstream string coupling.

The optimization search itself is non-mutating (autoOnPromote: "none").
The product already has a separate admin approval and transactional activation boundary.
Latest eval already has a fail-loud heldOutGate and a whole-run spending ledger.

Keep it only as current product-side protection while Track C replaces the old optimization engine.
It is not evidence that the public improvement path is complete.
Replacement remains:

• measure candidate behavior on real held-back tasks;
• include hostile tasks when the target has a safety requirement;
• enforce general content policy once at product delivery, not only inside one optimizer;
• persist unavailable checks as unavailable, never passed;
• keep human or tenant-policy approval separate from candidate search.

Delete the product-local scanner when those checks are wired through the current package set; do not grow a second general safety framework around it.

Draft PR #3501 must be closed and mined, not rebased

PR agent-dev-container#3501 is 277 develop commits behind, conflicts in eight files, changes 105 files, and adds 12,172 lines while deleting 10,207.
It pins eval 0.115.2, runtime 0.94.3, interface 0.25.0, and the pre-3.0 knowledge line rather than the compatible published set above.
Its product-local optimization engine, proposer wrappers, source research, source verification, and profile execution duplicate capabilities now owned by eval, runtime, knowledge, and interface.

Salvage only three things by re-deriving them against current develop:

• deletion of dead R3 worker, store, compose, launch, proposal-validation, and consultant-dispatch paths;
• the rule that unavailable measurements remain unavailable rather than passed;
• source pinning, license, and advisory checks contributed at the reusable package layer.

Close #3501 rather than resolving eight conflicts around an obsolete engine.

Current pull-request CI spends broadly and misses Intelligence

The #3727 diff changes five Intelligence files, but its remote run starts SDK packaging, web bundle size, React, design, sandbox create latency, deploy-script, ecosystem, and repository-wide jobs.
The workflow does not run the Intelligence test suites that changed.

This is the wrong trade:

• unrelated checks consume minutes on every pull request;
• the changed customer path can still merge without running;
• every job repeats dependency installation;
• skipped live checks create noise without evidence.

One Public Model

1. Observe

Normalize traces without inventing missing fields.
Every report marks input coverage as complete, degraded, corrupt, or uncaptured.
No cost, duration, tool, skill, MCP, or subagent conclusion is allowed when its source field is unavailable or internally inconsistent.

2. Choose the surface

The decision is evidence-driven:

Observed pattern	First surface to test
Missing or corrupt evidence	instrumentation or ingestion repair
Missing, stale, or unfaithful facts	knowledge, retrieval, or memory
Repeated instruction-following failure	prompt or skill
Missing capability	existing tool, MCP server, extension, or subagent
Wrong coordination or permissions	hook or full agent profile
Implementation defect	code

When evidence supports more than one surface, run the smallest credible candidates under one shared spending limit and compare them on the same held-back tasks.
When evidence supports none, propose better measurement rather than a mutation.

3. Generate candidates

A candidate generator is only a way to create alternatives for one mutable surface.
It does not own measurement, approval, or activation.
Each surface contributes two plug-ins to the same path: a materializer that constructs the exact runnable candidate and a generator that proposes alternatives.
AgentProfileDiff is the common profile change shape; changed surfaces are derived from the baseline and candidate rather than maintained as a second drifting list.

Surface	Existing candidate sources
Prompt	GEPA, FAPO, policy edit, trace analyst, composed generators
Skills	SkillOpt, skill mining, existing-skill search, authored skill candidate
Tools / MCP / extensions	open-source research, compatibility check, behavioral trial
Hooks / subagents / full profile	AgentProfileDiff plus explicit profile generator
Memory	memory curation and memory-to-evidence adapters
Knowledge / RAG / wiki	runKnowledgeImprovementJob()
Code	runtime-owned isolated coding and research agents

For capability gaps, search existing open-source skills, tools, MCP servers, and Pi extensions before generating a replacement.
Research agents may deepen the search iteratively, but their output enters the same candidate record and the same measurement path.

4. Freeze and measure

Seal one exact candidate before comparison.
Run baseline and candidate against the same held-back tasks, seeds, scorer, execution limits, model access, and spending owner.
Record quality, regressions, cost, tokens, latency, failures, and confidence intervals.
An absent signal is unavailable, not a pass.

5. Propose and review

The proposal binds findings, diff, measurements, and candidate digest.
Review binds approve, reject, request-changes, or feedback to that exact proposal.
Search never writes live state.

6. Activate and measure live

The product rechecks the expected current target digest, applies one authorized candidate exactly once, records the result, and measures the next live outcome.
The shared activation result has an idempotency key, before and after digests, and one typed outcome: applied, already-applied, conflict, expired, unsupported, failed, or indeterminate.
An external message or repository write requires explicit customer consent.
Returning to prior state creates another auditable activation; it never rewrites history.

Ownership

Layer	Owns	Must not own
traces	lossless parsing, integrity, normalized telemetry	recommendations or activation
agent-interface	portable records and digests	execution, databases, product policy
agent-eval	tasks, candidate generation, comparisons, statistics, spending	product writes or tenant delivery
agent-runtime	agent execution, candidate execution, public improvement composition	silent activation or product storage
agent-knowledge	sources, retrieval, memory evidence, knowledge candidates	generic agent orchestration or tenant approval
Intelligence	tenancy, billing, investigation UI, approval policy, activation adapters, notifications	duplicate optimizer contracts or package-local algorithms

Provider, storage, model, orchestration, and sandbox implementations remain injected ports.
The public path must run with local files and a caller-supplied executor; Tangle Sandbox and the paid Intelligence API are optional adapters.

Three Parallel Tracks

Track A - Make trace evidence truthful

Owner: traces.

1. Use the minimum and maximum timestamped Claude events across the root and nested agents.
2. Count provider usage once per provider message ID while preserving its text and tool fragments.
3. Discover recursive workflow-agent files and preserve parent-agent/tool relationships.
4. Reject ambiguous duplicate agent IDs rather than silently choosing a parent.
5. Reject impossible session durations and duplicated cumulative usage from analytical summaries.
6. Preserve corruption receipts and label every unavailable field.
7. Merge and publish traces#37, rerun the exact command through @latest, and then rerun Uri's account read-only.

Track B - Collapse the public improvement behavior

Owners: agent-runtime, agent-eval, and agent-interface only where the existing records are insufficient.

1. Make improve() return an immutable candidate result and leave the input profile and files unchanged.
2. Remove shipped, automatic profile application, and skill/memory writeBack from the search function.
3. Reuse proposeAgentImprovement, measured comparison, proposal, review, and activation authorization records.
4. Add only the missing generic activation result: idempotency key, compare-and-swap digests, and typed outcome; products still implement their own writes.
5. Use fail-loud held-back comparison; do not rely on optional checks that report missing inputs as passed.
6. Keep one shared spending ledger across search, baseline, candidate, judging, retries, and research.
7. Route knowledge improvement through the same proposal/review/authorization record chain without moving knowledge ownership into runtime.
8. Publish one compatible package set and prove a blank consumer installs one interface version.

Track C - Adopt it in Intelligence and prove the product

Owner: agent-dev-container.

1. Upgrade the current package set and remove product-local copies of shared records and algorithms.
2. Keep the product's existing tenant storage, transactional activation, outbox, explicit external-delivery consent, and return path.
3. Replace the static artifact scanner draft with real task measurement plus one product delivery policy.
4. Expose selected and skipped candidate generators, evidence completeness, diff, confidence, cost, tokens, latency, and review controls in the existing UI.
5. Run one isolated Drew-owned prompt improvement through the real API and browser, activate it, measure the next run, and return to the prior state.
6. Run Uri read-only with no customer mutation, trigger, outbound message, or customer billing.
7. Replace broad pull-request CI with changed-package checks and the relevant customer journey.

Track A starts immediately because trace parsing is independent.
Freeze and pack the interface contract before Track B and Track C branch from it.
Track B and Track C then proceed in parallel against that exact packed artifact.
Track C cannot claim release until the compatible packages are published and the product installs them normally.

Release Checks - 12 Total

• 1. A blank consumer installs the published package set with one resolved agent-interface, imports the public path, and needs no Tangle product or sandbox.
• 2. Intelligence installs that same published set without /tmp, workspace-package, patch, or lockfile skew and passes typecheck.
• 3. The real Claude trace command reports a plausible timestamp range and counts usage once per provider message ID; corrupt and uncaptured fields remain explicit.
• 4. Uri's current account produces a read-only report with trace/session counts, failures, task outcomes, tools, MCP, skills, subagents, tokens, cost, and latency all shown as measured or unavailable.
• 5. improve() cannot modify a profile, file, repository, memory document, or knowledge target; it returns a sealed candidate and evidence only.
• 6. Baseline and candidate receipts bind the same held-back tasks, scorer, limits, model access, seeds, and spending owner, and the comparison recomputes from those receipts.
• 7. One spending limit covers search, research, baseline, candidate, judging, retries, and failures; the run stops before exceeding the reserved amount.
• 8. Proposal, review, and activation authorization all bind the same candidate digest; duplicate requests return already-applied, stale target state returns conflict, and no write is repeated.
• 9. The Intelligence browser shows evidence completeness, exact diff, all measured dimensions, confidence interval, selected and skipped generators, and approve/reject/request-changes/feedback controls using real API data.
• 10. A Drew-owned isolated target completes observe through activation, the next live run is measured, and return-to-prior-state restores behavior without touching another tenant.
• 11. Uri's proof performs zero mutations, zero outbound delivery, zero customer-triggered jobs, and zero customer-billed model calls; the product's own spending and wall time are recorded.
• 12. Pull requests run cheap invariants plus changed packages and dependents; Intelligence changes run its deterministic full journey, UI changes run its browser flow, package changes run the blank-consumer proof, infrastructure checks run only for infrastructure paths, and the complete repository suite runs on develop, nightly, or release.

Checks to Delete or Move

Delete as release evidence:

• the 3,929-line stubbed dossier proof as proof of customer behavior;
• browser tests that stub the entire customer path;
• product-local candidate, comparison, proposal, or activation shapes already owned by the public packages;
• automatic write or shipped semantics inside search;
• optional checks recorded as passed when their inputs are absent;
• static red-team claims made from ordinary outputs that never ran hostile tasks;
• per-surface loop implementations.

Move off every pull request:

• SDK packaging for unrelated product changes;
• React and design ratchets for non-UI changes;
• sandbox create latency and Docker lifecycle for non-sandbox changes;
• deploy-script tests for non-deploy changes;
• repository-wide build, typecheck, and tests when changed-package plus dependent analysis is available;
• live external checks that cannot be deterministic before merge.

Keep on every pull request:

• fast repository invariants;
• changed-file formatting and security checks;
• affected package typecheck, build, and tests;
• the one deterministic customer journey for the changed product.

Keep after merge or deploy:

• the complete repository matrix;
• live service readiness;
• one real isolated product journey;
• durable result capture with exact serving revision.

Stop Rules

Do not claim complete when:

• telemetry contains impossible values;
• a proof is stubbed, route-only, or component-only;
• a candidate can write before review;
• a comparison cannot be recomputed from exact receipts;
• the active product uses unpublished or workspace-only package behavior;
• the UI is proven only with route interception;
• the changed customer path did not run;
• a skipped check is presented as success.

Tracking

This issue is the only cross-repository completion checklist.
Implementation pull requests link one of Tracks A-C and identify which of the 12 checks they advance.
Repository roadmaps may point here but must not copy the checklist.

[drewstone]

July 17 execution-planning evidence

Fresh checks behind the updated plan:

Check	Result
npm view for the six public packages	interface 0.30.0, eval 0.122.2, knowledge 3.0.1, materializer 0.5.1, traces 0.9.0, runtime 0.95.0
Blank npm install --package-lock-only of all six	Four interface versions resolved: root 0.30.0; traces subtree 0.26.1; eval/TCloud/sandbox subtree 0.17.1 and 0.13.0
Runtime 0.95.0 npm metadata	Current eval/interface/knowledge/materializer ranges; Tangle Sandbox and Playwright are optional peers
rg over src/candidate-execution	Nine injected leaf *Port interfaces plus two aggregate types; zero Intelligence, product, or Tangle Sandbox imports
Public execution contract	AgentEnvironmentProvider already models local, sandbox, fleet, or other providers in agent-interface
Runner ids	Interface defines sixteen ids; materializer defines a separate twelve-id union
Runtime mutation search	improve() still exposes writeBack, returns shipped, and writes skill or memory winners directly
Intelligence manifests and root overrides	Product requests runtime 0.93, eval 0.114, interface 0.25, knowledge 1.11, traces 0.8, and forces older runtime/eval/interface resolutions
Runtime PR #547	Merged; exact candidate execution is included in runtime 0.95.0

Smallest architectural conclusion: keep AgentEnvironmentProvider and AgentCandidateExecutorPort, add one tested bridge and one target-transition adapter, and remove provider/product dependencies from the public package-only path.

The full commands and temporary lockfile were run locally without changing customer, production, or repository state.