feat(llm): route operations by metadata

[cachanova]

Summary

• Add a metadata multi-LLM strategy for exact tag and metadata.* routes.
• Route synchronous, queued, and batch retain work from the full operation's classification, including stored append state. Selected lanes never fail over across providers.
• Keep reflect/review, mental-model refresh, consolidation, and consolidation dedup on the protected tag-routed member. Reflect is intentionally fail-closed because retrieval can expand a fact to its full document.
• Reject an operation before any LLM call when matching routes select different members.
• Document global and per-operation configuration and keep the generated Hindsight docs skill synchronized.

Configuration

export HINDSIGHT_API_LLM_PROVIDER=openai
export HINDSIGHT_API_LLM_API_KEY=sk-...
export HINDSIGHT_API_LLM_1_PROVIDER=ollama
export HINDSIGHT_API_LLM_1_MODEL=qwen3:8b
export HINDSIGHT_API_LLM_STRATEGY='{
  "mode": "metadata",
  "routes": [{"key": "tags", "value": "sensitive", "member": 1}]
}'

Unmatched retains use member 0. Retain can also route on user-defined values such as metadata.classification. Reflect and consolidation route from tags; with a tag route configured, they bind fail-closed to its member because they can admit new or transitive data during the operation. Routing values are ephemeral and are not written to LLM trace metadata.

Verification

• ./scripts/hooks/lint.sh
• 200 focused routing, config, provider, batch, reflect, mental-model delta, consolidation dedup, and cache-affinity tests
• 7 embedded env-template tests
• Docusaurus production build and generated-docs link validation
• 238 append/store regression tests passed; one parallel queue-claim timing failure passed on isolated rerun
• git diff --check
• Fable final review: no findings, PR-ready

Closes #1140

[strix-security]

Strix Security Review

Warning

This pull request has 1 commit after the last Strix review (7c4ed2d). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

No security issues found.

_{Updated for 7c4ed2d.}

---

Reviewed by Strix
_{Re-run review · Configure security review settings}

[nicoloboschi]

Reviewed and tested locally on 57f3876 (rebased check: the branch is 33 commits behind main).

Verdict: not mergeable as-is — it breaks 26 existing tests, and CI will not tell you. This is a fork PR, so the test-api jobs are skipped rather than run; the green checkmarks here cover a much smaller subset than they appear to.

Verification performed

Check	Result
./scripts/hooks/lint.sh	pass
Generated docs skill in sync (git status after lint)	clean
New tests: routing, config, provider, batch	84 passed
Full -k "retain or multi_llm or consolidation or reflect or tag or metadata or config"	27 failed, 1819 passed
Same suites on origin/main	101 passed, 0 failed

Of the 27 failures, 26 are introduced by this PR (24 consolidation + 2 async-retain-tags; see the two inline Must fix comments). The remaining two — test_worker.py::...test_claim_batch_skips_consolidation_when_same_bank_processing and test_fact_extraction_agent_experience.py::...test_user_interaction_classified_as_experience — both pass in isolation and are pre-existing flakes, not attributable here.

Summary of findings

Must fix

1. consolidator.py:2215 — with_config called on an already-configured wrapper; works in prod only via __getattr__ passthrough, breaks 24 tests.
2. memory_engine.py:18435 — unconditional self._retain_llm_config access breaks the test this PR itself edits.

Should fix

3. metadata.* classification is only persisted for items[0], so a multi-item document can silently lose its route and a later append lands on the primary.
4. Pooled DB connection held across external store I/O in _stored_append_routing_states, plus an N+1.
5. Two tag routes to different members construct fine but break every reflect and every consolidation at runtime — validate it in MultiLLMProvider.__init__.

Nits — comment/implementation mismatch on route ordering; document_tags=[] still dropped on the file-convert path.

What holds up well

The batch path is genuinely well covered — test_multi_llm_batch.py verifies the sensitive lane owns both submit and resume, which was the leak path I most expected to find untested. The fail-closed reasoning for reflect (expand can widen a fact to its full document) and for consolidation (facts arriving mid-job) is sound and well documented. Routing values correctly stay out of LLM trace metadata.

[cachanova]

Addressed every review item and rebased onto current main in 02399a53:

• derived consolidation dedup from the base provider and initialized the async-retain test fixture
• persisted metadata from every item in shared documents so later appends retain their route
• replaced append-state N+1 reads with one bank-scoped query, released the DB connection before store I/O, and used the bulk store API
• rejected conflicting tags routes at construction
• corrected all-match documentation and preserved explicit document_tags=[]
• added regression coverage for shared-document metadata, authoritative store routing, bulk store reads outside the DB lease, tag-route validation, and empty file-retain tags

Validation: repository lint and ty pass; 82 focused tests pass; broad deterministic selection completed with 1,858 passes. Its two remaining failures require the missing live OpenAI credential, and the one shared-worker failure passed in isolation.

@nicoloboschi this should be ready for re-review. @strix-security please review the latest commits.

[cachanova]

Rebased once more onto current main (ebcd88f9) after today’s upstream changes and updated the metadata batch tests for the current extraction API. New head: 0954c0de.

Fresh validation on the rebased branch:

• broad retain/routing/consolidation selection: 1,941 passed, 39 skipped, 1 xfailed
• focused metadata/multi-LLM/consolidation/mental-model set: 177 passed, 6 skipped before the two compatibility edits; the affected batch file then passed 18/18
• isolated shared-worker test: passed
• repository lint and ty check hindsight_api/: passed

All seven addressed review threads are now resolved, and GitHub reports the PR mergeable. @nicoloboschi ready for re-review when convenient. @strix-security please review the latest head.

[nicoloboschi]

superseded by #4188