Compliance PoC — PII redaction & data retention (#275)

[illeatmyhat]

Presentation PoC for #275 (Compliance — PII, Retention). For demonstrating the merged hook seam, not a merge candidate — hence draft. This branch is main plus a small set of demo files; every claim is backed by runnable examples. Full walkthrough in DEMO.md.

What changed

The original PoC predated the memory hook seam and imported detectors directly (PIIConfig / NullRedactor / backend.redactor) — all since removed from main and replaced by the seam. This branch has been reset to main + demo files and now shows how to USE the merged seam (PR #295): plugins configured via evolve hooks init → evolve.hooks.yaml (auto-discovered) or code-first HooksConfig(plugins=[HookPluginSpec(...)]), firing at the backend read/write choke points and before every LLM call.

The two #275 asks, plus secrets — three redaction methods on one seam

1. PII must never be saved into memory — two PII methods on memory_pre_write + llm_pre_call:
   • Regex (PIIFilterMemoryPlugin, [pii-regex]) — structured identifiers (email, phone, SSN, card, IP) at precision ~1.00; no NER.
   • Semantic (ReadiSemanticPIIPlugin, [pii-semantic]) — IBM READI NER; catches the names regex cannot.
2. Data-retention policies — age/unused flag-or-delete + session retention that cascade-deletes derived memories via provenance, with a why on every decision. CLI: evolve retention run --policy <file> [--apply] (dry-run by default).
3. Structured secrets (adjacent to Compliance - PII, Retention, etc. #275) — SecretsFilterMemoryPlugin ([secrets]) redacts credentials/tokens (AWS keys, GitHub/Slack tokens, private keys) on write and on LLM egress.

Running all three together is the recommended defence-in-depth posture.

Runnable demos

uv sync --extra hooks --extra pii-regex --extra pii-semantic --extra secrets
uv run --no-sync python examples/pii_redaction_demo.py   # regex + semantic; name caught only by semantic
uv run --no-sync python examples/secrets_demo.py         # AWS key + GitHub token redacted (write + egress)
uv run --no-sync python examples/hooks_demo.py           # normalizer + access-stamp + regex PII together
uv run --no-sync python examples/retention_demo.py       # flag / delete / provenance cascade, with "why"

Measured results (retained — they measure detector/model behavior)

English — ai4privacy, 43,501 records — per-entity recall:

PII type	regex ([pii-regex])	semantic (READI, en_core_web_trf)
Overall recall	0.12	0.47
Precision	1.00	0.99
email / ip	1.00 / 1.00	1.00 / 0.95
phone / ssn / credit card	0.46 / 0.22 / 0.04	0.60 / 0.43 / 0.03
first / last / middle name	0.00	0.92 / 0.97 / 0.94

Japanese — ai4privacy openpii-1.5m, 2,000 records — a language-matched model (one config line) is decisive:

PII type	regex	English NER	Japanese NER (ja_core_news_trf)
Overall recall	0.03	0.15	0.92
given / surname	0.00 / 0.00	0.07 / 0.04	0.82 / 0.96

Regex is high-precision on well-formed structured PII but blind to names and non-Western formats; semantic NER adds names/locations (~4× recall on English) but is language-bound. Hybrid (regex + language-matched NER + secrets) is the production shape.

Reviewer note

Branch = main + demo files only. Diff vs main: examples/pii_redaction_demo.py, examples/secrets_demo.py, DEMO.md (all new). Everything the demos exercise (the seam, the three plugins, retention, hooks_demo.py / retention_demo.py / pii_benchmark.py) is already on main. macOS/Apple-Silicon caveat: READI fails closed on the MPS backend, so the semantic e2e path needs a CPU/CUDA host; pii_redaction_demo.py falls back to regex-only with a printed note on darwin. Reproduce, don't trust the prose — full walkthrough in DEMO.md.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: adfaca3f-d345-4941-8545-e77f9c27b0c4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/275-pii-retention-poc

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[agent-toolkit-owl]

Checks Summary

Last run: 2026-07-24T17:46:00.646Z

Mend Unified Agent vulnerability scan found 4 vulnerabilities:

Severity	Identifier	Package	Details	Fix
🔺 High	CVE-2026-55685	react-router-7.17.0.tgz	This is a follow up to https://github.com/remix-run/react-router/security/advisories/GHSA-8x6r-g9mw-... This is a follow up to GHSA-8x6r-g9mw-2r78 that covers additional reported scenarios in which the manifest endpoint could be accessed via unauthenticated targeted requests that would put heavy load on the server and slow down response times. «[!NOTE] This only impacts Framework Mode applications. This does not impact your application if you are using Declarative or Data Mode.»	react-router - 7.18.0
🔷 Medium	CVE-2026-53669	react-router-7.17.0.tgz	This is a follow up to "CVE-2025-68470" (https://github.com/remix-run/react-router/security/advisori... This is a follow up to "CVE-2025-68470" (GHSA-9jcx-v3wj-wh4m). React Router was alerted to certain scenarios in which the fix there was incomplete so there still existed some scenarios where attacker supplied paths passed to navigation mechanisms could result in unexpected external navigations.	Upgrade to version https://github.com/remix-run/react-router.git - react-router@7.18.0
🔷 Medium	CVE-2026-53666	react-router-7.17.0.tgz	If application code allows attacker supplied input to overwrite certain aspects of errors caught by ... If application code allows attacker supplied input to overwrite certain aspects of errors caught by the SSR process, then it was possible for attacker to trigger unexpected constructor execution on the client which would trigger outbound network traffic. This is only possible with very specific (and unlikely) application layer code. «[!NOTE] This does not impact your application if you are using Declarative Mode. This only impacts Framework Mode and Data Mode applications doing manual SSR/hydration»	Upgrade to version https://github.com/remix-run/react-router.git - react-router@7.18.0
🔷 Medium	CVE-2026-53667	react-router-7.17.0.tgz	This is a follow up to https://github.com/remix-run/react-router/security/advisories/GHSA-8646-j5j9-... This is a follow up to GHSA-8646-j5j9-6r62. React Router was alerted of a code path in the (unstable) RSC error handling path in which redirects from untrusted sources could still result in an XSS vector via attacker-supplied redirect targets «[!NOTE] This only affects your application if you are using the unstable RSC APIs»	Upgrade to version https://github.com/remix-run/react-router.git - react-router@7.18.0