fix(decorators): key and name the registry set by a str namespace's exact value (LAB-6197)

docs/features/l1-invalidation.md

@@ -208,6 +208,13 @@ Things to know:
 
 **Custom `key=` functions.** Both forms work. `invalidate_cache(args...)` derives the key with the same `key=` function the write path used, so it deletes the exact entry from this process's L1 and from shared L2. No-args `invalidate_cache()` reaches what the table above says for the resolved backend — the registry tracks the key the write path actually wrote, custom or not. On the backends limited to "this process", tracked keys do not survive a restart, so after a deploy use the exact-args form.
 
+**`str` subclass namespaces.** A `str` subclass namespace is used as its plain `str` value everywhere, so a `StrEnum` or `(str, Enum)` member `USERS = "users"` is the namespace `users`. Earlier releases rendered a `(str, Enum)` member as `NS.USERS` in some places: in metrics labels on every Python version, and in custom `key=` keys, key registry set names and log-message prefixes on Python 3.11 and later. Plain `str` and `StrEnum` namespaces are unaffected. For a `(str, Enum)` namespace, upgrading changes the following.
+
+- **Custom `key=` entries move (Python 3.11+).** They move from `NS.USERS:k` to `users:k`, and the old entries are no longer served, so each distinct key is recomputed once. One no-args `invalidate_cache()` from an upgraded process deletes every old entry still tracked in the old registry set; run it once per tenant, since it reaches only the tenant set in `tenant_context`. Old entries that are not tracked, because their set expired seven days after its last write or because the backend does not track keys, retire only by TTL, and never if none was set. To erase them, run the `scan_iter` + `unlink` script from [Upgrading to 0.20.0](../backends/README.md#upgrading-to-0200) with `pattern = "t:*:NS.USERS:*"` on the tenant-scoped Redis backend (env auto-detection or `RedisBackendProvider`), or `pattern = "NS.USERS:*"` on a plain `RedisBackend`. A `redis-cli SCAN NS.USERS:*` against the tenant-scoped backend matches nothing, because its keys carry the `t:{tenant}:` prefix.
+- **The key registry set is renamed (Python 3.11+).** It moves from `ck:reg:NS.USERS:…` to `ck:reg:users:…`. Auto-mode keys do not move, and a no-args `invalidate_cache()` drains both sets, so entries tracked before the upgrade are still invalidated.
+- **Rolling deploys and rollbacks leave stale entries (Python 3.11+).** A no-args `invalidate_cache()` from a process on the earlier release drains only the old set, so it misses every entry the upgraded processes track in the new one. Those entries stay stale after the rollout completes, until their TTL (never, if none was set) or until an upgraded process drains the set. A rollback to the earlier release within seven days does the same. Once a rollout or a rollback completes, run one no-args `invalidate_cache()` per tenant from a process on this release (after a rollback, a one-off script) for each affected function.
+- **Observability names change.** The `namespace` metrics label reads `users` instead of `NS.USERS` on every Python version, 3.10 included, so update dashboards and alerts that match on it. On Python 3.11 and later, the log-message prefix `[NS.USERS]` becomes `[users]`.
+
 **Tenant scope:** with the tenant-scoped Redis backend (env auto-detection, or `RedisBackendProvider(...).get_shared_backend()`), each tenant's entries live under its own `t:{tenant}:` prefix. `invalidate_cache()` — with or without arguments — deletes only the L2 entries of the tenant set in `tenant_context` for the calling context (`default` when none is set); other tenants' entries stay cached and tracked. L1 is not tenant-scoped: within a process, all tenants share one L1 entry per cache key, so a tenant can be served the value another tenant cached, and `invalidate_cache()` evicts that entry for every tenant. Disable L1 on functions whose results differ by tenant: `@cache(..., l1_enabled=False)`, or with a preset `@cache.production(..., l1_enabled=False)`, which keeps the preset's other L1 settings.
 
 ---

src/cachekit/decorators/wrapper.py

@@ -583,11 +583,20 @@ def create_cache_wrapper(
 
     func_hash = function_hash(f"{func.__module__}.{func.__qualname__}")
 
+    # Rebind a str namespace to its exact str value before any use (LAB-6197). A str
+    # subclass renders through its own __format__/__eq__/startswith: a (str, Enum) member
+    # formats as "NS.USERS" on Python 3.11+ but "users" on 3.10, which split registry ids and
+    # key= keys across versions and could slip a crafted value past the "ck" check below.
+    # _legacy_namespace keeps the pre-fix f-string rendering for the registry drain below.
+    _legacy_namespace: str | None = None
+    if isinstance(namespace, str):
+        _legacy_namespace = f"{namespace}"
+        namespace = str.__str__(namespace)
+
     # INTEROP MODE (interop/v1, protocol spec/interop-mode.md): validate loudly at
     # decoration time. These checks also cover direct create_cache_wrapper callers
-    # that bypass DecoratorConfig validation. Runs before any other use of namespace
-    # and rebinds both segments to the exact str values it checked, so a str subclass
-    # (e.g. a (str, Enum) member) cannot render differently in a key.
+    # that bypass DecoratorConfig validation. Rebinds interop to the exact str value it
+    # checked; namespace is already exact from the block above.
     _interop_sig: inspect.Signature | None = None
     if interop is not None:
         try:
@@ -616,9 +625,19 @@ def create_cache_wrapper(
     # a key written under it could take the ck:reg: shape and overwrite a tracking set.
     if namespace == "ck" or (namespace or "").startswith("ck:"):
         raise ConfigurationError("namespace 'ck' (and 'ck:*') is reserved for cachekit's key registry")
-    _registry_id = (
-        f"ck:reg:{namespace if namespace is not None else ''}:"
-        f"{blake3_hash(f'{func.__module__}.{func.__qualname__}', digest_size=8)}"
+    _registry_hash = blake3_hash(f"{func.__module__}.{func.__qualname__}", digest_size=8)
+    _registry_id = f"ck:reg:{namespace if namespace is not None else ''}:{_registry_hash}"
+    # Pre-fix releases named the set with the namespace's f-string rendering. Auto-mode keys
+    # did not move, so entries tracked under the old name are still served; the no-args drain
+    # empties that set too, or they would outlive invalidate_cache() (LAB-5288 precedent).
+    # Interop is skipped: 0.20.0 shipped the registry with interop's exact-str rebind, so no
+    # release wrote a non-exact interop set. This drains the set name 0.20.x wrote: remove
+    # it only in a major release whose notes declare upgrades from 0.20.x unsupported (the
+    # rule get_legacy_cache_key follows).
+    _legacy_registry_id = (
+        f"ck:reg:{_legacy_namespace}:{_registry_hash}"
+        if interop is None and _legacy_namespace is not None and _legacy_namespace != namespace
+        else None
     )
 
     # ENCRYPTION + L1-ONLY (LAB-4665, protocol spec/intent-presets.md § L1 Posture rule 3:
@@ -2316,6 +2335,8 @@ def _drain_all() -> None:
                 scope = _l2_scope()
                 mine = {entry for entry in snap if entry[0] == scope}
                 deleted = _backend.drain_tracked(_registry_id, {key for _, key in mine})  # type: ignore[union-attr]
+                if _legacy_registry_id is not None:
+                    deleted |= _backend.drain_tracked(_legacy_registry_id, ())  # type: ignore[union-attr]
                 # Trim BEFORE evicting: _put_l1 puts then records, so a concurrent write can
                 # never leave an L1 entry whose key is no longer in _cached_keys.
                 trim = mine - watch

tests/unit/test_key_registry.py

@@ -13,8 +13,10 @@
 import contextvars
 import logging
 import os
+import sys
 import threading
 import time
+from enum import Enum
 from pathlib import Path
 from typing import Any, Optional
 
@@ -755,3 +757,135 @@ def __exit__(self, *exc: object) -> None:
         l1._lock = real_lock
         assert acquisitions == 3  # 1 000-key batches: a large drain never holds every get/put off at once
         assert l1.get("k2499") == (False, None)
+
+
+class _NS(str, Enum):
+    USERS = "users"
+
+
+class _StrEnumNS(str, Enum):
+    """enum.StrEnum's rendering (3.11+), spelled out so the test also runs on 3.10."""
+
+    USERS = "users"
+    __str__ = str.__str__
+    __format__ = str.__format__
+
+
+class _FormatsAs(str):
+    """A str whose __format__ lies: f-strings render ``rendered``, "".join the real value."""
+
+    rendered = "ck:reg:users"
+
+    def __format__(self, spec: str) -> str:
+        return self.rendered
+
+
+class _LegacyFormat(_FormatsAs):
+    rendered = "legacy"
+
+
+class _HidesCk(str):
+    """A str that claims not to be "ck" and not to start with it."""
+
+    def __eq__(self, other: object) -> bool:
+        return False
+
+    __hash__ = str.__hash__
+
+    def startswith(self, *args: Any, **kwargs: Any) -> bool:  # type: ignore[override]
+        return False
+
+
+@pytest.mark.unit
+class TestNamespaceExactStr:
+    """A str-subclass namespace keys and names its registry set by its underlying str (LAB-6197)."""
+
+    def test_str_enum_registry_id_uses_value(self) -> None:
+        backend = TrackingBackend()
+
+        @cache(backend=backend, ttl=60, namespace=_NS.USERS, l1_enabled=False)
+        def f(x: int) -> int:
+            return x
+
+        f(1)
+        (rid,) = _registry_ids(backend)
+        assert rid.startswith("ck:reg:users:")
+
+    def test_str_enum_custom_key_uses_value(self) -> None:
+        backend = TrackingBackend()
+
+        @cache(backend=backend, ttl=60, namespace=_NS.USERS, key=lambda *a, **kw: "k", l1_enabled=False)
+        def f(x: int) -> int:
+            return x
+
+        f(1)
+        assert set(backend.store) == {"users:k"}
+        (rid,) = _registry_ids(backend)
+        assert rid.startswith("ck:reg:users:")
+
+    def test_format_override_cannot_forge_registry_shape(self) -> None:
+        backend = TrackingBackend()
+
+        @cache(backend=backend, ttl=60, namespace=_FormatsAs("x"), key=lambda *a, **kw: "k", l1_enabled=False)
+        def f(x: int) -> int:
+            return x
+
+        f(1)
+        assert set(backend.store) == {"x:k"}
+        (rid,) = _registry_ids(backend)
+        assert rid.startswith("ck:reg:x:")
+
+    @pytest.mark.parametrize("value", ["ck", "ck:reg"])
+    def test_eq_and_startswith_override_cannot_bypass_ck_reservation(self, value: str) -> None:
+        ns = _HidesCk(value)
+        assert not ns == "ck" and not ns.startswith("ck:")  # the overrides the old check trusted
+        with pytest.raises(ConfigurationError, match="reserved"):
+
+            @cache(backend=TrackingBackend(), ttl=60, namespace=ns)
+            def f(x: int) -> int:
+                return x
+
+    def test_drain_also_empties_pre_fix_registry_set(self) -> None:
+        """Entries tracked under the pre-fix f-string registry id go on a no-args drain."""
+        backend = TrackingBackend()
+
+        @cache(backend=backend, ttl=60, namespace=_LegacyFormat("x"), l1_enabled=False)
+        def f(x: int) -> int:
+            return x
+
+        f(1)
+        (rid,) = _registry_ids(backend)
+        legacy_rid = "ck:reg:legacy:" + rid.rsplit(":", 1)[1]
+        backend.store["pre-upgrade-key"] = b"x"  # written and tracked by pre-fix code
+        backend.sets[legacy_rid] = {"pre-upgrade-key"}
+
+        f.invalidate_cache()
+        assert backend.store == {}
+        assert [r for r, _ in backend.drain_calls] == [rid, legacy_rid]
+
+    @pytest.mark.parametrize(
+        ("namespace", "interop", "legacy"),
+        [
+            ("users", None, None),
+            (_StrEnumNS.USERS, None, None),
+            (None, None, None),
+            # f"{member}" is "_NS.USERS" only from 3.11; on 3.10 the set name never moved.
+            (_NS.USERS, None, "_NS.USERS" if sys.version_info >= (3, 11) else None),
+            # 0.20.0 shipped the registry with interop's exact-str rebind: no pre-fix set exists.
+            (_NS.USERS, "get_user", None),
+        ],
+    )
+    def test_no_args_drain_ids(self, namespace: Optional[str], interop: Optional[str], legacy: Optional[str]) -> None:
+        """Only a namespace whose set name actually moved gets a second drain."""
+        backend = TrackingBackend()
+
+        @cache(backend=backend, ttl=60, namespace=namespace, interop=interop, l1_enabled=False)
+        def f(x: int) -> int:
+            return x
+
+        f(1)
+        (rid,) = _registry_ids(backend)
+        assert rid.startswith(f"ck:reg:{'users' if namespace is not None else ''}:")
+        f.invalidate_cache()
+        expected = [rid] if legacy is None else [rid, f"ck:reg:{legacy}:{rid.rsplit(':', 1)[1]}"]
+        assert [r for r, _ in backend.drain_calls] == expected