feat: pre-announcement hardening across kernel, adapters, lenses, CLI, and supply chain

.changeset/pre-announcement-hardening.md

@@ -0,0 +1,39 @@
+---
+"@libredb/libredb": minor
+---
+
+Durability, safety, and API-contract hardening across the kernel, adapters, lenses, and CLI (the pre-announcement audit wave).
+
+On-disk format: new databases now begin with an 8-byte `LRDB` magic/version header. Files written by earlier releases (headerless) keep opening through a legacy read path; files written by this release are not readable by older releases. The header is what lets `open()` refuse a file that is not a LibreDB database with a clear error instead of destroying it.
+
+Kernel:
+
+- `open({ path })` on a non-LibreDB file throws `NOT_A_DATABASE` and leaves the file byte-for-byte untouched (previously the file was silently truncated to zero).
+- Recovery classifies failures: a torn tail truncates (reported through the new `onRecovery` open option), while mid-log corruption throws `CORRUPT_WAL` and truncates nothing. Record payloads are structurally validated during replay.
+- A failed append/fsync latches the database: every later `transact()` throws `FAILED` until reopen, so an IO error can never lead recovery to silently drop later acknowledged commits.
+- `transact()` rejects async callbacks (`ASYNC_TRANSACTION`): writes after an `await` could never reach the log.
+- Keys and values are copied at the transaction boundary in both directions — caller buffer reuse and mutation of returned buffers can no longer corrupt the store.
+- `getRange` snapshots at first iteration, so delete-while-scanning visits every entry exactly once.
+- `close()` inside a transaction throws `CLOSE_IN_TRANSACTION` instead of surfacing a raw file error.
+- `open()` takes an exclusive per-file lock (`<path>.lock`, pid/host/nonce): a second writer throws `LOCKED` instead of silently diverging; locks from verifiably dead holders are reclaimed automatically. `FileSystem` gains an optional `lock()` seam method.
+- All kernel failures are now `LibreDbError` instances carrying a stable `code` (exported, with the `ErrorCode` and `RecoveryInfo` types).
+
+Adapters:
+
+- node-fs: creating a database fsyncs the parent directory (a fresh database can no longer vanish wholesale on power loss); recovery truncation is fsync'd; reads are positional on the WAL's own file descriptor instead of re-reading the whole file per call.
+- OPFS: reads loop until filled, so a legal short read can no longer masquerade as a torn tail; recovery treats an incomplete read as an IO fault (`INCOMPLETE_READ`), never as license to truncate.
+
+Lenses:
+
+- Collection/table names may not be empty or contain `:` (both broke namespace isolation); ids keep full freedom.
+- Strings that are not well-formed UTF-16 (lone surrogates) are rejected wherever they would become keys, ids, names, or kv values — distinct strings can no longer silently collide on one key.
+- Relational `number` columns reject `NaN` and the infinities (JSON would store them as `null`).
+- `doc()` refuses a name cataloged as a relational table (it would bypass schema validation); `table()` refuses a document collection's name.
+- `find()`/`where()` reject a predicate field explicitly set to `undefined`, which previously matched documents *missing* the field.
+
+CLI:
+
+- Write commands rely on the kernel's exclusive lock; `--force` removes a lock only when its holder is not verifiably alive, and never deletes a file that is not a libredb lock.
+- `get`/`scan` escape control characters by default so untrusted values cannot inject terminal escape sequences; `--raw` prints verbatim.
+
+Docker image now runs as a non-root user (distroless `:nonroot`, uid 65532).

.github/dependabot.yml

@@ -0,0 +1,25 @@
+# Dependabot keeps the SHA-pinned actions, the digest-pinned Docker bases, and
+# the npm devDependencies from going stale: pinning without an update loop
+# inverts over time (CVE fixes never arrive unless someone remembers to bump
+# digests by hand). Weekly PRs preserve the pinning discipline — every bump is
+# still a reviewed, pinned change.
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+  - package-ecosystem: "docker"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+  - package-ecosystem: "npm"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    # Zero runtime dependencies is a design fact (license tripwire enforces
+    # it), so everything here is devDependencies; group the noise.
+    groups:
+      dev-dependencies:
+        patterns:
+          - "*"

.github/workflows/ci.yml

@@ -60,5 +60,26 @@ jobs:
             echo "| License tripwire (runtime deps) | ${{ steps.license.outcome }} |"
             echo ""
             echo "- Coverage: 100% line/function/statement (enforced by bunfig.toml)"
-            echo "- Bundle: ${KB} kB (min+brotli) / 4 kB budget"
+            echo "- Bundle: ${KB} kB (min+brotli) / 5 kB budget"
           } >> "$GITHUB_STEP_SUMMARY"
+
+  node-smoke:
+    name: node 22 smoke
+    # package.json declares engines.node >= 22 and the docs advertise `npx
+    # libredb` and Node embedding, but the gate runs on Bun. This job keeps the
+    # Node half of the declared runtime surface off the honor system: build the
+    # real dist/, then open, write, lock, reopen, and read a file-backed
+    # database under Node, plus one pass of the CLI entry.
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+      - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
+        with:
+          bun-version-file: .bun-version
+      - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
+        with:
+          node-version: "22"
+      - run: bun install --frozen-lockfile
+      - run: bun run build
+      - name: Smoke-test the built package under Node
+        run: node scripts/node-smoke.mjs

.github/workflows/publish.yml

@@ -29,8 +29,23 @@ jobs:
     # `latest` channel or Docker `:latest`. Every job carries the same guard.
     if: ${{ !github.event.release.prerelease }}
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write # OIDC: signs the npm provenance attestation
     steps:
       - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+      # The release tag must name the version the tree actually carries: a tag
+      # created before `changeset:version` (or a typo'd tag) must fail loudly
+      # here, not ship binaries and images labeled with the wrong semver.
+      - name: Verify release tag matches package.json
+        env:
+          TAG: ${{ github.event.release.tag_name }}
+        run: |
+          PKG="v$(node -p 'require("./package.json").version')"
+          if [ "$TAG" != "$PKG" ]; then
+            echo "Release tag $TAG does not match package.json $PKG" >&2
+            exit 1
+          fi
       - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
         with:
           bun-version-file: .bun-version
@@ -45,9 +60,9 @@ jobs:
           node-version: "22"
           registry-url: "https://registry.npmjs.org"
       # npm publish runs prepublishOnly (build + attw + publint) automatically.
-      # Provenance is intentionally omitted while the repo is private; once it is
-      # public, add `--provenance` here and `id-token: write` to permissions.
-      - run: npm publish --access public
+      # --provenance attaches a signed attestation binding the tarball to this
+      # repo, commit, and workflow (verifiable via `npm audit signatures`).
+      - run: npm publish --access public --provenance
         env:
           NODE_AUTH_TOKEN: ${{ secrets.NPMJS_TOKEN }}
       # Run summary on the run page (no emoji, per repo convention).
@@ -81,7 +96,11 @@ jobs:
       - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
         with:
           node-version: "22"
-      - run: npx --yes jsr publish
+      # Pinned: an unpinned `jsr@latest` would hand the publish-capable OIDC
+      # token to whatever the registry serves at run time (supply-chain risk).
+      # Bump the version deliberately, like every other pin in this workflow
+      # (Dependabot does not track npx invocations).
+      - run: npx --yes jsr@0.14.3 publish
       - name: Job summary
         if: success()
         run: |

.size-limit.json

@@ -2,12 +2,17 @@
   {
     "name": "public entry (min+brotli)",
     "path": "dist/index.js",
-    "ignore": ["node:fs", "node:os", "node:path"],
-    "limit": "4 kB"
+    "ignore": [
+      "node:crypto",
+      "node:fs",
+      "node:os",
+      "node:path"
+    ],
+    "limit": "5 kB"
   },
   {
     "name": "browser entry (min+brotli)",
     "path": "dist/browser.js",
-    "limit": "4 kB"
+    "limit": "5 kB"
   }
 ]

Dockerfile

@@ -24,10 +24,12 @@ COPY src ./src
 RUN bun build --compile src/cli/main.ts --outfile /libredb
 
 # Stage 2: a minimal runtime carrying only the binary and the glibc/libstdc++ a
-# bun-compiled executable links against. distroless/cc has exactly that. Pinned
-# by digest because `cc-debian12` is a rolling tag (no version), so the digest is
-# what makes the runtime reproducible; refresh it periodically for base updates.
-FROM gcr.io/distroless/cc-debian12@sha256:d703b626ba455c4e6c6fbe5f36e6f427c85d51445598d564652a2f334179f96e
+# bun-compiled executable links against. distroless/cc has exactly that. The
+# :nonroot variant runs as uid 65532 so files the CLI creates in a bind-mounted
+# host directory are not root-owned (and a container escape holds no root).
+# Mount data writable by that uid, or pass `--user "$(id -u):$(id -g)"`. Pinned
+# by digest because the tag is rolling; refresh it periodically for base updates.
+FROM gcr.io/distroless/cc-debian12:nonroot@sha256:b0ae8e989418b458e0f25489bc3be523718938a2b70864cc0f6a00af1ddbd985
 COPY --from=build /libredb /usr/local/bin/libredb
 WORKDIR /data
 ENTRYPOINT ["/usr/local/bin/libredb"]

README.md

@@ -247,10 +247,11 @@ the store only through one narrow `transact` port. For the full tour, read
 
 **Do not use it (yet) when you need:**
 
-- A hardened production datastore at scale — it is **pre-alpha**; today's beachhead is test/dev.
+- A hardened production datastore at scale — it is an **early beta**; today's beachhead is test/dev.
 - Secondary indexes or a query planner — queries are O(n) scans by design in v1 (on the roadmap).
-- Concurrent multi-process access, replication, or a networked client/server — it is embedded and
-  in-process.
+- Concurrent multi-process access, replication, or a networked client/server — it is embedded,
+  in-process, and strictly single-writer (a second `open()` on the same file is refused by an
+  exclusive lock rather than silently corrupting it).
 - SQL wire compatibility or an existing-driver ecosystem.
 
 These limits are deliberate v1 scope, not hidden gaps — LibreDB's strength comes from what it refuses.
@@ -264,17 +265,36 @@ See the [Manifesto](./MANIFESTO.md).
 </picture>
 
 A transaction that returns has been written to a length-framed, CRC-32-checksummed write-ahead log and
-`fsync`'d *before* the commit becomes visible — so a committed write survives a crash, and a crash can
-only ever damage the last, un-fsync'd record (which recovery detects and truncates). This is not just
-asserted: the kernel's crash/recovery path is proven by **deterministic simulation testing**, running
-the real engine against a seeded in-memory filesystem that tears, corrupts, and crashes the log on
-command, then checking that recovery is always a valid committed prefix.
+`fsync`'d *before* the commit becomes visible — so on a healthy disk a committed write survives a
+crash, and a crash can only ever damage the last, un-fsync'd record (which recovery detects,
+truncates, and reports). The failure modes *outside* the clean-crash model are handled explicitly, not
+assumed away: a failed append/fsync latches the database instead of writing past a torn record, a
+second writer is refused by an exclusive open lock, a file that is not a LibreDB database is refused
+untouched (the `LRDB` header), mid-log corruption refuses to open rather than silently truncating, and
+a short read is an IO error, never data loss. This is not just asserted: the crash/recovery path is
+proven by **deterministic simulation testing** — the real engine against a seeded in-memory filesystem
+that tears, corrupts, errors, and crashes the log on command — plus a binary round-trip fuzz.
 
 ```sh
-bun run test    # includes a bounded 50-seed DST run
+bun run test    # includes a bounded 50-seed DST run and the fault-injection suites
 ```
 
-The full durability and DST walkthrough is in [`docs/RELIABILITY.md`](./docs/RELIABILITY.md).
+The precise durability contract and the DST walkthrough are in
+[`docs/RELIABILITY.md`](./docs/RELIABILITY.md).
+
+## Performance envelope
+
+Honesty about scale (comprehension is the budget in v1, not throughput):
+
+- **The whole store lives in memory** as one sorted array; the file on disk is the append-only log
+  that rebuilds it on open. The practical ceiling is data that comfortably fits in RAM — the test/dev
+  beachhead, not a server working set.
+- **Each `transact()` copies the store** before applying writes, so a per-row auto-commit loop is
+  quadratic in store size and will look hung on large seeds. **Wrap bulk loads in one `transact()`**
+  (or use `libredb import`, which already does): one copy, one fsync, one record for the whole batch.
+- **No secondary indexes**: a `find`/`where` is an O(n) scan by design in v1.
+- **The log grows without bound** until compaction lands (tracked in
+  [#12](https://github.com/libredb/libredb/issues/12)); reopening replays the whole log.
 
 ## Documentation
 
@@ -293,14 +313,18 @@ The full durability and DST walkthrough is in [`docs/RELIABILITY.md`](./docs/REL
 
 ## Project status & roadmap
 
-LibreDB is **pre-alpha** (`0.0.x`). The architecture is in place and every line of the core is tested,
-but the API may still change and it is not yet meant for production data.
+LibreDB is an **early beta** (`0.1.x`). The architecture is in place, every line of the core is
+tested, and the durability contract above is enforced — but the API may still change before 1.0, and
+the recommended home is still test/dev data.
 
-- **Done:** the ordered key-value kernel (transactions, WAL, crash recovery); the key-value, document,
-  and relational lenses; the self-describing catalog; the deterministic simulation testing harness;
-  100% line/function/statement coverage on the core.
+- **Done:** the ordered key-value kernel (transactions, WAL with a versioned on-disk header, crash
+  recovery that refuses corruption and foreign files, an IO-failure latch, an exclusive open lock);
+  the key-value, document, and relational lenses; the self-describing catalog; typed `LibreDbError`
+  codes; the DST harness with IO-fault injection and binary fuzz; 100% line/function/statement
+  coverage.
 - **Next:** secondary indexes and a richer query surface; more query operators; additional lenses;
-  production-hardening milestones (directory fsync on first create, WAL compaction/checkpointing).
+  WAL compaction/checkpointing ([#12](https://github.com/libredb/libredb/issues/12)); real-browser
+  OPFS verification ([#10](https://github.com/libredb/libredb/issues/10)).
 
 ## The LibreDB family
 

docs/BROWSER.md

@@ -57,6 +57,17 @@ maps onto an OPFS **sync access handle** (whose `read`/`write`/`getSize`/
 with no async core. Sync access handles are only available **inside a dedicated
 Web Worker**, so durable LibreDB *must* live in a Worker.
 
+One honest caveat on the word *durable*: the kernel's durability point maps to
+the handle's `flush()`, and the OPFS specification does not promise that
+`flush()` carries POSIX-`fsync` strength against **power loss** — the browser's
+storage layer decides when bytes reach stable media. In practice a committed
+write survives a tab crash, a page reload, and a browser restart; what a sudden
+power cut can lose is browser-and-OS dependent. Treat OPFS durability as "as
+strong as the browser's flush", not as a battery-backed guarantee (verifying
+this per engine is tracked in
+[#10](https://github.com/libredb/libredb/issues/10)). Storage may also be
+evicted under pressure unless you request persistence — see the checklist below.
+
 ---
 
 ## 3. In-memory: the 30-second start (main thread)

docs/CLI.md

@@ -41,7 +41,8 @@ Usage:
   libredb import <path> <file.json>  Bulk-set keys from a JSON object (one atomic commit)
 
 Options:
-  --force                            Override an existing write lock
+  --force                            Remove a write lock whose holder is no longer alive
+  --raw                              Print values verbatim (default escapes control characters)
 ```
 
 ---
@@ -141,18 +142,49 @@ The CLI touches real database files, so it is deliberately careful:
   (`inspect`/`stats`/`get`/`scan`) open through a **read-only filesystem adapter**:
   recovery drops a torn tail *in memory only*; the bytes on disk are left exactly
   as found.
-- **Writes take an advisory lock.** LibreDB is single-process with no internal
-  file locking, so two concurrent writers would corrupt a file. `set`/`delete`/
-  `import` create a `<path>.lock` first; if one is already held the command fails
-  loudly. Pass `--force` only to clear a **stale** lock left by a crashed writer
-  — it will not delete a file that isn't a libredb lock.
+- **A wrong path cannot destroy a file.** Opening a file that is not a LibreDB
+  database (a typo, a text file) fails with a clear error and leaves the file
+  byte-for-byte untouched — the on-disk `LRDB` header is checked before anything
+  is written.
+- **Writes hold the exclusive open lock.** The library itself locks the database
+  on open (`<path>.lock`, recording the holder's pid and host), so a second
+  writer — this CLI against a live app, or two CLI invocations — fails loudly
+  instead of silently corrupting the file. A lock whose holder is verifiably dead
+  is reclaimed automatically; `--force` additionally removes a lock that cannot
+  be verified (for example one from another machine), but refuses a verifiably
+  live holder and refuses to delete a file that is not a libredb lock.
+- **Output is escaped by default.** `get`/`scan` print stored values with control
+  characters escaped (`\x1b`, `\x07`, ...), so a value containing terminal escape
+  sequences cannot clear your screen, retitle your terminal, or write your
+  clipboard when you inspect an untrusted file. Pass `--raw` for the exact bytes.
 - **Reserved keys are refused.** Writes reject keys in LibreDB's reserved
   namespace (the `\x00`-prefixed catalog space), so the CLI cannot corrupt the
   catalog or another lens's layout.
 - **Bulk imports are atomic.** `import` commits all keys in one transaction.
 
 ---
 
+## Backup and restore
+
+The WAL **is** the database: one `.libredb` file holds everything, so backup is a
+file copy — with one rule.
+
+- **Backup:** copy the file while **no writer has it open** (no `<path>.lock`
+  present, or only your own closed session). A copy taken mid-write can split a
+  record in half; the copy would then open only up to the split.
+
+  ```sh
+  cp app.libredb backup/app-$(date -u +%Y%m%d).libredb
+  ```
+
+- **Restore:** copy the file back and open it — recovery replays it like any
+  reopen. Nothing else to do.
+- **Export as text:** `libredb scan <path> ""` is not supported (an empty prefix
+  is refused); scan per namespace prefix, or use the programmatic lenses for a
+  structured export. A first-class `export` command is on the roadmap.
+
+---
+
 ## Exit codes
 
 | Code | Meaning | Examples |