feat: pre-announcement hardening across kernel, adapters, lenses, CLI, and supply chain

.changeset/pre-announcement-hardening.md

@@ -0,0 +1,43 @@
+---
+"@libredb/libredb": minor
+---
+
+Durability, safety, and API-contract hardening across the kernel, adapters, lenses, and CLI (the pre-announcement audit wave).
+
+On-disk format: new databases now begin with an 8-byte `LRDB` magic/version header, and each record header carries a checksum of its own length field. Files written by earlier releases (headerless) keep opening through a legacy read path, and keep their legacy record framing on later appends. The header is what lets `open()` refuse a file that is not a LibreDB database with a clear error instead of destroying it; the record-header checksum is what lets recovery refuse a damaged length field instead of mistaking it for a torn tail.
+
+DOWNGRADE WARNING: a file written by this release must never be opened by 0.1.3 or older — the old recovery cannot parse the header, classifies the whole file as a torn tail, and silently truncates it to zero bytes. Back up before any downgrade. Two smaller legacy-behavior changes: a headerless file whose only record is torn/incomplete now refuses to open as `NOT_A_DATABASE` (0.1.3 recovered it to an empty database; refusing is the safe reading, since such a file is indistinguishable from a foreign one), and a legacy length-field corruption still reads as a torn tail (the legacy format has no header checksum — the v1 format exists to close exactly that gap).
+
+Kernel:
+
+- `open({ path })` on a non-LibreDB file throws `NOT_A_DATABASE` and leaves the file byte-for-byte untouched (previously the file was silently truncated to zero).
+- Recovery classifies failures: a torn tail truncates (reported through the new `onRecovery` open option), while mid-log corruption throws `CORRUPT_WAL` and truncates nothing. Record payloads are structurally validated during replay.
+- A failed append/fsync latches the database: every later `transact()` throws `FAILED` until reopen, so an IO error can never lead recovery to silently drop later acknowledged commits.
+- `transact()` rejects async callbacks (`ASYNC_TRANSACTION`): writes after an `await` could never reach the log.
+- Keys and values are copied at the transaction boundary in both directions — caller buffer reuse and mutation of returned buffers can no longer corrupt the store.
+- `getRange` snapshots at first iteration, so delete-while-scanning visits every entry exactly once.
+- `close()` inside a transaction throws `CLOSE_IN_TRANSACTION` instead of surfacing a raw file error.
+- `open()` takes an exclusive per-file lock (`<path>.lock`, pid/host/nonce): a second writer throws `LOCKED` instead of silently diverging; locks from verifiably dead holders are reclaimed automatically. `FileSystem` gains an optional `lock()` seam method.
+- All kernel failures are now `LibreDbError` instances carrying a stable `code` (exported, with the `ErrorCode` and `RecoveryInfo` types).
+
+Adapters:
+
+- node-fs: creating a database fsyncs the parent directory (a fresh database can no longer vanish wholesale on power loss); recovery truncation is fsync'd; reads are positional on the WAL's own file descriptor instead of re-reading the whole file per call.
+- OPFS: reads loop until filled, so a legal short read can no longer masquerade as a torn tail; recovery treats an incomplete read as an IO fault (`INCOMPLETE_READ`), never as license to truncate.
+
+Lenses:
+
+- Collection/table names may not be empty or contain `:` (both broke namespace isolation); ids keep full freedom.
+- Strings that are not well-formed UTF-16 (lone surrogates) are rejected wherever they would become keys, ids, names, or kv values — distinct strings can no longer silently collide on one key.
+- Relational `number` columns reject `NaN` and the infinities (JSON would store them as `null`).
+- `doc()` refuses a name cataloged as a relational table (it would bypass schema validation); `table()` refuses a document collection's name.
+- `find()`/`where()` reject a predicate field explicitly set to `undefined`, which previously matched documents *missing* the field.
+
+CLI:
+
+- Write commands rely on the kernel's exclusive lock; `--force` removes a lock only when its holder is not verifiably alive, and never deletes a file that is not a libredb lock.
+- `get`/`scan` escape control characters (including tab and newline) by default so untrusted values cannot inject terminal escape sequences — scripts that consumed values verbatim should pass `--raw`.
+
+New exports: `LibreDbError`, `ErrorCode`, `RecoveryInfo`, `nodeFileSystem`, and `readonlyFileSystem` (open a database for inspection with no lock and no writes — the supported way to read a file a live writer holds).
+
+Docker image now runs as a non-root user (distroless `:nonroot`, uid 65532): bind-mounted directories must be writable by that uid, or pass `--user "$(id -u):$(id -g)"`.

.github/dependabot.yml

@@ -0,0 +1,25 @@
+# Dependabot keeps the SHA-pinned actions, the digest-pinned Docker bases, and
+# the npm devDependencies from going stale: pinning without an update loop
+# inverts over time (CVE fixes never arrive unless someone remembers to bump
+# digests by hand). Weekly PRs preserve the pinning discipline — every bump is
+# still a reviewed, pinned change.
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+  - package-ecosystem: "docker"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+  - package-ecosystem: "npm"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    # Zero runtime dependencies is a design fact (license tripwire enforces
+    # it), so everything here is devDependencies; group the noise.
+    groups:
+      dev-dependencies:
+        patterns:
+          - "*"

.github/workflows/ci.yml

@@ -60,5 +60,26 @@ jobs:
             echo "| License tripwire (runtime deps) | ${{ steps.license.outcome }} |"
             echo ""
             echo "- Coverage: 100% line/function/statement (enforced by bunfig.toml)"
-            echo "- Bundle: ${KB} kB (min+brotli) / 4 kB budget"
+            echo "- Bundle: ${KB} kB (min+brotli) / 5 kB budget"
           } >> "$GITHUB_STEP_SUMMARY"
+
+  node-smoke:
+    name: node 22 smoke
+    # package.json declares engines.node >= 22 and the docs advertise `npx
+    # libredb` and Node embedding, but the gate runs on Bun. This job keeps the
+    # Node half of the declared runtime surface off the honor system: build the
+    # real dist/, then open, write, lock, reopen, and read a file-backed
+    # database under Node, plus one pass of the CLI entry.
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+      - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
+        with:
+          bun-version-file: .bun-version
+      - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
+        with:
+          node-version: "22"
+      - run: bun install --frozen-lockfile
+      - run: bun run build
+      - name: Smoke-test the built package under Node
+        run: node scripts/node-smoke.mjs

.github/workflows/publish.yml

@@ -29,8 +29,23 @@ jobs:
     # `latest` channel or Docker `:latest`. Every job carries the same guard.
     if: ${{ !github.event.release.prerelease }}
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write # OIDC: signs the npm provenance attestation
     steps:
       - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+      # The release tag must name the version the tree actually carries: a tag
+      # created before `changeset:version` (or a typo'd tag) must fail loudly
+      # here, not ship binaries and images labeled with the wrong semver.
+      - name: Verify release tag matches package.json
+        env:
+          TAG: ${{ github.event.release.tag_name }}
+        run: |
+          PKG="v$(node -p 'require("./package.json").version')"
+          if [ "$TAG" != "$PKG" ]; then
+            echo "Release tag $TAG does not match package.json $PKG" >&2
+            exit 1
+          fi
       - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
         with:
           bun-version-file: .bun-version
@@ -45,9 +60,9 @@ jobs:
           node-version: "22"
           registry-url: "https://registry.npmjs.org"
       # npm publish runs prepublishOnly (build + attw + publint) automatically.
-      # Provenance is intentionally omitted while the repo is private; once it is
-      # public, add `--provenance` here and `id-token: write` to permissions.
-      - run: npm publish --access public
+      # --provenance attaches a signed attestation binding the tarball to this
+      # repo, commit, and workflow (verifiable via `npm audit signatures`).
+      - run: npm publish --access public --provenance
         env:
           NODE_AUTH_TOKEN: ${{ secrets.NPMJS_TOKEN }}
       # Run summary on the run page (no emoji, per repo convention).
@@ -81,7 +96,11 @@ jobs:
       - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
         with:
           node-version: "22"
-      - run: npx --yes jsr publish
+      # Pinned: an unpinned `jsr@latest` would hand the publish-capable OIDC
+      # token to whatever the registry serves at run time (supply-chain risk).
+      # Bump the version deliberately, like every other pin in this workflow
+      # (Dependabot does not track npx invocations).
+      - run: npx --yes jsr@0.14.3 publish
       - name: Job summary
         if: success()
         run: |

.size-limit.json

@@ -2,12 +2,17 @@
   {
     "name": "public entry (min+brotli)",
     "path": "dist/index.js",
-    "ignore": ["node:fs", "node:os", "node:path"],
-    "limit": "4 kB"
+    "ignore": [
+      "node:crypto",
+      "node:fs",
+      "node:os",
+      "node:path"
+    ],
+    "limit": "5 kB"
   },
   {
     "name": "browser entry (min+brotli)",
     "path": "dist/browser.js",
-    "limit": "4 kB"
+    "limit": "5 kB"
   }
 ]

Dockerfile

@@ -24,10 +24,12 @@ COPY src ./src
 RUN bun build --compile src/cli/main.ts --outfile /libredb
 
 # Stage 2: a minimal runtime carrying only the binary and the glibc/libstdc++ a
-# bun-compiled executable links against. distroless/cc has exactly that. Pinned
-# by digest because `cc-debian12` is a rolling tag (no version), so the digest is
-# what makes the runtime reproducible; refresh it periodically for base updates.
-FROM gcr.io/distroless/cc-debian12@sha256:d703b626ba455c4e6c6fbe5f36e6f427c85d51445598d564652a2f334179f96e
+# bun-compiled executable links against. distroless/cc has exactly that. The
+# :nonroot variant runs as uid 65532 so files the CLI creates in a bind-mounted
+# host directory are not root-owned (and a container escape holds no root).
+# Mount data writable by that uid, or pass `--user "$(id -u):$(id -g)"`. Pinned
+# by digest because the tag is rolling; refresh it periodically for base updates.
+FROM gcr.io/distroless/cc-debian12:nonroot@sha256:b0ae8e989418b458e0f25489bc3be523718938a2b70864cc0f6a00af1ddbd985
 COPY --from=build /libredb /usr/local/bin/libredb
 WORKDIR /data
 ENTRYPOINT ["/usr/local/bin/libredb"]

README.md

@@ -247,10 +247,11 @@ the store only through one narrow `transact` port. For the full tour, read
 
 **Do not use it (yet) when you need:**
 
-- A hardened production datastore at scale — it is **pre-alpha**; today's beachhead is test/dev.
+- A hardened production datastore at scale — it is an **early beta**; today's beachhead is test/dev.
 - Secondary indexes or a query planner — queries are O(n) scans by design in v1 (on the roadmap).
-- Concurrent multi-process access, replication, or a networked client/server — it is embedded and
-  in-process.
+- Concurrent multi-process access, replication, or a networked client/server — it is embedded,
+  in-process, and strictly single-writer (a second `open()` on the same file is refused by an
+  exclusive lock rather than silently corrupting it).
 - SQL wire compatibility or an existing-driver ecosystem.
 
 These limits are deliberate v1 scope, not hidden gaps — LibreDB's strength comes from what it refuses.
@@ -264,17 +265,36 @@ See the [Manifesto](./MANIFESTO.md).
 </picture>
 
 A transaction that returns has been written to a length-framed, CRC-32-checksummed write-ahead log and
-`fsync`'d *before* the commit becomes visible — so a committed write survives a crash, and a crash can
-only ever damage the last, un-fsync'd record (which recovery detects and truncates). This is not just
-asserted: the kernel's crash/recovery path is proven by **deterministic simulation testing**, running
-the real engine against a seeded in-memory filesystem that tears, corrupts, and crashes the log on
-command, then checking that recovery is always a valid committed prefix.
+`fsync`'d *before* the commit becomes visible — so on a healthy disk a committed write survives a
+crash, and a crash can only ever damage the last, un-fsync'd record (which recovery detects,
+truncates, and reports). The failure modes *outside* the clean-crash model are handled explicitly, not
+assumed away: a failed append/fsync latches the database instead of writing past a torn record, a
+second writer is refused by an exclusive open lock, a file that is not a LibreDB database is refused
+untouched (the `LRDB` header), mid-log corruption refuses to open rather than silently truncating, and
+a short read is an IO error, never data loss. This is not just asserted: the crash/recovery path is
+proven by **deterministic simulation testing** — the real engine against a seeded in-memory filesystem
+that tears, corrupts, errors, and crashes the log on command — plus a binary round-trip fuzz.
 
 ```sh
-bun run test    # includes a bounded 50-seed DST run
+bun run test    # includes a bounded 50-seed DST run and the fault-injection suites
 ```
 
-The full durability and DST walkthrough is in [`docs/RELIABILITY.md`](./docs/RELIABILITY.md).
+The precise durability contract and the DST walkthrough are in
+[`docs/RELIABILITY.md`](./docs/RELIABILITY.md).
+
+## Performance envelope
+
+Honesty about scale (comprehension is the budget in v1, not throughput):
+
+- **The whole store lives in memory** as one sorted array; the file on disk is the append-only log
+  that rebuilds it on open. The practical ceiling is data that comfortably fits in RAM — the test/dev
+  beachhead, not a server working set.
+- **Each `transact()` copies the store** before applying writes, so a per-row auto-commit loop is
+  quadratic in store size and will look hung on large seeds. **Wrap bulk loads in one `transact()`**
+  (or use `libredb import`, which already does): one copy, one fsync, one record for the whole batch.
+- **No secondary indexes**: a `find`/`where` is an O(n) scan by design in v1.
+- **The log grows without bound** until compaction lands (tracked in
+  [#12](https://github.com/libredb/libredb/issues/12)); reopening replays the whole log.
 
 ## Documentation
 
@@ -293,14 +313,18 @@ The full durability and DST walkthrough is in [`docs/RELIABILITY.md`](./docs/REL
 
 ## Project status & roadmap
 
-LibreDB is **pre-alpha** (`0.0.x`). The architecture is in place and every line of the core is tested,
-but the API may still change and it is not yet meant for production data.
+LibreDB is an **early beta** (`0.1.x`). The architecture is in place, every line of the core is
+tested, and the durability contract above is enforced — but the API may still change before 1.0, and
+the recommended home is still test/dev data.
 
-- **Done:** the ordered key-value kernel (transactions, WAL, crash recovery); the key-value, document,
-  and relational lenses; the self-describing catalog; the deterministic simulation testing harness;
-  100% line/function/statement coverage on the core.
+- **Done:** the ordered key-value kernel (transactions, WAL with a versioned on-disk header, crash
+  recovery that refuses corruption and foreign files, an IO-failure latch, an exclusive open lock);
+  the key-value, document, and relational lenses; the self-describing catalog; typed `LibreDbError`
+  codes; the DST harness with IO-fault injection and binary fuzz; 100% line/function/statement
+  coverage.
 - **Next:** secondary indexes and a richer query surface; more query operators; additional lenses;
-  production-hardening milestones (directory fsync on first create, WAL compaction/checkpointing).
+  WAL compaction/checkpointing ([#12](https://github.com/libredb/libredb/issues/12)); real-browser
+  OPFS verification ([#10](https://github.com/libredb/libredb/issues/10)).
 
 ## The LibreDB family
 

docs/BROWSER.md

@@ -57,6 +57,17 @@ maps onto an OPFS **sync access handle** (whose `read`/`write`/`getSize`/
 with no async core. Sync access handles are only available **inside a dedicated
 Web Worker**, so durable LibreDB *must* live in a Worker.
 
+One honest caveat on the word *durable*: the kernel's durability point maps to
+the handle's `flush()`, and the OPFS specification does not promise that
+`flush()` carries POSIX-`fsync` strength against **power loss** — the browser's
+storage layer decides when bytes reach stable media. In practice a committed
+write survives a tab crash, a page reload, and a browser restart; what a sudden
+power cut can lose is browser-and-OS dependent. Treat OPFS durability as "as
+strong as the browser's flush", not as a battery-backed guarantee (verifying
+this per engine is tracked in
+[#10](https://github.com/libredb/libredb/issues/10)). Storage may also be
+evicted under pressure unless you request persistence — see the checklist below.
+
 ---
 
 ## 3. In-memory: the 30-second start (main thread)