Make cluster consensus, crash recovery, and restore crash-safe

.config/nextest.toml

@@ -121,9 +121,12 @@ threads-required = 'num-test-threads'
 # suite while passing in 16.5s on its own, a 9x margin that rules out slowness.
 # `max-threads = 1` alone does not help, because the competition is the
 # unit-test fan-out rather than the other server tests. The cost is bounded —
-# two tests, one of which the default filter already excludes.
+# three tests, one of which the default filter already excludes.
+#
+# `pitr_restore` boots three servers in sequence and runs the restore three
+# times, so it needs the same raised kill.
 [[profile.default.overrides]]
-filter = 'binary(crash_wal_truncation) | binary(crash_ilp_timeseries_write)'
+filter = 'binary(crash_wal_truncation) | binary(crash_ilp_timeseries_write) | binary(pitr_restore)'
 test-group = 'server-process-serial'
 threads-required = 'num-test-threads'
 slow-timeout = { period = "30s", terminate-after = 8 }
@@ -147,7 +150,7 @@ slow-timeout = { period = "30s", terminate-after = 8 }
 # hide the cause rather than fix it. The tail this costs is bounded — roughly a
 # dozen crash/shutdown tests at about ten seconds each.
 [[profile.default.overrides]]
-filter = 'binary(wal_direct_io) | binary(ilp_client_address) | binary(crash_recovery) | binary(crash_recovery_overlays) | binary(crash_recovery_analytics) | binary(crash_resp_kv_write) | binary(crash_metadata_applier_wedge) | binary(crash_dropped_collection_reclaim) | binary(crash_purge_not_resurrected) | binary(crash_mid_replay) | binary(crash_checkpoint_corruption) | binary(crash_checkpoint_truncate_window) | binary(crash_refused_write_not_resurrected) | binary(crash_replay_fail_stop) | binary(crash_core_stall) | binary(crash_replay_stamp) | binary(crash_replay_stamp_calvin) | binary(calvin_hold_liveness) | binary(apply_pipeline_group_independence) | binary(crash_kv_atomic_autocommit) | test(/^cases::startup_failure::/) | test(/^cases::shutdown_in_flight::/) | test(/^cases::shutdown_budget::/) | test(/^cases::shutdown_abort_offender::/) | test(/^cases::shutdown_idempotent::/)'
+filter = 'binary(wal_direct_io) | binary(ilp_client_address) | binary(timeseries_write_events) | binary(crash_recovery) | binary(crash_recovery_overlays) | binary(crash_recovery_analytics) | binary(crash_resp_kv_write) | binary(crash_metadata_applier_wedge) | binary(crash_dropped_collection_reclaim) | binary(crash_purge_not_resurrected) | binary(crash_mid_replay) | binary(crash_checkpoint_corruption) | binary(crash_checkpoint_truncate_window) | binary(crash_refused_write_not_resurrected) | binary(crash_replay_fail_stop) | binary(crash_core_stall) | binary(crash_replay_stamp) | binary(crash_replay_stamp_calvin) | binary(calvin_hold_liveness) | binary(apply_pipeline_group_independence) | binary(crash_kv_atomic_autocommit) | test(/^cases::startup_failure::/) | test(/^cases::shutdown_in_flight::/) | test(/^cases::shutdown_budget::/) | test(/^cases::shutdown_abort_offender::/) | test(/^cases::shutdown_idempotent::/)'
 test-group = 'server-process'
 threads-required = 'num-test-threads'
 

CHANGELOG.md

@@ -14,6 +14,7 @@ NodeDB uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - Native-protocol `SELECT` returns nested objects and arrays as structured values, not JSON text. `nodedb_types::conversion::json_to_value_display` is replaced by `json_to_value_ref`.
 - JWT `metadata` claims keep their JSON type instead of being coerced to strings.
 - `document_get` for a missing id returns `Ok(None)` instead of a serialization error.
+- **`[server] single_node_calvin` is removed.** A server without a `[cluster]` section always runs the single-node Calvin sequencer, so cross-core (cross-vShard) transactions always commit atomically. A config that still sets the key fails to load as an unknown field. Remove the line.
 
 ### Added
 

docs/architecture.md

@@ -162,7 +162,7 @@ SET cross_shard_txn = 'best_effort_non_atomic';
 
 (Bare `best_effort` is deliberately rejected; invalid values return SQLSTATE `22023`.)
 
-**Single-node deployments run Calvin by default** (`[server] single_node_calvin = true`): a standalone node synthesizes a one-node sequencer group so transactions spanning multiple cores (vShards) commit atomically instead of being rejected. Set it `false` to force the legacy fast path. Uncontended single-shard point writes bypass the sequencer entirely and go directly through the relevant data-group Raft; contended or predicate/bulk writes route through the deterministic scheduler.
+**Single-node deployments always run Calvin.** A node with no `[cluster]` section synthesizes a one-node cluster with its own sequencer group, so transactions spanning multiple cores (vShards) commit atomically. Uncontended single-shard point writes bypass the sequencer entirely and go directly through the relevant data-group Raft; contended or predicate/bulk writes route through the deterministic scheduler.
 
 **Overlay hygiene.** Per-transaction staging overlays are kept alive by every staged write/read; overlays orphaned by vanished clients are reaped after a 6-hour lease. The `nodedb_active_txn_overlays` Prometheus gauge tracks live overlays. Data-Plane resource rejection surfaces as SQLSTATE `53200` (backpressure — retry when pressure subsides).
 

docs/databases.md

@@ -389,8 +389,8 @@ Database operations are gated by role:
 | `CLONE DATABASE`               | `Superuser`                    |
 | `MIRROR DATABASE`              | `Superuser`                    |
 | `MOVE TENANT`                  | `Superuser`                    |
-| `BACKUP DATABASE`              | `DatabaseOwner` or higher      |
-| `RESTORE DATABASE`             | `Superuser`                    |
+| `BACKUP DATABASE`              | `DatabaseOwner` or `Superuser` |
+| `RESTORE DATABASE`             | `DatabaseOwner` or `Superuser`; `Superuser` when the database does not exist |
 
 See [Roles & Permissions](security/rbac.md) for full role definitions.