Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions distribution/self-hosted/release-manifest.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -129,9 +129,9 @@
"required": ["sha256", "count", "latest", "forwardOnly"],
"properties": {
"sha256": { "$ref": "#/$defs/sha256" },
"count": { "const": 10 },
"latest": { "const": "010" },
"forwardOnly": { "const": ["010"] }
"count": { "const": 11 },
"latest": { "const": "011" },
"forwardOnly": { "const": ["010", "011"] }
}
},
"catalog": {
Expand Down
33 changes: 26 additions & 7 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,24 @@ schema. An active forgotten writer causes a bounded lock-timeout failure; the
transaction and migration registration roll back, leaving schema 009 usable.
Stop that writer and rerun the unchanged migration.

Migration 011 reconciles the historical
`external_source_snapshots.decoded_bytes` counter. Native ingestion before 011
counted UTF-8 instruction bytes plus resource bytes, while the restore gate and
migration-006 backfill defined the same derived counter as the sum of immutable
canonical revision-object bytes. The migration accepts only an exact match to
one of those two representations, recomputes the canonical total through
immutable snapshot observations (including reused revisions), and records every
prior and resulting value in the append-only
`external_snapshot_byte_total_reconciliations` ledger. It does not change bundle
or content hashes, revision identities, provenance, resources, classifications,
or advisory history. Any other total, malformed object, hash mismatch, overflow,
or duplicate accounting aborts the transaction and leaves schema 010 unchanged.
Migration 011 is safe to rerun through the normal idempotent migrator, but it is
also a forward-only boundary. A deferred database constraint rejects every new
snapshot unless its canonical projection and reconciliation evidence agree at
commit, so a pre-011 writer fails rather than advancing mixed-format history.
Never start that writer against schema 011.

### Backup, restore, and rollback for migration 010

The pre-upgrade backup must include the complete PostgreSQL database and be
Expand Down Expand Up @@ -287,13 +305,14 @@ corrections. Do not edit the migration table.

`Database migration ... is newer than binary migration ...` means the selected
image is too old for that database; select a compatible post-migration image. Do
not bypass the guard. A lock or statement timeout during migration 010 means
either a writer was not drained or the rehearsed bound is too small. Confirm
writers are stopped before changing a bound. Because each migration and its
registration are transactional, a failed 010 leaves schema 009 in place and is
safe to rerun after the cause is fixed. A historical candidate-attribution
failure is a data-integrity stop: keep traffic down, preserve evidence, and
repair it only through a separately reviewed forward migration or restore.
not bypass the guard. A lock or statement timeout during migration 010 or 011
means either a writer was not drained or the rehearsed bound is too small.
Confirm writers are stopped before changing a bound. Because each migration and
its registration are transactional, a failed migration leaves the preceding
schema in place and is safe to rerun after the cause is fixed. A historical
candidate-attribution failure is a data-integrity stop: keep traffic down,
preserve evidence, and repair it only through a separately reviewed forward
migration or restore.

### Catalog verification fails

Expand Down
38 changes: 38 additions & 0 deletions docs/upgrades/v0.2.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# v0.2.0 upgrade: canonical snapshot byte totals

v0.2.0 adds migration 011 to reconcile one derived imported-snapshot counter.
The immutable catalog objects, hashes, revisions, provenance, classifications,
and advisory history remain authoritative and unchanged.

Before migration 011, two exact counter representations can exist:

- native ingestion counted UTF-8 instruction bytes plus resource bytes;
- the migration-006 backfill counted canonical revision-object bytes.

Both are derived exclusively from immutable objects. Migration 011 accepts only
an exact match to one of them, records the prior representation and both totals
in an append-only ledger, and stores the canonical total. Shared revisions are
accounted through their immutable per-snapshot observations, not only through
the snapshot that first created the revision. Quarantined and missing
observations contribute no revision bytes. After migration, a deferred database
constraint recomputes the immutable projection at commit and rejects missing or
fabricated reconciliation evidence, including writes from a pre-011 binary.

Treat this as a maintenance upgrade:

1. Stop and drain every SkillWire, ingestion, and administration writer.
2. Take a protected PostgreSQL backup and restore-validate it in an isolated
PostgreSQL 17.10 instance while it is still on schema 009 or 010.
3. Run the v0.2.0 migrator. Schema 009 upgrades through 010 and 011; schema 010
applies only 011.
4. Rerun the migrator to prove idempotence, then require schema 011 readiness,
catalog verification, advisory verification, and the exact MCP smoke tests.
5. Start only a schema-011-compatible application.

An arbitrary counter difference is not a compatibility case. Malformed or
missing catalog objects, content or bundle hash drift, overflow, duplicate
revision accounting, invalid attribution, or inconsistent audit evidence aborts
the migration. Preserve the failed database and backup; do not edit either in
place. Crossing back over migration 010 or 011 requires the validated
pre-upgrade database backup and its matching application image, not an
image-only rollback.
Loading