fix(genesis-writer): flush and drain before checkpointing a step

[rickyrombo]

The checkpoint can outrun the data

Step completion was recorded as soon as step.fn returned:

if err := step.fn(ctx); err != nil { ... }
// Record step completion for resume.
INSERT INTO genesis_writer_progress ...

But rows accumulate in w.blockTxs and are written by an async pipeline through a buffered channel (blockWriteCh, size 4). A step returning does not mean its rows are durable — its trailing block may still be unflushed, or queued and unwritten.

So the progress row can land while the step's tail has not. A later resume then skips a step that never finished, and nothing detects it: the step reads as complete.

What it cost

On the 2026-08-07 snapshot the events step was recorded complete holding only the first 73 of 112 rows.

That it is a clean prefix is the signature of a lost queue tail rather than a filtered source — event_id order correlates with time at only 0.154, and the missing rows span the same date range (2025-04 to 2026-08) as the migrated ones.

The cascade:

lost	rows
events	39
event subscriptions	606
comments on events	63
total	708

Fix

Flush the current block and drain the writer before the progress insert, so the checkpoint means what the README already claims:

If interrupted mid-step, the entire step is rerun on resume.

That was the intent; it just wasn't enforced. Each step now ends its block early, costing a few partial blocks — already true of the last block of any run.

Testing

The existing suite passes. There is no direct test: exercising this needs a live database and an interruption injected precisely between step return and queue drain. The change is structural — the barrier is unconditional — but I want to be straight that the failure path itself is not covered.

🤖 Generated with Claude Code