From 0e7c720dcac6ee2038c4743d63fd6b8039351e32 Mon Sep 17 00:00:00 2001 From: Pascal Grange Date: Fri, 9 Oct 2026 09:32:39 +0200 Subject: [PATCH] InMemory: reject writeEbBody for unregistered points SQLite rejects a body written for a point that writeEbPoint never inserted. InMemory instead registered the point silently, so InMemory-based tests could not catch a missing registration. Today processLeiosBlock always registers the point first, so nothing hits this and the tests still pass. The guard matters once this TODO lands: -- TODO remove the 'writeEbPoint' call below once no important node's -- VolatileDB still holds an announcing RB that it fetched without this -- patch. [...] pointWritten <- writeEbPoint writer point ebBytesSize' After that, any body reaching processLeiosBlock without a centrally-processed announcement would fail on SQLite. With this change InMemory fails the same way, so the tests surface it instead of passing: ``` $> cabal test ouroboros-consensus:consensus-test --test-options='--pattern "/LeiosDemoLogic.Invariants/"' Test suite consensus-test: RUNNING... ouroboros-consensus Leios LeiosDemoLogic.Invariants curated sequences forge purges a body it already holds (offered first): FAIL ouroboros-consensus/test/consensus-test/Test/LeiosDemoLogic/Invariants.hs:136: expected: Right [] but got: Left "exception on Forge [0,1] 12: LeiosDbWriteException {writeJob = \"WriteEbBody (12, 47f6c6404a56ea658d0b40e31c47eba73beaad3d3d71a5e61fc0e90ea48d6d99)\", submittedFrom = \"CallStack (from HasCallStack):\\n resolved, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:197:11 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory\\n writeEbBody, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/Common.hs:135:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.Common\\n a use of `writeEbBody', called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs:1112:22 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoLogic\", writeFailure = LeiosDbException {errorMessage = \"writeEbBody: point not registered (programmer error)\", callStack = \"CallStack (from HasCallStack):\\n throwLeiosDbException, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:297:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory\"}}" Use -p '/LeiosDemoLogic.Invariants/&&/forge purges a body it already holds (offered first)/' to rerun this test only. an offer of a self-forged EB is not re-fetched (forged first): FAIL ouroboros-consensus/test/consensus-test/Test/LeiosDemoLogic/Invariants.hs:138: expected: Right [] but got: Left "exception on Forge [0,1] 12: LeiosDbWriteException {writeJob = \"WriteEbBody (12, 47f6c6404a56ea658d0b40e31c47eba73beaad3d3d71a5e61fc0e90ea48d6d99)\", submittedFrom = \"CallStack (from HasCallStack):\\n resolved, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:197:11 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory\\n writeEbBody, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/Common.hs:135:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.Common\\n a use of `writeEbBody', called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs:1112:22 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoLogic\", writeFailure = LeiosDbException {errorMessage = \"writeEbBody: point not registered (programmer error)\", callStack = \"CallStack (from HasCallStack):\\n throwLeiosDbException, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:297:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory\"}}" Use -p '/LeiosDemoLogic.Invariants/&&/an offer of a self-forged EB is not re-fetched (forged first)/' to rerun this test only. a body is claimed acquired only by a settled write: OK acquired EB kept until its greatest slot is below the immutable tip: OK an announcement raises a forged EB's max slot (so it isn't pruned early): OK an announcement's onset is recorded (earliest kept); an offer never clobbers it: OK start-up seeding marks each completed EB held, with an empty pool: OK start-up seeding: a peer's offer of a seeded EB is not re-fetched: OK a big-ledger peer has a larger, but still finite, closure budget: OK job assignment draws within the least-requested bucket, at random, respecting exclusions: OK only the announcement that takes an election's focus is tracked: OK EB-hash collision (ported from #2309) forged, then the same EB announced at a new slot: both points registered and notified: FAIL Exception: WriteEbBody (5, 47f6c6404a56ea658d0b40e31c47eba73beaad3d3d71a5e61fc0e90ea48d6d99) failed: writeEbBody: point not registered (programmer error) CallStack (from HasCallStack): throwLeiosDbException, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:297:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory submitted from: CallStack (from HasCallStack): resolved, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:197:11 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory writeEbBody, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/Common.hs:135:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.Common a use of `writeEbBody', called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs:1112:22 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoLogic HasCallStack backtrace: collectBacktraces, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:169:13 in ghc-internal:GHC.Internal.Exception toExceptionWithBacktrace, called at libraries/ghc-internal/src/GHC/Internal/IO.hs:260:11 in ghc-internal:GHC.Internal.IO throwIO, called at ./Control/Concurrent/Async.hs:78:13 in tsty-1.5.4-843ddea7:Control.Concurrent.Async Use -p '/LeiosDemoLogic.Invariants/&&/forged, then the same EB announced at a new slot: both points registered and notified/' to rerun this test only. body held, then the same EB announced at a new slot: both points registered: OK both announced before the fetch: both points registered: OK forged, then the same EB merely offered at a new slot: the offer registers nothing: FAIL Exception: WriteEbBody (5, 47f6c6404a56ea658d0b40e31c47eba73beaad3d3d71a5e61fc0e90ea48d6d99) failed: writeEbBody: point not registered (programmer error) CallStack (from HasCallStack): throwLeiosDbException, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:297:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory submitted from: CallStack (from HasCallStack): resolved, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:197:11 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory writeEbBody, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/Common.hs:135:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.Common a use of `writeEbBody', called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs:1112:22 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoLogic HasCallStack backtrace: collectBacktraces, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:169:13 in ghc-internal:GHC.Internal.Exception toExceptionWithBacktrace, called at libraries/ghc-internal/src/GHC/Internal/IO.hs:260:11 in ghc-internal:GHC.Internal.IO throwIO, called at ./Control/Concurrent/Async.hs:78:13 in tsty-1.5.4-843ddea7:Control.Concurrent.Async Use -p '/LeiosDemoLogic.Invariants/&&/forged, then the same EB merely offered at a new slot: the offer registers nothing/' to rerun this test only. a certificate tracks an endorser block no announcement did: OK an endorser block that references too many tx bytes is dropped, not fetched: OK an endorser block at exactly its bound is fetched: FAIL ouroboros-consensus/test/consensus-test/Test/LeiosDemoLogic/Invariants.hs:476: expected BodyAcquired, got Just (MkEbState (SlotNo 5) SNothing NoBody) Use -p '/LeiosDemoLogic.Invariants/&&/an endorser block at exactly its bound is fetched/' to rerun this test only. a certificate leaves a body we already hold held: OK outstanding-state invariants hold across arbitrary sequences: FAIL *** Failed! Falsified (after 4 tests and 1 shrink): [ArriveBody [2,3,4] 11,Announce [1,2] 11] Left "exception on Announce [1,2] 11: ExceptionInLinkedThread \"ThreadId [1]\" (LeiosDbWriteException {writeJob = \"WriteEbBody (11, ff3e90aa6fbe36b630f58be98da978061e94c0a998da2376f0b4abf615082391)\", submittedFrom = \"CallStack (from HasCallStack):\\n resolved, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:197:11 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory\\n writeEbBody, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/Common.hs:135:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.Common\\n a use of `writeEbBody', called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs:1112:22 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoLogic\", writeFailure = LeiosDbException {errorMessage = \"writeEbBody: point not registered (programmer error)\", callStack = \"CallStack (from HasCallStack):\\n throwLeiosDbException, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:297:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory\"}})" /= Right () Use --quickcheck-replay="(SMGen 14245007439037612623 12544114665117986533,3)" to reproduce. Use -p '/LeiosDemoLogic.Invariants/&&/outstanding-state invariants hold across arbitrary sequences/' to rerun this test only. the fetch logic never requests an already-held EB body: FAIL *** Failed! Falsified (after 4 tests and 1 shrink): [ArriveBody [2,3,4] 10,Disconnect] exception on Disconnect: ExceptionInLinkedThread "ThreadId [1]" (LeiosDbWriteException {writeJob = "WriteEbBody (10, ff3e90aa6fbe36b630f58be98da978061e94c0a998da2376f0b4abf615082391)", submittedFrom = "CallStack (from HasCallStack):\n resolved, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:197:11 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory\n writeEbBody, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/Common.hs:135:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.Common\n a use of `writeEbBody', called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs:1112:22 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoLogic", writeFailure = LeiosDbException {errorMessage = "writeEbBody: point not registered (programmer error)", callStack = "CallStack (from HasCallStack):\n throwLeiosDbException, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:297:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory"}}) Use --quickcheck-replay="(SMGen 10024954925442282638 16722750794533007117,3)" to reproduce. Use -p '/LeiosDemoLogic.Invariants/&&/the fetch logic never requests an already-held EB body/' to rerun this test only. a concurrent offer, announcement and body arrival keep the reverse index in sync (IOSimPOR): FAIL *** Failed! Falsified (after 1 test): Schedule control: ControlDefault No thread delayed Failure: FailureException (ExceptionInLinkedThread "RacyThreadId [2,2,1]" (LeiosDbWriteException {writeJob = "WriteEbBody (12, 47f6c6404a56ea658d0b40e31c47eba73beaad3d3d71a5e61fc0e90ea48d6d99)", submittedFrom = "CallStack (from HasCallStack):\n resolved, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:197:11 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory\n writeEbBody, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/Common.hs:135:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.Common\n a use of `writeEbBody', called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs:1112:22 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoLogic", writeFailure = LeiosDbException {errorMessage = "writeEbBody: point not registered (programmer error)", callStack = "CallStack (from HasCallStack):\n throwLeiosDbException, called at ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs:297:5 in ouroboros-consensus-3.0.1.0-inplace:LeiosDemoDb.InMemory"}})) Use --quickcheck-replay="(SMGen 7877884416199475454 17480834384350021,0)" to reproduce. Use -p '/LeiosDemoLogic.Invariants/&&/a concurrent offer, announcement and body arrival keep the reverse index in sync (IOSimPOR)/' to rerun this test only. 8 out of 22 tests failed (0.01s) ``` --- .../ouroboros-consensus/LeiosDemoDb/InMemory.hs | 6 ++++++ .../test/consensus-test/Test/LeiosDemoDb.hs | 16 ++++++++++++++++ .../test/consensus-test/Test/LeiosDemoLogic.hs | 6 +++++- 3 files changed, 27 insertions(+), 1 deletion(-) diff --git a/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs b/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs index 8d0415995c..29fb03f5ca 100644 --- a/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs +++ b/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoDb/InMemory.hs @@ -22,6 +22,7 @@ import Control.Concurrent.Class.MonadSTM.Strict , newBroadcastTChan , newTVarIO , readTVar + , readTVarIO , writeTChan ) import Data.ByteString (ByteString) @@ -289,6 +290,11 @@ imInsertEbBody stateVar notificationChan point eb fills = do ebBytesSize = encodeLeiosEbSize eb when (null items) $ throwLeiosDbException "writeEbBody: empty EB body (programmer error)" + -- As on SQLite: the point MUST already be present (inserted via + -- 'writeEbPoint' on the announcement path). + known <- Map.member point . imEbPoints <$> readTVarIO stateVar + when (not known) $ + throwLeiosDbException "writeEbBody: point not registered (programmer error)" atomically $ do let entries = IntMap.fromList diff --git a/ouroboros-consensus/test/consensus-test/Test/LeiosDemoDb.hs b/ouroboros-consensus/test/consensus-test/Test/LeiosDemoDb.hs index 2084782cb8..e80942b0e2 100644 --- a/ouroboros-consensus/test/consensus-test/Test/LeiosDemoDb.hs +++ b/ouroboros-consensus/test/consensus-test/Test/LeiosDemoDb.hs @@ -126,6 +126,10 @@ tests = "orphanhood" [ testCase "dropping the handle does not kill its owner" test_droppingTheHandleSpreadsNoException ] + , -- InMemory only: on SQLite a failed ingest write also kills the + -- writer (see 'startWriter'), which is linked to the test thread. + testCase "InMemory: inserting a body for an unregistered point fails, as on SQLite" $ + withFreshDb InMemory test_ebBodyWithoutPointFails ] -- | Database creation strategy for different implementations. @@ -845,6 +849,18 @@ test_noReNotifyOnRelatedTxReinsert db = do -- per (slot, hash) when the closure completes, regardless of how many -- slots reference the same hash. Conflating the slots loses a -- notification. +-- | A body can only be persisted for a point already registered (via +-- 'writeEbPoint', on the announcement path). +test_ebBodyWithoutPointFails :: LeiosDbHandle IO -> IO () +test_ebBodyWithoutPointFails db = withRW db $ \con -> do + result <- tryDb $ rwInsertEbBody con (mkTestPoint (SlotNo 1) 1) (mkTestEb 2) + case result of + Left _ -> pure () + Right _ -> assertFailure "writeEbBody succeeded for a point that was never registered" + where + tryDb :: IO a -> IO (Either LeiosDbException a) + tryDb = try + test_multipleSlotsSameHash :: LeiosDbHandle IO -> IO () test_multipleSlotsSameHash db = do chan <- subscribeEbNotifications db diff --git a/ouroboros-consensus/test/consensus-test/Test/LeiosDemoLogic.hs b/ouroboros-consensus/test/consensus-test/Test/LeiosDemoLogic.hs index bb0a05b58f..cb71131c76 100644 --- a/ouroboros-consensus/test/consensus-test/Test/LeiosDemoLogic.hs +++ b/ouroboros-consensus/test/consensus-test/Test/LeiosDemoLogic.hs @@ -64,6 +64,7 @@ import LeiosDemoTypes , WhetherTxsClosureOffered (..) , demoLeiosFetchStaticEnv , emptyLeiosOutstanding + , encodeLeiosEbSize , focusElectionIfUnfocused , hashLeiosEb , hashLeiosTx @@ -418,7 +419,10 @@ serveStoredBody n = do db <- newLeiosDBInMemory let body = MkLeiosEb $ V.generate n $ \i -> (hashLeiosTx (MkLeiosTx (BS8.pack (show i))), 100) bodyPoint = MkLeiosPoint (SlotNo 0) (hashLeiosEb body) - withWriter db $ \w -> void . await =<< writeEbBody w bodyPoint body [] + -- The point first: a body can only be written for a registered point. + withWriter db $ \w -> do + void . await =<< writeEbPoint w bodyPoint (encodeLeiosEbSize body) + void . await =<< writeEbBody w bodyPoint body [] withReader db $ \r -> do ctx <- newLeiosFetchContext r msgLeiosBlockRequest nullTracer ctx bodyPoint