fix: defer retain completion outbox until store commit - #4203
Merged
nicoloboschi merged 2 commits intoSep 8, 2026
Conversation
koriyoshi2041
approved these changes
Sep 8, 2026
koriyoshi2041
left a comment
Contributor
There was a problem hiding this comment.
The store/SQL boundary looks sound at d760a00: reached callbacks are deferred only for store-owned sessions, the store commit happens before the fresh SQL outbox transaction, and retain or commit failures cannot publish a success event. The factory path also preserves per-document callback contents, while unchanged retains count the committed document total rather than newly created IDs.
The focused matrix covers direct/factory callbacks, previous/committed counts, and both failure points. I also checked that the session path does not sub-batch, so the deferred list does not introduce duplicate per-document completion events. Approved.
Queue reached callbacks while a retain session buffers memories, then publish document counts after its successful commit. Cover document factories, unchanged and zero-fact retains, and commit failures.
…write fails The deferred completion outbox now runs after the store session has committed, so a failure there cannot be undone by failing the retain — it would only report a stored document as lost and invite a duplicate re-submit. Log the dropped retain.completed event and let the retain report the truth.
nicoloboschi
force-pushed
the
fix/store-owned-retain-webhook
branch
from
September 8, 2026 15:26
ce02c54 to
7bfbacd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #4189.
A store-owned retain session can buffer its facts until
commit(). The completion outbox currently queries that store from inside the retain pipeline, before the session commits, so a successful retain can announcememory_unit_count: 0even though the document ends up with four memories.This change records the callbacks actually reached by the pipeline and runs them after successful session commit, in a fresh SQL outbox transaction. It preserves document-specific callback factories and keeps counts based on the document's stored total, not the IDs created by this retain. The SQL path and stores without a retain session keep their existing callback timing.
The SQL outbox and an external store are still separate transactional systems; this does not introduce cross-store atomicity. A failed retain/session commit does not publish a success event.
Tests
FactRecordcorrectly, preserves document fields and allocated IDs, and honors the document filter used by the count assertion.pytest tests/test_retain_outbox_session.py tests/test_webhooks.py tests/test_memories_extension.py— 238 passed, 67 skipped. Existing local-ML-dependent tests skip in the slim environment; the new count regressions run with deterministic embeddings and a real isolated pg0 database.0 != 4and unchanged-retain counts[4, 0].git diff --checkpassed.Local validation limits
The complete API typecheck could not pass in the slim environment because optional ML/provider imports are missing. The all-workspace lint shell script could not start from the CRLF Windows checkout; the Python checks above were run directly. The full repository suite and full free-threaded API environment were not validated locally. A separate source-extracted lifecycle smoke check passed 96 threaded event-loop scenarios on Python 3.14t with the GIL disabled, but it is not a substitute for the free-threaded CI job.