perf(ama): drain durable operations inline, relax cron cadence - #299
Merged
Conversation
The */5 AMA work cron kept the Neon endpoint awake around the clock: each run issues an unconditional hold-release UPDATE plus a claimDue query, resetting the 5-minute autosuspend timer every cycle (~183 CU-hrs/month of always-on compute). Mutating endpoints now kick a background drain of the durable operations queue via waitUntil in the same invocation that enqueued the work, so booking emails, Finalizing Booking recovery, refunds, and admin-triggered retries start immediately instead of waiting for the next sweep. The lease-based runner already tolerates concurrent drains, so the kick and the scheduled sweep cannot double-execute. With request-triggered work handled inline, the cron becomes a pure clock fallback (reminders, retry backoff, expired Slot Hold bookkeeping) and drops to */30, with media reconcile hourly and aligned so wakeups consolidate. Reminders and retries may now land up to 30 minutes later; everything guest-facing gets faster, and the database can suspend between wakeups.
Preview deployment ready
Branch: |
Address review feedback on the inline operations drain: - The Stripe webhook handler now surfaces the processed outcome via an onOutcome callback, and the route kicks the drain only for booking_created. Duplicate, ignored, orphaned, booking-exists, and hold-release deliveries return 200 without enqueueing work, so they no longer start a runner pass. - A single runner pass caps at its batch size, so older due work could crowd out the operation the triggering mutation just enqueued. The kick now re-runs the drain while full batches come back, bounded by a 30-second inline budget; leftovers stay with the scheduled sweep.
A single slow first pass (ten due operations with provider calls) could exhaust the 30-second outer budget before the drain reached the operation the triggering mutation enqueued, deferring it to the scheduled sweep. Raise the mutating routes to the platform-default 300s maxDuration (the explicit 60 had actually lowered it) and extend the drain budget to 240s, so 240s of passes plus one worst-case 45s pass still finish inside the ceiling. Passes are claim-ordered by nextAttemptAt, so each one moves the queue strictly toward the newest operation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Neon compute usage has been ~183 CU-hrs/month — the endpoint never autosuspends. The
*/5 * * * *AMA work cron issues an unconditional hold-releaseUPDATEplus aclaimDuequery on every run, resetting Neon's 5-minute autosuspend timer exactly at the threshold, 288 times a day. Page visits are not a factor (public routes are fully static under Cache Components and never touch the database).What changed
Inline drain on mutation. New
kickAmaOperations()inlib/ama/booking/server.tsrunsrunner.run()viawaitUntilin the background of the current invocation. It is called after a successful response in the five routes that enqueue durable work:Each of these routes now sets
maxDuration = 60so the runner's 45-second time budget always fits.Sparser crons. AMA work drops from
*/5to*/30; media reconcile drops from*/15to hourly on the hour, aligned so wakeups consolidate. The cron is now purely the clock fallback for reminders, retry backoff, and expired Slot Hold bookkeeping.Reviewer notes
claimDue+ lease token), so a concurrent inline drain and scheduled sweep cannot run the same operation twice. An interrupted drain's lease expires and the next run reclaims it.expiresAt > now.releaseExpiredHoldsis bookkeeping only.Verification
npx tsc --noEmitcleanpnpm test:ama— 668 tests + 9 migration tests passpnpm test:unit— 1245 tests passGreptile Summary
The PR starts AMA durable-operation drains from successful enqueueing mutations and reduces cron frequency to allow the database to autosuspend.
Confidence Score: 5/5
The PR appears safe to merge.
No blocking failure remains.
Important Files Changed
Sequence Diagram
sequenceDiagram participant Mutation as AMA mutation/webhook participant Queue as Durable operations participant Drain as Inline runner participant Provider as Email/Calendar/Stripe participant Cron as 30-minute cron Mutation->>Queue: Enqueue durable work Mutation->>Drain: kickAmaOperations via waitUntil loop While batch is full and budget remains Drain->>Queue: Claim due work under leases Drain->>Provider: Execute side effects Drain->>Queue: Complete or schedule retry end Cron->>Queue: Fallback sweepReviews (3): Last reviewed commit: "fix(ama): give the inline drain room to ..." | Re-trigger Greptile
Context used (3)