Skip to content

detect resume from sleep without logind (#2360, phase 3) - #2537

Merged
m3nu merged 2 commits into
borgbase:masterfrom
ebuzerdrmz44:refactor/scheduler-resume-detection
Aug 11, 2026
Merged

detect resume from sleep without logind (#2360, phase 3)#2537
m3nu merged 2 commits into
borgbase:masterfrom
ebuzerdrmz44:refactor/scheduler-resume-detection

Conversation

@ebuzerdrmz44

Copy link
Copy Markdown
Contributor

Description

Phase C of #2360, first of two. Fixes one of the four reliability defects the issue lists: "DBus sleep/resume detection fails silently."

At startup Vorta checks whether org.freedesktop.login1 is registered. If it is not, it logs a warning and gives up permanently. So without logind there has never been any resume detection, and after a wake every profile's timer stays stale until the 15-minute poll comes around.

This adds a detector that does not need logind, making the DBus signal a fast path rather than the only path.

What's in it

  • wake_timer ticks every 60s and only compares two timestamps. QTimer does not fire while suspended, so a gap far larger than one tick means the machine was frozen and the schedules are stale.
  • It measures wall clock, not time.monotonic(). man 2 clock_gettime: CLOCK_BOOTTIME is "identical to CLOCK_MONOTONIC, except that it also includes any time that the system is suspended". A monotonic detector would see a 60s gap after an 8-hour suspend and never fire. The cost is that an NTP step or DST shift can trip it, which buys one redundant reload_all_timers(), exactly what the poll already does unconditionally.
  • loginSuspendNotify and checkForResume now share _handle_resume(), previously inline in the former. It resets the detector's baseline so a logind resume does not also trip the timer on the next tick.
  • The logind failure log drops from warning to info, since it is no longer a failure.
  • Tests cover both branches of the detector and of the logind slot, through the existing clockmock fixture and the real timeout signal. loginSuspendNotify had no coverage before.

Independent of #2530 and #2532: branches off master.

Need help in these decisions

1. Is 60s the right tick? Each tick is two datetime subtractions, so the cost is the wakeup, not compute. 5 minutes still beats today by 3x, and CLOCK_BOOTTIME would allow a much lazier check at the price of a Linux-only branch plus a macOS fallback. One-line change either way.

2. Where does this land after Phase B? I expect wake_timer, checkForResume and _handle_resume to go to the State component rather than Scheduling, since none of it computes a next run. Confirm and I will carry them there in PR4.

@ebuzerdrmz44 ebuzerdrmz44 changed the title Detect resume from sleep by watching for wall clock gaps, so scheduli… detect resume from sleep without logind (#2360, phase 3) Aug 8, 2026
@ebuzerdrmz44

ebuzerdrmz44 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

@m3nu Map of where this sits, since #2360 is now four PRs wide and this one jumps ahead of Phase B.

Phase C does not depend on Phase A or B, so it does not have to wait in line. What does gate everything else: fork PRs cannot be stacked, so PR4 onwards each have to branch off a merged master.

Phase PR What State
A Persistence #2530 JobModel + skip reasons at all 3 scheduler exit points open, all review points addressed
A Persistence #2532 Pause state persisted and restored on restart open, no review yet
B Split PR4 Extract State: pause/status/startup crash sweep needs #2532 merged
B Split PR5 Extract Scheduling: set_timer_for_profile, reload_all_timers, next_job* needs PR4
B Split PR6 Extract Execution, sequential post-backup tasks, intent to result link, atomic check to submit needs PR5
C Reliability this PR Resume detection without logind
C Reliability next QTimer ~24.8 day overflow needs this
D Jobs View PR9-11 JobsModel + Jobs page, filters, cancel/re-queue needs questions below

Still open from earlier rounds, no rush on any of them:

  1. db_lock (asked on add JobModel: persistent scheduler job store (#2360, phase 1) #2530):
  2. Jobs view placement (issue open question 1): replace the Log page, or a separate tab? Shapes PR9.
  3. Filtering priority (issue open question 2): which of profile, repo, type, status matters first? Shapes PR10. Worth knowing that a type filter needs PR6 first, since JobModel.Type only has BACKUP until Execution is split.

One note from re-reading the issue against what is built, not needing a reply unless you disagree:

Goal 1's "Pending/scheduled jobs (survive restarts)" and your "don't materialise scheduled rows" are compatible, as I read it. set_timer_for_profile recomputes every profile's next run at startup, so pending jobs do survive a restart, derived rather than stored. The consequence for later: PR9 has to merge the in-memory timers with JobModel rows to give Goal 3 its "pending and completed in one view".

@ebuzerdrmz44
ebuzerdrmz44 force-pushed the refactor/scheduler-resume-detection branch from f03937a to 99c5a61 Compare August 11, 2026 16:27

@m3nu m3nu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase looks clean — re-anchored onto #2530 with the content unchanged, CI green. Reviewed against the merged master.

The core reasoning is right, and it's the part I checked hardest. CLOCK_MONOTONIC doesn't advance across suspend, so a monotonic detector would see one tick's worth of gap after an 8-hour sleep and never fire; wall clock is the only signal available without a Linux-specific CLOCK_BOOTTIME branch plus a macOS fallback. The false-positive sources you accept — NTP step, DST shift, an event loop blocked past the threshold — each cost exactly one redundant reload_all_timers(), which the periodic poll already does unconditionally. That's the correct trade.

Also good: _handle_resume resetting _last_wake_check so a logind resume doesn't leave the detector primed to fire again on the next tick, and starting wake_timer unconditionally rather than only when logind is missing. Keep it unconditional — PrepareForSleep doesn't fire for a hypervisor pause and doesn't exist on macOS at all, so the timer is a genuine backstop rather than a fallback. loginSuspendNotify having tests for the first time is a nice bonus.

Two things before I approve, both small.

1. Make the tick 5 minutes, threshold 10. Answering your question 1: the cost isn't the two datetime subtractions, it's that a 60s timer wakes the process 1440 times a day and defeats tickless idle on Linux and App Nap on macOS. What the latency buys is how soon after resume a missed backup gets picked up — and for a backup app, 5 minutes versus 1 minute is not a difference anyone notices, while 15 minutes versus 5 is. So take the 3x improvement at a fifth of the wakeups. Your existing test gaps (2 hours and 1 minute) still land on the right side of a 10-minute threshold, so this should be constants-only.

2. Stop wake_timer for every scheduler the module builds, not just these three. You spotted the hazard — the comment about qapp keeping every scheduler alive is exactly right — but the fix is applied per-test, and the other tests in this file that construct a VortaScheduler still leak one with a 2020 baseline. Once clockmock is torn down, such a timer computes a six-year gap and calls reload_all_timers() on a stale scheduler in the middle of an unrelated test. The pre-existing qt_timer has the same leak and has never bitten because 15 minutes outlives the suite; at 60s (or even 5 min) that stops being true. An autouse fixture in test_scheduler.py that stops wake_timer on construction would cover it, and Phase B is about to add a lot more scheduler tests to this file.

Answering your question 2: yes, wake_timer, checkForResume and _handle_resume go to State, not Scheduling — none of them computes a next run, they observe machine state and poke the scheduler. Carry them there in PR4.

Tiny nit, take or leave: the new log line reads "watching for clock gaps instead", but the detector runs whether or not logind is present. "as well" would be more accurate.

Push those two and I'll approve. Note that #2532 and this one overlap in VortaScheduler.__init__ — you're ahead of it right now, so if this lands first, #2532 picks up the manual rebase there rather than the other way round.

@m3nu m3nu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three in, approving.

5 min / 10 min threshold, and the fixture. The fixture is a better shape than what I suggested — patching __init__ catches every scheduler the module builds rather than relying on each test to remember, and stopping qapp.scheduler's timer explicitly closes the one instance the patch can't reach.

Take-or-leave, don't hold the PR for it: the fixture only applies to test_scheduler.py, so qapp.scheduler's timer is still live during earlier files — test_schedule.py runs first alphabetically and has its own clockmock, which is the same 2020-baseline setup. At a 5-minute interval the odds of a tick landing inside that file are low enough that I'm not worried, but tests/unit/conftest.py is the natural home for it whenever you next touch that file.

@m3nu
m3nu merged commit afd36f9 into borgbase:master Aug 11, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants