Skip to content

fix(desktop): pause screen capture while another app is sharing the screen#10204

Merged
undivisible merged 10 commits into
mainfrom
fix/10143-pause-capture-during-share
Jul 21, 2026
Merged

fix(desktop): pause screen capture while another app is sharing the screen#10204
undivisible merged 10 commits into
mainfrom
fix/10143-pause-capture-during-share

Conversation

@skanderkaroui

@skanderkaroui skanderkaroui commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Fixes #10143 — an active screen share in a call (Zoom/Teams/Meet) stops at the moment Omi takes its periodic Rewind screenshot.

Root cause

The Rewind capture path issues one-shot SCScreenshotManager.captureImage requests every ~1s. When another app is streaming the screen (an outgoing call share), the concurrent ScreenCaptureKit request contends in WindowServer capture arbitration; this is the same cause family as the CleanShot/Shottr 20–60s stall fixed by #6819, but the observed outcome — the user's share terminating — is worse. Apple documents no contract for concurrent capture (checked SCScreenshotManager/SCStream docs), so the mitigation is to not capture at all while a share is active.

Fix

  • ConferencingApps.isShareIndicatorWindow(ownerName:title:) + activeScreenSharePresent(): detect the share-only chrome windows conferencing apps create while the user is presenting (Zoom "zoom share statusbar/toolbar window" floating controls, Teams "Screen sharing toolbar", the browser "<site> is sharing your screen/a tab/a window" stop-sharing bubble — this covers Google Meet and Teams web). Adding another app is a 2-line signature + test.
  • New ProactiveExternalCaptureYield (Core): one seam that runs both "yield to an external capture" gates — the existing screenshot-app gate (desktop: stop CleanShot/Shottr/macOS screenshots from stalling 20-60s #6819) and the new share gate — each a ProactiveScreenshotCaptureGate with its own 10s post-condition backoff. The share check is an autoclosure so its CGWindowList enumeration is skipped when the screenshot gate already pauses the tick. Extracted (rather than grown in place) because ProactiveAssistantsPlugin.swift is frozen by the product file line-count ratchet; the plugin now makes a single shouldYield call.
  • Declares a new failure class, FC-concurrent-capture-contention (definition added in this PR, as Failure-Class: new requires): second instance of this cause after desktop: stop CleanShot/Shottr/macOS screenshots from stalling 20-60s #6819; the reusable guard surface is the shared gate state machine + the new yield seam.

Failure-Class: new

Why per-app window signatures instead of a native "screen is being shared" API

macOS has no public API that reports another app capturing/sharing the screen. UIScreen.isCaptured is iOS/Catalyst-only; the only macOS equivalent is the private SkyLight call CGSIsScreenWatcherPresent(). Even if we accepted a private API, any generic "screen watcher present" signal is self-confounded: Omi itself is a screen watcher every ~1s, so the signal would read true because of our own capture, not the call app's share.

Follow-up worth validating live (needs a GUI session): whether one-shot SCScreenshotManager captures do not light the macOS 15 system capture indicator while persistent SCStream shares do. If that holds, enumerating the system indicator window becomes an app-agnostic signal covering Discord/Slack/any social app — qualified by a persistence threshold (N consecutive ticks) so a short-lived capture (a screenshot tool, a one-shot grab) is never misread as a video share. The ProactiveExternalCaptureYield seam is where that signal would plug in.

Carried CI repair

4a8244742d (#10181) removed floating-bar typing but left source-inspection tripwires anchored on the deleted composer (AgentPillLifecycleTests, FloatingBarTimingSignalTests), so the desktop Swift test target fails on every merge with main. This branch carries the tripwire re-anchor commit; the same repair rides in #10210, and #10211 carries an equivalent one — whichever merges first repairs main and the later copies merge as no-ops.

The branch previously also carried a beta production-identity restore for INV-DATA-1; main has since been repaired canonically (beta is a channel on the production bundle id, #10219) and the updated production-data-plane-routing guard rejects the divergent identity, so that commit is reverted here — the branch no longer touches AppBuild/DesktopBackendEnvironment/OmiApp.

(The single Failure-Class declaration above stays new for this PR's own FC-concurrent-capture-contention definition.)

Verification

  • xcrun swift build -c debug --package-path Desktop — clean build.
  • xcrun swift test --filter "ConferencingAppsTests|ProactiveAssistantOrchestrationPolicyTests" — 23/23 pass: new share-indicator classification tests (positive Zoom/Teams/browser signatures; negatives for ordinary call windows, a Teams chat titled about screen sharing, non-conferencing owners) and new ProactiveExternalCaptureYield behavioral tests (pause across a share then backoff then resume; screenshot gate short-circuits the share check; reset clears both gates).
  • Full desktop Swift suite (scripts/swift-test-suites.sh, 344 suites): same result as pristine origin/main on this machine — 3 suites fail identically on both (AgentPillLifecycleTests, FloatingBarTimingSignalTests, ScreenRecordingPermissionPolicyTests; headless-environment failures, no GUI session), i.e. no regression from this change.
  • make preflight — 19/19 checks pass.
  • Live verification (named bundle + real Google Meet share): built this branch as a named test bundle and ran a real Meet screen share against it. Log shows Active screen share detected — pausing capture until the share ends at share start, zero capture activity for the full ~2.5-minute share, Screen share ended on stop, and the encoder/indexer resuming frames ~9s later. The Meet share survived the entire session — the Desktop: active screen share in a call stops when Omi captures a screenshot #10143 failure mode did not reproduce.

Review in cubic

Skander Karoui added 3 commits July 21, 2026 10:41
Add ConferencingApps.isShareIndicatorWindow / activeScreenSharePresent:
recognize the chrome windows conferencing apps create only while the
user is presenting (Zoom 'zoom share statusbar/toolbar window' floating
controls, Teams 'Screen sharing toolbar', the browser
'<site> is sharing your screen/a tab/a window' stop-sharing bubble for
Meet/Teams web). Pure classification is unit-tested with positive and
negative signatures (ordinary call windows, a Teams chat titled about
screen sharing, and non-conferencing owners must not match).
…creen

An active Zoom/Teams/Meet screen share stopped at the moment Omi took its
periodic Rewind screenshot: the one-shot SCScreenshotManager.captureImage
request contends with the share stream in WindowServer capture arbitration
(same cause family as the CleanShot stall fixed in #6819, worse outcome).

Pause the capture loop while a share-indicator window is present, plus a
10s backoff after the share ends. Both external-capture yield gates
(screenshot app from #6819 + the new screen-share gate) now live in
ProactiveExternalCaptureYield, extracted from the ratchet-frozen plugin
file; the plugin makes a single shouldYield call and the ratchet baseline
ratchets down (1641 -> 1628). The share check is an autoclosure so its
CGWindowList enumeration is skipped when the screenshot gate already
pauses the tick.

Verification: xcrun swift build clean; swift test --filter
'ConferencingAppsTests|ProactiveAssistantOrchestrationPolicyTests' 23/23
pass (share-indicator signature tests + yield-gate behavioral tests:
pause across a share then backoff then resume, screenshot gate
short-circuits the share check, reset clears both gates). Live share
repro NOT yet exercised - no GUI session on this machine; live
validation plan documented in the PR body.

Failure-Class: new

Fixes #10143
Two fixes share this cause: #6819 (CleanShot/Shottr 20-60s stall while
Omi's capture loop contends in WindowServer) and the #10143 fix in this
PR (active call screen share stopping when Omi captures). Register the
class so future concurrent-capture fixes declare it and reuse the
ProactiveScreenshotCaptureGate / ProactiveExternalCaptureYield guard
surface instead of adding one-off pauses.

@undivisible undivisible left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes

Share-yield seam is the right fix for #10143 (ProactiveExternalCaptureYield + Zoom/Teams/browser signatures), but this can’t land on a red Desktop Swift gate.

Please

  1. Get Desktop Swift Test Suites green (or prove AgentPillLifecycleTests / FloatingBarTimingSignalTests fail identically on pristine main in the same runner and file the flake separately).
  2. One live GUI share smoke (Zoom or Meet): confirm the “Active screen share detected — pausing capture” log and that the share survives Omi’s capture loop.
  3. Optional follow-up: browser bubble match is English-only — note locale gap or add a second signal later.

…oval

4a82447 removed typing from the floating bar (pill steering now
routes to the main app via Continue in Omi) but updated none of the
source-inspection tripwires anchored on the deleted composer, so
Desktop Swift CI is red on main and the fail-closed release candidate
gate is blocked. Re-encode the new contract:

- testProviderRequestsStayInTheModelLoop: the in-model-loop anchor is
  now the Continue-in-Omi route to openMainAppWindow(); the no-local-
  parsing negatives are unchanged.
- testSubagentComposerOnlyContinuesItsCanonicalSession →
  testSubagentFollowUpsOnlyContinueTheCanonicalSession: anchors on the
  manager's canonical-session guard and
  DesktopCoordinatorService.continueAgent, since the view composer no
  longer exists.
- testTypedSendDelegatesResponseSizingToWindow: archiving moved into
  the window with sizing; assert the window archives via
  historyChatProvider and the view no longer archives at all.
- FloatingBarTimingSignalTests.testFollowUpFocusUsesViewLifecycle:
  deleted — the follow-up field it guarded no longer exists; the
  asyncAfter baseline test still guards the timing-signal class.

Verification: xcrun swift test --filter
"AgentPillLifecycleTests|FloatingBarTimingSignalTests" — 77 tests,
0 failures (was 4 failures, matching the red Desktop Swift CI run on
main at 928b396); scripts/check_desktop_test_quality.py at/below
baseline; agent-logic-harness filter references only class names, and
the class still exists.
…artifact refactor

dba3af2 (restore single macOS artifact) removed the beta production
identity from AppBuild while #10216 was in flight; #10216 then landed
INV-DATA-1 plus guards that pin com.omi.computer-macos.beta to the
production customer data plane. Their merge left main red and, worse,
routed installed Omi Beta apps to the development backend.

Restore the boundary the locked invariant demands:

- AppBuild: betaProductionBundleIdentifier + productionFamilyBundleIdentifiers
  are back; isNonProduction / isProductionBundle key off the family set.
- DesktopBackendEnvironment: production-family bundles (not just stable)
  fall through to production routing, so installed beta apps keep their
  customer data plane.
- Re-restore the stable-only legacy Omi Computer.app cleanup gate from
  d5a8ba9 (also dropped by dba3af2): with beta production-family
  again, an ungated cleanup from a beta install would force-terminate the
  running stable app.
- AppBuildProductionFamilyTests covers the family set and the cleanup gate.

The retirement of the separately packaged beta artifact (channel pointer
over immutable Omi.zip/omi.dmg) is untouched: this restores identity and
routing semantics for already-installed beta bundles only.
skanderkaroui added a commit that referenced this pull request Jul 21, 2026
…ed composer

testFollowUpFocusUsesViewLifecycle pinned the floating-bar follow-up composer
that #10181 deleted, so it fails on every merge with main. Same deletion as the
repair carried by #10210/#10204.

Verified: xcrun swift test --filter "FloatingBarTimingSignalTests|AgentPillLifecycleTests" — 77/77 pass.
@undivisible
undivisible dismissed their stale review July 21, 2026 23:50

All 3 review asks addressed: CI green, live Meet share smoke verified, English-only browser bubble documented as known limitation. Approving.

@undivisible undivisible left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve

Code review passed. Share-yield seam is clean: correctly gates both screenshot-app and screen-share capture with autoclosure short-circuit. 23/23 tests pass, CI green, live Meet share smoke verified. English-only browser bubble match is correct trade-off (window titles are OS-level identifiers, not UI text). No i18n needed — no user-facing strings in this PR.

@undivisible
undivisible merged commit ce98bb2 into main Jul 21, 2026
35 of 38 checks passed
@undivisible
undivisible deleted the fix/10143-pause-capture-during-share branch July 21, 2026 23:51
skanderkaroui added a commit that referenced this pull request Jul 22, 2026
#10210)

## What

Onboarding integration icons (the "Use Omi memory where you work" step,
and the matching home stack — same `ConnectorBrandIcon`) didn't show up
for most users: Notion, ChatGPT, and Claude marks resolved **only** from
a hardcoded `/Applications/<App>.app` path, so any user without the app
installed at exactly that path saw a generic gray SF Symbol tile.

- **Bundled brand marks for every onboarding row.** Ships
`notion_logo.png`, `chatgpt_logo.png`, `claude_logo.png` (Simple Icons
brand marks, rasterized at 512px with alpha and recolored for the dark
tile: white Notion/OpenAI, Claude coral `#D97757`). Codex reuses the
ChatGPT mark and Claude Code the Claude mark, mirroring the existing
app-icon reuse. Every destination the onboarding introduces (Notion,
Obsidian, Claude / Claude Code, ChatGPT / Codex, OpenClaw, Hermes, plus
Gemini behind "More") now has a real logo with zero apps installed.
- **Installed apps found by bundle identifier.**
`installedApplicationURL` now asks LaunchServices
(`urlForApplication(withBundleIdentifier:)`) first — finds apps in
`~/Applications`, Setapp, or renamed copies — keeping the fixed
`/Applications` path as a fallback for unregistered copies.
- **Sturdier resource-bundle probe.** The SwiftPM
`"<pkg>_<target>.bundle"` scan now checks nested bundle locations under
every candidate bundle (not just `Bundle.main.resourceURL`), so the
bundled-logo fallback also resolves outside the packaged-app layout —
this is what lets the new regression test load the real resources
hermetically.

## Verification

- `xcrun swift build -c debug --package-path Desktop` — clean.
- New `ConnectorBrandIconResourceTests` passes (`xcrun swift test
--filter ConnectorBrandIconResourceTests`): every onboarding brand
resolves a loadable bundled logo. The build machine has none of
Notion/Obsidian/ChatGPT/Claude installed, so this exercises exactly the
previously-broken "no apps installed" scenario end-to-end through the
production loader.
- Rendered-logo visual check: composited all three new PNGs on the dark
tile background and inspected — crisp white OpenAI knot, coral Claude
spark, white Notion mark.
- `make preflight` green.

## Carried CI repair

`4a8244742d` (#10181) removed floating-bar typing but left
source-inspection tripwires anchored on the deleted composer
(AgentPillLifecycleTests, FloatingBarTimingSignalTests), so the desktop
Swift test target fails on every merge with main. This branch carries
the tripwire re-anchor commit; the same repair rides in #10204, and
#10211 carries an equivalent one — whichever merges first repairs main
and the later copies merge as no-ops.

The branch previously also carried a beta production-identity restore
for INV-DATA-1; main has since been repaired canonically (beta is a
channel on the production bundle id, #10219) and the updated
`production-data-plane-routing` guard rejects the divergent identity, so
that commit is reverted here — the branch no longer touches
AppBuild/DesktopBackendEnvironment/OmiApp.

## Product invariants

- INV-INT-1 — touched `ConnectorBrandIcon.swift` (matches
`**/*Connector*`). Icon rendering only: no connector contract, probe, or
"Connected" semantics change. The bundle identifiers added are stable
app identity (same class of fact as the pre-existing hardcoded install
paths), not volatile provider knowledge.

## Failure class (fixes)

Failure-Class: none

(The icon fix introduces no failure class; the previously declared
FC-customer-data-plane-divergence rode with the carried data-plane
repair, which is now reverted in favor of main's canonical fix from
#10219.)


<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/BasedHardware/omi/pull/10210?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Desktop: active screen share in a call stops when Omi captures a screenshot

2 participants