Skip to content

feat(macos): implement SCKit-based capture and prefer it to AVFoundation where available - #5511

Open
martona wants to merge 4 commits into
LizardByte:masterfrom
martona:feature/macos-sckit
Open

feat(macos): implement SCKit-based capture and prefer it to AVFoundation where available#5511
martona wants to merge 4 commits into
LizardByte:masterfrom
martona:feature/macos-sckit

Conversation

@martona

@martona martona commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Description

Replaces AVFoundation's AVCaptureScreenInput as the default macOS capture backend with a ScreenCaptureKit-based implementation (macOS 14+; AVFoundation remains as fallback for older systems or SCKit setup failures).

Capture runs through SCScreenshotManager polling paced to the session frame rate, rather than an SCStream.

Two reasons:

  • SCStream's update detection misses or delays small screen changes (e.g. a blinking terminal cursor), producing visible latency for keystroke echo.
  • Mixing SCStream frames with screenshot frames (streaming plus polling as a fallback) causes visible flicker on translucent surfaces such as toolbar materials, because the two paths composite slightly differently.

Polling a single consistent source resolves both, at capture latency measured equal to AVFoundation. Since polling goes through the zero-copy VideoToolbox path by default, there's no CPU cost to it.

Fixes

  • Cursor visibility: SCKit composites the cursor differently, fixing the long-standing AVFoundation bug where a cursor hidden by an application (e.g. while typing in a text field) never reappears in the stream.
  • Host processing latency reporting on macOS (first commit): capture timestamps are derived from sample buffer PTS and survive encoder pipelining via PTS-matched bookkeeping, so Moonlight's ctrl+alt+shift+S host latency stat works on macOS.

Testing

Tested on macOS 26 (Apple silicon, VM, Mac Studio and MBP): cursor reappearance, static-screen behavior, display reconfiguration, latency stat parity, extended interactive sessions at 1080p60/4K60 with hardware (VideoToolbox) and software (x264) encoders.

Screenshot

Issues Fixed or Closed

Closes #3433

Roadmap Issues

Type of Change

  • feat: New feature (non-breaking change which adds functionality)
  • fix: Bug fix (non-breaking change which fixes an issue)
  • docs: Documentation only changes
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
  • refactor: Code change that neither fixes a bug nor adds a feature
  • perf: Code change that improves performance
  • test: Adding missing tests or correcting existing tests
  • build: Changes that affect the build system or external dependencies
  • ci: Changes to CI configuration files and scripts
  • chore: Other changes that don't modify src or test files
  • revert: Reverts a previous commit
  • BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

  • Code follows the style guidelines of this project
  • Code has been self-reviewed
  • Code has been commented, particularly in hard-to-understand areas
  • Code docstring/documentation-blocks for new or existing methods/components have been added or updated
  • Unit tests have been added or updated for any new or modified functionality

AI Usage

See our AI usage policy.

  • None: No AI tools were used in creating this PR
  • Light: AI provided minor assistance (formatting, simple suggestions)
  • Moderate: AI helped with code generation or debugging specific parts
  • Heavy: AI generated most or all of the code changes

@martona

martona commented Aug 16, 2026

Copy link
Copy Markdown
Contributor Author

The remaining QualityGate issue is not mine; it's from 4 months ago. I can certainly fix it but I would rather keep this PR on topic.

@ReenigneArcher

Copy link
Copy Markdown
Member

Agree on the sonar issue. Could you fix the doxygen errors before I review this? https://app.readthedocs.org/projects/sunshinestream/builds/34086280/ You can expand the failed section and ctrl + F for error: to find them.

@martona

martona commented Aug 16, 2026

Copy link
Copy Markdown
Contributor Author

Done. Sorry I missed them last night.

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
1 New issue
1 New Code Smells (required ≤ 0)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

macOS 15: The cursor disappears and does not reappear

2 participants