feat: report where the user is typing, so a client can keep it on screen

[harrison001]

Description

On a phone the on-screen keyboard covers half the picture, and the field you're typing into is usually in the covered half. The client has no way to know which half matters. The host does.

GET /caret answers in fractions of the streamed display, so the client doesn't need the host's resolution:

{"x":0.42,"y":0.71,"w":0.01,"h":0.02,"source":"caret"}
{"x":0.42,"y":0.71,"w":0,"h":0,"source":"pointer"}
{}

source says which it is. caret is the focused element's insertion point via Accessibility, macOS only. pointer is the cursor position, used when the focused application reports no caret, which most don't; that half works on Windows, Linux/X11 and macOS. {} means neither was available and the client should leave the picture where it is.

HTTPS only, so it sits behind the same client-certificate check as applist and launch.

platf::pointer_location() is new in platform/common.h. I didn't reuse get_mouse_loc(): it's documented as existing only for tests, and it takes the input backend of a running stream, which an HTTP handler doesn't have.

Three things on the macOS side that aren't obvious from the docs:

• AXUIElementCreateSystemWide() with kAXFocusedUIElementAttribute returns nothing for applications that answer when asked directly, iTerm2 among them. It has to ask the application.
• Accessibility uses whole-desktop coordinates, so a display above the main one gives negative values. A caret on a display that isn't being streamed is dropped.
• An element with no insertion point returns an empty rect at the origin rather than an error, and taking that at face value puts the caret in the top-left corner.

Nothing upstream calls this yet. The consumer is TracePort, an open-source Moonlight fork of mine that this was built for. The endpoint is additive and does nothing if no client asks.

Tested on macOS 26.6.1, M1 Max, streaming to that client:

• iTerm2 and Xcode report a caret. Brave, VS Code, Sublime Text and Telegram don't, and fall back to the pointer. 528 requests over one three-minute session, all past client-certificate verification.
• Pointer on a second display returns {}. On the centre of the streamed display it returns exactly {"x":0.500,"y":0.500}.

The Windows and Linux implementations of pointer_location() are compile-tested only. CI builds both, but I have neither platform to run them on.

Screenshot

Issues Fixed or Closed

Roadmap Issues

Type of Change

• feat: New feature (non-breaking change which adds functionality)
• fix: Bug fix (non-breaking change which fixes an issue)
• docs: Documentation only changes
• style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
• refactor: Code change that neither fixes a bug nor adds a feature
• perf: Code change that improves performance
• test: Adding missing tests or correcting existing tests
• build: Changes that affect the build system or external dependencies
• ci: Changes to CI configuration files and scripts
• chore: Other changes that don't modify src or test files
• revert: Reverts a previous commit
• BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

• Code follows the style guidelines of this project
• Code has been self-reviewed
• Code has been commented, particularly in hard-to-understand areas
• Code docstring/documentation-blocks for new or existing methods/components have been added or updated
• Unit tests have been added or updated for any new or modified functionality

AI Usage

See our AI usage policy.

• None: No AI tools were used in creating this PR
• Light: AI provided minor assistance (formatting, simple suggestions)
• Moderate: AI helped with code generation or debugging specific parts
• Heavy: AI generated most or all of the code changes

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud