Skip to content

Removed OSMesa, replaced with BUILD_BACKEND_HEADLESS_SOFTWARE - #252

Closed
kerberjg wants to merge 1 commit into
v2.0from
feature/backend-headless-software
Closed

Removed OSMesa, replaced with BUILD_BACKEND_HEADLESS_SOFTWARE#252
kerberjg wants to merge 1 commit into
v2.0from
feature/backend-headless-software

Conversation

@kerberjg

@kerberjg kerberjg commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

OSMesa is removed from Mesa 25.1+ and no longer a viable headless rendering path.

This PR replaces the headless EGL backend (which required libosmesa6, libgles2, and libegl1) with a zero-dependency headless mode built on the existing SoftwareBackend (Flutter kSoftware renderer, NoneSink by default).

New CMake flag: BUILD_BACKEND_HEADLESS_SOFTWARE

  • Mutually exclusive with all other backends (same as BUILD_BACKEND_SOFTWARE)
  • Forces BUILD_SOFTWARE_SINK_DRM, BUILD_SOFTWARE_SINK_FBDEV, and BUILD_SOFTWARE_INPUT_LIBINPUT OFF; no system library deps required
  • No OSMesa/EGL/Wayland linkage: IVI_HAVE_EGL and waypp include gates no longer include the headless case; waypp subproject is not built

SoftwareBackend gains a const bool is_headless constructor parameter:

  • Ctor: rejects hardware sinks (NativeSize() != nullopt) in headless mode, swapping them for NoneSink with a warning
  • GetVsyncCallback(): returns nullptr in headless mode (no vblank source)
  • SetCursor(): no-op in headless mode (no display to composite onto)
  • CreateSurface(): trace-logs the no-op in headless mode
  • New GetSink() accessor exposes the active sink for in-process pixel capture (used by the reworked getViewRenderBuf API)

flutter_view.cc constructs SoftwareBackend with is_headless=true under BUILD_BACKEND_HEADLESS_SOFTWARE, using MakeSinkFromEnv() so test fixtures can set IVI_SW_SINK=memory to enable pixel capture without recompiling.

App::getViewRenderBuf() is reimplemented via MemorySink::SnapshotLatest(), returning std::vector<uint8_t> with row_bytes/height out-params instead of a raw GLubyte* into an OSMesa framebuffer.

Also - all references to OSMesa are removed.

@kerberjg kerberjg self-assigned this Jun 22, 2026
@kerberjg
kerberjg force-pushed the feature/backend-headless-software branch 2 times, most recently from e4922f8 to 72022d6 Compare June 25, 2026 19:43
OSMesa is deprecated and no longer a viable headless rendering path.
Replace the headless EGL backend (which required libosmesa6, libgles2,
and libegl1) with a zero-dependency headless mode built on the existing
SoftwareBackend (Flutter kSoftware renderer, NoneSink by default).

New CMake flag: BUILD_BACKEND_HEADLESS_SOFTWARE
  - Mutually exclusive with all other backends (same as BUILD_BACKEND_SOFTWARE)
  - Forces BUILD_SOFTWARE_SINK_DRM, BUILD_SOFTWARE_SINK_FBDEV, and
    BUILD_SOFTWARE_INPUT_LIBINPUT OFF; no system library deps required
  - No OSMesa/EGL/Wayland linkage: IVI_HAVE_EGL and waypp include gates
    no longer include the headless case; waypp subproject is not built

SoftwareBackend gains a `const bool is_headless` constructor parameter:
  - Ctor: rejects hardware sinks (NativeSize() != nullopt) in headless
    mode, swapping them for NoneSink with a warning
  - GetVsyncCallback(): returns nullptr in headless mode (no vblank source)
  - SetCursor(): no-op in headless mode (no display to composite onto)
  - CreateSurface(): trace-logs the no-op in headless mode
  - New GetSink() accessor exposes the active sink for in-process pixel
    capture (used by the reworked getViewRenderBuf API)

flutter_view.cc constructs SoftwareBackend with is_headless=true under
BUILD_BACKEND_HEADLESS_SOFTWARE, using MakeSinkFromEnv() so test fixtures
can set IVI_SW_SINK=memory to enable pixel capture without recompiling.

App::getViewRenderBuf() is reimplemented via MemorySink::SnapshotLatest(),
returning std::vector<uint8_t> with row_bytes/height out-params instead of
a raw GLubyte* into an OSMesa framebuffer.
@kerberjg
kerberjg force-pushed the feature/backend-headless-software branch from 72022d6 to abf1a7a Compare June 25, 2026 23:06
@kerberjg kerberjg changed the title Removed OSMesa, replaced with BUILD_BACKEND_HEADLESS_SOFTWARE (WIP) Removed OSMesa, replaced with BUILD_BACKEND_HEADLESS_SOFTWARE Jun 25, 2026
@kerberjg
kerberjg marked this pull request as ready for review June 25, 2026 23:28
@kerberjg
kerberjg requested a review from jwinarske as a code owner June 25, 2026 23:28
@jwinarske

Copy link
Copy Markdown
Contributor

@kerberjg There seems to be some confusion.

e.g.

// In headless mode hardware sinks (DRM dumb / fbdev) are meaningless —
// there is no display to scan out to. If the caller passed one (e.g. via
// a stale IVI_SW_SINK env var), swap it for a NoneSink so the process
// never tries to open /dev/dri/* or /dev/fb*.

These are the software sinks as documented in shell/backend/software/README.md

Supported Software Sinks

Sink Spec syntax What it does Vsync
NoneSink none (default) Discards every frame. CI engine-only smoke.
MemorySink memory Mutex-guarded std::vector<uint8_t> snapshot of the most recent frame. SnapshotLatest(&row_bytes, &height) exposes it to in-process test fixtures.
FileSink file:<path-pattern> Writes each frame as a NetPBM PAM (P7) file. Pattern with %d / %05d interpolates the frame index. Pattern without % writes the first frame only. Parent directories auto-created.
FbDevSink fbdev[:<device>] Opens /dev/fb0 (or operator path), mmaps, packs each Present. Auto-detects 32-bpp BGRA/BGRX or 16-bpp RGB565 from the driver's FBIOGET_VSCREENINFO offsets; refuses palettized / nonstd != 0 / other layouts with a clear error.
DrmDumbSink drm-dumb[:<device>] Opens /dev/dri/card0, picks first connected connector + its CRTC + preferred mode, allocates 2 dumb buffers (XRGB8888 by default, RGB565 via IVI_SW_DRM_FORMAT=rgb565 when the plane advertises it), modesets onto buffer 0. Per-frame: pack into the back buffer, drmModePageFlip. PAGE_FLIP_EVENT drives Flutter's vsync_callback with the kernel-provided scanout timestamp.

Hold off on this PR until further notice. Major changes are incoming.

@kerberjg

kerberjg commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator Author

@jwinarske Yeah, that looks like something my Claude spat out. Thanks for the correction!

Please note this one is blocking for #239

@jwinarske

Copy link
Copy Markdown
Contributor

closing in favor of #254

@jwinarske jwinarske closed this Jun 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants