Skip to content

fix(rootfs/qemu-static): use direct armhf exec probe instead of broken arch-test gate#117

Open
iav wants to merge 1 commit into
mainfrom
fix/native-armhf-detection
Open

fix(rootfs/qemu-static): use direct armhf exec probe instead of broken arch-test gate#117
iav wants to merge 1 commit into
mainfrom
fix/native-armhf-detection

Conversation

@iav
Copy link
Copy Markdown
Owner

@iav iav commented May 14, 2026

Problem

prepare_host_binfmt_qemu_cross_arm64_host_armhf_target (added in 6755e9190, 2024-12-28, for Apple Silicon) gates on arch-test arm. On success → skip qemu-arm import; on failure → write a hand-rolled /usr/share/binfmts/qemu-arm magic with interpreter /usr/bin/qemu-arm-static and enable it.

This is broken on most modern aarch64 hosts:

  • arch-test arm probes ARMv5 EABI. Current toolchains and the kernel's COMPAT layer target armhf (EABI v5+); the two surfaces diverge.
  • On Ubuntu Noble 6.8 / Ampere Altra (Hetzner CAX), arch-test arm returns failure even with CONFIG_COMPAT=y and a fully-working COMPAT path. The gate then unconditionally routes every aarch64 host through the Apple-Silicon import branch — qemu-arm gets registered, mmdebstrap and chroot apt-get / dpkg --configure / customize_image run through qemu-user-static emulation, ~10× slower than native.
  • The unconditional update-binfmts --enable qemu-arm at function entry could re-enable a pre-existing registration even on hosts the operator deliberately wanted to keep without qemu.

Codex (#117 P1 round 1) also flagged: the hand-rolled descriptor overwrites a packaged /usr/share/binfmts/qemu-arm (resolute's qemu-user-binfmt ships /usr/bin/qemu-arm without -static) → enable fails → armhf builds break on resolute Apple-Silicon-like hosts.

Empirically: helios4 BUILD_MINIMAL=yes RELEASE=trixie PREFER_DOCKER=yes finishes in 1:07 min on Hetzner CAX21 when qemu-arm stays unregistered (mmdebstrap native via binfmt_elf), vs ~10× longer when emulation routes mmdebstrap.

Fix

Replace the single arch-test arm gate with a 4-tier detection that picks the safest path on each host:

  1. /usr/share/binfmts/qemu-arm exists (packaged by qemu-user-binfmt on resolute, or by qemu-user-static install): update-binfmts --enable qemu-arm — loads the packaged descriptor into the kernel. Preserves the packaged interpreter path. Hard-error if enable fails (descriptor present but broken — host needs operator attention).
  2. /proc/sys/fs/binfmt_misc/qemu-arm exists without descriptor: trust if enabled, hard-error if disabled (no descriptor to re-enable from).
  3. No registration: probe with a direct exec /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 --help (ld-linux-armhf is shipped by gcc-arm-linux-gnueabihf, an armbian aarch64 host build dep). Authoritative test of CONFIG_COMPAT functionality. If succeeds → kernel runs armhf natively → no qemu setup needed.
  4. Native probe fails (Apple Silicon / no COMPAT): hand-roll /usr/share/binfmts/qemu-arm magic with /usr/bin/qemu-arm-static, import, enable — the original behaviour, now only reached when actually needed.

Also removed the unconditional update-binfmts --enable qemu-arm "|| true" at function entry — its job is subsumed by step (1).

Empirical validation

  • Native aarch64 (CAX21 / Ampere Altra, Ubuntu Noble 6.8): probe succeeds, no qemu-arm registered, helios4 trixie minimal builds 0:57 (PREFER_DOCKER=no) / 1:07 (PREFER_DOCKER=yes), mmdebstrap native.
  • No-CONFIG_COMPAT kernel (custom helios64 build with opts_n+=("COMPAT" "COMPAT_VDSO" "ARM64_32BIT_EL0")): ld-linux-armhf.so.3 execs with Exec format error; probe correctly fails → step (4) fires.
  • Pre-existing qemu-arm registration (manually added /usr/share/binfmts/qemu-arm with /usr/bin/qemu-arm-static): step (1) early-returns, descriptor md5 unchanged before/after, kernel entry left enabled.

Test plan

  • Hetzner CAX21 Noble 6.8 — native aarch64 with COMPAT, mmdebstrap native, helios4 image OK.
  • helios64 no-COMPAT kernel — ld-linux-armhf exec → "Exec format error", step (4) reachable.
  • Manual qemu-arm registration → step (1) preserves descriptor + enabled state, no clobber.
  • Resolute aarch64 with qemu-user-binfmt package — covered by inspection (step 1 enables packaged descriptor). No resolute Apple Silicon hardware available; happy to defer to community verification.

Assisted-by: Claude:claude-opus-4.7

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a8209d26d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/functions/rootfs/qemu-static.sh
@iav iav force-pushed the fix/native-armhf-detection branch from 4a8209d to 934a2f0 Compare May 14, 2026 02:02
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 934a2f0769

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/functions/rootfs/qemu-static.sh
Comment thread lib/functions/rootfs/qemu-static.sh Outdated
@iav iav force-pushed the fix/native-armhf-detection branch from 934a2f0 to d5b76d4 Compare May 14, 2026 02:15
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5b76d4f23

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/functions/rootfs/qemu-static.sh
@iav iav force-pushed the fix/native-armhf-detection branch from d5b76d4 to 4be6288 Compare May 14, 2026 02:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant