Skip to content

Port exa perf and bugfix patches onto NixOS/nix 2.36 - #1

Closed
ethancedwards8 wants to merge 8 commits into
masterfrom
exa/giga-3360-perf-fixes
Closed

Port exa perf and bugfix patches onto NixOS/nix 2.36#1
ethancedwards8 wants to merge 8 commits into
masterfrom
exa/giga-3360-perf-fixes

Conversation

@ethancedwards8

@ethancedwards8 ethancedwards8 commented Sep 7, 2026

Copy link
Copy Markdown
Member

Summary

Part 1 of 3 for GIGA-3360: re-implementation (not cherry-pick) of the exa/patched perf + bugfix series from exa-labs/nix-src against upstream 2.36.0 (36a6e9d6d). One commit per patch; each message links the original nix-src commit and the GIGA-2837 sub-issue.

  • max-http2-streams setting (required — runners set it to 1 and the daemon won't start without it)
  • nix eval fast path: serve cached flake string attrs straight from the eval cache (GIGA-2848)
  • Boehm GC free-space divisor defaults to 1 (~20% faster full-nixpkgs eval, +30% RSS) but now respects GC_FREE_SPACE_DIVISOR env override, which the original patch clobbered (GIGA-2849)
  • base16 / nix32 encoders write into presized buffers (GIGA-2839/2840)
  • Callback idempotent instead of assert-aborting on double invoke; HttpBinaryCacheStore::getFile try/catch split (Nix daemon crashes with assertion failure when building multiple mlibc cross-compiled packages NixOS/nix#13484, GIGA-2851)
  • Daemon: MonitorFdHup treats POLLERR|POLLNVAL as hangup; forked workers restore signal mask + start a handler thread (Fix MonitorFdHup 100% CPU spin and unresponsive daemon workers NixOS/nix#15691, GIGA-2838)
  • GC: catch Interrupted in the GC-roots client thread (was std::terminate-ing the daemon)
  • GC: autoGC(sync) no longer blocks store adds behind a running pass when above min-free

Dropped vs nix-src (per GIGA-2837 verdicts / already upstream): checkName LUT (GIGA-2853), printString scan-and-copy (GIGA-2852), nix run eval-cache fix (upstream now clears evalCaches before exec in run/develop/env/formatter), forward-aws-credentials + its revert.

Adaptations to 2.36: NixStringContextElem::Path variant is gone; getCursor takes AutoCall; the fast path only swallows Error, not Interrupted; Callback uses ignoreExceptionInDestructor() rather than a silent catch(...).

Test plan

  • Full meson build clean (aarch64-darwin, debugoptimized)
  • Unit suites: nix-util-tests, nix-store-tests, nix-expr-tests, nix-fetchers-tests, nix-flake-tests all pass
  • Functional: eval, eval-cache, flakes, gc-auto, gc-non-blocking, gc-concurrent, binary-cache, config, hash-*, eval-store (main + ca) pass
  • nix config show shows max-http2-streams = 100; --option max-http2-streams 1 accepted
  • nix eval .#attr on a flake: 2nd run logs using cached string attribute; --raw/--json match; --apply bypasses the fast path
  • GC_FREE_SPACE_DIVISOR=3 nix eval --expr 1+1 works
  • CI on x86_64-linux / aarch64-linux via exa-build.yml (lands in the plumbing PR)
  • Re-run nix-perf-benchmarks against this base (follow-up)

Generated with Devin

ethancedwards8 and others added 8 commits September 7, 2026 19:07
@ethancedwards8 @devin-ai-integration
libstore: add max-http2-streams setting
b8fb36c 
Expose libcurl's CURLMOPT_MAX_CONCURRENT_STREAMS as a `max-http2-streams`
setting (default 100, libcurl's default). Setting it to 1 keeps HTTP/2
framing and header compression but disables multiplexing, which avoids
head-of-line blocking when a single paused stream (e.g. a slow store
import) throttles every other stream on the same TCP connection.

Our CI runners set `max-http2-streams = 1` in nix.conf, so the daemon
fails to start without this setting (GIGA-2850, GIGA-3360).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@ethancedwards8 @devin-ai-integration
nix eval: serve cached flake string attributes from the eval cache
728dd6b 
Add AttrCursor::cachedGetStringWithContext(), a cache-only variant of
getStringWithContext() that returns std::nullopt on a miss instead of
falling through to forceValue(). Use it in `nix eval` for plain flake
attribute evaluations (no --apply / --write-to): if the attribute is a
string that is already in the eval-cache database and every store path
in its context is still valid, print it without evaluating the flake at
all. This skips derivationStrict for e.g. `nix eval .#foo.outPath`
whenever the flake inputs haven't changed.

Temp roots are added for context paths, as the existing cached branch of
getStringWithContext() does, and ensureLazyPathsCopied() is still run
on the cached context.

Re-implementation of exa-labs/nix-src 8948813 against NixOS/nix 2.36
(GIGA-2848, GIGA-3360). Differences from the original: the `Path`
string-context variant no longer exists upstream; errors other than
`Error` (notably Interrupted) are not swallowed by the fast path.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@ethancedwards8 @devin-ai-integration
libexpr: default the Boehm GC free space divisor to 1
7d9256b 
The libgc default free space divisor is 3, i.e. the heap grows ~33%
after each collection. Setting it to 1 makes the heap double instead,
roughly halving the number of collections. Measured on a full-nixpkgs
`nix-env -qa --out-path`: ~104s -> ~84s wall (-20%) for 15.2 GB ->
19.8 GB peak RSS (+30%). This one change accounts for essentially the
whole end-to-end win of the exa perf series (GIGA-2849).

Unlike the original fork patch, the value is only applied when the
GC_FREE_SPACE_DIVISOR environment variable is unset, matching how
GC_INITIAL_HEAP_SIZE is already handled here. libgc reads that variable
in GC_INIT(); unconditionally overriding it afterwards left users with
no way to opt back out on memory-constrained machines
(`GC_FREE_SPACE_DIVISOR=3 nix ...` now restores stock behaviour).

Re-implementation of exa-labs/nix-src 319801a against NixOS/nix 2.36
(GIGA-3360).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@ethancedwards8 @devin-ai-integration
libutil: presize output in base16 and nix32 encoders
ab18a35 
Both encoders built their result with reserve() + push_back(), paying a
capacity check per character. Allocate the exact-length string up front
and write through a pointer instead. base16::encode additionally uses a
constexpr byte -> two-digit table so each input byte is a single lookup
rather than two nibble lookups.

These are hot in hash printing (store path computation, narinfo
handling, eval-cache keys). Output is byte-for-byte unchanged; the
existing unit tests in src/libutil-tests cover both encoders.

Re-implementation of exa-labs/nix-src e781363 and 196a532 against
NixOS/nix 2.36 (GIGA-2839, GIGA-2840, GIGA-3360).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@ethancedwards8 @devin-ai-integration
libutil: make Callback idempotent instead of aborting on a second call
29dae6e 
Callback::operator() and Callback::rethrow() asserted that they were
the first invocation. Under load, HttpBinaryCacheStore::getFile could
complete its callback twice: once from the FileTransfer completion
handler and once from the surrounding catch(...) when enqueueing
raced with a failure, taking the daemon down with SIGABRT
(NixOS#13484, still open upstream).

A callback that has been fulfilled cannot be un-fulfilled, so a second
invocation is now a no-op. Since both entry points are noexcept, an
exception escaping the wrapped lambda is now routed through
ignoreExceptionInDestructor() (logged, not terminating) rather than
calling std::terminate. Also split the try/catch in
HttpBinaryCacheStore::getFile so a disabled cache reports through the
callback before anything is enqueued, narrowing the window in which two
paths can race for the same callback.

This is deliberately a hardening fix, not a root-cause fix; see
GIGA-2851 for the follow-up. Re-implementation of exa-labs/nix-src
8a8eedd against NixOS/nix 2.36 (GIGA-3360).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@ethancedwards8 @devin-ai-integration
daemon: detect broken client sockets and handle signals in forked wor…
2a4541e 
…kers

Two related fixes for daemon workers that outlive their client
(NixOS#15691, still open upstream):

* MonitorFdHup (poll() variant): also treat POLLERR and POLLNVAL as a
  hangup. A client socket that errors out rather than closing cleanly is
  never reported as POLLHUP, so poll() returned immediately forever and
  the monitor thread spun at 100% CPU while the worker kept going.

* nix-daemon: after fork(), the worker inherits the daemon's blocked
  signal mask but not its signal handler thread, so it could never
  observe SIGTERM/SIGINT/SIGPIPE. Restore the mask saved at daemon
  startup and start a fresh handler thread in the worker. The order
  matters: startSignalHandlerThread() re-saves the current mask, and
  saving the already-blocked mask would later be reapplied to build
  children via restoreProcessContext(), leaving signals blocked in
  builders.

Re-implementation of exa-labs/nix-src 56de571 against NixOS/nix 2.36
(GIGA-2838, GIGA-3360).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@ethancedwards8 @devin-ai-integration
libstore: don't crash the daemon when a GC client read is interrupted
7ffbf49 
The per-client thread in LocalStore::collectGarbage that reads temp
roots from connected clients only caught Error. Interrupted derives from
BaseError, not Error, so an interrupt raised while blocked in
readLine() (e.g. the daemon worker being told to shut down mid-GC)
escaped the std::thread and took the whole nix-daemon down via
std::terminate. Catch it and leave the loop like any other client
disconnect.

Re-implementation of exa-labs/nix-src aff8949 against NixOS/nix 2.36
(GIGA-3360).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@ethancedwards8 @devin-ai-integration
libstore: don't block store adds behind a running auto-GC above min-free
2b3cf17 
When autoGC(sync=true) found a GC pass already running it always waited
for it to finish. A pass over a large store can take hours, and during
that time every store add on the machine (builds, substitutions,
`nix copy` into the store) was stalled, even with plenty of free space.

Adds are safe to run concurrently with GC via the temp-roots protocol,
so only wait when free space is actually below the `min-free` floor;
otherwise return immediately and let the add proceed. The `!sync` case
is unchanged in effect (it never waited).

Re-implementation of exa-labs/nix-src 2b9074b against NixOS/nix 2.36
(GIGA-3360).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@linear-code

linear-code Bot commented Sep 7, 2026

Copy link
Copy Markdown

GIGA-3360

@ethancedwards8
ethancedwards8 deleted the exa/giga-3360-perf-fixes branch September 7, 2026 23:34
@ethancedwards8

ethancedwards8 commented Sep 7, 2026

Copy link
Copy Markdown
Member Author

Superseded by a PR against the exa integration branch (GitHub closed this one when the head branch was renamed). See #4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ethancedwards8