Testing improvements, break up repl/plugins tests, add TAP output to repl characterisation tests - #16421
Open
xokdvium wants to merge 6 commits into
Open
Testing improvements, break up repl/plugins tests, add TAP output to repl characterisation tests#16421xokdvium wants to merge 6 commits into
xokdvium wants to merge 6 commits into
Conversation
…bdirectory It was already a separate test suite, just inexplicably located in the top-level directory. Split up the tests and simplify the common.sh setup machinery with the help of BASH_ENV.
To get rid of: hint: Using 'master' as the name for the initial branch. This default branch name hint: will change to "main" in Git 3.0. Changing that from master breaks the tests. Also dedups test to use createGitRepo
That file is entirely too large and unmanageable. The stack size ulimit issue was fixed too, though I'm not too sure how. I was able to reproduce that but it was gone after the reorganisation and changing the directory structure. The remaining repl.sh test stay as is because they do building and are also relevant to CA test suite, which re-runs it with a different config.
github-actions
Bot
added
the
with-tests
xokdvium
force-pushed
the
break-up-repl-plugins-tests-tap
branch
from
September 3, 2026 21:08
e83b498 to
35c5468
Compare
This provides much more legible output: 1/1 repl - nix-functional-tests:characterisation OK 1.40s 35 subtests passed Or with verbose mode: ▶ 1/1 multiline-string-function-binding OK ▶ 1/1 multiple-bindings-same-line OK ▶ 1/1 nested-attr-path OK ▶ 1/1 pretty-print-idempotent OK ▶ 1/1 printing OK 1/1 repl - nix-functional-tests:characterisation OK 1.40s 35 subtests passed
xokdvium
force-pushed
the
break-up-repl-plugins-tests-tap
branch
from
September 3, 2026 21:11
35c5468 to
316bbdd
Compare
xokdvium
commented
Sep 7, 2026
Comment on lines
-28
to
-34
| # FIXME: repl tests fail on systems with stack limits | ||
| stack_ulimit="$(ulimit -Hs)" | ||
| stack_required="$((64 * 1024 * 1024))" | ||
| if [[ "$stack_ulimit" != "unlimited" ]]; then | ||
| ((stack_ulimit < stack_required)) && skipTest "repl tests cannot run on systems with stack size <$stack_required ($stack_ulimit)" | ||
| fi | ||
|
|
Contributor
Author
There was a problem hiding this comment.
This got fixed somehow.... maybe worth digging into
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
I decided to start doing more things to battle our test suite, which is starting to look more like an incoherent collection of disorganized trash.
Things done:
Context
For SHA256 git support that I want to do once libgit2 2.0 is finally out (#16349 was the latest bit) I'd also want to apply the same treatment to all git fetching tests so that we can exercise both object formats equally well. Hence why I decided to start small and do the smaller test suites first. Everything else should easily follow.
lang/ and other characterisation tests should also benefit pretty nicely from TAP support, but I'm going to leave that for later.
Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.