Skip to content

proposal - unbreak test cli arg.md

Latest commit

 

History

History
340 lines (258 loc) · 5.96 KB

proposal - unbreak test cli arg.md

File metadata and controls

340 lines (258 loc) · 5.96 KB

Proposal: un/break --test

Currently, this flag's behaviour is very unexpected, often resulting in silent-failure footguns, and precludes command nesting, as demonstrated in a very simple and typical CI setup:

- name: tests with coverage
  run: node --run test -- --test-reporter lcov

That results in

node --test --test-reporter lcov

In most cases, --test-reporter gets lost (the lcov reporter is not enabled); in a worse case, this unexpectedly causes tests within a directory named lcov to be run.

More info

Currently (since always):

--test receives everything after it (space-delimited).

--test currently does 2 things:

  • enables the test runner
  • accepts paths for the runner to consume

This is similar to another existing feature with which we want to improve interop: watch mode.

--watch currently does 2 things:

  • enables watch mode
  • optionally accepts 1 value to override the entrypoint
node
  --watch
  --watch-path ./src/**/*.ts
  --watch-path ./test/**/*.ts

Proposed options

Option: n-number of --tests + --watch-path

--test is no-longer positional, accepting 1 value per occurrance of the flag. Supplying any value will override the default glob path.

When combined with test mode, --watch's value is ignored (treated as an "on" flag)

--watch-path supplies additional paths to trigger the test-runner to re-run.

Description Code sample Resulting --test value
Test & watch modes enabled with defaults 
node
  --test
  --watch

**/*.test.{cjs,cts,mjs,mts,js,ts} etc

https://nodejs.org/api/test.html#running-tests-from-the-command-line
Test & watch modes enabled, overriding default 
node
  --test ./src/foo/*.test.js
  --watch

./src/foo/*.test.js
Test & watch modes enabled, multiple test paths & additional watch paths 
node
  --test ./src/foo/*.test.js
  --test ./src/bar/*.test.js
  --watch
  --watch-path ./src/quz/fixt.json

./src/foo/*.test.js ./src/bar/*.test.js

In addition to changes within the graph of those test, tests will also re-run if ./src/quz/fixt.json changes.

Option: Break all the things

Currently watch has 2 flags (which are redundant):

node --watch
node --watch ./src/not-pjson-main.js
node
  --watch ./src/not-pjson-main.js
  --watch-path ./src/**/*.js

This could be simplified into just 1 --watch flag (where the default value is a single-element array of the derived entrypoint).

node --watch
node --watch ./src/not-pjson-main.js
node --watch ./src/**/*.js
node
  --watch ./assets/**
  --watch ./src/**/*.js
  ./src/not-pjson-main.js

An explicit entrypoint, like all node commands, must come last and must be a relative or absolute path (not a glob/pattern).

test would then work the same way:

node --test
node --test ./src/**/*.test.js
node
  --test ./src/foo/*.test.js
  --test ./src/bar/*.test.js
  ./src/not-pjson-main.js

All together

--watch receives a clone of --test's resolved value(s) plus --watch's own values—with 1 exception: when both test and watch modes are enabled, --watch does not include a default entrypoint (it's irrelevant and it would likely result in perf waste at best, and unexpected behaviour at worst).

All examples are sequence-independent (all within the same heading behave the same).

Watch + test paths & main entrypoint:
node
  --test ./src/foo/*.test.js
  --test ./src/bar/*.test.js
  --watch
  ./src/not-pjson-main.js
node
  --watch
  --test ./src/foo/*.test.js
  --test ./src/bar/*.test.js
  ./src/not-pjson-main.js
Without an entrypoint (just paths):
node
  --test ./src/foo/*.test.js
  --test ./src/bar/*.test.js
  --watch
node
  --watch
  --test ./src/foo/*.test.js
  --test ./src/bar/*.test.js
Additional watch paths
node
  --test ./src/foo/*.test.js
  --test ./src/bar/*.test.js
  --watch ./src/**/*.js
node
  --watch ./src/**/*.js
  --test ./src/foo/*.test.js
  --test ./src/bar/*.test.js

Option: Apply --watch's current design to --test

--test is optional and optionally accepts 1 value, whose value defaults to node's built-in glob defaults (./src/**/*.test.(c|m)?(j|t)s, etc). Additional paths are supplied via a new --test-path flag:

Description Code sample Resulting --test value
Override default 
node --test ./src/foo/*.test.js

./src/foo/*.test.js
Override default & Set additional path 
node
  --test ./src/foo/*.test.js
  --test-path ./src/bar/*.test.js

./src/foo/*.test.js ./src/bar/*.test.js
Defaults + Additional paths 
node
  --test
  --test-path ./src/foo/*.test.js
  --test-path ./src/bar/*.test.js

./src/**/*.test.(c|m)?(j|t)s ./src/foo/*.test.js ./src/bar/*.test.js

which reduces to only the default: ./src/**/*.test.(c|m)?(j|t)s
No defaults & Additional paths 
node
  --test-path ./src/foo/*.test.js
  --test-path ./src/bar/*.test.js

./src/foo/*.test.js ./src/bar/*.test.js