Unexpected Failure When Replacing Variable

[Alexhuszagh]

Summary

While trying to use insert_var to replace data for cargo --help's output, in order to add UI testing to cross for our integrationg tests, I noticed that I was unable to use insert_var to actually replace data for very specific corner cases. Here's a sample project reproducing the failing test case. The minimum, failing example test string is:

const DATA: &str = r#"'
:"#;

Detailed Information

The test case uses:

$ trycmd-test
[REPLACE]

And the test script is:

const DATA: &str = r#"'
:"#;

#[test]
fn cli_tests() {
    trycmd::TestCases::new()
        .case("tests/cmd/failed-replacement.md")
        .insert_var("[REPLACE]", DATA)
        .unwrap();
}

Where main.rs is:

fn main() {
    println!("{DATA}");
}

const DATA: &str = r#"'
:"#;

It will fail with the following output:

running 1 test
Testing tests/cmd/failed-replacement.md:2 ... failed
Exit: success

---- expected: stdout
++++ actual:   stdout
   1      - [REPLACE]
        1 + '
        2 + :
stderr:

Update snapshots with `TRYCMD=overwrite`
Debug output with `TRYCMD=dump`
test cli_tests ... FAILED

Note that all of the following actually work:

const EX1: &str = r#"':"#;
const EX2: &str = r#"X's USAGE:"#;

It seems to require the following regex pattern: '.*\n.*:. If the newline is not present, everything works as expected.

Original Use

The original failing output, and what we were originally trying to match, is as follows:

Rust's package manager

USAGE:
    cargo [+toolchain] [OPTIONS] [SUBCOMMAND]

OPTIONS:
    -V, --version               Print version info and exit
        --list                  List installed commands
        --explain <CODE>        Run `rustc --explain CODE`
    -v, --verbose               Use verbose output (-vv very verbose/build.rs output)
    -q, --quiet                 Do not print cargo log messages
        --color <WHEN>          Coloring: auto, always, never
        --frozen                Require Cargo.lock and cache are up to date
        --locked                Require Cargo.lock is up to date
        --offline               Run without accessing the network
        --config <KEY=VALUE>    Override a configuration value (unstable)
    -Z <FLAG>                   Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for
                                details
    -h, --help                  Print help information

Some common cargo commands are (see all commands with --list):
    build, b    Compile the current package
    check, c    Analyze the current package and report errors, but don't build object files
    clean       Remove the target directory
    doc, d      Build this package's and its dependencies' documentation
    new         Create a new cargo package
    init        Create a new cargo package in an existing directory
    add         Add dependencies to a manifest file
    run, r      Run a binary or example of the local package
    test, t     Run the tests
    bench       Run the benchmarks
    update      Update dependencies listed in Cargo.lock
    search      Search registry for crates
    publish     Package and upload this package to the registry
    install     Install a Rust binary. Default location is $HOME/.cargo/bin
    uninstall   Uninstall a Rust binary

See 'cargo help <command>' for more information on a specific command.

[epage]

Yes, newlines are not supported in substitutions. If nothing else, we should assert if the user provides one.

Processing is done on a per line basis. This helps with our multi-line glob and with allowing matching literal ones as well as ones with variables.

I didn't quite catch, what is the case you are needing a newline for?

[Alexhuszagh]

Yes, newlines are not supported in substitutions. If nothing else, we should assert if the user provides one.

Ah that makes more sense. Thank you.

As for why... Since we don't know exactly what the output is from cargo --help (or say, some other command), since it might ostensibly change with new flags in a later version, our wrapper might have different output at some point but we want to check our cargo wrapper works correctly. cross effectively wraps cargo to invoke the commands within a docker container, and we've added some diagnostics, version output, but otherwise try to keep it as similar to cargo as possible, so we'd like to be able to use the output from cargo itself when testing the UI.

We could definitely handle it on our own with custom logic (see this PR for a very bad attempt at my own custom logic), but I'd rather use this excellent tool if possible.

[Alexhuszagh]

I'll probably just use snapbox directly for this, since it has assert_matches and cargo_bin, and this way it doesn't require any complex replacements on your part. Thanks for the quick feedback on the lack of support for newlines, since this remedies it quite quickly.