Skip to content
Joel Natividad edited this page Nov 16, 2026 · 17 revisions

Troubleshooting

Tier: Beginner

Common errors and fixes. When in doubt, also check the Discussions Q&A category — many issues have been hashed out there.

Install & launch

Illegal instruction (core dumped) / SIGILL on x86_64

The x86_64 prebuilts target the x86-64-v3 level, so they require a CPU with AVX2, FMA and BMI1/BMI2 (#4649). Intel Haswell (2013) / AMD Excavator (2015) and newer qualify, but some later low-end chips (Intel Pentium/Celeron/Atom up to ~2020) and VMs with a conservative CPU model (QEMU/Proxmox kvm64/qemu64) don't. On such a CPU, Illegal instruction is the expected result, not a bug.

Check with grep -o avx2 /proc/cpuinfo | head -1 on Linux (no output means no AVX2), or Coreinfo on Windows. Without AVX2:

  • Compile from source, which uses the portable baseline by default:

    cargo install qsv --locked --features all_features
  • or use qsvdp, the one x86_64 prebuilt that stays on the portable baseline (it has only DataPusher+'s commands);

  • or use a 23.0.x release, the last without the AVX2 requirement.

If qsv --update already put you on a binary that won't start, it can't update itself back: re-download 23.0.x from the releases page, or build from source. Apple Silicon, Windows-on-ARM, IBM Power and IBM Z prebuilts are unaffected. Details: Installation.

If your CPU does have AVX2 and a current prebuilt still dies with SIGILL, that's a bug: please file an issue with your CPU model (lscpu on Linux, Get-WmiObject Win32_Processor on Windows).

Note

There is no portable qsvp variant to switch to: the p-suffixed binaries are in no release archive (Binary Variants).

Windows: the Easy Installer said "Successfully installed" but qsv is missing or stale

Easy Installer v1.1.1 and earlier extract a hardcoded qsvp.exe — a portable binary that releases don't include — and its update path returns no error, so it reports success even when nothing was installed.

Fix: install Easy Installer v1.1.2 or later and re-run it, then verify with qsv --version. Or skip the installer entirely: download the x86_64-pc-windows-msvc archive from releases and extract qsv.exe to a directory on your PATH.

qsv <cmd> file.csv -o file.csv errors out

qsv refuses to write its output over its input: it detects that they are the same file (by file identity, not by string comparison, so ./x.csv and x.csv are caught too) and errors out rather than truncating the data. Write to a different path and rename afterwards.

macOS: "qsv cannot be opened because the developer cannot be verified"

qsv binaries are signed with an ad-hoc signature. Either remove the quarantine attribute before the first run:

# replace qsv with qsvmcp, qsvlite, qsvdp or qsvpy* for those variants
sudo xattr -d com.apple.quarantine qsv

…or right-click the binary → Open → confirm in the Gatekeeper dialog. See Apple's Gatekeeper docs.

Windows: "qsv is not recognized"

The MSI Easy Installer adds qsv to your PATH — but only for new terminals. Close and reopen your terminal after install. If still not found, the installer may have failed; check the install log.

qsv --version reports unexpected features

The binary's enabled features depend on the build. Run qsv --version to see them — package-manager builds differ from the official prebuilt binaries. For the full feature set, install the prebuilt or build from source with -F all_features. See Binary Variants.

Want the luau feature on musl Linux?

Not in the prebuilt: the musl binaries are cross-compiled on a glibc-based Ubuntu runner, and building luau that way would require statically linking the host's libc. Build from source on your musl distro (Alpine, Void, …) with cargo build --release --locked --bin qsv -F all_features.

Encoding & format

"invalid UTF-8 sequence" / mojibake characters

The input isn't UTF-8. Two paths:

# Path A: Lossy normalization (substitute � for invalid bytes)
qsv input --encoding-errors replace messy.csv > utf8.csv

# Path B: Proper transcoding (recommended)
iconv -f ISO-8859-1 -t UTF-8 messy.csv -o utf8.csv
qsv input utf8.csv > clean.csv

qsv requires UTF-8 input (ASCII is a subset). qsv input does a "lossy save" to UTF-8, replacing invalid sequences with � (U+FFFD); to truly transcode, use a utility like iconv first. See Transform & Reshape → input.

Mojibake in a fetch/fetchpost response column is a different problem: the server sent latin-1/windows-1252 text without a charset in its Content-Type, so it was decoded as UTF-8. Pass --default-encoding windows-1252 (see HTTP & Web).

Windows: Excel puts everything in the first column

In PowerShell, redirecting qsv's output (qsv stats wcp.csv > wcpstats.csv) writes the file as UTF-16LE, Windows' default encoding, which Excel doesn't recognize as a CSV. Write the file with --output instead, so it stays UTF-8:

qsv stats wcp.csv --output wcpstats.csv

Or set QSV_OUTPUT_BOM=1 so qsv adds a UTF-8 Byte Order Mark that Excel on Windows recognizes. Excel on macOS, and files produced on other operating systems, aren't affected.

Wrong delimiter detected

# Explicit
qsv stats --delimiter ';' data.ssv

# Or: tell qsv to sniff every input
export QSV_SNIFF_DELIMITER=1
qsv stats data.weird-delim

For a one-off normalization to comma:

qsv fmt --out-delimiter ',' data.weird > data.csv

"CSV record has X fields, but expected Y" (ragged rows)

The CSV has rows of inconsistent width. Fix with:

qsv fixlengths ragged.csv > fixed.csv

Or pass --flexible to commands that accept it (count --flexible).

stats and frequency accept --flexible too; without it, they report a ragged record as an error (#4614).

BOM in the first column header

qsv strips a leading UTF-8 BOM when it reads a file, so the first header comes through clean (qsv headers shows name, not \ufeffname). If another tool shows the BOM, it's that tool reading the original file.

For generating Excel-friendly CSVs with a BOM:

QSV_OUTPUT_BOM=1 qsv stats data.csv --output stats_excel.csv

Index & cache

Index out of date after the CSV changed

You don't need to do anything: when the CSV is newer than its .idx, qsv rebuilds the index automatically (qsv index itself "will also automatically update stale indices"). To rebuild by hand, run qsv index data.csv again. To have qsv create indexes for large files on its own, set QSV_AUTOINDEX_SIZE=<bytes> — see Performance Tuning.

Stats cache not being picked up

"Automagical" (🪄) commands (frequency, schema, pivotp, sqlp, …) consume <file>.stats.csv.data.jsonl, not <file>.stats.csv. Regenerate with:

qsv stats --stats-jsonl <file>

Or set QSV_STATSCACHE_MODE=force. See Stats Cache & Caching.

Polars schema drift / "schema inference error"

Polars commands (sqlp, joinp, pivotp) inferred a column as Int64 but a later row has 12.5. Fix:

qsv schema --polars --force data.csv
# Hand-edit data.pschema.json: change the column's dtype to Float64

Polars commands pick up the corrected .pschema.json automatically.

sqlp errors on aliased tables or date casts

Polars SQL errors to know (details):

  • no table or struct column named '<t>' found means you aliased a table (FROM t AS x) but still refer to it as t.col. Use x.col.

  • A CAST to a date/time type infers one format from the column's values:

    • could not find an appropriate format to parse dates means no format fits any value.
    • conversion from `str` to `date` failed in column '<col>' for N out of M values means a format was found, but some values don't match it. The error lists them.

    Use TRY_CAST to get NULL for the values that don't parse. For US-style dates, use STRPTIME(col, '%m/%d/%Y'). A plain CAST can read 01/05/2024 day-first as 1 May without an error.

  • --try-parsedates has the same day-first problem. Polars only infers day-first (%d/%m/%Y, %d-%m-%Y, %d.%m.%Y) and year-first formats, so 01/05/2024 loads as 2024-05-01. If the column also holds a date like 12/25/2024, it stays a string, with no error.

  • invalid series dtype: expected `String`, got `date` from STRPTIME means --try-parsedates already turned the column into a date. For month-first (US) dates, leave --try-parsedates off and use STRPTIME(col, '%m/%d/%Y').

Memory

OOM killer / qsv exited unexpectedly on a big file

In-memory commands (🤯 in the README) — color, dedup, pragmastat, readstat, reverse, sort, stats (advanced statistics, e.g. --everything), table, transpose and writestat — can run out of memory on huge files. qsv's Out-of-Memory prevention checks first, in NORMAL mode by default (details). Three fixes: Three fixes:

  1. Switch to the streaming variant:

    • sort → extsort
    • dedup → extdedup
    • stats with --cardinality-method approx --quantile-method approx, plus --mode-cardinality-cap under --everything/--mode (modes keep an exact table of every distinct value; only the cap bounds it)
  2. Switch the memory check to CONSERVATIVE mode, which also counts free swap and applies a safety margin:

export QSV_MEMORY_CHECK=1
export QSV_FREEMEMORY_HEADROOM_PCT=10
  1. Constrain stats / frequency chunk memory:
export QSV_STATS_CHUNK_MEMORY_MB=2048
export QSV_FREQ_CHUNK_MEMORY_MB=1024

See Recipe: Larger-than-RAM CSV.

frequency runs out of memory on an ID column

Pre-populate the stats cache so frequency knows the column is ALL_UNIQUE and short-circuits it:

qsv stats --cardinality --stats-jsonl data.csv
qsv frequency data.csv

Or switch to the heavy-hitters mode:

qsv frequency --sketch-method frequent_items --sketch-map-size 16384 data.csv

Specific commands

qsv stats: "infer_dates is set but no date-like columns found"

--infer-dates only checks columns whose names match the --dates-whitelist patterns (default: sniff). To force date inference on a column:

qsv stats --infer-dates --dates-whitelist 'mydate,timestamp,created_at' data.csv

qsv search returns no matches but you can see them

Regex is case-sensitive by default. Add -i:

qsv search -i 'brooklyn' data.csv

For non-ASCII characters, enable Unicode mode:

QSV_REGEX_UNICODE=1 qsv search 'café' data.csv

(Unicode mode is slower; off by default.)

"regex parse error": look-around or backreference not supported

qsv uses the Rust regex crate (the same engine as ripgrep), which leaves out look-around and backreferences in exchange for guaranteed linear-time matching. Rewrite the pattern without them; test it on regex101 with the Rust flavor. In JSON Schema patterns, validate --fancy-regex does support look-around and backreferences.

qsv fetch returns errors against a known-good API

Likely a rate limit. Check the report:

# --report takes d|s and writes a sidecar (data.csv.fetch-report.tsv), not stdout
qsv fetch URL data.csv --report d --new-column response > out.csv
qsv frequency --select qsv_fetch_status data.csv.fetch-report.tsv

Add --rate-limit N to throttle, or --disk-cache to dedupe repeated calls. See Recipe: Fetch & Cache.

qsv luau not available

The luau feature isn't enabled in qsvlite / qsvdp, the musl prebuilds, or the Linux ARM64 prebuilds (run qsv --list to check). Install the standard qsv or qsvpy* variant, or build from source. See Binary Variants.

Diagnostics

Quick env dump for bug reports

qsv --version
qsv --envlist

Paste both in your Discussions / Issues post.

Enable verbose logging

export QSV_LOG_LEVEL=debug
export QSV_LOG_DIR=./qsv-logs
qsv stats data.csv
ls qsv-logs/

See docs/Logging.md.

Where do I report bugs?

Always include qsv --version and qsv --envlist output, plus a minimal reproducer.

See also

Home · Why qsv? · Tier legend

Quick links

Get Started

Command Reference

Cookbook

Tuning & Internals

Ecosystem

Reference

Legacy

Clone this wiki locally