Repository navigation
Troubleshooting
Tier: Beginner
Common errors and fixes. When in doubt, also check the Discussions Q&A category — many issues have been hashed out there.
The x86_64 prebuilts target the x86-64-v3 level, so they require a CPU with AVX2, FMA and BMI1/BMI2 (#4649). Intel Haswell (2013) / AMD Excavator (2015) and newer qualify, but some later low-end chips (Intel Pentium/Celeron/Atom up to ~2020) and VMs with a conservative CPU model (QEMU/Proxmox kvm64/qemu64) don't. On such a CPU, Illegal instruction is the expected result, not a bug.
Check with grep -o avx2 /proc/cpuinfo | head -1 on Linux (no output means no AVX2), or Coreinfo on Windows. Without AVX2:
-
Compile from source, which uses the portable baseline by default:
cargo install qsv --locked --features all_features
-
or use
qsvdp, the one x86_64 prebuilt that stays on the portable baseline (it has only DataPusher+'s commands); -
or use a 23.0.x release, the last without the AVX2 requirement.
If qsv --update already put you on a binary that won't start, it can't update itself back: re-download 23.0.x from the releases page, or build from source. Apple Silicon, Windows-on-ARM, IBM Power and IBM Z prebuilts are unaffected. Details: Installation.
If your CPU does have AVX2 and a current prebuilt still dies with SIGILL, that's a bug: please file an issue with your CPU model (lscpu on Linux, Get-WmiObject Win32_Processor on Windows).
Note
There is no portable qsvp variant to switch to: the p-suffixed binaries are in no release archive (Binary Variants).
Easy Installer v1.1.1 and earlier extract a hardcoded qsvp.exe — a portable binary that releases don't include — and its update path returns no error, so it reports success even when nothing was installed.
Fix: install Easy Installer v1.1.2 or later and re-run it, then verify with qsv --version. Or skip the installer entirely: download the x86_64-pc-windows-msvc archive from releases and extract qsv.exe to a directory on your PATH.
qsv refuses to write its output over its input: it detects that they are the same file (by file identity, not by string comparison, so ./x.csv and x.csv are caught too) and errors out rather than truncating the data. Write to a different path and rename afterwards.
qsv binaries are signed with an ad-hoc signature. Either remove the quarantine attribute before the first run:
# replace qsv with qsvmcp, qsvlite, qsvdp or qsvpy* for those variants
sudo xattr -d com.apple.quarantine qsv…or right-click the binary → Open → confirm in the Gatekeeper dialog. See Apple's Gatekeeper docs.
The MSI Easy Installer adds qsv to your PATH — but only for new terminals. Close and reopen your terminal after install. If still not found, the installer may have failed; check the install log.
The binary's enabled features depend on the build. Run qsv --version to see them — package-manager builds differ from the official prebuilt binaries. For the full feature set, install the prebuilt or build from source with -F all_features. See Binary Variants.
Not in the prebuilt: the musl binaries are cross-compiled on a glibc-based Ubuntu runner, and building luau that way would require statically linking the host's libc. Build from source on your musl distro (Alpine, Void, …) with cargo build --release --locked --bin qsv -F all_features.
The input isn't UTF-8. Two paths:
# Path A: Lossy normalization (substitute � for invalid bytes)
qsv input --encoding-errors replace messy.csv > utf8.csv
# Path B: Proper transcoding (recommended)
iconv -f ISO-8859-1 -t UTF-8 messy.csv -o utf8.csv
qsv input utf8.csv > clean.csvqsv requires UTF-8 input (ASCII is a subset). qsv input does a "lossy save" to UTF-8, replacing invalid sequences with � (U+FFFD); to truly transcode, use a utility like iconv first. See Transform & Reshape → input.
Mojibake in a fetch/fetchpost response column is a different problem: the server sent latin-1/windows-1252 text without a charset in its Content-Type, so it was decoded as UTF-8. Pass --default-encoding windows-1252 (see HTTP & Web).
In PowerShell, redirecting qsv's output (qsv stats wcp.csv > wcpstats.csv) writes the file as UTF-16LE, Windows' default encoding, which Excel doesn't recognize as a CSV. Write the file with --output instead, so it stays UTF-8:
qsv stats wcp.csv --output wcpstats.csvOr set QSV_OUTPUT_BOM=1 so qsv adds a UTF-8 Byte Order Mark that Excel on Windows recognizes. Excel on macOS, and files produced on other operating systems, aren't affected.
# Explicit
qsv stats --delimiter ';' data.ssv
# Or: tell qsv to sniff every input
export QSV_SNIFF_DELIMITER=1
qsv stats data.weird-delimFor a one-off normalization to comma:
qsv fmt --out-delimiter ',' data.weird > data.csvThe CSV has rows of inconsistent width. Fix with:
qsv fixlengths ragged.csv > fixed.csvOr pass --flexible to commands that accept it (count --flexible).
stats and frequency accept --flexible too; without it, they report a ragged record as an error (#4614).
qsv strips a leading UTF-8 BOM when it reads a file, so the first header comes through clean (qsv headers shows name, not \ufeffname). If another tool shows the BOM, it's that tool reading the original file.
For generating Excel-friendly CSVs with a BOM:
QSV_OUTPUT_BOM=1 qsv stats data.csv --output stats_excel.csvYou don't need to do anything: when the CSV is newer than its .idx, qsv rebuilds the index automatically (qsv index itself "will also automatically update stale indices"). To rebuild by hand, run qsv index data.csv again. To have qsv create indexes for large files on its own, set QSV_AUTOINDEX_SIZE=<bytes> — see Performance Tuning.
"Automagical" (🪄) commands (frequency, schema, pivotp, sqlp, …) consume <file>.stats.csv.data.jsonl, not <file>.stats.csv. Regenerate with:
qsv stats --stats-jsonl <file>Or set QSV_STATSCACHE_MODE=force. See Stats Cache & Caching.
Polars commands (sqlp, joinp, pivotp) inferred a column as Int64 but a later row has 12.5. Fix:
qsv schema --polars --force data.csv
# Hand-edit data.pschema.json: change the column's dtype to Float64Polars commands pick up the corrected .pschema.json automatically.
Polars SQL errors to know (details):
-
no table or struct column named '<t>' foundmeans you aliased a table (FROM t AS x) but still refer to it ast.col. Usex.col. -
A
CASTto a date/time type infers one format from the column's values:-
could not find an appropriate format to parse datesmeans no format fits any value. -
conversion from `str` to `date` failed in column '<col>' for N out of M valuesmeans a format was found, but some values don't match it. The error lists them.
Use
TRY_CASTto get NULL for the values that don't parse. For US-style dates, useSTRPTIME(col, '%m/%d/%Y'). A plainCASTcan read01/05/2024day-first as 1 May without an error. -
-
--try-parsedateshas the same day-first problem. Polars only infers day-first (%d/%m/%Y,%d-%m-%Y,%d.%m.%Y) and year-first formats, so01/05/2024loads as2024-05-01. If the column also holds a date like12/25/2024, it stays a string, with no error. -
invalid series dtype: expected `String`, got `date`fromSTRPTIMEmeans--try-parsedatesalready turned the column into a date. For month-first (US) dates, leave--try-parsedatesoff and useSTRPTIME(col, '%m/%d/%Y').
In-memory commands (🤯 in the README) — color, dedup, pragmastat, readstat, reverse, sort, stats (advanced statistics, e.g. --everything), table, transpose and writestat — can run out of memory on huge files. qsv's Out-of-Memory prevention checks first, in NORMAL mode by default (details). Three fixes: Three fixes:
-
Switch to the streaming variant:
-
Switch the memory check to CONSERVATIVE mode, which also counts free swap and applies a safety margin:
export QSV_MEMORY_CHECK=1
export QSV_FREEMEMORY_HEADROOM_PCT=10- Constrain
stats/frequencychunk memory:
export QSV_STATS_CHUNK_MEMORY_MB=2048
export QSV_FREQ_CHUNK_MEMORY_MB=1024See Recipe: Larger-than-RAM CSV.
Pre-populate the stats cache so frequency knows the column is ALL_UNIQUE and short-circuits it:
qsv stats --cardinality --stats-jsonl data.csv
qsv frequency data.csvOr switch to the heavy-hitters mode:
qsv frequency --sketch-method frequent_items --sketch-map-size 16384 data.csv--infer-dates only checks columns whose names match the --dates-whitelist patterns (default: sniff). To force date inference on a column:
qsv stats --infer-dates --dates-whitelist 'mydate,timestamp,created_at' data.csvRegex is case-sensitive by default. Add -i:
qsv search -i 'brooklyn' data.csvFor non-ASCII characters, enable Unicode mode:
QSV_REGEX_UNICODE=1 qsv search 'café' data.csv(Unicode mode is slower; off by default.)
qsv uses the Rust regex crate (the same engine as ripgrep), which leaves out look-around and backreferences in exchange for guaranteed linear-time matching. Rewrite the pattern without them; test it on regex101 with the Rust flavor. In JSON Schema patterns, validate --fancy-regex does support look-around and backreferences.
Likely a rate limit. Check the report:
# --report takes d|s and writes a sidecar (data.csv.fetch-report.tsv), not stdout
qsv fetch URL data.csv --report d --new-column response > out.csv
qsv frequency --select qsv_fetch_status data.csv.fetch-report.tsvAdd --rate-limit N to throttle, or --disk-cache to dedupe repeated calls. See Recipe: Fetch & Cache.
The luau feature isn't enabled in qsvlite / qsvdp, the musl prebuilds, or the Linux ARM64 prebuilds (run qsv --list to check). Install the standard qsv or qsvpy* variant, or build from source. See Binary Variants.
qsv --version
qsv --envlistPaste both in your Discussions / Issues post.
export QSV_LOG_LEVEL=debug
export QSV_LOG_DIR=./qsv-logs
qsv stats data.csv
ls qsv-logs/See docs/Logging.md.
- Questions / how-to: Discussions Q&A
- FAQ-worthy: Discussions FAQ
- Confirmed bugs: Issues
- Security: see SECURITY.md
Always include qsv --version and qsv --envlist output, plus a minimal reproducer.
qsv — GitHub · Releases · Discussions · crates.io · qsv pro · Try it online · Benchmarks · DeepWiki · DOI · Licensed MIT · Third-party notices · AI Policy
qsv was made possible by datHere: standards-based, best-of-breed, open source solutions to make your Data Useful, Usable & Used.
Edit this page: Contributing to the Wiki
Home · Why qsv? · Tier legend
- Whirlwind Tour · Notebooks · Lessons & Exercises
- Benchmarks · Interactive Benchmarks
- Visualizations Gallery
- Neuro-symbolic, Localizable, Data Schematics
- Feature Flags · Goals/Non-goals · Testing
- Talks: NYC SOD 2022 · csv,conf,v8 · PyCon US 2025 · csv,conf,v9 · NYC SOD 2026
- "Have we achieved ACI?" series: 1 · 2 · 3 · 4
- Sponsor
- All Commands (index)
- Selection & Inspection
- Transform & Reshape
- Aggregation & Statistics
- Joins & Set Ops
- SQL & Polars
- Validation & Schema
- Metadata Profiling (profile)
- Conversion & I/O
- Geospatial
- Visualization (viz)
- HTTP & Web
- Get & Disk Cache
- Scripting (Luau / Python)
- Indexing, Compression & Diff
- AI & Documentation
- Recipes index
- Inspect an Unknown CSV
- Clean & Normalize
- Geographic Enrichment
- Date Enrichment
- CKAN Integration
- DCAT-US v3 for data.gov
- JSON Schema Validation
- Round-Trip SAS / Stata / SPSS
- Build a Data Pipeline
- Stats → Insights
- Data-Quality Signals with moarstats
- Group By
- Fetch & Cache
- Larger-than-RAM CSV
- Diff & Audit
- Multi-table Joins
- Synthesize Fake Data
- Performance Tuning
- Benchmarks
- Tuned csv Fork Benchmarks
- Security & Dependency Audits
- Environment Variables
- Stats Cache & Caching
- Lookup Tables
- MiniJinja Templating