Skip to content

Revisions

  • wiki: sqlp --try-parsedates reads US dates day-first; STRPTIME needs it off Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Nov 16, 2026
  • wiki: stats .data.jsonl reuse rules and sqlp CAST date errors, per 29.0.0 - Stats-Cache: consumers also regenerate the .data.jsonl when it predates the .stats.csv.json sidecar, the CSV size changed, or (schema modes) it lacks --infer-dates (src/util/stats_cache.rs at 29.0.0) - not only on CSV mtime / parsing options. - Troubleshooting: CAST to a date infers one format from the column; a partial mismatch gives "conversion from `str` to `date` failed", and US-style dates can be read day-first without an error (tested on 29.0.0). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Nov 15, 2026
  • wiki: describe the latest release only — drop older-version notes Removes "new in X", "(X+)", "since X", "fixed in X", "in X and earlier" notes, "Requires: qsv X+" lines and per-release changelog sections across 43 pages. Facts a 29.0.0 reader still needs are restated in the present tense; 29.0.0 breaking changes stay, minus older backstory. Adds a "Describe the latest release only" convention to Contributing-to-the-Wiki, with *(unreleased)* for master-only features. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Nov 15, 2026
  • Troubleshooting: match the README; drop advice for errors that no longer occur - SIGILL: lead with the AVX2 requirement (expected on non-AVX2 CPUs); source builds are portable, not CPU-tuned; qsvdp and 23.0.x options - from the README: PowerShell UTF-16LE / Excel first-column note, regex look-around/backreference limits (validate --fancy-regex), OOM prevention modes, the README's in-memory command list, sudo xattr - stale indexes are rebuilt automatically ("please rebuild" error and index --force don't exist); a leading UTF-8 BOM is stripped on read - luau also missing from Linux ARM64 prebuilts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Nov 2, 2026
  • wiki: stats approx cardinality under --everything (#4806), to parquet inference and --all-strings (#4807) - Aggregation-and-Statistics, Performance-Tuning, Recipe-Larger-than-RAM, Troubleshooting: under --everything/--mode the exact distinct-value table is kept for modes regardless of --cardinality-method, so only --mode-cardinality-cap bounds that memory (24.0.0, NYC 311 1M rows: exact 625 MB, approx 617 MB, approx+cap 197 MB). Examples gain the cap; the OOM auto-switch notes say it doesn't bound mode tables. Unreleased note for #4806: exact counts kept when available, estimates clamped to the row count. - Conversion-and-IO: the NYC 311 to-parquet example failed on its own dataset (default 1,000-row inference vs a later ZIP+4); now uses --infer-len 0, explains inference and no-overwrite. Unreleased note for #4807 (--all-strings reads as text). Same fix for the Recipe-Larger-than-RAM DuckDB hand-off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Oct 13, 2026
  • wiki: update for 24.0.0 "Evidence Act" release - Home: 24.0.0 headline, 80 commands, 24.0.0 breaking-changes box - Flip all "Unreleased (after 23.0.1)" markers to 24.0.0 across ~28 pages - New: writestat section, readstat --dictionary/--select/--offset/--limit/--sample, --help --format json|md and --export-tool-definitions, aarch64 musl and Windows ARM64 gnullvm prebuilts, MSRV 1.99 - Variant counts: qsv 80, qsvpy 81, qsvmcp 70; 57 MCP tools - frequency/stats fixes and speedups, symlinked moarstats fix, #4603 fixed - Visualization: "Recent viz changes (24.0.0)"; earlier section retitled 23.0.1 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Oct 5, 2026
  • wiki: correct sqlp/profile claims against both binaries - CAST string->date was already strict on 23.0.1 (verified); drop it from the breaking list, note only the changed error text. - CORR/NULLS FIRST in windows and DATE literals already work on 23.0.1; keep only features verified to fail there as new. - CKAN step 9: use the repo's dcat-init-context.json seed; --strict passes on master, still rejected on 23.0.1. - Mark the pipeline recipe's stage-4 block illustrative. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Sep 28, 2026
  • wiki: fetch/HTTP - upcoming --default-encoding, cache-key change, OS proxy Marked Unreleased (after 23.0.1): fetch/fetchpost --default-encoding (example + one-time cache invalidation), cache key includes encoding, qsvlite/qsvdp/qsvmcp honor OS proxy settings and response charsets, and a Troubleshooting pointer for mojibake in fetch responses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Sep 28, 2026
  • wiki: sqlp - upcoming Polars SQL breaking changes and new surface - Add a correlated EXISTS example (works on 23.0.1 already). - Marked Unreleased (after 23.0.1): strict inference-based string->date CAST, Postgres alias scope, scalar-subquery aggregate binding, literal:N constants; ROLLUP/GROUPING, APPROX_QUANTILE, date parts, DATE literals. Every example run against master; the breaking ones also against 23.0.1. - Troubleshooting entry for queries that break after upgrading; CKAN recipe CAST caveat; pivotp totals cross-link to ROLLUP. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Sep 28, 2026
  • wiki: document unreleased moarstats, pragmastat, cache and index changes Marked Unreleased (after 23.0.1): - moarstats 56 -> 73 measures, JB/BC/kurtosis now match scipy, version- stale caches recompute, duplicate headers (#4651, #4655, #4663); new --bivariate example (cramersv, regression) and #4654/#4656 behavior. - pragmastat ps_* row matching (#4665); pivotp --agg smart bimodal guard. - Stats/frequency cache: qsv_version check + self-heal (#4618), --flexible in the key (#4616), permissions follow the source (#4622). - dc: frequency caches persist (#4641); atomic index writers (#4631); stats/frequency --flexible and ragged-CSV errors (#4614). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Sep 28, 2026
  • wiki: warn that the next release's x86_64 prebuilts require AVX2 Marked Unreleased (after 23.0.1). Adds the x86-64-v3 requirement (#4649), the qsvdp exemption, the /proc/cpuinfo check and - most urgently - the warning not to qsv --update on a non-AVX2 CPU, next to the SIGILL text that is still true for 23.0.1 (Installation, Troubleshooting, Binary-Variants, FAQ, Performance-Tuning, MCP-Server). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Sep 28, 2026
  • wiki: fix claims that are wrong for the released 23.0.1 - The 23.0.0 tag and release no longer exist; "Context is King" is 23.0.1, so every releases/tag/23.0.0 link 404'd. Relabel 23.0.0 -> 23.0.1. - moarstats count: 55 / 25+ -> 56 (73 on master, marked unreleased). - moarstats example: `> file` captured 0 bytes (moarstats updates the stats CSV in place); use -o. - FAQ: MSRV is 1.98; linux-aarch64 is not built with target-cpu=native. - Binary-Variants: target-cpu=native also survives on Windows-on-ARM, ppc64le and s390x, not just Apple Silicon. - Date recipes: julianday / SQLite-order strftime fail on Polars SQL; rewritten with STRPTIME/STRFTIME/EXTRACT, verified on 23.0.1 and master. - Environment-Variables: add the 5 MCP/Cowork vars missing vs docs. - SQL-and-Polars: an IN subquery was mislabeled "correlated". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Sep 28, 2026
  • docs: 23.0.0 breaking changes, retired portable binaries & installer warning The highest-consequence half of the 23.0.0 catch-up, landed first. Breaking in 23.0.0 (qsvmcp only): qsvmcp no longer ships `get`, no longer accepts `dc:` cache-prefix inputs, and no longer reads cloud (s3://, gs://, az://) sources. Noted on Binary-Variants, MCP-Server and Get-and-Disk-Cache, each scoped so a `qsv`/`qsvdp` user doesn't think they lost cloud support. Stale since 22.0.1, not 23.0.0: the "portable" binaries (qsvp, qsvplite, qsvpdp, qsvpmcp) were dropped a day before the wiki's last update, so three pages still told users to switch to a binary no release ships. They were redundant — `-C target-cpu=native` had already been removed from the x86_64 publish targets, so the regular prebuilts are the portable ones. Rewritten as a retirement note with the measured evidence, and the SIGILL advice on Installation/Troubleshooting/Performance-Tuning now points at building from source instead. That retirement is also why the Windows Easy Installer breaks: v1.1.1 and earlier extract a hardcoded qsvp.exe and report success anyway. Added the upgrade-to-v1.1.2-first warning to Installation and Troubleshooting. Also: the `--output`-truncates-its-own-input data-loss bug (fixed in 23.0.0) as a Troubleshooting entry, and `get cache-list`'s new age/staleness reporting with the TTL-driven (not policy-driven) definition of "stale". Counts re-derived by running `--list` on each freshly built binary rather than incremented: qsv 78->79, qsvpy 79->80, qsvmcp 69 (68 + readstat, `get` gone), qsvlite 51, qsvdp 44->43 (was wrong), MCP skills 55->56. Dropped the `qsv_fetch`/`qsv_fetchpost` tool bullet from MCP-Server — those skills do not exist and fetch is not in the qsvmcp binary at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Sep 12, 2026
  • docs: wiki drift audit — fix stale counts, flags, viz/MapLibre, dep forks Audited all wiki pages against current qsv source (v21.1.0): - Command counts, version strings, stats/vocab/token counts corrected - Removed non-existent flags (diff --select, fetch --redis-cache-conn, QSV_TMPDIR, QSV_MEMORY_HEADROOM_MB) and fictional apply ops (cast, wkt_point) / luau helpers; fixed luau col[] access + positional script arg - viz page migrated Mapbox->MapLibre (DensityMap, token-free --style list, --dict-info/--dataset-pid/--cluster/--heatmap-density) - Security page dep-fork updates (csvlens YS-L, calamine/plotly/spreadsheet-ods), geoconvert formats, validate keywords, Integrations release URL - Recipe fixes: template arg order, Tieske date TAT, joinp --asof, --language Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

    @jqnatividad jqnatividad committed Jul 6, 2026
  • docs(wiki): update for qsv 20.1.0 — "Synthetic Data" release - AI & Documentation: add synthesize section; expand describegpt with Content Types (47-token vocab), --two-pass cross-field refinement, deterministic unique_id tag, --markdown-template, lower-LLM-cost notes - Aggregation & Statistics: document auto-fallback to approximate modes on OOM (stats: --quantile/cardinality-method approx; frequency: --sketch-method frequent_items) with little-endian gating - Recipe-Larger-than-RAM: reframe explicit approx flags as auto-on-OOM (override-to-lock semantics) with stats cache mode-key note - SQL & Polars: pivotp --agg quantile@<p> / q@<p> with p95 example - Command Reference: add synthesize row in AI & Documentation table - Cookbook: new "Generate" section; new Recipe-Synthesize-Fake-Data.md with end-to-end describegpt --two-pass --infer-content-type → synthesize walkthrough, locale switching, --consistent-fakes variant, caveats - Sidebar: link the new Synthesize Fake Data recipe - Home: bump "70+ commands" → "73 commands across four binary variants"; add 20.1.0 highlight strip with deep-links - FAQ: refresh MSRV anchor 20.0.0 → 20.1.0 (Rust 1.95 unchanged) - Troubleshooting: link both 20.0.0 and 20.1.0 changelogs in excel section Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

    @jqnatividad jqnatividad committed May 18, 2026
  • wiki: Phase E complete - all 6 polish pages Troubleshooting: organized by category - install/launch (SIGILL, macOS Gatekeeper, Windows PATH, musl Luau), encoding/format (UTF-8, wrong delimiter, ragged rows, BOM), index/cache (stale, missing JSONL, Polars schema drift), memory (OOM, ext-* alternatives, approx algorithms), specific commands (stats date-whitelist, search case sensitivity, fetch rate limits, excel edge cases). Plus diagnostics with --version, --envlist, QSV_LOG_LEVEL, and where to report bugs. FAQ: short answers to the most-asked questions. qsv vs pandas/DuckDB/csvkit/ miller/xsv, Windows/ARM/IBM platform support, streaming vs in-memory, what an index/stats-cache/automagical command is, how to update, what features my qsv has, AI/LLM privacy (no data leaves your machine unless you explicitly send it), bug-reporting flow. Comparison: head-to-head with xsv (qsvlite is the drop-in), csvkit (10x faster), Miller (overlap, both excellent), DuckDB (complement, not replace - to-parquet hand-off), pandas (qsv as the preprocessing layer), Visidata (qsv lens is the closest equivalent), awk/sed (qsv understands CSV quoting). Ends with a "which tool when" cheat sheet. Glossary: 30+ terms - antimode, asof join, automagical, BLAKE3, cardinality, CKAN, DataSketches, dyncols, lazy frame, lookup table, MAD, MCP, MiniJinja, neuro-symbolic, non-equi join, Polars, Polars schema, Pragmastat, pseudonymization, reservoir sampling, RFC 4180, sortiness, stats cache, streaming, TOON, t-digest, WKT, XSD. External-Resources: try-it-online, 100.dathere.com lessons, 5 conference talks, "Have we achieved ACI?" 3-part blog series, canonical /docs/ links, live benchmark dashboard, Discussions categories, datHere ecosystem (qsv pro, qsv-recipes, qsv-lookup-tables, qsv-cookiecutter, DataPusher+), 4 contributed notebooks, packaging across 10 distros, underlying library credits, Zenodo DOI + DeepWiki + FOSSA, sponsor link. Contributing-to-the-Wiki: clone+commit+push workflow, page conventions (H1, tier badge, See also), templates for command-reference and recipe pages, when-to-update matrix, 10-item verification checklist, commit- message convention, don't-duplicate-/docs/ guidance, anchor dataset conventions, slug rules, asset handling, asciinema casts. All 49 wiki pages now have substantive content. Plan complete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

    @jqnatividad jqnatividad committed May 13, 2026
  • wiki: add stubs for Phase B/C/D/E pages so sidebar links resolve Adds 39 placeholder pages so every sidebar entry resolves to real content rather than a 404. Each stub declares its tier, the phase it will be filled in, and a one-paragraph preview of what's coming. They link back to Home / Getting-Started / Command-Reference / Cookbook for navigation. Pages added: - Phase B (Command Reference, 13): Command-Reference, Selection-and- Inspection, Transform-and-Reshape, Aggregation-and-Statistics, Joins- and-Set-Ops, SQL-and-Polars, Validation-and-Schema, Conversion-and-IO, Geospatial, HTTP-and-Web, Scripting-Luau-Python, Indexing-Compression- Diff, AI-and-Documentation - Phase C (Cookbook recipes, 12): Recipe-Inspect-Unknown-CSV, Recipe- Clean-and-Normalize, Recipe-Geographic-Enrichment, Recipe-Date- Enrichment, Recipe-CKAN-Integration, Recipe-JSON-Schema-Validate, Recipe-Build-a-Data-Pipeline, Recipe-Stats-to-Insights, Recipe-Fetch- and-Cache, Recipe-Larger-than-RAM, Recipe-Diff-and-Audit, Recipe-Multi- Table-Joins - Phase D (Tuning + ecosystem, 8): Performance-Tuning, Environment- Variables, Stats-Cache-and-Caching, Lookup-Tables, Claude-Cowork-Plugin, MCP-Server, qsv-pro-Spotlight, Integrations - Phase E (Polish, 6): Troubleshooting, FAQ, Comparison, Glossary, External-Resources, Contributing-to-the-Wiki Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

    @jqnatividad jqnatividad committed May 13, 2026