Skip to content

Fix custom CSV dialect delimiter being silently overridden - #680

Open
agu2347 wants to merge 1 commit into
jazzband:masterfrom
agu2347:fix-csv-dialect-delimiter-override-issue-622
Open

Fix custom CSV dialect delimiter being silently overridden#680
agu2347 wants to merge 1 commit into
jazzband:masterfrom
agu2347:fix-csv-dialect-delimiter-override-issue-622

Conversation

@agu2347

@agu2347 agu2347 commented Aug 24, 2026

Copy link
Copy Markdown

Fixes #622.

Problem

As diagnosed by @claudep in the issue thread: CSVFormat.import_set() and export_stream_set() both unconditionally run

kwargs.setdefault('delimiter', cls.DEFAULT_DELIMITER)

before calling csv.reader(in_stream, **kwargs) / csv.writer(stream, **kwargs). Python's csv module documents that when both a dialect and explicit fmtparams (like delimiter) are given, the fmtparams take precedence over the dialect's own attributes. So if a caller passes a custom dialect (with, say, a colon delimiter) but no explicit delimiter, this setdefault still injects delimiter=',' into kwargs, which then silently overrides the dialect's delimiter back to the default comma:

>>> import csv, io, tablib
>>> class ColonDialect(csv.excel):
...     delimiter = ":"
>>> tablib.import_set(io.StringIO("a:b\n1:2\n"), format="csv", dialect=ColonDialect).headers
['a:b']   # wrong -- whole line read as one column

Fix

Only apply the default delimiter when no dialect was supplied, in both import_set() and export_stream_set():

if 'dialect' not in kwargs:
    kwargs.setdefault('delimiter', cls.DEFAULT_DELIMITER)

This leaves existing behavior unchanged for the common case (no dialect, optional explicit delimiter) and for explicit dialect + explicit delimiter (still overrides, per csv module semantics -- that's a deliberate, explicit request). It only changes the case where a dialect is passed without an explicit delimiter, which now correctly uses the dialect's own delimiter instead of silently falling back to comma.

Testing

Added CSVTests.test_csv_import_custom_dialect_delimiter_not_overridden, covering both directions (import and export) with a colon-delimited custom dialect. Confirmed it fails with the exact symptom from the issue (data parsed as a single column) when reverted, and passes with the fix.

Ran the full existing suite (pytest tests/): 188 passed (187 baseline + this new test), with the same 2 pre-existing ruff/ruff::format failures present identically before and after this change (unrelated formatting/lint issues elsewhere in tests/test_tablib.py, not touched by this PR).

CSVFormat.import_set() and export_stream_set() unconditionally did
kwargs.setdefault('delimiter', cls.DEFAULT_DELIMITER) before calling
csv.reader()/csv.writer(). Python's csv module lets explicit fmtparams
(like delimiter) override the corresponding attribute of a passed
dialect, so this silently reset any delimiter configured on a custom
dialect back to the comma default -- even though the caller never
asked for a delimiter override, only a dialect.

Only apply the default delimiter when no dialect was supplied.

Fixes jazzband#622
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simple csv file can't be parsed correctly because of Sniffer() parameters

1 participant