Skip to content

Move the sqlglot floor to 30.18.0 and delete what it makes dead - #142

Merged
jbylund merged 1 commit into
mainfrom
chore/sqlglot-30.18-floor
Sep 4, 2026
Merged

jbylund merged 1 commit into
mainfrom
chore/sqlglot-30.18-floor

Conversation

@jbylund

@jbylund jbylund commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Closes #109, closes #72. Strikes four rows off #49 and seven items off #58; #123 keeps only item 3.

sqlglot v30.18.0 shipped on 2026-09-03 and thirteen of the strict-xfail tripwires in test_sqlglot_workarounds.py XPASSed on the version bump — which is the whole point of that file, working for the first time at scale. Four were repairs in tables.py, so the repairs go.

Deleted

gone fixed by
_rewrite_not_in, with _anti_join and _null_count tobymao/sqlglot#8190, #8236
_rewrite_null_ordering #8158
the nested-OFFSET arm of _reject_silently_ignored, and the offset arm of rows_sliced_here #8219
the SetOperation arm of _take_result_order #8214
ENV.setdefault("DPIPE", ...) in examples/git_sql.py, plus a stale DPIPE comment in catalog.py #8146

Net 150 lines out of tables.py. No answer changes — every row a client gets is the same one, with less pg_mimic between the query and it. The one behaviour change is a widening: a nested OFFSET (SELECT ... FROM (SELECT ... OFFSET 2) AS page, or in a CTE or a UNION branch) is now answered rather than refused with 0A000, because the executor applies one wherever it appears.

_take_result_order survives, and #109 said why wrongly

This is the part worth reviewing. #109's checklist predicted the SELECT DISTINCT half of _take_result_order was a pg_mimic layering artifact — there to keep the IS NULL key _rewrite_null_ordering spliced in out of DISTINCT's dedup — and would therefore become deletable along with it.

It is a live upstream defect instead. With _rewrite_null_ordering gone, the executor still discards a SELECT DISTINCT's ORDER BY entirely and sorts ascending by the select list:

query sqlglot v30.18.0 postgres
SELECT DISTINCT a FROM t ORDER BY a DESC [(1,), (2,), (3,)] [(3,), (2,), (1,)]
SELECT DISTINCT a FROM t ORDER BY a DESC LIMIT 1 [(1,)] [(3,)]
SELECT DISTINCT b, a FROM t ORDER BY a sorted by (b, a) sorted by a

The LIMIT row is the severe one — wrong rows, not a wrong order — and the third has no DESC in it at all, so it is not a direction bug. Cause is in the planner: the DISTINCT aggregate is stacked above the Sort step and then re-sorts, because it groups by sorting. GROUP BY builds the same two steps the other way round and is correct. Reported as jbylund/sqlglot#54 with the plan dumps.

So the function stays, narrowed to the SELECT DISTINCT case, and gets its own tripwire instead of sharing one with the set operation that got fixed. That sharing is exactly why this went unnoticed: a single assertion over two causes cannot XPASS when only one is fixed — the #107 lesson, and the same shape as the btrim/ltrim split.

Tests

963 passed, 20 xfailed (from 961/33). Thirteen marks came off; the tests stay as plain assertions, which is what pins the floor.

New:

Renamed test_is_not_null_answers_a_boolean to test_an_operator_after_a_case_binds_to_the_whole_case, per #109: IS NOT NULL was never the defect. #8176 fixed an unparenthesized CASE in the generated Python, so any operator after a CASE bound wrong; IS NOT NULL was the vehicle the fuzzer arrived in.

Confirmed with tools/fuzz --target mimic --count 4000 against PostgreSQL 18: 142 divergence buckets on a C-locale database, against 193 on the old floor and the 144 #109 predicted for this change. (153 on an en_US.UTF-8 database — the extra 11 are collation findings, which is what the --locale=C note in tools/fuzz/README.md is for.)

ruff check and ruff format --check clean.

Not in scope

Three #49 rows are untouched and still needed: canonicalize_table_aliases, _flatten_parenthesized (a parenthesized top-level query's own row window is still ignored — (SELECT a FROM t) LIMIT 1 returns every row), and the TABLESAMPLE refusal. DISTINCT ON is still finished in Python and still refused inside a subquery, since the executor ignores it entirely.

Nothing else was deleted on the strength of #8190 alone, per #109's warning: jbylund/sqlglot#35 is unchanged by this release, so the unsound unnest_subqueries rewrites are still out there.

#123 item 3 (CAST(1 AS DECIMAL) / 0.5 raising on Decimal / float) is untouched — it is #58 item 14 reached through our own _cast, and nothing upstream addressed it.

The daily Upstream sqlglot job did not announce this release, and that is filed separately as #141: the job is fine and went red on exactly the right run, but the main leg is permanently red by design, so the workflow's conclusion has been failure every day and the one actionable transition was invisible.

Closes #109, closes #72. Strikes four rows off #49 and seven items off
#58; #123 keeps only item 3.

sqlglot v30.18.0 fixes thirteen of the executor bugs the strict-xfail
tripwires in tests/test_sqlglot_workarounds.py were watching for. Four of
them were repairs in tables.py, so the repairs go:

  - _rewrite_not_in, with _anti_join and _null_count -- 8190, 8236
  - _rewrite_null_ordering -- 8158
  - the nested-OFFSET arm of _reject_silently_ignored, and the offset arm
    of rows_sliced_here -- 8219. A nested OFFSET is answered now rather
    than refused.
  - the SetOperation arm of _take_result_order -- 8214
  - ENV.setdefault("DPIPE", ...) in examples/git_sql.py, and a stale
    DPIPE comment in catalog.py -- 8146

Net 150 lines out of tables.py. No answer changes; there is less pg_mimic
between the query and them.

_take_result_order and _sorted_rows survive, for a reason #109 recorded
wrongly. It expected the SELECT DISTINCT half to be a pg_mimic layering
artifact -- keeping the IS NULL key _rewrite_null_ordering added out of
DISTINCT's dedup -- and so to become deletable with it. It is a live
upstream defect instead: the executor discards a SELECT DISTINCT's ORDER
BY and sorts ascending by the select list, so a LIMIT returns the wrong
rows and `SELECT DISTINCT b, a FROM t ORDER BY a` is wrong with no DESC
in it. Reported as jbylund/sqlglot#54, and it now has its own tripwire
rather than sharing one with the set operation that got fixed -- a single
assertion over two causes cannot XPASS, which is the #107 lesson again.

Confirmed with tools/fuzz against PostgreSQL 18: 142 divergence buckets
on a C-locale database, against 193 on the old floor and the 144 #109
predicted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jbylund
jbylund merged commit b53be1a into main Sep 4, 2026
6 checks passed
@jbylund
jbylund deleted the chore/sqlglot-30.18-floor branch September 4, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

When the next sqlglot release lands: bump the floor and delete what it makes dead information_schema: || (string concatenation) still returns no rows

1 participant