DEV-1712 Stage 8: order-only hidden slots + plan-time validations (order-only refs, CMA trim, partition_by) - #274
Open
Open
Conversation
Implements the DEV-1703 Stage 8 slice: ORDER BY refs not declared as dimensions/measures, hidden cross-model-aggregate projection trim, and the rank-family partition_by grain guard. Order-only refs (Law 2), classified at plan time in stage_planner.plan_query: - aggregate (local or cross-model): materialised hidden, sorted on, stripped from the result / StageSchema (never rejected). - local row column: split emission `orders.<col>` in the raw-rows case; a grouped query raises ValueError (not in GROUP BY). - joined row column: UnresolvableOrderColumnError. - inline transform/composite (change(...) etc.): ValueError -> declare as a measure. Full support deferred to DEV-1733 (worktree + strict-xfail future tests). Composite arithmetic is unexpressible via OrderItem (Pydantic). The generator's old NotImplementedError becomes a defensive assertion. Hidden cross-model aggregate trim (DEV-1495 bug 2): an order-only CMA is hidden=True/public_alias=None and skips the combined projection, with a CTE-qualified ORDER BY term -- gated on `not transform_layers` so a hidden CMA feeding a cumsum step stays projected for the step CTE. The malformed `._sum` alias half was already fixed by Stage 9's naming module. partition_by grain guard (DEV-1497): a pre-intern pass (planning.rewrite_rank_partition_keys, mirroring lower_sugar_transforms) validates each rank-family partition_by key is a query dimension/time-dim by exact ValueKey membership, and rewrites a time-dim source column to its TimeTruncKey bucket (no more raw-timestamp grain widening / duplicate alias); a non-dimension raises the restored legacy message. DEV-1645 Flavor-A ORDER BY: ported main's lost legacy _OrderColRef / _order_split_sql / _resolve_order_column fix (split-not-composite + UnresolvableOrderColumnError) into the legacy generator so the 7 pinned unit tests + 1 integration pin promote. tests/parity_xfails.py is now empty (the DEV-1485 Stage 11 end-state). Deliberate typed-pipeline divergence: a grouped raw-row order is rejected at plan time (HTTP 400) rather than emitting SQL the database rejects at execution. Docs: queries.md ORDER BY semantics, formulas.md partition_by note, slayer-query skill, DECISIONS.md entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughChangesOrder planning and SQL generation
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Sequence Diagram(s)sequenceDiagram
participant Query
participant StagePlanner
participant SQLGenerator
participant SQLResult
Query->>StagePlanner: submit order and partition_by targets
StagePlanner->>StagePlanner: validate and classify targets
StagePlanner->>SQLGenerator: provide planned order references
SQLGenerator->>SQLResult: render aliases, qualified columns, and CTE references
SQLResult-->>Query: return ordered projection and rows
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (3)
slayer/engine/planning.py (1)
405-431: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMake
rewrite_fnkeyword-only.
rewrite_rank_partition_keystakes two parameters and both are passed positionally atslayer/engine/stage_planner.pyline 605. The coding guidelines require keyword arguments for functions with more than one parameter. A keyword-only callback also documents intent at every recursive call site.Also consider annotating the callback type so the contract described in the docstring is machine-checkable.
♻️ Proposed signature change
-def rewrite_rank_partition_keys(key: ValueKey, rewrite_fn) -> ValueKey: +def rewrite_rank_partition_keys( + key: ValueKey, *, rewrite_fn: Callable[[TransformKey], frozenset], +) -> ValueKey:Every recursive call and the
stage_planner._rwcall site then passrewrite_fn=....🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@slayer/engine/planning.py` around lines 405 - 431, Update rewrite_rank_partition_keys so rewrite_fn is a keyword-only parameter, and add a callable type annotation matching its TransformKey-to-frozenset contract. Change every recursive invocation and the stage_planner _rw call site to pass rewrite_fn by keyword, preserving existing behavior.Source: Coding guidelines
slayer/sql/generator.py (1)
6258-6273: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCompute the trimmed-plan set once.
The trim predicate
plan.hidden and not planned_query.transform_layersappears twice: at line 6140 (trim_hidden, which drops the aggregate fromcombined_parts) and again at line 6266 (which decides whether the ORDER BY term is CTE-qualified). The two must always agree. If they diverge, the projection drops the column while the ORDER BY still names the bare alias, and the emitted SQL is invalid.Note also that this block runs after the
return self._render_cross_model_transform_chain(...)at line 6238, sonot planned_query.transform_layersis always true here.Build one set of trimmed plan ids in the projection loop and reuse it.
♻️ Proposed refactor
+ trimmed_cma_plan_ids: Set[str] = set() for plan in planned_query.cross_model_aggregate_plans: ... trim_hidden = plan.hidden and not planned_query.transform_layers + if trim_hidden: + trimmed_cma_plan_ids.add(plan.aggregate_slot_id)hidden_cma_order_ref: Dict[str, str] = {} for plan in planned_query.cross_model_aggregate_plans: - # Only CMAs actually trimmed from the projection (hidden + no - # transform chain) need the CTE-qualified ORDER BY reference. - if not (plan.hidden and not planned_query.transform_layers): + # Only CMAs actually trimmed from the projection need the + # CTE-qualified ORDER BY reference. + if plan.aggregate_slot_id not in trimmed_cma_plan_ids: continue🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@slayer/sql/generator.py` around lines 6258 - 6273, Compute a set of trimmed cross-model aggregate plan IDs in the projection loop using the existing trim predicate, then reuse that set when building hidden_cma_order_ref instead of repeating the predicate. Since this block runs after the transform-chain return, retain the projection loop’s established conditions and use the shared IDs to CTE-qualify exactly the aggregates removed from combined_parts.tests/test_dev1712_order_only_hidden_slots.py (1)
298-317: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAssert the error text for the joined-column rejection.
Both tests assert only the exception type.
UnresolvableOrderColumnErrorembeds the column and the qualifier in its message, and the planner builds those two fields atslayer/engine/stage_planner.pyline 677. A type-only assertion cannot detect a malformed message there — see the separate comment on that line, where the qualifier is currently duplicated into the rendered text.Add a message assertion so the user-facing text is pinned.
💚 Proposed test tightening
- with pytest.raises(UnresolvableOrderColumnError): - await _sql(engine, query) + with pytest.raises(UnresolvableOrderColumnError) as ei: + await _sql(engine, query) + msg = str(ei.value) + assert "'customers.region'" in msg, f"malformed qualifier/column: {msg}"Apply the same change to
test_joined_row_column_grouped_raises.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_dev1712_order_only_hidden_slots.py` around lines 298 - 317, Update both test_joined_row_column_ungrouped_raises and test_joined_row_column_grouped_raises to assert the exact user-facing message of UnresolvableOrderColumnError, including the joined column and qualifier, while preserving the existing exception-type assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@slayer/engine/stage_planner.py`:
- Around line 673-677: Update the UnresolvableOrderColumnError construction in
the shown partition-key branch to pass the bare leaf column as column and the
full joined row-key path as qualifier, avoiding duplicated qualifiers in the
formatted message. Prefer deriving both values directly from the ColumnKey,
ColumnSqlKey, and TimeTruncKey structure via a small helper adjacent to
_row_key_path rather than splitting _partition_key_display output.
- Around line 580-594: Update the time-dimension lookup near `_td_by_source` to
detect source columns mapped to multiple `TimeTruncKey` granularities, storing
those columns in `_td_ambiguous_sources` while preserving unambiguous mappings.
In `_validate_partition_keys`, check ambiguous source columns before resolving
through `_td_by_source` and raise an error naming the column and competing
granularities, directing the user to specify the qualified time bucket.
In `@slayer/sql/generator.py`:
- Around line 11018-11045: Update the hidden local ColumnSqlKey handling in the
split-emission path so derived SQL expressions are expanded and their join paths
registered before generating the order expression. Ensure local expressions such
as customers.region bind the required relation instead of producing an unbound
ORDER BY reference, while preserving existing ColumnKey and TimeTruncKey
behavior. Add an ungrouped order-only regression test covering this case.
In `@tests/test_dev1712_order_only_hidden_slots.py`:
- Around line 691-697: Update
test_composite_order_string_rejected_at_construction to expect
pydantic.ValidationError instead of the broad Exception type, and add the
ValidationError import with the module’s existing top-level imports. Keep the
test’s current construction and boundary assertion unchanged.
---
Nitpick comments:
In `@slayer/engine/planning.py`:
- Around line 405-431: Update rewrite_rank_partition_keys so rewrite_fn is a
keyword-only parameter, and add a callable type annotation matching its
TransformKey-to-frozenset contract. Change every recursive invocation and the
stage_planner _rw call site to pass rewrite_fn by keyword, preserving existing
behavior.
In `@slayer/sql/generator.py`:
- Around line 6258-6273: Compute a set of trimmed cross-model aggregate plan IDs
in the projection loop using the existing trim predicate, then reuse that set
when building hidden_cma_order_ref instead of repeating the predicate. Since
this block runs after the transform-chain return, retain the projection loop’s
established conditions and use the shared IDs to CTE-qualify exactly the
aggregates removed from combined_parts.
In `@tests/test_dev1712_order_only_hidden_slots.py`:
- Around line 298-317: Update both test_joined_row_column_ungrouped_raises and
test_joined_row_column_grouped_raises to assert the exact user-facing message of
UnresolvableOrderColumnError, including the joined column and qualifier, while
preserving the existing exception-type assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 402dea2a-27a0-4970-b673-bff8d384a3f2
📒 Files selected for processing (12)
.claude/skills/slayer-query.mdDECISIONS.mddocs/concepts/formulas.mddocs/concepts/queries.mdslayer/engine/planning.pyslayer/engine/stage_planner.pyslayer/sql/generator.pytests/parity_xfails.pytests/test_dev1712_order_only_hidden_slots.pytests/test_nested_dag_cross_stage_refs.pytests/test_projection_trim.pytests/test_sql_generator.py
💤 Files with no reviewable changes (1)
- tests/test_projection_trim.py
Comment on lines
+580
to
+594
| _td_by_source = { | ||
| dm.bound.value_key.column: dm.bound.value_key | ||
| for dm in _td_dms | ||
| if isinstance(dm.bound.value_key, TimeTruncKey) | ||
| } | ||
| _td_key_set = set(_td_by_source.values()) | ||
| _available_dims = [dm.declared_name for dm in (*_dim_dms, *_td_dms)] | ||
|
|
||
| def _validate_partition_keys(tk: TransformKey) -> frozenset: | ||
| new_pks = [] | ||
| for pk in tk.partition_keys: | ||
| if pk in _dim_key_set or pk in _td_key_set: | ||
| new_pks.append(pk) # already a query dim / td bucket | ||
| elif pk in _td_by_source: | ||
| new_pks.append(_td_by_source[pk]) # td source col -> bucket |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Two time dimensions over one source column collapse in _td_by_source.
_td_by_source is keyed by the TimeTruncKey.column. TimeTruncKey treats the same column at different granularities as distinct slots (see the class docstring in slayer/core/keys.py). A query with time_dimensions=[{created_at, month}, {created_at, day}] therefore produces two entries with the same key, and the later one overwrites the earlier one.
rank(amount:sum, partition_by=created_at) then partitions by whichever bucket happens to be last in declared_measures order. The choice is silent and order-dependent.
Reject the ambiguous case instead, and point the user at the qualified bucket they mean.
🛡️ Proposed guard
- _td_by_source = {
- dm.bound.value_key.column: dm.bound.value_key
- for dm in _td_dms
- if isinstance(dm.bound.value_key, TimeTruncKey)
- }
+ _td_by_source: dict = {}
+ _td_ambiguous_sources: set = set()
+ for dm in _td_dms:
+ vk = dm.bound.value_key
+ if not isinstance(vk, TimeTruncKey):
+ continue
+ if vk.column in _td_by_source:
+ _td_ambiguous_sources.add(vk.column)
+ _td_by_source[vk.column] = vkThen in _validate_partition_keys, raise when pk in _td_ambiguous_sources with a message that names the column and the competing granularities.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@slayer/engine/stage_planner.py` around lines 580 - 594, Update the
time-dimension lookup near `_td_by_source` to detect source columns mapped to
multiple `TimeTruncKey` granularities, storing those columns in
`_td_ambiguous_sources` while preserving unambiguous mappings. In
`_validate_partition_keys`, check ambiguous source columns before resolving
through `_td_by_source` and raise an error naming the column and competing
granularities, directing the user to specify the qualified time bucket.
Comment on lines
+673
to
+677
| if isinstance(okey, (ColumnKey, ColumnSqlKey, TimeTruncKey)): | ||
| disp = _partition_key_display(okey) | ||
| path = _row_key_path(okey) | ||
| if path: | ||
| raise UnresolvableOrderColumnError(column=disp, qualifier=path[0]) |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
The UnresolvableOrderColumnError message repeats the join qualifier.
UnresolvableOrderColumnError.__init__ formats the message as f"ORDER BY column '{qualifier}.{column}' ..." (see slayer/core/errors.py lines 516-519). It expects column to be the bare leaf. Here disp is the full dotted path from _partition_key_display, and qualifier is path[0].
For order=[{"column": "customers.region"}] the message reads ORDER BY column 'customers.customers.region' cannot be resolved. For a multi-hop path it reads 'customers.customers.regions.name'.
Pass the leaf as column and the joined path as qualifier.
🐛 Proposed fix
if isinstance(okey, (ColumnKey, ColumnSqlKey, TimeTruncKey)):
disp = _partition_key_display(okey)
path = _row_key_path(okey)
if path:
- raise UnresolvableOrderColumnError(column=disp, qualifier=path[0])
+ leaf = disp.split(".")[-1]
+ raise UnresolvableOrderColumnError(
+ column=leaf, qualifier=".".join(path),
+ )A cleaner variant adds a small helper next to _row_key_path that returns (path, leaf) for the three row key shapes, so the leaf is read from the key instead of re-split from the display string.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if isinstance(okey, (ColumnKey, ColumnSqlKey, TimeTruncKey)): | |
| disp = _partition_key_display(okey) | |
| path = _row_key_path(okey) | |
| if path: | |
| raise UnresolvableOrderColumnError(column=disp, qualifier=path[0]) | |
| if isinstance(okey, (ColumnKey, ColumnSqlKey, TimeTruncKey)): | |
| disp = _partition_key_display(okey) | |
| path = _row_key_path(okey) | |
| if path: | |
| leaf = disp.split(".")[-1] | |
| raise UnresolvableOrderColumnError( | |
| column=leaf, qualifier=".".join(path), | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@slayer/engine/stage_planner.py` around lines 673 - 677, Update the
UnresolvableOrderColumnError construction in the shown partition-key branch to
pass the bare leaf column as column and the full joined row-key path as
qualifier, avoiding duplicated qualifiers in the formatted message. Prefer
deriving both values directly from the ColumnKey, ColumnSqlKey, and TimeTruncKey
structure via a small helper adjacent to _row_key_path rather than splitting
_partition_key_display output.
Comment on lines
+11018
to
+11045
| # DEV-1712 (Law 2, split emission): a hidden LOCAL ROW column | ||
| # ordered in an UNGROUPED query. The plan-time order validation | ||
| # (``plan_query``) guarantees the only hidden ROW slot that | ||
| # reaches here is a local (empty-path) column in a query with no | ||
| # GROUP BY — grouped row columns and joined columns are rejected | ||
| # up front, aggregates take the branch above. Emit a SPLIT | ||
| # ``<relation>.<column>`` reference (mixed-case-aware) against | ||
| # the base FROM scope, identical to how the column would render | ||
| # if it were a projected dimension. | ||
| key = slot.key | ||
| if source_model is not None and isinstance( | ||
| key, (ColumnKey, ColumnSqlKey, TimeTruncKey) | ||
| ): | ||
| row_key = key.column if isinstance(key, TimeTruncKey) else key | ||
| if isinstance(row_key, ColumnKey): | ||
| path, leaf = row_key.path, row_key.leaf | ||
| else: # ColumnSqlKey | ||
| path, leaf = row_key.path, row_key.column_name | ||
| if not path: | ||
| order_col = self._joined_or_local_dim_expr( | ||
| path=(), leaf=leaf, source_model=source_model, | ||
| source_relation=source_relation, bundle=bundle, | ||
| ) | ||
| ascending = order_entry.direction == "asc" | ||
| select = select.order_by( | ||
| self._ordered(order_col, ascending=ascending), | ||
| ) | ||
| continue |
Contributor
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Find order-only tests that use a derived column whose Column.sql crosses a join.
set -euo pipefail
# 1. Locate the Stage 8 order tests and the sibling projection-side expansion helper.
fd -t f 'test_dev1712_order_only_hidden_slots.py' tests
# 2. Look for order items over derived columns in the order-only suites.
rg -n -C 5 'distinct_dimension_values=False' tests/test_dev1712_order_only_hidden_slots.py
# 3. Show every model fixture column that sets a dotted Column.sql (a crossing derived column).
rg -nP -C 3 'Column\(\s*name=[^)]*sql="[A-Za-z_]+\.[A-Za-z_]+"' tests
# 4. Confirm the projection path expands derived dims while the order path does not.
ast-grep run --pattern $'def _dim_column_expr_from_planned($$$) {
$$$
}' --lang python slayer/sql/generator.py || \
rg -n -A 15 'def _dim_column_expr_from_planned' slayer/sql/generator.pyRepository: MotleyAI/slayer
Length of output: 36502
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- generator order branch ---'
sed -n '10940,11070p' slayer/sql/generator.py
printf '%s\n' '--- derived-dimension and planned-expression helpers ---'
sed -n '9480,9655p' slayer/sql/generator.py
printf '%s\n' '--- order classification and row-key path helpers ---'
rg -n -C 12 '_row_key_path|ColumnSqlKey|_expand_derived_row_dims|plan_query|hidden.*ROW|LOCAL ROW' slayer/sql/generator.py
printf '%s\n' '--- relevant order-only tests and fixtures ---'
sed -n '1,380p' tests/test_dev1712_order_only_hidden_slots.py
printf '%s\n' '--- related derived-order tests ---'
rg -n -C 12 'order=.*(derived|cust_|region_|ColumnRef)|OrderItem\(column=.*(derived|cust_|region_)|ColumnSqlKey' testsRepository: MotleyAI/slayer
Length of output: 50372
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import re
path = Path("slayer/sql/generator.py")
lines = path.read_text().splitlines()
names = {
"_dim_column_expr_from_planned",
"_joined_or_local_dim_expr",
"_expand_derived_row_dims",
"_row_key_path",
"plan_query",
}
for name in names:
hits = [i for i, line in enumerate(lines) if re.search(rf"^\s*(async\s+)?def\s+{re.escape(name)}\b", line)]
for start in hits:
indent = len(lines[start]) - len(lines[start].lstrip())
end = start + 1
while end < len(lines):
text = lines[end]
stripped = text.lstrip()
current_indent = len(text) - len(stripped)
if stripped and current_indent <= indent and re.match(r"^(async\s+)?def\s+", stripped):
break
end += 1
print(f"\n--- {name}: lines {start + 1}-{end} ---")
print("\n".join(f"{i + 1:5}: {lines[i]}" for i in range(start, min(end, start + 180))))
PY
printf '%s\n' '--- exact order-emission context ---'
sed -n '10980,11065p' slayer/sql/generator.py
printf '%s\n' '--- targeted order/derived references in tests ---'
rg -n -C 8 \
'ColumnSqlKey|derived.*order|order.*derived|cust_signup|region_payment|customer_score|deep_pop|OrderItem\(column=ColumnRef\(name="[^"]+"\)' \
tests/test_dev1712_order_only_hidden_slots.py \
tests/test_sql_generator.py \
tests/test_dev1476_first_last_explicit_time.py \
tests/test_dev1474_time_shift_cross_model_partition.pyRepository: MotleyAI/slayer
Length of output: 50371
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- generator symbols ---'
rg -n \
'_row_key_path|_order|UnresolvableOrderColumnError|ColumnSqlKey|hidden|GROUP BY|group_by' \
slayer/sql/generator.py | head -n 240
printf '%s\n' '--- planner symbols across source ---'
rg -n \
'_row_key_path|UnresolvableOrderColumnError|ColumnSqlKey|order.*path|path.*order|hidden.*ROW|ROW.*hidden' \
slayer --glob '*.py' | head -n 320
printf '%s\n' '--- order-only fixture definitions ---'
sed -n '1,205p' tests/test_dev1712_order_only_hidden_slots.py
printf '%s\n' '--- order-only tests after joined-column cases ---'
sed -n '295,390p' tests/test_dev1712_order_only_hidden_slots.py
printf '%s\n' '--- all direct order references to derived fixture names ---'
rg -n -C 6 \
'order=\[OrderItem\(column=ColumnRef\(name="(cust_signup|customer_score|deep_pop|region_payment|cust_balance|region_weight|li_weight|b_display|net_amount|signup_at_alias)' \
tests --glob '*.py'Repository: MotleyAI/slayer
Length of output: 50371
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- stage-planner row-path and order validation ---'
sed -n '150,215p' slayer/engine/stage_planner.py
sed -n '620,705p' slayer/engine/stage_planner.py
printf '%s\n' '--- binding of local and joined derived columns ---'
sed -n '500,625p' slayer/engine/binding.py
sed -n '680,715p' slayer/engine/binding.py
printf '%s\n' '--- base-select scope registration and derived expansion order ---'
sed -n '4200,4420p' slayer/sql/generator.py
printf '%s\n' '--- narrow test-name inventory for order and derived columns ---'
rg -n '^(\s+async\s+def|\s+def)\s+.*(order|derived|ColumnSqlKey|source_sql|cust_signup|deep_pop|region_payment)' \
tests/test_dev1712_order_only_hidden_slots.py \
tests/test_sql_generator.py \
tests/test_dev1476_first_last_explicit_time.py \
tests/test_dev1708_stage4_cte_scope.pyRepository: MotleyAI/slayer
Length of output: 46876
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- construction of base_render_order and hidden order slots ---'
sed -n '3235,3385p' slayer/sql/generator.py
sed -n '3790,4005p' slayer/sql/generator.py
printf '%s\n' '--- current hidden-row regression coverage ---'
sed -n '7145,7195p' tests/test_sql_generator.py
sed -n '200,325p' tests/test_dev1712_order_only_hidden_slots.py
printf '%s\n' '--- derived expansion and join collection contracts ---'
sed -n '7750,7845p' slayer/sql/generator.py
sed -n '9425,9520p' slayer/sql/generator.py
printf '%s\n' '--- static behavioral verifier ---'
python3 - <<'PY'
from pathlib import Path
import re
stage = Path("slayer/engine/stage_planner.py").read_text()
binding = Path("slayer/engine/binding.py").read_text()
generator = Path("slayer/sql/generator.py").read_text()
tests = Path("tests/test_dev1712_order_only_hidden_slots.py").read_text()
checks = {
"local derived refs become ColumnSqlKey(path=())":
"return ColumnSqlKey(path=(), model=model.name, column_name=col.name)" in binding,
"row-path helper returns the key path":
'return tuple(getattr(key, "path", ()))' in stage,
"empty derived path is accepted by order validation":
"if path:\n raise UnresolvableOrderColumnError" in stage
and "continue # ungrouped local row column -> split emission" in stage,
"derived expansion registers SQL-crossed joins":
"_expand_derived_column_sql(" in generator
and "_joined_paths_in_sql(" in generator
and "scope.join_paths.add(path)" in generator,
"order split uses direct planned resolver":
"return self._dim_column_expr_from_planned(" in generator
and "sql=col.sql, name=col.name, model_name=source_relation" in generator,
"order-only test file has no derived-column order target":
not bool(re.search(
r'order=\[OrderItem\(column=ColumnRef\(name="(?:[A-Za-z_]+)"',
tests
)) or "Column(name=" not in tests,
}
for name, result in checks.items():
print(f"{'PASS' if result else 'FAIL'}: {name}")
PYRepository: MotleyAI/slayer
Length of output: 34113
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
import ast
from pathlib import Path
source = Path("tests/test_dev1712_order_only_hidden_slots.py").read_text()
tree = ast.parse(source)
order_refs = []
for node in ast.walk(tree):
if isinstance(node, ast.Call) and isinstance(node.func, ast.Name):
if node.func.id != "OrderItem":
continue
for kw in node.keywords:
if kw.arg != "column":
continue
value = kw.value
if (
isinstance(value, ast.Call)
and isinstance(value.func, ast.Name)
and value.func.id == "ColumnRef"
and value.args
and isinstance(value.args[0], ast.Constant)
):
order_refs.append(value.args[0].value)
print("order-only ColumnRef targets:", order_refs)
print(
"derived order regression present:",
any(name in {"cust_signup", "customer_score", "deep_pop", "region_payment"}
for name in order_refs),
)
PY
poetry run ruff check slayer/ tests/Repository: MotleyAI/slayer
Length of output: 271
Expand joins for hidden local ColumnSqlKey order targets.
A local derived column such as sql="customers.region" has path=() and passes plan validation, but hidden row targets are excluded from base_render_order. The split-emission path then calls _resolve_sql without registering customers, producing an unbound ORDER BY reference. Expand the derived expression and register its join paths before emission. Add an ungrouped order-only regression test.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@slayer/sql/generator.py` around lines 11018 - 11045, Update the hidden local
ColumnSqlKey handling in the split-emission path so derived SQL expressions are
expanded and their join paths registered before generating the order expression.
Ensure local expressions such as customers.region bind the required relation
instead of producing an unbound ORDER BY reference, while preserving existing
ColumnKey and TimeTruncKey behavior. Add an ungrouped order-only regression test
covering this case.
…t cleanups Codex (correctness): a joined ORDER BY ref whose leaf collides with a local declared dimension/measure (order=owners.status when `status` is a local dim) silently bound to the local column via the bare-leaf shortcut in the order-binding loop, sorting by the wrong field. Guard the bare-name shortcuts (stage_planner.plan_query) so they apply only to unqualified refs or refs qualified with the host model; a foreign-qualified ref falls through to the dotted/bind_expr path, where a truly-joined order ref is then rejected by the plan-time order validation. Regression test added. Sonar S3776 (planning.py): flattened rewrite_rank_partition_keys (cognitive complexity 26 -> under threshold) using a local recursion helper + ternary rebuilds; also made rewrite_fn keyword-only + type-annotated (CodeRabbit). Sonar S5685 (stage_planner.py): replaced the walrus-in-argument-list in the bound_filters rebuild with an explicit loop. Sonar S9073 (tests): split composite `assert a and b` into separate asserts. Sonar S5958 (tests): pytest.raises(Exception) -> pytest.raises(ValidationError). Full non-integration suite green (8971 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DEV-1703 Stage 8. Closes DEV-1712; closes DEV-1472, DEV-1495 bug 2, DEV-1497; promotes the Stage-8 DEV-1645 Flavor-A ORDER-BY pins.
What lands
Order-only hidden slots (Law 2) — an ORDER BY ref not declared as a dimension/measure is classified at plan time (
stage_planner.plan_query, right after_bucket_slots):StageSchema(never rejected)distinct_dimension_values=false)ORDER BY orders.<col>(mixed-case-aware)ValueError(HTTP 400) — not inGROUP BY; add to dims or order by an aggregateUnresolvableOrderColumnError(HTTP 400)change(amount:sum))ValueError— declare it as a measure. Deferred to DEV-1733 (worktree + strict-xfail future tests). Composite arithmetic is unexpressible viaOrderItem(Pydantic rejects at construction).The grouping predicate is planner-semantic (
agg_slotspresent, or dims/tds with dedup), so a hidden order aggregate that induces grouping correctly forces a row column in the same query to be rejected. The generator's oldNotImplementedErrorbecomes a defensive assertion — the plan-time pass guarantees only the split shape reaches it.Hidden cross-model-aggregate trim (DEV-1495 bug 2) — an order-only CMA is
hidden=True/public_alias=Noneand skips the combined projection, with a CTE-qualified ORDER BY term. Gated onnot transform_layersso a hidden CMA feeding acumsum(...)step stays projected for the step CTE (the transform outer-wrap trims there). The malformedorders.customers._sumalias half was already fixed by Stage 9's naming module.partition_by grain guard (DEV-1497) — a pre-intern pass (
planning.rewrite_rank_partition_keys, mirroringlower_sugar_transforms' identity-preserving rebuild) validates every rank-familypartition_bykey resolves to a query dimension/time-dimension by exact ValueKey membership (the typed binder resolves it to a ValueKey before validation, so legacy string-match ambiguity can't arise). A time-dim source column is rewritten to itsTimeTruncKeybucket (kills the raw-timestamp grain widening + duplicate alias); a non-dimension raises the restored legacy message (transform + column + available dims).DEV-1645 Flavor-A ORDER BY — ported main's lost legacy
_OrderColRef/_order_split_sql/_resolve_order_columnfix (split-not-composite +UnresolvableOrderColumnError) into the legacy generator across all three order-emission sites, so the 7 pinned unit tests + 1 integration pin promote.tests/parity_xfails.pyis now empty — the DEV-1485 (Stage 11) end-state.Deliberate divergence from main: the typed pipeline rejects a grouped raw-row order at plan time (HTTP 400) instead of emitting SQL the database rejects at execution.
Tests
New
tests/test_dev1712_order_only_hidden_slots.py(the contract table incl. execution/response-strip, diamond CMA, partition matrix, transform deferral + DEV-1733 xfail). Un-pinned: 7 Flavor-A unit + 1 Postgres integration + the DEV-1495/DEV-1497 xfails. The skipped cross-stage test became 3 passing variants. Full non-integration suite green (8970 passed, 0 failed, 31 xfailed);ruffclean.Out of scope
Result-key naming (Stage 9, already landed); windowed measures (Stage 10); inline transform/composite order targets (DEV-1733).
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes
Documentation