Skip to content

Support role-qualified dimension attributes in /djsql #2480

Description

@shangyian

In /djsql, using a role-qualified dimension attribute (<dimension>.<column>[<role>]) in WHERE, and in GROUP BY works only if it's not in the SELECT list. Putting it into SELECT fails with:

Only direct columns are allowed in DJ SQL queries, found: default.special_country_dim.name[user_birth_country]

Repro:

SELECT
  default.special_country_dim.name[user_birth_country],   -- rejected
  default.avg_user_age
FROM metrics
GROUP BY default.special_country_dim.name[user_birth_country]

The same metric and dimension pair work fine through the structured JSON endpoint:

await client.get(
    "/sql/default.avg_user_age",
    params={
        "dimensions": ["default.special_country_dim.name[user_birth_country]"],
        "filters": ["default.special_country_dim.name[user_birth_country] = 'United States'"],
    },
)
# 200

The role resolution itself is fine, so this is a limitation of the DJ SQL surface only.

Root cause

api/djsql.py requires every projection item to be an ast.Column, but a role-qualified attribute doesn't parse to a Column. The [role] suffix is indistinguishable from array/map subscripting to the grammar, so it parses to a Subscript:

>>> q = ("SELECT default.special_country_dim.name[user_birth_country], default.avg_user_age "
...      "FROM metrics GROUP BY default.special_country_dim.name[user_birth_country]")
>>> t = parse(q)

That accounts for the behavior here:

  • Projection fails the isinstance(col, ast.Column) guard.
  • GROUP BY also parses to Subscript, but is handled by str(exp), which renders back to default.special_country_dim.name[user_birth_country], exactly the string the structured API expects. So grouping by a roled attribute works on its own.
  • WHERE clause filters are carried through as strings and never reach this guard.

Suggested fix

The representation already exists on the AST, so this is a gap in the DJ SQL handler front end.

ast.Column has a role attribute on it, and parse_dimension_ref already parses the string form into a role-qualified DimensionRef.

What's missing is only that the grammar yields a Subscript for col[role], and nothing lowers that into a Column with role populated. GROUP BY, WHERE, and ORDER BY get away with it by stringifying and letting parse_dimension_ref sort it out downstream.

So the fix is to lower the Subscript form to a Column with role set, reusing the same role/name split parse_dimension_ref implements, rather than stringifying to match GROUP BY. That also removes the metric-vs-dimension hazard: the projection currently splits metrics from dimensions by testing col.identifier(False) not in dimensions, where dimensions was built with str(exp). Comparing structured references instead of strings makes that robust, whereas a stringify-only fix leaves it dependent on two code paths rendering the same text.

Role path separator

The role path separator isn't consistent across the codebase:

location separator
sql/parsing/ast.py self.role.split(" -> ")
construction/build_v2.py dimension_attr.role.split("->")
construction/build_v3/loaders.py joins with "->"

This should be normalized.

Tests

There is no coverage for role-qualification in the DJ SQL surface. Some cases worth adding to djsql_test.py:

  • single role: v3.customer.name[order]
  • role path: v3.date.month[customer->registration]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions