Context
Follow-up from review on PR #1485 (ODCS type: library quality metric support): comment.
DQX's type: library data-contract rule generator maps ODCS quality metrics (rowCount, nullValues, missingValues, invalidValues, duplicateValues) onto DQX aggregate checks where possible. Four of the eight ODCS threshold fields have no exact-fit DQX aggregate check today, so the generator falls back to a dataset-level sql_query check instead:
mustBeGreaterThan (metric > X)mustBeLessThan (metric < X)mustBeBetween (lo < metric < hi, both bounds exclusive per ODCS)mustNotBeBetween (metric <= lo OR metric >= hi, per ODCS's exclusive-bounds semantics)
Proposal
Add native DQX check functions for these, so the data-contract generator (and any other caller) can use a structured, exact-fit check instead of a raw SQL string:
is_aggr_greater_than(column, limit, aggr_type=..., ...)is_aggr_less_than(column, limit, aggr_type=..., ...)is_aggr_in_range(column, min_limit, max_limit, aggr_type=..., ...) (exclusive bounds)is_aggr_not_in_range(column, min_limit, max_limit, aggr_type=..., ...) (exclusive bounds)
These would mirror the existing is_aggr_equal / is_aggr_not_equal / is_aggr_not_less_than / is_aggr_not_greater_than family in check_funcs.py (same aggr_type, group_by, row_filter, tolerance parameters, etc.), and the DataContractRulesGenerator in contract_rules_generator.py could then replace its sql_query fallback for these four ODCS threshold fields with the new structured checks.
Out of scope for this issue
The generated sql_query fallback that's in place today is reviewed and correct — this is a nice-to-have follow-up to make the generated rules more structured, not a bug fix.
Context
Follow-up from review on PR #1485 (ODCS
type: libraryquality metric support): comment.DQX's
type: librarydata-contract rule generator maps ODCS quality metrics (rowCount,nullValues,missingValues,invalidValues,duplicateValues) onto DQX aggregate checks where possible. Four of the eight ODCS threshold fields have no exact-fit DQX aggregate check today, so the generator falls back to a dataset-levelsql_querycheck instead:mustBeGreaterThan(metric > X)mustBeLessThan(metric < X)mustBeBetween(lo < metric < hi, both bounds exclusive per ODCS)mustNotBeBetween(metric <= lo OR metric >= hi, per ODCS's exclusive-bounds semantics)Proposal
Add native DQX check functions for these, so the data-contract generator (and any other caller) can use a structured, exact-fit check instead of a raw SQL string:
is_aggr_greater_than(column, limit, aggr_type=..., ...)is_aggr_less_than(column, limit, aggr_type=..., ...)is_aggr_in_range(column, min_limit, max_limit, aggr_type=..., ...)(exclusive bounds)is_aggr_not_in_range(column, min_limit, max_limit, aggr_type=..., ...)(exclusive bounds)These would mirror the existing
is_aggr_equal/is_aggr_not_equal/is_aggr_not_less_than/is_aggr_not_greater_thanfamily incheck_funcs.py(sameaggr_type,group_by,row_filter, tolerance parameters, etc.), and theDataContractRulesGeneratorincontract_rules_generator.pycould then replace itssql_queryfallback for these four ODCS threshold fields with the new structured checks.Out of scope for this issue
The generated
sql_queryfallback that's in place today is reviewed and correct — this is a nice-to-have follow-up to make the generated rules more structured, not a bug fix.