Skip to content

feat: add source-aware diagnostics to the native CLI and REPL #95

Description

@dodok8

Objective

Add source-aware diagnostics for schema and query parsing, query resolution,
script validation, and SQLite execution in the native CLI and REPL while
preserving no_std compatibility in the engine crates.

Use miette only in the tooling/presentation layer. Engine crates should
continue to return concrete typed errors and source byte ranges without
depending on a terminal renderer.

Motivation or use case

The lexers currently record byte, line, and column positions, but most of that
provenance is lost before an error reaches a user:

  • query AST nodes do not retain source ranges, so resolver errors cannot point
    to the failing type, field, path step, assignment, or expression;
  • parse_script returns query source slices that compile_script parses
    again, making inner ranges relative to each slice;
  • CompiledScriptStatement drops its original statement span, so runtime
    failures can report only a statement number;
  • gelite-commands::CommandError flattens typed errors into strings;
  • the CLI and REPL print those strings without source excerpts or labels.

Source-aware diagnostics should make syntax, semantic, and execution failures
actionable without introducing std-only requirements into parser, AST,
resolver, IR, planning, or runner contracts.

Expected outcome

  • Source locations propagated across crate boundaries use absolute UTF-8 byte
    ranges into the original input. Line and column rendering is derived from
    the source at the presentation boundary.
  • Query AST nodes that can cause resolver failures retain optional source
    ranges. Programmatically constructed AST values remain supported without
    requiring source locations.
  • ResolveError retains its typed error category and an optional primary
    source range.
  • Query scripts produce typed parsed statements with absolute ranges instead
    of reparsing per-statement source slices.
  • Compiled query and transaction statements retain their original statement
    ranges. SQLite execution failures are associated with the failing statement.
  • CommandError preserves structured parser, resolver, transaction, planning,
    and runner failures instead of reducing them to debug-formatted strings.
  • Schema parsing keeps enough temporary declaration provenance to associate
    catalog validation failures with the relevant declaration without adding
    source metadata to schema-model.
  • The native CLI and REPL render source excerpts and labels through miette.
    Use a renderer configuration without backtrace support for user-input
    diagnostics.
  • Source text and source names remain owned by the CLI/REPL boundary and are
    attached only when constructing a report. AST, IR, plans, and runner errors
    do not own complete source files.

Scope

  • engine/query-ast
    • retain optional byte ranges on resolver-relevant query nodes
  • engine/query-parser
    • populate AST ranges and return typed, source-ranged script statements
  • engine/query-resolver
    • attach the most specific available AST range to each ResolveError
  • engine/schema-parser
    • preserve syntax locations and map catalog validation failures back to
      declarations while parsing
  • tools/gelite-commands
    • preserve structured error kinds and statement ranges through compilation
      and execution
  • tools/gelite-cli and tools/repl
    • attach source names/content and render miette diagnostics
  • focused parser, resolver, command, CLI, REPL, and query-pipeline tests
  • update the relevant tooling/parser plan documentation with the final error
    and source-range contracts

Prefer core::ops::Range<usize> or the existing parser span representation
over introducing a general source-map abstraction. Add a shared span type only
if the implementation demonstrates that the existing types cannot express the
required cross-crate contract cleanly.

Related specs and plans

  • CONTRIBUTING.md: requires typed recoverable errors and preservation of
    no_std engine crates.
  • spec/query.md: defines the syntax and semantic failures owned by the parser
    and resolver; this issue does not change query semantics.
  • spec/ir.md: keeps Semantic IR backend-independent; source provenance should
    stop before IR unless required by an existing contract.
  • plan/query-parser-implementation-plan.md: defines lexer/parser source-span
    tracking and parser error ownership.
  • plan/schema-parser-implementation-plan.md: defines schema parser errors and
    direct construction of SchemaCatalog without a separate schema AST.
  • plan/cli-and-tooling-plan.md: assigns diagnostic output to the thin native
    CLI/tooling boundary.
  • plan/new-db-engine-design.md: identifies source-span tracking, syntax
    diagnostics, and miette as intended developer-experience components.
  • feat: add multi-statement query scripts and transactions #49: introduced multi-statement scripts and transaction execution whose
    statement provenance must now be retained.

Boundaries and non-goals

  • Do not add miette or another std-only diagnostic dependency to
    engine/* crates.
  • Do not add WASM, browser, or JavaScript-facing diagnostic rendering or API
    contracts in this issue. Those integrations will be designed separately.
  • Do not propagate detailed source locations into Semantic IR, SQLite plans,
    generated SQL, or SQLiteRunnerError.
  • Do not attempt to map SQLite error offsets or constraint messages back to an
    exact Gelite subexpression. Runtime failures should label the originating
    statement.
  • Do not build a multi-file source database, parser recovery framework,
    diagnostic registry, or general provenance system.
  • Do not add source locations to schema-model::SchemaCatalog or persisted
    schema metadata.
  • Multiple related labels, diagnostic documentation URLs, and stable public
    diagnostic codes are deferred unless needed to satisfy the primary
    source-location behavior.

Acceptance criteria

  • Invalid schema/query syntax rendered by the native CLI and REPL includes the
    source name, source excerpt, and a label at the parser-reported byte range.
  • Representative resolver failures (UnknownObjectType, UnknownField, an
    incompatible operand type, and an invalid assignment) label the most
    specific relevant AST node.
  • A resolver test using multibyte UTF-8 before the failing node verifies that
    byte offsets remain correct.
  • Multi-statement scripts preserve absolute ranges into the complete original
    script, including statements after the first line.
  • A runner failure in a later script statement is rendered against that full
    statement's original range, while the existing transaction rollback behavior
    remains unchanged.
  • Programmatically constructed AST values with no source range continue to
    resolve and return useful unlocated errors.
  • Schema catalog validation failures can identify the relevant source
    declaration without storing source locations in schema-model.
  • CommandError no longer uses debug-formatted parser or resolver errors as
    its only retained representation.
  • All existing engine/* crates remain no_std and have no dependency on
    miette.
  • Diagnostic rendering tests use deterministic color/backtrace settings and
    assert stable semantic details rather than snapshotting terminal-dependent
    decoration.
  • cargo fmt --all -- --check passes.
  • cargo clippy --workspace --all-targets -- -D warnings passes.
  • cargo test --workspace passes.
  • Native CLI smoke checks cover at least one parse error, one resolver error,
    and one runtime statement error.

Branch

Use issue-95-source-aware-diagnostics.

Checks

  • I searched existing issues for the same request or task.
  • I checked the relevant spec/ or plan/ document when one exists.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions