fix: ingest resolves a schema scope and qualifies non-default schemas (DEV-1758)

.claude/skills/slayer-models.md

@@ -88,6 +88,15 @@ Model names cannot contain `__` (reserved for join-path aliases), but
 `sql_table` can. Ingestion sanitizes only the name: object
 `reports__patient__drug` → model `reports_patient_drug`, `sql_table` unchanged.
 
+`sql_table` is emitted verbatim into the generated SQL, so anything outside the
+connection's default schema **must** be schema-qualified (`analytics.orders`)
+or the query fails with table-not-found. Ingestion qualifies exactly when the
+object's schema differs from the default, or when the schema was named
+explicitly — default-schema objects stay bare, so both forms coexist in one
+datasource by design. The schema is everything before the final dot, so
+`project.dataset.table` works. A missing qualifier is repaired by re-ingesting
+that schema; an existing one is never rewritten.
+
 Auto-ingestion sets `hidden: true` on recognised ELT/migration bookkeeping
 tables — prefixes `_dlt_`, `_airbyte_`, plus exact names like
 `alembic_version`, `flyway_schema_history`, `databasechangelog`,

.claude/skills/slayer-overview.md

@@ -12,7 +12,7 @@ SLayer is a lightweight, agent-first semantic layer. Instead of writing raw SQL,
 - **SQLGenerator** — takes an EnrichedQuery (not SlayerQuery) and converts it to SQL via sqlglot (dialect-aware: postgres, mysql, bigquery, etc.)
 - **SlayerSQLClient** — executes SQL via SQLAlchemy with retry logic and statement timeouts
 - **Storage** — YAML or SQLite backends for model and datasource configs
-- **Ingestion** — auto-generates models from DB schema with rollup-style FK joins (denormalized LEFT JOINs). It can be triggered manually (`slayer ingest`, `ingest_datasource_models`, `POST /ingest`) or **on every server boot** via `slayer serve --ingest-on-startup` / `slayer mcp --ingest-on-startup` (also `SLAYER_INGEST_ON_STARTUP=1`, or `create_app/create_mcp_server(ingest_on_startup=True)` programmatically). It is idempotent and continues on per-datasource failures.
+- **Ingestion** — auto-generates models from DB schema with rollup-style FK joins (denormalized LEFT JOINs). It can be triggered manually (`slayer ingest`, `ingest_datasource_models`, `POST /ingest`) or **on every server boot** via `slayer serve --ingest-on-startup` / `slayer mcp --ingest-on-startup` (also `SLAYER_INGEST_ON_STARTUP=1`, or `create_app/create_mcp_server(ingest_on_startup=True)` programmatically). It is idempotent and continues on per-datasource failures. One pass covers **one schema** — `--schema a,b` / `--all-schemas` (and the `schemas` / `all_schemas` equivalents on MCP, REST and the Python API) opt into more; precedence is explicit flag → `datasource.schema_name` → the connection default. Non-default-schema objects get a schema-qualified `sql_table`.
 - **Interfaces** — MCP server (stdio via `slayer mcp`, SSE via `slayer serve` at `/mcp/sse`), REST API (FastAPI on port 5143), Python SDK, and two read-only wire-protocol facades for BI tools: Arrow Flight SQL (`slayer flight-serve`, port 5144) and Postgres (`slayer pg-serve`, port 5145; the connection `database` selects the SLayer datasource)
 
 ## Key Models

docs/concepts/ingestion.md

@@ -68,12 +68,36 @@ Non-SQLite datasources (Postgres, MySQL, DuckDB, ClickHouse, SQL Server) skip th
 
 Already-persisted v7 SQLite models with the wrong `INT` type are **not** auto-repaired on `storage.get_model()` load (running a full table scan per column on every load would be too expensive). Re-ingest is the auto-heal path: `slayer ingest` or `slayer serve --ingest-on-startup`. The DEV-1361 DOUBLE → INT narrowing on legacy-dict migration is also gated on the probe on SQLite — it only fires when the probe positively certifies INT.
 
+## Schema scope
+
+One ingest pass covers one schema unless told otherwise. The scope is, in
+order of precedence: the schemas named on the call; the datasource's
+`schema_name`; the connection's default schema. Naming several schemas, or
+every schema, is opt-in on all four surfaces:
+
+| Surface | One schema | Several | Every schema |
+|---|---|---|---|
+| CLI | `--schema public` | `--schema public,analytics` | `--all-schemas` |
+| Python | `schemas=["public"]` | `schemas=["public", "analytics"]` | `all_schemas=True` |
+| MCP | `schema_name="public"` | `schemas="public,analytics"` | `all_schemas=True` |
+| REST | `"schema_name": "public"` | `"schemas": ["public","analytics"]` | `"all_schemas": true` |
+
+Combining two of them is rejected (a `ValueError`, a 422, or an error string)
+rather than silently preferring one. When exactly one schema is scanned and
+others exist, the result carries a hint naming them.
+
+Objects outside the connection's default schema are written with a
+schema-qualified `sql_table`; default-schema objects stay unqualified. See
+[`slayer ingest`](../reference/cli.md#which-schemas-get-ingested) for the full
+rules, including qualifier repair and the cross-schema guard.
+
 ## Usage
 
 ### CLI
 
 ```bash
 slayer ingest --datasource my_postgres --schema public --storage ./slayer_data
+slayer ingest --datasource my_postgres --all-schemas --storage ./slayer_data
 ```
 
 ### Python
@@ -86,13 +110,15 @@ async def main():
     result = await ingest_datasource_idempotent(
         datasource=ds,
         storage=storage,
-        schema="public",
+        schemas=["public"],                       # or all_schemas=True
         include_tables=["orders", "customers"],  # Optional filter
         exclude_tables=["migrations"],            # Optional exclusion
     )
-    # result.additions  — what was added (new models / columns / joins)
-    # result.to_delete  — pending validate_models drift entries
-    # result.errors     — per-model failures (best-effort, doesn't abort)
+    # result.additions   — what was added (new models / columns / joins)
+    # result.to_delete   — pending validate_models drift entries
+    # result.errors      — per-model failures (best-effort, doesn't abort)
+    # result.skipped     — live objects we declined to model, with reasons
+    # result.schema_hint — set when other schemas were left out
     return result
 
 asyncio.run(main())
@@ -103,6 +129,7 @@ asyncio.run(main())
 ```
 create_datasource(name="mydb", type="postgres", ...)
 ingest_datasource_models(datasource_name="mydb", schema_name="public")
+ingest_datasource_models(datasource_name="mydb", all_schemas=True)
 ```
 
 ### REST API
@@ -111,6 +138,10 @@ ingest_datasource_models(datasource_name="mydb", schema_name="public")
 curl -X POST http://localhost:5143/ingest \
   -H "Content-Type: application/json" \
   -d '{"datasource": "my_postgres", "schema_name": "public"}'
+
+curl -X POST http://localhost:5143/ingest \
+  -H "Content-Type: application/json" \
+  -d '{"datasource": "my_postgres", "all_schemas": true}'
 ```
 
 ## Querying Rolled-Up Models

docs/concepts/models.md

@@ -63,6 +63,18 @@ generated SQL, but `sql_table` has no such restriction. Auto-ingestion uses
 this: an object named `reports__patient__drug` becomes a model named
 `reports_patient_drug` whose `sql_table` is still `reports__patient__drug`.
 
+`sql_table` may be schema-qualified (`analytics.orders`), and for anything
+outside the connection's default schema it has to be — the generated SQL uses
+the value verbatim, so an unqualified name resolves through the search path
+and a non-default-schema table is simply not found. Auto-ingestion writes the
+qualifier whenever the object's schema differs from the connection's default,
+or whenever the schema was named explicitly; default-schema objects stay
+unqualified. Within one datasource the two forms therefore coexist, which is
+intended: it keeps widening the ingest scope from rewriting models that
+already exist. Snowflake `db.schema.table` and BigQuery
+`project.dataset.table` are accepted too — the schema is everything before the
+final dot.
+
 ## Columns
 
 A column is the unit of structure on the model. The same column entry can serve as a group-by key in one query and as input to an aggregation in another — the role is decided per query, not declared up front. What the column *carries* is its identity (name), how to compute it from the source (`sql`), what data type to expect, and a handful of policy fields (which aggregations are allowed, whether it's a primary key, whether it's hidden).

docs/configuration/datasources.md

@@ -171,6 +171,24 @@ Statement-level timeout is enforced via
 !!! note
     Both `username` and `user` field names are accepted. The `user` alias is automatically mapped to `username` for compatibility with common database tooling conventions.
 
+### `schema_name` and ingestion
+
+`schema_name` is the default schema for both `slayer ingest` and `slayer
+validate-models`, so the two always look at the same tables. It is a
+*fallback*: an explicit `--schema` / `--all-schemas` on the command line wins,
+and neither combination is an error. With `schema_name` unset, ingest uses the
+connection's default schema.
+
+`slayer datasources create --schema X --ingest` persists `schema_name: X`. A
+comma-separated list or `--all-schemas` persists nothing — there is no single
+value to record, and writing the first one would silently narrow every later
+bare `slayer ingest`.
+
+Ingesting more than one schema is opt-in because it changes what `sql_table`
+holds: objects outside the connection's default schema are written
+schema-qualified (`analytics.orders`), which is what makes them queryable.
+See [`slayer ingest`](../reference/cli.md#slayer-ingest).
+
 ### BigQuery credentials
 
 Three ways to authenticate, in the order SLayer prefers them:

docs/reference/cli.md

@@ -83,6 +83,8 @@ Auto-generate models from a datasource.
 ```bash
 slayer ingest --datasource my_postgres
 slayer ingest --datasource my_postgres --schema public
+slayer ingest --datasource my_postgres --schema public,analytics
+slayer ingest --datasource my_postgres --all-schemas
 slayer ingest --datasource my_postgres --include orders,customers
 slayer ingest --datasource my_postgres --exclude migrations,django_session
 slayer ingest --datasource my_postgres --no-views
@@ -92,13 +94,51 @@ slayer ingest --datasource my_postgres --surface-internals
 | Flag | Required | Description |
 |------|----------|-------------|
 | `--datasource` | Yes | Datasource name |
-| `--schema` | No | Database schema to inspect |
+| `--schema` | No | Comma-separated schemas to inspect |
+| `--all-schemas` | No | Inspect every non-system schema in the current database. Mutually exclusive with `--schema` |
 | `--include` | No | Comma-separated tables to include |
 | `--exclude` | No | Comma-separated tables to exclude |
 | `--no-views` | No | Skip views and materialized views (ingested by default) |
 | `--surface-internals` | No | Ingest recognised ELT/migration internals visible instead of hidden |
 | `--storage` | No | Storage path |
 
+#### Which schemas get ingested
+
+With neither flag, ingest covers exactly one schema, resolved in this order:
+
+1. `--schema` / `--all-schemas`, when given;
+2. the datasource's persisted `schema_name`
+   ([datasource config](../configuration/datasources.md));
+3. the connection's default schema.
+
+If other schemas exist, ingest names them and exits 0 — a hint, not a failure:
+
+```
+Note: ingested schema 'main' only. Other schemas in this datasource: openfda_rest.
+Re-run with --schema openfda_rest, or --all-schemas, to ingest them.
+```
+
+Objects outside the connection's default schema get a schema-qualified
+`sql_table` (`openfda_rest.reports`), which is what makes them queryable.
+Default-schema objects stay unqualified, so widening the scan never rewrites
+models that already exist. A schema named explicitly is always written
+verbatim, so `--schema public` keeps producing `public.orders`.
+
+`--all-schemas` covers the **current database only**. Schemas belonging to an
+`ATTACH`ed DuckDB catalog are reported as skipped, naming the
+`--schema <catalog>.<schema>` invocation that would ingest them.
+
+A model whose `sql_table` is missing its schema qualifier is repaired on the
+next ingest of that schema, and the repair is reported:
+
+```
+Updated: reports (sql_table: reports → openfda_rest.reports)
+```
+
+An existing qualifier is never rewritten, and two schemas' same-named tables
+are never merged into one model — the second is skipped with a `cross-schema`
+reason rather than silently repointing the first.
+
 #### Views
 
 Views and materialized views are ingested alongside tables by default — dbt
@@ -271,7 +311,8 @@ slayer datasources create demo --ingest        # bundled Jaffle Shop demo
 | `--name` | No | Override the auto-derived name (default for the demo: `jaffle_shop`) |
 | `--description` | No | Human-readable description |
 | `--ingest` | No | Run auto-ingestion immediately after creating the datasource |
-| `--schema` | No | (with `--ingest`) Schema to ingest from |
+| `--schema` | No | (with `--ingest`) Comma-separated schemas to ingest from. A single schema is also persisted as the datasource's `schema_name`, so later bare `slayer ingest` runs use it |
+| `--all-schemas` | No | (with `--ingest`) Ingest every non-system schema. Mutually exclusive with `--schema`; persists no `schema_name` |
 | `--include` | No | (with `--ingest`) Comma-separated tables to include |
 | `--exclude` | No | (with `--ingest`) Comma-separated tables to exclude |
 | `--no-views` | No | (with `--ingest`) Skip views and materialized views (ingested by default) |

slayer/api/server.py

@@ -5,7 +5,7 @@
 from typing import Any
 
 from fastapi import FastAPI, HTTPException
-from pydantic import BaseModel, ConfigDict, Field
+from pydantic import BaseModel, ConfigDict, Field, model_validator
 
 from slayer.mcp.server import create_mcp_server
 from slayer.core.errors import (
@@ -103,11 +103,28 @@ class IngestRequest(BaseModel):
     datasource: str
     include_tables: list[str] | None = None
     exclude_tables: list[str] | None = None
+    # Kept for backward compatibility; folded into ``schemas=[schema_name]``.
     schema_name: str | None = None
+    schemas: list[str] | None = None
+    all_schemas: bool = False
     # Ingest recognised ELT/migration internals visible rather than hidden.
     # Governs models this call creates; unhide an existing one via edit_model.
     surface_internals: bool = False
 
+    @model_validator(mode="after")
+    def _one_way_to_say_it(self) -> "IngestRequest":
+        """Reject conflicting scope arguments at the edge, so every caller of
+        the endpoint gets the engine's rule (not whichever the handler reads
+        first)."""
+        from slayer.engine.ingestion import _resolve_scope_args
+
+        _resolve_scope_args(
+            schema=self.schema_name,
+            schemas=self.schemas,
+            all_schemas=self.all_schemas,
+        )
+        return self
+
 
 class ValidateModelsRequest(BaseModel):
     data_source: str | None = None
@@ -653,6 +670,8 @@ async def ingest(request: IngestRequest) -> dict[str, Any]:
                 include_tables=request.include_tables,
                 exclude_tables=request.exclude_tables,
                 schema=request.schema_name,
+                schemas=request.schemas,
+                all_schemas=request.all_schemas,
                 surface_internals=request.surface_internals,
             )
         except SQLAlchemyError as exc: