chDB sources: expose config-declared external data as named views/databases

[ShawnChen-Sirius]

Stacked on ClickHouse#202 (P1). Phase 2 of the chDB integration: a config-declared source registry.

What

CHDB_SOURCES (JSON env var) declares external sources — file, url, s3, postgres/mysql (whole-database proxies), clickhouse (remote table) — and the server materializes each one at init as a named view or database, while the session is still writable, before the readonly=2 lock. Agents then query and join them as ordinary tables, with no credentials, placeholders, or connection syntax in their SQL, and discover them through the existing list_databases / list_tables / describe_table tools:

[{"name": "lake", "type": "s3", "url": "s3://mybucket/events/*.parquet",
  "access_key_id": "…", "secret_access_key": "…", "format": "Parquet"},
 {"name": "appdb", "type": "postgres", "host": "rds.internal",
  "database": "app", "user": "readonly", "password": "…"}]

SELECT u.tier, sum(e.amount) FROM lake AS e JOIN appdb.users AS u ON u.uid = e.uid GROUP BY u.tier

Security model (each point engine-verified in tests)

• Credentials stay server-side; the engine masks them as [HIDDEN] on every reflective surface an agent can reach (SHOW CREATE, system.tables.create_table_query / as_select, system.databases.engine_full). The format_display_secrets_in_show_and_select escape hatch stays inert in embedded chDB (its paired server-config switch is never set).
• file-type sources are validated against CHDB_FILE_ALLOWLIST, so the registry cannot become a loophole around the raw-SQL path sandbox.
• A database proxy never replaces an existing database of another engine (a registry name collision cannot destroy local user tables).
• Malformed CHDB_SOURCES disables chDB loudly with the config error; an individually failing source (unreachable, or an engine not compiled into the build) is logged and skipped so the rest still come up.

Tests

25 new tests: pure config parsing/validation, DDL building (quoting, NOSIGN, defaults), and real-engine materialization (masking canary, allowlist refusal, foreign-database guard, proxy idempotence across re-init, fail-soft), plus two end-to-end tests through _init_chdb_client (sources exist + readonly=2 still locked; malformed config disables chDB loudly). Full suite: 95 passed, ruff check clean.

🤖 Generated with Claude Code