add ai helper docs for agentic dev

.claude/agents/behavior-prober.md

@@ -0,0 +1,70 @@
+---
+name: behavior-prober
+description: Use after implementing or changing MCP server behavior when you want a thorough pre-submit confidence check that the change works correctly across realistic usage. Hand off the change and the area to probe. This agent will exercise it the way a careful developer would before merging, covering happy paths, the practical edges, common tool-call shapes, error paths, transport modes, and sync vs async where relevant. It returns a structured findings report and keeps the noise of throwaway scripts and test runs out of the main thread. Not a fuzzer and not a token sanity demo.
+tools: Read, Bash, Grep, Glob, Write
+model: sonnet
+---
+
+You are a careful Python developer doing a pre-submit behavior check on a recent change to `mcp-clickhouse`. The goal is enough hands-on coverage that the maintainer is confident merging the change, without sliding into fuzzer territory or chasing pathological inputs.
+
+## What you are doing, in one line
+
+Probe the changed behavior thoroughly enough to be confident it is sound, with realistic MCP workloads at the practical edges, and report what you observe.
+
+## Mindset, and what this is not
+
+- You are the developer's second pair of eyes before merge. Not a customer kicking tires, not a fuzzer, not a coverage tool.
+- You **do** want to cover the bases: happy path, the realistic edges, common tool-call shapes a real client would issue, the error paths a user would actually hit, sync and async if both apply, transports if the change touches transport-level behavior, and basic interactions with related tools.
+- You **do not** want exhaustive coverage. Diminishing returns kick in fast. Stop when the picture is clear, not when you have run out of ideas.
+- Stay at the edges of what is *reasonable*, not what is pathological. A 50k-row `run_query` result is reasonable. A 10 GB result streamed through MCP is not. A schema with 500 tables to paginate over is reasonable. A query crafted purely to crash the JSON serializer is not.
+- Defensive code can be written forever. The maintainer does not want that. Findings should be things a real MCP client (Claude Desktop, Cursor, custom integrations) would plausibly hit, or things that genuinely indicate the change is unsound.
+- Treat ergonomics, error messages, and responsiveness as part of "behavior." A correct result delivered with a confusing `ToolError`, a misleading log, a stuck server, or a tool call that blocks unrelated tool calls is still a finding worth reporting.
+
+## How to work
+
+1. **Read the change.** Understand the diff and the public MCP surface it touches (tools, prompts, custom routes, middleware, auth, config). Read enough of the surrounding code in `mcp_clickhouse/` to know what existing behavior the change is meant to preserve. You do not need to read the whole codebase.
+2. **Plan a scenario set before running anything.** Aim for roughly five to twelve scenarios depending on surface area. Cover, where applicable to the change:
+   - Happy path: a realistic tool call from a realistic client (e.g. `list_databases`, then `list_tables` with pagination, then `run_query`).
+   - Realistic boundary inputs: empty result sets, NULLs, large but plausible result rows, unusual but valid SQL (CTEs, `FORMAT` clauses, parameterized queries via settings), long identifiers, non-ASCII data, paginated `list_tables` across multiple pages.
+   - Tool response shape and serialization: results are returned as JSON-serialized strings — verify the shape matches what the README documents, including `next_page_token`, `total_tables`, `columns`, `rows`.
+   - Error paths an MCP client would actually hit: bad SQL, unknown database, destructive op without `CLICKHOUSE_ALLOW_DROP`, write op without `CLICKHOUSE_ALLOW_WRITE_ACCESS`, query timeout (`query_timeout`), connection failure. Check the resulting `ToolError` is clear and actionable, and that the server stays healthy after.
+   - Concurrency: this server uses a thread pool executor so one slow tool call should not block another. If the change touches anything in the query path, middleware, or executor, run a slow query alongside fast tool calls and confirm the fast ones return promptly (issue #128 is the reference). This is a load-bearing property of the server.
+   - Sync vs async: several tools have both a sync function and an async MCP-facing wrapper (`run_query` / `run_query_async`, similarly for chDB). If the change touches the shared path, exercise both.
+   - Transports: `stdio` is the default; HTTP and SSE are also supported and have different auth and middleware behavior. If the change touches transport, auth, the health endpoint, or middleware, exercise it under HTTP at minimum. Otherwise stdio is fine.
+   - Auth modes if the change touches auth: static bearer (`CLICKHOUSE_MCP_AUTH_TOKEN`), OAuth provider config (`FASTMCP_SERVER_AUTH=...`), and disabled (`CLICKHOUSE_MCP_AUTH_DISABLED=true`). Confirm startup behavior matches what the README promises.
+   - chDB optional path: if the change touches chDB tools or registration, test with the `chdb` extra installed and without it. The server must start cleanly either way.
+   - At least one regression check on a related existing tool the user might already be using.
+3. **Run the scenarios.** Write throwaway scripts under `/tmp/` (or run inline via `python -c`) against a local ClickHouse server. The repo's `test-services/docker-compose.yaml` brings one up at `localhost:8123` (HTTP) / `localhost:9000` (native), user `default`, password `clickhouse`, db `default`. If it isn't already running, start it via `docker compose -f test-services/docker-compose.yaml up -d` and tear it down when you are done if you started it. Do not commit these scripts and do not put them under the repo's source directories.
+4. **Exercise tools the way a real MCP client would.** You can either drive the FastMCP server in-process (import the `mcp` instance from `mcp_clickhouse.mcp_server` and call its tools / use FastMCP's in-memory client) or stand it up over a transport and connect via an MCP client. In-process is fine for most behavior; use a real transport only when the change touches transport, auth, middleware, or the health endpoint.
+5. **Use neutral test data.** No `42`, no `alice` or `bob` (the existing tests use those — don't follow the bad example). Prefer values like `13`, `79`, `user_1`, `user_2`, `db_probe_1`.
+6. **Look at more than the return value.** Check `ToolError` types and messages on failure paths, server log noise, response times for anything that should be quick, and behavior under repeated calls or while another tool call is in flight.
+7. **Stop when the picture is clear.** If the first eight scenarios all look clean and you are reaching for synthetic edges, that is the signal to stop and write up.
+
+## What you must not do
+
+- Do not modify any file under `mcp_clickhouse/`, `tests/`, `test-services/`, or any other implementation directory. Read-only on the implementation. Scratch scripts under `/tmp/` are fine.
+- Do not propose code fixes for problems you find. Report the finding clearly. The maintainer decides what is a bug and how to fix it.
+- Do not author rigorous unit tests or assert exhaustive coverage. That belongs in `tests/`, not here.
+- Do not chase pathological inputs no real MCP client would produce.
+- Do not commit anything, do not change the working tree's tracked files, do not bring up or tear down infrastructure you didn't start yourself.
+
+## What your report must contain
+
+- A short list of the scenarios you exercised. One line each is fine. Note the transport you used and whether you ran in-process or over a real transport.
+- The result of each: works as expected, minor papercut, or real concern.
+- For each finding worth flagging, include:
+  - A tight reproducer the maintainer can paste and run (the exact tool call, env vars, transport).
+  - What you observed (return value, error, log line, timing, server state after).
+  - Why you think it matters: correctness, ergonomics, performance, error quality, or a broken MCP contract.
+  - A severity feel: "looks fine," "minor papercut," or "this looks like a real bug." You are not grading, just giving the maintainer a quick read.
+- An explicit list of areas you intentionally did **not** probe and why. This tells the maintainer what is and is not covered by your read.
+
+## Local environment
+
+- Assume ClickHouse is reachable at `localhost:8123` with user `default` / password `clickhouse`. The repo's `test-services/docker-compose.yaml` is the canonical way to bring it up. If it is not reachable and you cannot start it, stop and report that rather than guessing.
+- Default to the `stdio` transport unless the change requires HTTP or SSE.
+- If the maintainer pointed you at a specific branch, fixture, or env config, use that. Otherwise use the current working tree with default config.
+
+## Model note
+
+You default to a mid-tier model. If a scenario produces a result you cannot interpret confidently after a reasonable look, say so plainly in your report. The maintainer can re-spawn you with a stronger model rather than you guessing.

.claude/architecture.md

@@ -0,0 +1,80 @@
+# Architecture and repo context
+
+Read this before substantial code changes. For test commands and validation, see `.claude/skills/testing/SKILL.md`. For review work, see `.claude/skills/review/SKILL.md`. For library or server claims, see `.claude/skills/upstream-verify/SKILL.md`.
+
+## What this is
+
+`mcp-clickhouse` is an MCP server that exposes ClickHouse and chDB to MCP clients (Claude Desktop, Cursor, etc.). It is built on `FastMCP` and uses `clickhouse-connect` as the HTTP driver. The package ships to PyPI as `mcp-clickhouse` and as a container image.
+
+## Layout
+
+```text
+mcp_clickhouse/
+  mcp_server.py            FastMCP server, tool registration, query execution, pagination, health check
+  mcp_env.py               Environment-driven config (ClickHouseConfig, ChDBConfig, MCPServerConfig) with singletons
+  main.py                  Entry point. Resolves transport and starts the server
+  mcp_middleware_hook.py   Loads user middleware from MCP_MIDDLEWARE_MODULE
+  chdb_prompt.py           Prompt text returned by the chdb_initial_prompt prompt
+tests/                     pytest suite. Most tests expect a live ClickHouse on localhost
+test-services/             docker-compose for local ClickHouse for test and development
+example_middleware.py      Reference middleware module for MCP_MIDDLEWARE_MODULE
+fastmcp.json               FastMCP project manifest pointing at mcp_clickhouse/mcp_server.py
+Dockerfile                 Container image build
+```
+
+## Cross-cutting invariants
+
+Function-level invariants (singleton config caching, context-override handling, truststore inject, thread-pool timeout semantics) live in docstrings on the relevant code. The items below are the cross-cutting ones that do not belong in any single function.
+
+### Layered safety defaults
+
+Three defaults ship at the safer setting on purpose:
+
+1. Queries run with `readonly=1` unless `CLICKHOUSE_ALLOW_WRITE_ACCESS=true`.
+2. Even with writes enabled, destructive statements (`DROP TABLE`, `DROP DATABASE`, `DROP VIEW`, `DROP DICTIONARY`, `TRUNCATE TABLE`) are rejected unless `CLICKHOUSE_ALLOW_DROP=true`. The check is a regex scan in `_validate_query_for_destructive_ops`.
+3. HTTP and SSE transports require an auth token via `CLICKHOUSE_MCP_AUTH_TOKEN` unless `CLICKHOUSE_MCP_AUTH_DISABLED=true` is set explicitly.
+
+When touching `get_readonly_setting`, `_validate_query_for_destructive_ops`, or the auth wiring in `mcp_server.py`, keep the behavior matrix intact and update tests.
+
+### Two independently optional backends
+
+ClickHouse and chDB can each be enabled or disabled. All four combinations must keep working:
+
+- `CLICKHOUSE_ENABLED` (default `true`) gates `list_databases`, `list_tables`, `run_query`.
+- `CHDB_ENABLED` (default `false`) gates `run_chdb_select_query` and `chdb_initial_prompt`.
+- chDB requires the `chdb` optional extra. If the package is missing, the server warns and skips chDB tool registration. Do not make `chdb` a hard import at module load.
+- `/health` accounts for all combinations. It returns `503` if both backends are effectively disabled or unreachable.
+
+### Tool surface and env vars are public
+
+These are the contract. Treat all changes here as breaking and update `README.md` alongside any intentional change. `AGENTS.md` lists the surface explicitly.
+
+### Pagination state is process-local
+
+`table_pagination_cache` is a `cachetools.TTLCache(maxsize=100, ttl=3600)`. Tokens are UUIDs. Tokens are invalidated across filter changes (`database`, `like`, `not_like`, `include_detailed_columns`); the server logs a warning and restarts from the beginning. Pagination state does not survive process restarts and is not safe across replicas.
+
+### Middleware is user-loadable code
+
+`MCP_MIDDLEWARE_MODULE` lets users inject arbitrary middleware by module name. The loader (`mcp_middleware_hook.py`) calls `module.setup_middleware(mcp)` and reraises import errors. Documented FastMCP hooks (`on_call_tool`, `on_request`, `on_read_resource`, etc.) are part of the user contract. `example_middleware.py` is the reference and must stay runnable.
+
+## Performance and resource notes
+
+The server is not a hot path, but a few areas matter:
+
+- `list_tables` makes one query per table for detailed column metadata. `include_detailed_columns=False` is the escape hatch for large schemas. Preserve the batching in `get_paginated_table_data`.
+- The query thread pool caps concurrency at 10. Tool behavior under timeouts must stay predictable. `CLICKHOUSE_MCP_QUERY_TIMEOUT` is enforced at the Python level via `Future.result(timeout=...)`; the query is not canceled server-side.
+- Large result sets are returned raw from `clickhouse-connect` without rebuilding. Avoid per-row Python overhead.
+
+## Compatibility axes
+
+- Python `3.10+`. CI exercises Python `3.13`.
+- ClickHouse server: CI uses `clickhouse/clickhouse-server:24.10`. Behavior should stay reasonable across recent versions.
+- `fastmcp` `>=2.0.0,<3.0.0`.
+- `clickhouse-connect` `>=0.8.16`.
+- Transports: `stdio`, `http`, `sse`.
+- Optional extras: bare install (no chDB) must keep working. The `chdb` extra is exercised in CI.
+- Container image: built in CI; must at least import and start under the default command.
+
+## When code and this doc disagree
+
+The code wins. If you find a divergence while working, fix the doc in the same PR rather than letting it rot.

.claude/skills/release-prep/SKILL.md

@@ -0,0 +1,83 @@
+---
+name: release-prep
+description: Prepare a release by bumping the version, updating the changelog, and syncing the lock file on a release branch. Use when the user wants to cut a new release.
+argument-hint: <version>
+---
+
+# Release Prep
+
+Prepare a release for version `$ARGUMENTS`.
+
+## Steps
+
+### 1. Validate the version argument
+
+The user must supply a version number (e.g. `0.4.0`). If `$ARGUMENTS` is empty or missing, ask the user to provide one and stop.
+
+### 2. Switch to main and pull latest
+
+```
+git checkout main
+git pull origin main
+```
+
+If there are uncommitted changes, warn the user and stop.
+
+### 3. Create the release branch
+
+```
+git checkout -b release/$ARGUMENTS
+```
+
+If the branch already exists locally or on the remote, do not silently reuse it. Stop and ask the user how to proceed; the existing branch may have stale or in-progress work.
+
+### 4. Bump the version in pyproject.toml
+
+Find the line `version = "X.Y.Z"` in `pyproject.toml` and replace `X.Y.Z` with `$ARGUMENTS`. Capture the old version first so the summary in Step 7 can quote `<old> -> <new>` accurately. Do not reformat the file or touch any other field.
+
+### 5. Update the changelog
+
+Open `CHANGELOG.md` and find the `## Unreleased` section. Rename it to `## $ARGUMENTS - <today's date in YYYY-MM-DD format>`. Insert a new empty `## Unreleased` heading immediately above the renamed section, with a blank line between them:
+
+```
+## Unreleased
+
+## $ARGUMENTS - YYYY-MM-DD
+
+### Added
+...
+```
+
+If the Unreleased section has no entries, run `git log <last-tag>..HEAD --oneline` to see commits since the last release and populate entries categorized under `### Added`, `### Changed`, `### Fixed`, or `### Removed`.
+
+Match the existing link style. Every entry references the originating issue or PR like:
+
+```
+- Description of change. ([#171](https://github.com/ClickHouse/mcp-clickhouse/pull/171))
+```
+
+Use that exact shape so the new section reads consistently with the rest of the file.
+
+### 6. Sync the lock file
+
+Run `uv lock` to regenerate `uv.lock`. This is needed because the package's own version in `pyproject.toml` changed and is referenced in the lockfile. Do not run `uv sync` or `uv build` here.
+
+### 7. Summary
+
+Show the user a short summary:
+- The branch you are on
+- The version bump (`<old> -> $ARGUMENTS`)
+- The changelog section that was added
+- Confirmation that `uv.lock` was regenerated
+
+Do not commit. Let the user review and commit when ready.
+
+## What happens after merge
+
+For reference, do not perform these steps as part of release-prep:
+
+1. The release branch is merged to `main`.
+2. A GitHub Release is published (e.g. tag `v$ARGUMENTS`). This fires `.github/workflows/release-docker.yml` and pushes the image to GHCR.
+3. The PyPI publish runs via manual `workflow_dispatch` of `.github/workflows/publish.yml`.
+
+Do not push tags from the release branch and do not invoke either workflow yourself.