Feature/configurable business tools

CHANGELOG.md

@@ -5,6 +5,30 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+- **Declarative business tools (opt-in, additive, experimental):** define business-facing MCP tools in YAML/JSON
+  via `COSMOS_TOOLS_CONFIG` (or `CosmosMcp:ToolsConfigPath`). Supports point-read, query,
+  text/vector/hybrid search, create/replace/patch/delete, optimistic concurrency, transactional
+  batch, and bounded Cosmos-only `sequence` composition with assertions, generated ids, and system
+  timestamps.
+- Per-tool authorization (scopes/roles/claims), tenant isolation with anti-spoofing, and governance
+  (read-only default, write/delete/cross-partition opt-in, RU/timeout/maxItems/topK budgets, patch
+  allow-lists).
+- Hierarchical (subpartitioned) partition key support (`partitionKeys: [...]`).
+- Shared `ICosmosGateway` provider surface, injection-resistant parameter binding, closed input
+  schema generation, and output projection/redaction.
+- `samples/banking/cosmos-tools.yaml`, `samples/ecommerce/cosmos-tools.yaml` (a non-banking example
+  proving the engine is domain-agnostic), and `docs/declarative-tools/*` (YAML reference, security &
+  governance, GA compatibility matrix, banking migration walkthrough).
+
+### Compatibility
+- Fully backward compatible. The declarative runtime is dormant unless a configuration file is
+  supplied; no existing tool, schema, default, or environment variable was changed.
+- The declarative layer is **experimental** and may change in a future release; it is opt-in and
+  dormant by default.
+
 ## [1.1.2] - 2026-05-29
 
 ### Added

README.md

@@ -26,6 +26,12 @@ This toolkit provides a **production-ready, fully automated MCP server** that ha
 
 **See it in action:** 📚 [Real-world use cases](docs/USE-CASES.md) • 🗺️ [Roadmap](ROADMAP.md) • 🤝 [Contributing](CONTRIBUTING.md)
 
+> 🧪 **Declarative business tools (opt-in, experimental):** define secure, governed, business-facing MCP tools in
+> YAML — point reads, queries, search, writes, and transactional batches — without writing handler
+> code. This is additive and dormant unless you supply a config file, so existing deployments are
+> unaffected. **This feature is experimental and may change in a future release.**
+> See [docs/declarative-tools](docs/declarative-tools/README.md).
+
 ## Prerequisites
 
 - Azure subscription ([Free account](https://azure.microsoft.com/free/))

docs/declarative-tools/README.md

@@ -0,0 +1,101 @@
+# Declarative Business Tools (vNext)
+
+> ⚠️ **Experimental.** This declarative layer is experimental and may change in a future release.
+> It is additive and opt-in (dormant unless you supply a configuration file). Review the security,
+> authorization, tenant-isolation, and governance settings before using it in production.
+
+The Azure Cosmos DB MCP Toolkit can expose **business-facing, governed MCP tools** defined in a
+YAML (or JSON) file — no bespoke handler code required. This layer is **additive and opt-in**:
+if you do not provide a configuration file, the toolkit exposes only its GA built-in tools and
+behaves exactly as before.
+
+## Contents
+
+- [YAML configuration reference](./yaml-reference.md) — every field and one example per operation type
+- [Security & governance](./security-governance.md) — auth, tenant isolation, RU/timeout budgets
+- [GA compatibility matrix](./compatibility-matrix.md) — evidence that existing behavior is unchanged
+- [Banking migration walkthrough](./banking-migration.md) — tool-by-tool classification and `bank_transfer` analysis
+
+## What you can define
+
+Point reads, parameterised queries, full-text/vector/hybrid search, create/replace/patch/delete,
+optimistic concurrency, transactional batches, and short **Cosmos-only** bounded composition
+(`sequence`) with assertions, generated ids, and system timestamps.
+
+It is deliberately **not** a workflow engine, saga orchestrator, or scripting runtime.
+
+## Quick start
+
+1. Author a config file, e.g. `cosmos-tools.yaml`:
+
+   ```yaml
+   version: "1.0"
+   sources:
+     app:
+       type: cosmos
+       endpoint: "${COSMOS_ENDPOINT}"
+       database: "${COSMOS_DATABASE}"
+       authentication:
+         type: managed-identity
+   defaults:
+     source: app
+     governance:
+       readOnly: true          # writes must be explicitly enabled per tool
+       timeoutMs: 5000
+       maxItems: 100
+   tools:
+     get_account_balance:
+       description: Returns the current balance for an account.
+       operation:
+         type: point-read
+         container: accounts
+         id: "${accountId}"
+         partitionKey: "${customerId}"
+       input:
+         type: object
+         required: [customerId, accountId]
+         properties:
+           customerId: { type: string }
+           accountId: { type: string }
+       output:
+         select:
+           accountId: accountId
+           balance: balance
+   ```
+
+2. Point the toolkit at it (either form works):
+
+   ```powershell
+   $env:COSMOS_TOOLS_CONFIG = "C:\path\to\cosmos-tools.yaml"
+   ```
+
+   or in `appsettings.json`:
+
+   ```json
+   { "CosmosMcp": { "ToolsConfigPath": "cosmos-tools.yaml" } }
+   ```
+
+3. Start the server. Configured tools appear alongside the built-in tools in `tools/list`.
+
+## Fail-closed behavior
+
+- Writes (`create`, `replace`, `patch`, `delete`, `transactional-batch`, `sequence`) require
+  `governance.readOnly: false` on the tool. Read-only is the default.
+- `delete` additionally requires `governance.allowDelete: true`.
+- An invalid configuration prevents startup with a clear diagnostic — the server never starts
+  with a partially valid tool set.
+- If `COSMOS_TOOLS_CONFIG` points at a missing file, startup fails rather than silently ignoring it.
+
+## A complete example
+
+The toolkit engine is **domain-agnostic** — it executes whatever a config file describes. See the
+[samples overview](../../samples/README.md), which includes two configs built on the identical engine
+with no code differences:
+
+- [`samples/banking/cosmos-tools.yaml`](../../samples/banking/cosmos-tools.yaml) — retail banking
+  (hierarchical partition keys, tenant isolation, point-read, query, vector-search, create, transactional-batch).
+- [`samples/ecommerce/cosmos-tools.yaml`](../../samples/ecommerce/cosmos-tools.yaml) — a different
+  domain (catalog/orders) using point-read, query, hybrid-search, patch with an allow-list, and a
+  bounded `sequence` with an assertion.
+
+Anything expressible with the supported operation types works for any Cosmos-backed application.

docs/declarative-tools/banking-migration.md

@@ -0,0 +1,83 @@
+# Banking Multi-Agent Workshop — Migration Walkthrough
+
+This document classifies the Banking workshop's MCP tools and records how each maps to the
+declarative toolkit. Source analyzed: the workshop `mcp` branch
+(`csharp/src/BankingAPI/Services/BankingDataService.cs`, the Semantic Kernel plugins, and
+`python/src/app/tools/mcp_server.py`).
+
+## Data model
+
+- Container **`accounts`** — hierarchical partition key **`[tenantId, accountId]`**, `type`
+  discriminator (`BankAccount`, `BankTransaction`, `ServiceRequest`).
+- Container **`offers`** — partition key **`[tenantId]`**, holds `Offer` and `OfferTerm` (with a
+  `Vector` embedding property).
+
+## Tool classification
+
+| Tool | Purpose | Backend | Uses Cosmos | Complexity | Declarative today | Target form | Recommended location | Reason |
+|---|---|---|:--:|---|:--:|---|---|---|
+| `bank_balance` | Read one account | Cosmos | ✅ | simple data op | ✅ | `point-read` | Cosmos MCP Toolkit | Direct point read on `[tenantId, accountId]` |
+| `get_transaction_history` | List account transactions between dates | Cosmos | ✅ | simple data op | ✅ | `query` (parameterised, partition-scoped) | Cosmos MCP Toolkit | Pure parameterised query |
+| `get_offer_information` | Semantic search of product offers | Cosmos (+embeddings) | ✅ | simple data op | ✅ | `vector-search` | Cosmos MCP Toolkit | Embedding + vector search on `offers` |
+| `create_account` | Open a new account | Cosmos | ✅ | simple data op | ✅ | `create` (write opt-in) | Cosmos MCP Toolkit | Single document create |
+| `service_request` | File a service request | Cosmos | ✅ | simple data op | ✅ | `create` (write opt-in) | Cosmos MCP Toolkit | Single document create |
+| `bank_transfer` | Request a funds transfer | Cosmos | ✅ | constrained compound | ✅ (request form) | `create` (request) + `transactional-batch` demo | Cosmos MCP Toolkit | See analysis below |
+| `calculate_monthly_payment` | Loan math | pure calculation | ❌ | simple calc | n/a | stays code | Banking MCP Server | No Cosmos involvement |
+| `get_branch_location` | Branch lookup by state | static data | ❌ | simple data | n/a | stays code | Banking MCP Server | Static/non-Cosmos data |
+| `transfer_to_*_agent` | Agent hand-off | agent framework | ❌ | orchestration | n/a | stays code | Host / agent framework | Not a data operation |
+| `health_check` | Liveness | n/a | ❌ | n/a | n/a | stays code | Either server | Infra concern |
+
+The migrated Cosmos-backed tools are implemented in
+[`samples/banking/cosmos-tools.yaml`](../../samples/banking/cosmos-tools.yaml) and verified by
+`BankingSampleConfigTests` and the emulator integration tests.
+
+## `bank_transfer` — detailed analysis
+
+`bank_transfer` is Cosmos-backed and is treated as a first-class case (not excluded for containing
+business logic).
+
+**Reads/writes involved in a "real" transfer:** read source account, validate funds, debit source,
+credit destination, create a transaction record.
+
+**Partition reality:** the `accounts` container is partitioned by `[tenantId, accountId]`. The
+source and destination are **different `accountId` values → different logical partitions**.
+
+1. **Can it be one transactional batch?** **No.** A Cosmos transactional batch is scoped to a single
+   logical partition. Debit (source partition) + credit (destination partition) span two partitions,
+   so they cannot be one atomic batch under this (unchanged, GA) partition design.
+2. **Can it be bounded Cosmos-only composition?** Partially. A `sequence` can read-validate-debit-credit
+   with optimistic concurrency and bounded compensation, but it is **not atomic** across the two
+   partitions — a crash between debit and credit needs compensation, which is best-effort.
+3. **What exact primitive would make it atomic?** A cross-partition (multi-partition) ACID
+   transaction / two-phase commit.
+4. **Is that in scope for a Cosmos toolkit?** **No.** Azure Cosmos DB does not provide cross-partition
+   ACID transactions; providing one would require a cross-system saga/2PC engine, which the product
+   boundary explicitly excludes.
+5. **Conclusion:** the workshop's **request-based** model is the correct, fully-declarative
+   representation and is what we migrate: `bank_transfer` records a `FundTransfer` service request
+   (a single create in the source account's partition) that is fulfilled asynchronously. This is the
+   `bank_transfer` tool in the sample.
+
+To still demonstrate atomic multi-write capability, the sample also provides
+`post_account_transaction`, a **transactional batch** that adjusts a balance **and** appends the
+matching transaction record atomically — valid precisely because both writes share one logical
+partition. This is exercised end-to-end against the emulator in
+`EmulatorIntegrationTests.Transactional_batch_adjusts_balance_and_records_transaction`.
+
+## Semantic differences to note
+
+- `get_offer_information` is scoped to the tenant partition; the workshop additionally filters
+  `type = 'Term'` and `accountType`. Those metadata filters can be layered on with a filtered
+  `VectorDistance` query if strict parity is required.
+- Identity fields (`tenantId`) are enforced from the caller's token claim (`tid`) rather than trusted
+  from the model, which is a security improvement over passing `tenantId` as a plain argument.
+
+## Single vs. multiple MCP servers
+
+The host aggregates two sibling servers:
+
+- **Cosmos DB MCP Toolkit** — serves the Cosmos-backed configured tools above.
+- **Banking MCP Server** — retains genuinely out-of-scope tools (`calculate_monthly_payment`,
+  `get_branch_location`, agent hand-offs).
+
+The Banking server does **not** proxy the Cosmos toolkit; both are exposed to the host directly.

docs/declarative-tools/compatibility-matrix.md

@@ -0,0 +1,58 @@
+# GA Compatibility Matrix
+
+The declarative layer is **additive and opt-in**. This document records the evidence that existing
+GA behavior is unchanged.
+
+## Principle
+
+- No existing tool is renamed or removed.
+- No input or output schema of a built-in tool is changed.
+- No default is changed; no new setting is mandatory.
+- The declarative runtime is dormant unless `COSMOS_TOOLS_CONFIG` (or `CosmosMcp:ToolsConfigPath`)
+  is provided.
+
+## Evidence
+
+| Existing capability | GA behavior | New behavior | Compatible | Evidence |
+|---|---|---|:--:|---|
+| Tool discovery (`tools/list`) | 8 built-in tools advertised | Same 8 tools; configured tools only added when a config file is supplied | ✅ | Registration is a no-op without config (`ConfiguredToolsRegistration.AddConfiguredCosmosTools`); existing tests unchanged |
+| `list_databases` / `list_collections` | Unchanged static `[McpServerTool]` methods | Not modified | ✅ | `Program.cs` `CosmosDbTools` untouched |
+| `get_recent_documents` (1–20) | Range validation unchanged | Not modified | ✅ | `CosmosDbToolsTests.GetRecentDocuments_Should_Validate_Count_Parameter` still passes |
+| `text_search` property validation | Regex identifier check | Not modified | ✅ | Existing unit tests pass |
+| `find_document_by_id` | Unchanged | Not modified | ✅ | Existing unit tests pass |
+| `get_approximate_schema` | Unchanged | Not modified | ✅ | Existing unit tests pass |
+| `vector_search` | Unchanged; explicit `selectProperties`, no wildcard | Not modified; configured vector-search also forbids wildcard | ✅ | Existing tests pass; `ConfigurationValidator` rejects `*` in `select` |
+| `hybrid_search` | Unchanged | Not modified | ✅ | Existing tests pass |
+| Input schemas | Closed (`additionalProperties: false`) | Configured tools also generate closed schemas | ✅ | `JsonSchemaGeneratorTests` |
+| Authentication modes (Entra ID / DEV_BYPASS_AUTH) | Unchanged | Reused; configured tools honor the same principal and bypass flag | ✅ | `CallerContext.FromPrincipal`, `AuthorizationEvaluatorTests` |
+| Transports (SSE + Streamable HTTP at `/mcp`) | Unchanged | Configured tools registered via the same `AddMcpServer()` builder | ✅ | `Program.cs` wiring after `WithToolsFromAssembly` |
+| Environment variables | Reinterpreted? No | New optional `COSMOS_TOOLS_CONFIG` only | ✅ | Only read when present |
+| `CosmosClientFactory` / `EmbeddingClientFactory` | Unchanged | Reused by `CosmosGateway` | ✅ | No edits to these files |
+
+## Test evidence
+
+- Baseline at the base commit: **18 pass / 5 fail** (the 5 failures are pre-existing and unrelated
+  to this work — see below).
+- After this change: **63 pass / 5 fail**. The same 5 pre-existing failures remain; all 45 new
+  tests pass, and every originally-passing test still passes.
+
+### Pre-existing failures (present before this work)
+
+These fail at the base commit `6ebcd31` and are **not** caused by this change:
+
+1. `CosmosDbToolsTests.HybridSearch_Should_Reject_Wildcard_SelectProperties` — the test asserts on
+   `'*'` but `JsonSerializer` escapes `'` to `\u0027`, so the substring match fails.
+2–5. `McpProtocolControllerIntegrationTests.*` — these POST to the SDK `/mcp` endpoint without an
+   `Accept: text/event-stream` header, so the Streamable HTTP transport returns `406 Not Acceptable`
+   before the asserted JSON-RPC error path is reached.
+
+They were left untouched to avoid altering baseline test expectations.
+
+## How to re-verify
+
+```powershell
+cd Q:\repos\MCPToolKit
+dotnet test AzureCosmosDB.MCP.Toolkit.sln
+```
+
+(The `net9.0` runtime is required to execute the tests.)

docs/declarative-tools/security-governance.md

@@ -0,0 +1,68 @@
+# Security & Governance
+
+The declarative layer is designed to **fail closed** and to keep identity decisions server-side.
+
+## Authentication
+
+Configured tools run inside the same ASP.NET pipeline as the built-in tools and honor the same
+authentication configuration (Entra ID JWT bearer, or the `DEV_BYPASS_AUTH=true` development
+bypass). The caller's `ClaimsPrincipal` is read per invocation from the request `HttpContext`.
+
+## Authorization
+
+Per-tool `authorization` supports:
+
+- `requiredScopes` — every listed scope must be present (`scp` claim).
+- `requiredRoles` — every listed role must be present (`roles` claim).
+- `claims` — claim type must equal the required value.
+- `tenantClaim` + `tenantField` — tenant isolation (below).
+- `partitionKeyFromClaim` — partition restriction derived from identity.
+
+When any authorization rule is present and the caller is unauthenticated (and not in dev bypass),
+the tool is denied.
+
+## Tenant isolation & anti-spoofing
+
+The tenant identity is **always** taken from a validated token claim, never from model-supplied
+input:
+
+1. If the model supplies a `tenantField` value that differs from the `tenantClaim`, the call is
+   **denied** (`tenant isolation violation`).
+2. Before binding, the trusted claim value is **overlaid** onto the input, so the executed
+   operation uses the caller's real tenant regardless of what the model sent.
+
+The same mechanism applies to `partitionKeyFromClaim` for partition-level restriction.
+
+## Injection resistance
+
+- Query text comes only from configuration. Caller input is bound as **parameters**
+  (`@name`), ids, and partition keys — never concatenated into SQL.
+- Identifier paths used to build SQL fragments (search property, vector/text paths, projection
+  fields) come from configuration and are validated; wildcard projection is rejected.
+- Unit tests assert that SQL-looking input (`'; DROP TABLE ...`, `A1' OR '1'='1`) is passed through
+  as a literal parameter value and never appears in the statement text.
+
+## Governance (fail closed)
+
+- **Read-only by default.** Writes require `governance.readOnly: false`; `delete` additionally
+  requires `allowDelete: true`. This is enforced at load time — an offending tool makes the whole
+  configuration invalid and the server does not start.
+- **Cross-partition** queries require `allowCrossPartition: true`; otherwise queries are scoped to
+  the tool's partition key.
+- **Limits:** `maxItems` caps result counts and `MaxItemCount`; `maxTopK` caps vector/hybrid `topK`.
+- **Timeouts:** `timeoutMs` wraps each invocation in a linked cancellation token; a timeout returns
+  a structured `timeout` error rather than hanging.
+- **Patch allow-list:** when `allowedPatchPaths` is set, patch operations must target one of the
+  listed JSON paths.
+
+## Observability
+
+Each configured invocation logs tool name, version, operation, database, container, latency, and a
+result category (`ok`, `validation`, `authorization`, `not_found`, `conflict`, `timeout`,
+`cosmos`, `internal`). Sensitive document content is not logged by default.
+
+## Error taxonomy
+
+All failures are returned as structured, client-safe JSON: `{ "error": "...", "category": "...",
+"details": [...] }`. Categories include `validation`, `authorization`, `binding`, `assertion`,
+`not_found`, `conflict` (ETag/precondition), `timeout`, `cosmos`, and `internal`.