Skip to content

feat: add MCP tool contract drift monitoring blueprint - #287

Open
ayush-singh-0601 wants to merge 3 commits into
kestra-io:mainfrom
ayush-singh-0601:feat/mcp-tool-contract-monitor
Open

ayush-singh-0601 wants to merge 3 commits into
kestra-io:mainfrom
ayush-singh-0601:feat/mcp-tool-contract-monitor

Conversation

@ayush-singh-0601

@ayush-singh-0601 ayush-singh-0601 commented Sep 21, 2026 •

Copy link
Copy Markdown

What

Adds a production-oriented Blueprint that snapshots the agent-facing contracts of Kestra flows exposed through McpToolTrigger and detects future drift.

Why

MCP tool inputs, outputs, descriptions, server assignment, and safety annotations form an interface consumed by AI agents. Changes to that interface can be breaking even when the underlying flow remains valid.

How

The Blueprint:

  • inventories flows from a target namespace
  • extracts and normalizes MCP tool contracts
  • compares them with a KV-backed accepted baseline
  • generates JSON and Markdown drift reports
  • preserves the previous baseline until changes are explicitly accepted
  • can optionally fail on breaking contract changes
  • can send a Slack digest for unaccepted drift, with severity counts, affected tool paths, and a pointer to the full reports
  • includes a disabled six-hour schedule for recurring monitoring

Slack alerts are disabled by default. Enabling them requires the Slack Notifications plugin and a SLACK_WEBHOOK_URL secret. The baseline and reporting path does not require an external API, LLM, database, or SaaS service.

Compatibility adjustment

The current flows.List typed output omits plugin-specific McpToolTrigger fields. The Blueprint keeps flows.List as the namespace inventory and adds flows.Export to obtain the authoritative saved YAML. The Python analyzer cross-checks both inventories and parses the exported definitions with PyYAML.

Validation

  • imported and executed successfully on Kestra 2.0.2
  • exercised all 15 PRD scenarios, including baseline creation, unchanged contracts, drift classification, explicit acceptance, breaking gate behavior, zero MCP tools, and malformed baseline handling
  • verified KV get/set round trips and persisted JSON/Markdown report artifacts
  • passed 20 focused analyzer tests
  • passed YAML, metadata, tag, task-structure, and whitespace checks
  • validated the follow-up YAML and embedded Python syntax; the Slack branch still needs a live webhook check

@github-project-automation github-project-automation Bot moved this to To review in Pull Requests Sep 21, 2026
@MilosPaunovic MilosPaunovic added kind/external Pull requests raised by community contributors area/docs Issues related to documentation, plugin examples, blueprints, and guides labels Sep 22, 2026

@aj-emerich aj-emerich left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a cool idea and a good showcase of the MCPToolTrigger! Would it be possible to add some downstream ingestion or logic that needs this information? Or some type of alerting/reporting to showcase some additional Kestra plugins?

@ayush-singh-0601

Copy link
Copy Markdown
Author

@aj-emerich Thanks for the suggestion. I added a downstream Slack alert for unaccepted contract drift. It uses the analyzer output to report the change and severity counts, the first five affected tool paths, and the execution ID so the team can open the full Markdown and JSON reports.

The alert is optional (notify_on_drift=true with a SLACK_WEBHOOK_URL secret). It stays quiet for the initial baseline, unchanged contracts, and changes that have already been accepted. I updated the Blueprint documentation and PR description with the setup. The YAML and embedded Python syntax checks pass; I could not exercise the Slack delivery without a webhook.

Could you take another look when you have a chance?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Issues related to documentation, plugin examples, blueprints, and guides kind/external Pull requests raised by community contributors

Projects

Status: To review

Development

Successfully merging this pull request may close these issues.

3 participants