feat(pluggable-widgets-mcp): update readme and agents.md, add security.md, clarify notifications

Author: rahmanunver

packages/pluggable-widgets-mcp/AGENTS.md

@@ -1,257 +1,113 @@
-# Pluggable Widgets MCP Server - AI Agent Guide
+# Pluggable Widgets MCP Server
 
-This document provides context for AI development assistants working on the MCP (Model Context Protocol) server for Mendix pluggable widgets.
+MCP server enabling AI assistants to scaffold and manage Mendix pluggable widgets via STDIO (default) or HTTP transport.
 
-## Overview
+## Quick Reference
 
-This package implements an MCP server that enables AI assistants to scaffold and manage Mendix pluggable widgets programmatically. It supports both HTTP and STDIO transports for flexible integration with various MCP clients.
-
-### Key Characteristics
-
-- **MCP SDK**: Built on `@modelcontextprotocol/sdk` for standardized AI tool integration
-- **Dual Transport**: HTTP (Express) for web clients, STDIO for CLI clients (Claude Desktop, etc.)
-- **TypeScript**: Fully typed with Zod schemas for runtime validation
-- **Widget Generator**: Wraps `@mendix/generator-widget` via PTY for interactive scaffolding
+```bash
+pnpm dev          # Development with hot reload
+pnpm build        # TypeScript compilation + path alias resolution
+pnpm start        # Build and run (STDIO mode, default)
+pnpm start:http   # Build and run (HTTP mode, port 3100)
+pnpm lint         # ESLint check
+```
 
 ## Project Structure
 
 ```
 src/
-├── index.ts              # Entry point - transport mode selection
-├── config.ts             # Server configuration and constants
-├── security/
-│   ├── guardrails.ts     # Security validation (path traversal, extension whitelist)
-│   └── index.ts          # Security module exports
-├── server/
-│   ├── server.ts         # MCP server factory and tool/resource registration
-│   ├── http.ts           # HTTP transport setup (Express)
-│   ├── stdio.ts          # STDIO transport setup
-│   ├── routes.ts         # Express route handlers
-│   └── session.ts        # HTTP session management
-├── resources/
-│   ├── index.ts          # Resource registration
-│   └── guidelines.ts     # Widget development guidelines
-└── tools/
-    ├── index.ts          # Tool registration aggregation
-    ├── types.ts          # MCP tool type definitions
-    ├── scaffolding.tools.ts      # Widget creation (create-widget)
-    ├── file-operations.tools.ts  # File read/write/list operations
-    ├── build.tools.ts            # Widget building and validation
-    └── utils/
-        ├── generator.ts      # Widget generator PTY wrapper
-        ├── progress-tracker.ts # Progress/logging helper
-        ├── notifications.ts  # MCP notification utilities
-        └── response.ts       # Tool response helpers
-```
-
-## Architecture
-
-### Transport Layer
-
-The server supports two transport modes selected via CLI argument:
-
-- **STDIO** (default): Single-session stdin/stdout for CLI integration (Claude Code, Claude Desktop)
-- **HTTP**: Multi-session Express server on port 3100 for web clients and testing
-
-### Tool Registration
-
-Tools are registered directly with the MCP server using the SDK's `server.tool()` method. The current architecture uses category-based registration functions:
-
-```typescript
-// src/tools/index.ts
-export function registerAllTools(server: McpServer): void {
-    registerScaffoldingTools(server); // Widget creation
-    registerFileOperationTools(server); // File operations
-    registerBuildTools(server); // Building & validation
-}
-```
-
-**Available Tools**:
-
-- **Scaffolding**: `create-widget` - Scaffolds new widgets via PTY interaction
-- **File Operations**:
-    - `list-widget-files` - Lists files in widget directory
-    - `read-widget-file` - Reads widget file contents
-    - `write-widget-file` - Writes single file
-    - `batch-write-widget-files` - Writes multiple files atomically
-- **Build**: `build-widget` - Compiles widget and parses errors (TypeScript, XML, dependencies)
-
-### Resources
-
-MCP resources provide read-only documentation that clients can fetch on-demand:
-
-```typescript
-// src/resources/index.ts
-export function registerResources(server: McpServer): void {
-    registerGuidelineResources(server); // Widget development guidelines
-}
-```
-
-Resources are loaded from `docs/` directory and exposed via URIs like `resource://guidelines/property-types`.
-
-### Widget Generator Integration
-
-The `create-widget` tool uses `node-pty` to interact with the Mendix widget generator CLI. Key implementation details:
-
-- **PTY Simulation**: Required because the generator uses interactive prompts
-- **Prompt Detection**: Matches expected prompts in terminal output
-- **Answer Automation**: Sends pre-configured answers based on user input
-- **Progress Tracking**: Reports progress via MCP notifications
-
-## Development Commands
-
-```bash
-pnpm dev          # Development mode with hot reload (tsx watch)
-pnpm build        # TypeScript compilation + path alias resolution (preserves shebang)
-pnpm start        # Build and run (HTTP mode on port 3100)
-pnpm start:stdio  # Build and run (STDIO mode)
-pnpm lint         # ESLint check
+├── index.ts          # Entry point - transport mode selection
+├── config.ts         # Server configuration and constants
+├── security/         # Path traversal & extension validation
+├── server/           # HTTP and STDIO transport setup
+├── resources/        # MCP resources (guidelines)
+├── generators/       # XML and TSX code generators
+└── tools/            # MCP tool implementations
 ```
 
-## Adding New Tools
+## Adding Tools
 
-1. **Create tool file**: `src/tools/my-feature.tools.ts`
+1. Create `src/tools/my-feature.tools.ts`:
 
 ```typescript
 import type { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
 import { z } from "zod";
 
-const myToolSchema = z.object({
-    param: z.string().describe("Parameter description for LLM")
-});
-
 export function registerMyTools(server: McpServer): void {
     server.tool(
-        "my-tool", // Tool name
-        "Description shown to LLM", // Tool description
-        myToolSchema, // Input validation schema
-        async ({ param }) => {
-            // Handler with typed args
-            // Implementation
-            return {
-                content: [
-                    {
-                        type: "text",
-                        text: "Success message"
-                    }
-                ]
-            };
-        }
+        "my-tool",
+        "Description shown to LLM",
+        z.object({ param: z.string().describe("Parameter description") }),
+        async ({ param }) => ({
+            content: [{ type: "text", text: "Success" }]
+        })
     );
-
-    console.error("[my-feature] Registered 1 tool");
 }
 ```
 
-2. **Register in index**: Update `src/tools/index.ts`
+2. Register in `src/tools/index.ts`:
 
 ```typescript
 import { registerMyTools } from "./my-feature.tools";
 
 export function registerAllTools(server: McpServer): void {
-    registerScaffoldingTools(server);
-    registerFileOperationTools(server);
-    registerBuildTools(server);
-    registerMyTools(server); // Add here
+    // ... existing registrations
+    registerMyTools(server);
 }
 ```
 
-## Code Conventions
+## Code Patterns
 
-### Imports
+- **Imports**: Use `@/` path alias for absolute imports from `src/`
+- **Schemas**: All tool inputs require Zod schemas
+- **Errors**: Use `createErrorResponse()` from `@/tools/utils/response`
+- **Long operations**: Use `ProgressTracker` from `@/tools/utils/progress-tracker`
 
-- Use `@/` path alias for absolute imports from `src/`
-- Prefer specific file imports over barrel exports when dealing with circular dependencies
-- Group imports: node builtins → external packages → internal modules
+## Notification Behavior (Important for AI Agents)
 
-### Error Handling
+When using this MCP server, understand where different types of output appear:
 
-- Use `createErrorResponse()` for user-facing errors
-- Log to `console.error` (not stdout) in STDIO mode
-- Use `ProgressTracker` for long-running operations
+| Output Type                | Visibility                 | Purpose                                                 |
+| -------------------------- | -------------------------- | ------------------------------------------------------- |
+| **Tool Results**           | ✅ Visible in conversation | Final outcomes, structured data, success/error messages |
+| **Progress Notifications** | ❌ Not in conversation     | Client UI indicators only (spinners, progress bars)     |
+| **Log Messages**           | ❌ Not in conversation     | Debug console/MCP Inspector only                        |
 
-### Type Safety
+**Key Implications for AI Agents:**
 
-- All tool inputs must have Zod schemas
-- Tool handlers receive fully typed arguments via Zod inference
-- Use `McpServer` methods directly for type-safe tool registration
+1. **Don't expect intermediate progress in chat**: Long operations (scaffolding, building) will show results only when complete. The conversation won't contain step-by-step progress updates.
 
-## Testing
+2. **Tool results are authoritative**: Only tool result content appears in the conversation history. Use this for:
 
-Use MCP Inspector for interactive testing:
+    - Success confirmations with file paths
+    - Structured error messages with suggestions
+    - Any information the AI needs to continue the workflow
 
-```bash
-# STDIO mode
-npx @modelcontextprotocol/inspector node dist/index.js stdio
+3. **Progress tracking is for humans**: `sendProgress()` and `sendLogMessage()` are for human observers using MCP Inspector or UI indicators, not for AI decision-making.
 
-# HTTP mode
-pnpm start
-npx @modelcontextprotocol/inspector
-# Connect to http://localhost:3100/mcp
-```
+4. **When debugging**:
+    - If operations seem to "hang", check MCP Inspector's Notifications/Logs panels
+    - Progress notifications confirm the server is working, even if the chat is quiet
+    - This is per MCP specification, not a bug
 
-## Security
-
-All security validation is centralized in `src/security/guardrails.ts` for easy auditing:
+**Example Workflow:**
 
 ```typescript
-import { validateFilePath, ALLOWED_EXTENSIONS } from "@/security";
+// ❌ This progress won't appear in AI's context
+await sendProgress(context, 50, "Scaffolding widget...");
 
-// Validates path traversal and extension whitelist
-validateFilePath(widgetPath, filePath, true); // true = check extension
+// ✅ This result WILL appear in AI's context
+return createToolResponse(`Widget created at ${widgetPath}`);
 ```
 
-### Security Measures
-
-| Protection          | Function                  | Description                                                         |
-| ------------------- | ------------------------- | ------------------------------------------------------------------- |
-| Path Traversal      | `validateFilePath()`      | Blocks `..` sequences and resolved path escapes                     |
-| Extension Whitelist | `isExtensionAllowed()`    | Only allows: `.tsx`, `.ts`, `.xml`, `.scss`, `.css`, `.json`, `.md` |
-| Directory Boundary  | `isPathWithinDirectory()` | Ensures files stay within widget directory                          |
-
-When adding file operation tools, always use `validateFilePath()` from the security module.
-
-## Key Files Reference
-
-| File                       | Purpose                                          |
-| -------------------------- | ------------------------------------------------ |
-| `config.ts`                | Server constants (ports, timeouts, paths)        |
-| `security/guardrails.ts`   | Security validation (path traversal, extensions) |
-| `tools/index.ts`           | Tool registration aggregation                    |
-| `tools/utils/generator.ts` | Widget generator PTY prompts and defaults        |
-| `resources/guidelines.ts`  | Widget development guideline resources           |
-| `server/session.ts`        | HTTP session lifecycle management                |
-| `server/server.ts`         | MCP server factory and registration entry point  |
-
-## Common Patterns
-
-### Progress Notifications
-
-```typescript
-const tracker = new ProgressTracker({
-    context,
-    logger: "my-tool",
-    totalSteps: 5
-});
+## Testing
 
-tracker.start("initializing");
-await tracker.progress(25, "Step 1 complete");
-await tracker.info("Detailed log message", { key: "value" });
-tracker.stop();
+```bash
+npx @modelcontextprotocol/inspector node dist/index.js
 ```
 
-### Long-Running Operations
-
-- Use `ProgressTracker` for heartbeat and stuck detection
-- Set appropriate timeouts (see `SCAFFOLD_TIMEOUT_MS`)
-- Call `tracker.markComplete()` before expected long waits (e.g., npm install)
-
-## Roadmap Context
-
-Current focus is widget scaffolding. Planned additions:
+## Security
 
-- Widget property editing
-- XML configuration management
-- Build and deployment automation
+**Read before implementing file operations**: [docs/agent/security.md](docs/agent/security.md)
 
-When adding features, maintain the existing patterns for tool registration, progress tracking, and transport-agnostic design.
+All file operation tools must use `validateFilePath()` from `@/security` to prevent path traversal attacks.

packages/pluggable-widgets-mcp/README.md

@@ -197,6 +197,59 @@ npx @modelcontextprotocol/inspector
 
 This is useful for verifying tool behavior without needing a full AI client integration.
 
+## Understanding Feedback and Notifications
+
+This server uses MCP's notification system to provide progress updates and logging. However, **different types of feedback appear in different places**—not all feedback shows up in your chat conversation.
+
+### Where Different Types of Feedback Appear
+
+| Feedback Type              | Where It Appears                       | Example                                                           |
+| -------------------------- | -------------------------------------- | ----------------------------------------------------------------- |
+| **Tool Results**           | ✅ Chat conversation                   | Widget created at `/path/to/widget`, Build completed successfully |
+| **Progress Notifications** | ⚙️ Client UI (spinners, progress bars) | "Scaffolding widget...", "Building widget..."                     |
+| **Log Messages**           | 🔍 Debug console (MCP Inspector)       | Detailed operation logs, debug info                               |
+
+### Why Progress Doesn't Show in Chat
+
+**This is by design per the MCP specification**, not a bug. The MCP architecture separates concerns:
+
+- **`notifications/progress`** → Routed to client UI indicators (loading spinners, status bars)
+- **`notifications/message`** → Routed to debug/inspector consoles for developers
+- **Tool results** → Returned to the conversation when operations complete
+
+This means:
+
+- Long operations (scaffolding, building) will show **results** when complete
+- You won't see intermediate progress steps in the chat history
+- MCP Inspector shows all notifications in real-time (bottom-right panel)
+
+### Viewing Debug Output
+
+**With MCP Inspector:**
+
+1. Run: `npx @modelcontextprotocol/inspector node dist/index.js stdio`
+2. Execute a tool (e.g., `create-widget`)
+3. Watch the **Notifications panel** (bottom-right) for progress updates
+4. Check the **Logs panel** for detailed debug output
+
+**With Claude Desktop:**
+
+- Progress notifications may appear as UI indicators (client-dependent)
+- Check Claude Desktop's developer console for log messages (if available)
+- Tool results will always appear in the conversation
+
+### Expected Behavior Examples
+
+**During widget scaffolding:**
+
+- Chat shows: "Starting scaffolding..." → (wait) → "Widget created at `/path`"
+- Inspector shows: Step-by-step progress notifications for all 14 prompts
+
+**During widget building:**
+
+- Chat shows: "Building..." → (wait) → "Build successful" or structured error
+- Inspector shows: TypeScript compilation progress, dependency resolution
+
 ## Roadmap
 
 - [x] Widget scaffolding (`create-widget`)