Add VNet enterprise tests for Foundry IQ KB MCP server#545
Closed
farzad528 wants to merge 1164 commits intomicrosoft-foundry:mainfrom
Closed
Add VNet enterprise tests for Foundry IQ KB MCP server#545farzad528 wants to merge 1164 commits intomicrosoft-foundry:mainfrom
farzad528 wants to merge 1164 commits intomicrosoft-foundry:mainfrom
Conversation
Update to me (for now), team not available here yet.
…ey-patch-1 Add CODEOWNERS
…r/fix-role-assignment Fixed Workspace GUID and References
…tion based on user feedback.
…to yulin/voice-live-update
…-1rp Adapting Semantic Kernel agents to use Foundry project endpoint
Tweaked readme and comments in config files
Delete empty archive directory
…ve-update update voice live agent sample for FDP projects
…le_api Fix sample
…amples Update C# samples to use latest SDK
* [Hosted Agents] fix the dependency version * misc: bump version
* Automatic fixes * Mengla/add af foundry tools sample (microsoft-foundry#1) * Bump hosted agents version to 1.0.0b8 * remove agent_framework dependency in requrements.txt * add a sample for agentframework with foundry tool * align naming convention --------- Co-authored-by: junanchen <[email protected]> * use latest from_agent_framework signature (microsoft-foundry#2) * use latest from_agent_framework signature * remove keyword * lg tools (microsoft-foundry#3) * add lg tools sample, bump agentserver version to 1.0.0b9 * rename FoundryToolsReactAgent * add desc FoundryToolsReactAgent * remove unused file --------- Co-authored-by: foundry-samples automation <[email protected]> Co-authored-by: junanchen <[email protected]> Co-authored-by: Jun'an Chen <[email protected]>
…undry#491) * Automatic fixes * Mengla/add af foundry tools sample (microsoft-foundry#1) * Bump hosted agents version to 1.0.0b8 * remove agent_framework dependency in requrements.txt * add a sample for agentframework with foundry tool * align naming convention --------- Co-authored-by: junanchen <[email protected]> * use latest from_agent_framework signature (microsoft-foundry#2) * use latest from_agent_framework signature * remove keyword * lg tools (microsoft-foundry#3) * add lg tools sample, bump agentserver version to 1.0.0b9 * rename FoundryToolsReactAgent * add desc FoundryToolsReactAgent * remove unused file * restore the correct readme --------- Co-authored-by: foundry-samples automation <[email protected]> Co-authored-by: junanchen <[email protected]> Co-authored-by: Jun'an Chen <[email protected]>
… version pinning. Change to use Acr image build, to not require local docker. (microsoft-foundry#494)
…osoft-foundry#496) * Initial plan * Update FoundryA365 README to clarify agent blueprint approval workflow Co-authored-by: fosteramanda <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: fosteramanda <[email protected]>
…tandard-agent-setup (microsoft-foundry#503) - Added dnsZonesSubscriptionId parameter to allow DNS zones from different subscriptions - Updated all DNS zone references in private-endpoint-and-dns.bicep to support cross-subscription scope - Added comprehensive documentation in README.md about cross-subscription requirements - Updated main.bicepparam with clear usage examples and warnings - Maintains backward compatibility (empty parameter uses current subscription) This enables users to reference existing private DNS zones located in a different Azure subscription than the deployment subscription, which is useful in enterprise environments with centralized DNS management.
* update agent usage detail * add use agent in response --------- Co-authored-by: bobogogo1990 <[email protected]>
* Automatic fixes * Mengla/add af foundry tools sample (microsoft-foundry#1) * Bump hosted agents version to 1.0.0b8 * remove agent_framework dependency in requrements.txt * add a sample for agentframework with foundry tool * align naming convention --------- Co-authored-by: junanchen <[email protected]> * use latest from_agent_framework signature (microsoft-foundry#2) * use latest from_agent_framework signature * remove keyword * lg tools (microsoft-foundry#3) * add lg tools sample, bump agentserver version to 1.0.0b9 * rename FoundryToolsReactAgent * add desc FoundryToolsReactAgent * LG HITL example * remove unused file * validated HITL * af ai-function based HITL * af workflow hitl * workflow hitl * restore the correct readme * add workflow for python sample test * modify triggering condition * fix requirements.txt * add missing env * fix workflow * fix type checking issue * Fix workflow hitl issue * fix requirements.txt * fix sample * fix sample * chore: Add C# hosted agent testing to CI/CD pipeline * minor fix for samples * Automatic fixes * fix: SystemUtilityAgent sample * chore: improve C# build error reporting in CI workflow * test: ci error details * Revert "test: ci error details" This reverts commit e48fe24. * misc: increate error message limit * Reapply "test: ci error details" This reverts commit 7c6e1ed. * Revert "Reapply "test: ci error details"" This reverts commit d416fd5. * bump to 1.0.0b10 (microsoft-foundry#8) * exclude CI related changes * sync latest manually --------- Co-authored-by: foundry-samples automation <[email protected]> Co-authored-by: junanchen <[email protected]> Co-authored-by: Jun'an Chen <[email protected]> Co-authored-by: Declan <[email protected]> Co-authored-by: lixiaoli <[email protected]>
…oundry#500) * Fixes requirements.txt * Fixes evaluation implementation * Corrected C# Evaluations implementation * Correct lost section tags
* Add RFT sample for score_model grader, delete the old incorrect sample * Add RFT samples for python grader * Add RFT sample for string_check grader * Add RFT sample for text similarity grader * Add RFT sample for multi grader * Update README * Fix for reasoning model doesn't support temperature * Remove multi-grader sample as the support is not there in CLI yet * Refatoring, update datasets, and fix errors * Update medmcq dataset to match the file formats - reference_answer was missing
Merging on behalf of hosted agent PM + eng reviewers * Sample for local tool calls in agent * Remove image since it is not relevant.
…ies when calling tools (microsoft-foundry#514) * Sample for local tool calls in agent * Remove image since it is not relevant. * Updating sample to use hotels instead of date and time for local tool calls
…oyment. Include sample launch.json if using VS Code (microsoft-foundry#516) * Sample for local tool calls in agent * Remove image since it is not relevant. * Updating sample to use hotels instead of date and time for local tool calls * -Align PROJECT_ENDPOINT and MODEL_DEPLOYMENT_NAME with azd conventions for agents and add VS Code launch.json for easier local debugging
Removed the questions.jsonl file from CODEOWNERS and updated the CLI fine-tuning ownership.
… substitution (microsoft-foundry#518) * Sample for local tool calls in agent * Remove image since it is not relevant. * Updating sample to use hotels instead of date and time for local tool calls * -Align PROJECT_ENDPOINT and MODEL_DEPLOYMENT_NAME with azd conventions for agents and add VS Code launch.json for easier local debugging * Updating model to use chat reference from resources
…y#519) - Move C# samples into AgentFramework/ subfolder - Update .gitignore Note: python/hosted-agents/agent-framework/agent-with-local-tools/ and python/hosted-agents/code-interpreter-custom/ unchanged (foundry-samples is source of truth)
…ry#520) * Sample for local tool calls in agent * Remove image since it is not relevant. * Updating sample to use hotels instead of date and time for local tool calls * -Align PROJECT_ENDPOINT and MODEL_DEPLOYMENT_NAME with azd conventions for agents and add VS Code launch.json for easier local debugging * Updating model to use chat reference from resources * Adding AgentWithLocalTools csharp sample * Updating default model * Moving to AgentFramework folder
…required. (microsoft-foundry#524) * Sample for local tool calls in agent * Remove image since it is not relevant. * Updating sample to use hotels instead of date and time for local tool calls * -Align PROJECT_ENDPOINT and MODEL_DEPLOYMENT_NAME with azd conventions for agents and add VS Code launch.json for easier local debugging * Updating model to use chat reference from resources * Adding AgentWithLocalTools csharp sample * Updating default model * Moving to AgentFramework folder * Remove credential for agent framework as it is not required.
…foundry#525) * Update Python enterprise agent tutorial to v2 SDK Migrate from azure-ai-agents (v1) to azure-ai-projects 2.0.0b3 (v2): main.py: - Replace AgentsClient with AIProjectClient + openai_client - Replace create_agent() with agents.create_version() + PromptAgentDefinition - Replace SharepointTool with SharepointPreviewTool + SharepointGroundingToolParameters - Replace McpTool with MCPTool (require_approval='always') - Replace thread/message/run pattern with Responses API - Replace RunHandler/ToolApproval with McpApprovalResponse - Use context manager pattern for client lifecycle evaluate.py: - Standardize env vars: AZURE_AI_PROJECT_ENDPOINT -> PROJECT_ENDPOINT - Standardize env vars: AZURE_AI_MODEL_DEPLOYMENT_NAME -> MODEL_DEPLOYMENT_NAME - Update usage comment to reference azure-ai-projects==2.0.0b3 requirements.txt: - Remove azure-ai-agents==1.2.0b6 - Update azure-ai-projects from 1.0.0 to 2.0.0b3 - Add openai dependency * Address review: output name/version, rename chat_with_assistant to create_agent_response * Use context managers for credential, project client, and openai client * chore: migrate C# ModernWorkplaceAssistant to v2 SDK (Azure.AI.Projects + Responses API) - Replace PersistentAgentsClient with AIProjectClient + PromptAgentDefinition - Replace threads/runs with Responses API (ProjectResponsesClient) - Replace SharepointToolDefinition with SharepointPreviewTool + SharePointGroundingToolOptions - Replace MCPToolDefinition with McpTool from OpenAI.Responses - Replace SubmitToolApprovalAction with McpToolCallApprovalRequestItem pattern - Update csproj: Azure.AI.Projects 2.0.0-beta.1 + Azure.AI.Projects.OpenAI 1.0.0-beta.5 - Remove Azure.AI.Agents.Persistent dependency - Preserve all snippet ID tags for doc system * fix: use correct v2 SDK type names and package versions for C# sample - SharepointPreviewTool -> SharepointAgentTool (actual type in Azure.AI.Projects.OpenAI) - GetProjectOpenAIClient() -> .OpenAI property accessor - Azure.AI.Projects 2.0.0-beta.1 -> 1.2.0-beta.5 (available on azure-sdk-dev feed) - Azure.Identity 1.13.1 -> 1.17.1 (required by Azure.AI.Projects 1.2.0-beta.5) - Add NoWarn OPENAI001 to csproj (preview API suppression) - GetConnectionAsync now includes required includeCredentials parameter - Build verified: dotnet build succeeds with azure-sdk-dev feed packages --------- Co-authored-by: Jon Burchel <[email protected]>
…t-foundry#526) * Sample for local tool calls in agent * Remove image since it is not relevant. * Updating sample to use hotels instead of date and time for local tool calls * -Align PROJECT_ENDPOINT and MODEL_DEPLOYMENT_NAME with azd conventions for agents and add VS Code launch.json for easier local debugging * Updating model to use chat reference from resources * Adding AgentWithLocalTools csharp sample * Updating default model * Moving to AgentFramework folder * Remove credential for agent framework as it is not required. * Pin azure-ai-agentserver-agentframework to 1.0.0b10
Co-authored-by: foundry-samples-repo-sync[bot] <foundry-samples-repo-sync[bot]@users.noreply.github.com>
…sources) - Reverted 15-private-network-standard-agent-setup to match upstream/main exactly - Added 19-hybrid-private-resources-agent-setup: hybrid architecture with public AI Services endpoint and private backend resources (AI Search, Cosmos DB, Storage) - Template 19 enables portal-based agent testing while keeping data resources private - Includes TESTING-GUIDE.md, test scripts, and architecture diagrams in template 19
Updates based on actual testing: - Fixed SDK usage: use AzureAISearchTool class, not dict format - Added note about portal limitation with network injection - Added correct API version (2025-05-15-preview) for REST API - Added MCP HTTP server implementation (Streamable HTTP transport) - Updated MCP deployment instructions with proper HTTP-based server - Added connection name lookup command - Fixed test script to use AzureAISearchTool class MCP Server: - Added mcp-http-server/server.py - Flask-based MCP server - Added mcp-http-server/Dockerfile for containerization - Implements JSON-RPC over HTTP as required by Azure AI Agents
Add test scripts and infrastructure for validating the Foundry IQ Knowledge Base MCP server with Azure AI Foundry Agent Service V2 in a VNet/private endpoint scenario (template 19). New files: - foundry-iq-kb-mcp/Dockerfile: Container image for the MCP server - tests/create_sample_kb.py: Seed AI Search with sample KB data - tests/test_kb_api_connectivity.py: Direct REST tests against AI Search KB API - tests/test_foundry_iq_kb_mcp.py: Agent V2 integration tests via MCP tool - tests/TESTING-GUIDE-KB-MCP.md: Full testing guide with VNet setup guidance The MCP server must be containerized inside the VNet when AI Search has public access disabled, as it makes direct HTTP calls to the KB API endpoint. Co-authored-by: Copilot <[email protected]>
Covers 11 enterprise scenarios with pre-configured variables: 1. Public (basic), 2. PNA disabled, 3. VNet injection, 4. PNA+PLE+VPN, 5. CMK, 6. Custom subdomain, 7. Disable local auth, 8. RBAC only, 9. UAI, 10. BYO Storage, 11. Actionable error messaging Use with VS Code REST Client extension. Co-authored-by: Copilot <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds test scripts and infrastructure for validating the Foundry IQ Knowledge Base MCP server with Azure AI Foundry Agent Service V2 in a VNet/private endpoint scenario (template 19).
This mirrors the testing approach from #450 but specifically targets the Foundry IQ KB MCP server and Azure AI Search Knowledge Base API (\2025-11-01-preview).
New Files
Architecture
\
Test Client → Agent Service → Data Proxy → (networkInjection) → VNet
├── Foundry IQ KB MCP Server (Container App)
└── Azure AI Search (Private Endpoint)
\\
Key finding: The MCP server must be containerized inside the VNet when AI Search has public access disabled, because the MCP server makes direct HTTP \ etch()\ calls to the AI Search KB API endpoint. A hosted MCP server (e.g., on Vercel) cannot reach private endpoints.
Test Coverage
Related