Skip to content
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
33e03db
feat: mount MCP servers as local CLIs after gateway starts
Copilot Apr 4, 2026
f3d7205
refactor: convert mount_mcp_as_cli to github-script, wrap prompt in x…
Copilot Apr 4, 2026
a6cfa91
feat: add tools.mount-as-clis flag, enable in smoke-copilot, filter f…
Copilot Apr 4, 2026
578ae43
fix: implement proper MCP session protocol in CLI wrappers
Copilot Apr 4, 2026
6183bd6
feat: move MCP CLI tools prompt to actions/setup/md template file
Copilot Apr 4, 2026
b15ce1b
fix: skip backtick-wrapped placeholder names in prompt validation
Copilot Apr 4, 2026
73ad591
feat: mount built-in MCPs (github, playwright, etc.) as CLIs
Copilot Apr 4, 2026
8e98c46
fix: use json.Marshal for GH_AW_MCP_CLI_SERVERS and add jq error hand…
Copilot Apr 4, 2026
5a24ced
docs: update smoke-copilot prompt to use CLI syntax for mounted MCP s…
Copilot Apr 4, 2026
5283e4a
fix: generate MCP CLI scripts in RUNNER_TEMP/gh-aw/mcp-cli/ to be ins…
Copilot Apr 4, 2026
d22ea30
Merge remote-tracking branch 'origin/main' into copilot/add-automatic…
Copilot Apr 4, 2026
2946954
fix: address code review feedback on mcp_cli_mount comments and mcpsc…
Copilot Apr 4, 2026
85f06a0
fix: bake gateway API key into generated CLI wrapper scripts at gener…
Copilot Apr 4, 2026
3a7b43c
Merge remote-tracking branch 'origin/main' into copilot/add-automatic…
Copilot Apr 4, 2026
7526d82
fix: bash syntax error in jq expression and agentic-workflows name mi…
Copilot Apr 4, 2026
52e4485
Merge remote-tracking branch 'origin/main' into copilot/add-automatic…
Copilot Apr 5, 2026
efc9b05
chore: merge main and recompile all 180 workflows
Copilot Apr 5, 2026
9698016
feat: improve mcp-clis prompt clarity and smoke-copilot instructions
Copilot Apr 5, 2026
17bff9f
Merge remote-tracking branch 'origin/main' into copilot/add-automatic…
Copilot Apr 5, 2026
35bf4d2
chore: merge main and recompile all workflows
Copilot Apr 5, 2026
acd44a6
fix: pass container-accessible gateway URL to CLI wrapper scripts
Copilot Apr 5, 2026
3214e78
fix: persist GH_AW_MCP_CLI_SERVERS to GITHUB_ENV so conversion script…
Copilot Apr 5, 2026
b04813a
Merge remote-tracking branch 'origin/main' into copilot/add-automatic…
Copilot Apr 5, 2026
81db7fa
chore: merge main and recompile 181 workflows
Copilot Apr 5, 2026
dceff35
Merge remote-tracking branch 'origin/main' into copilot/add-automatic…
Copilot Apr 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 45 additions & 27 deletions .github/workflows/smoke-copilot.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 7 additions & 6 deletions .github/workflows/smoke-copilot.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ tools:
- pelikhan
playwright:
web-fetch:
mount-as-clis: true
runtimes:
go:
version: "1.25"
Expand Down Expand Up @@ -125,12 +126,12 @@ strict: false

## Test Requirements

1. **GitHub MCP Testing**: Review the last 2 merged pull requests in ${{ github.repository }}
1. **GitHub CLI Testing**: Use the `github` CLI command to list the last 2 merged pull requests in ${{ github.repository }} (e.g., `github pull_request_read --method list --owner <owner> --repo <repo> --state closed --per_page 2`)
2. **MCP Scripts GH CLI Testing**: Use the `mcpscripts-gh` tool to query 2 pull requests from ${{ github.repository }} (use args: "pr list --repo ${{ github.repository }} --limit 2 --json number,title,author")
3. **Serena MCP Testing**:
- Use the Serena MCP server tool `activate_project` to initialize the workspace at `${{ github.workspace }}` and verify it succeeds (do NOT use bash to run go commands - use Serena's MCP tools)
- After initialization, use the `find_symbol` tool to search for symbols (find which tool to call) and verify that at least 3 symbols are found in the results
4. **Playwright Testing**: Use the playwright tools to navigate to <https://github.com> and verify the page title contains "GitHub" (do NOT try to install playwright - use the provided MCP tools)
3. **Serena CLI Testing**:
- Use the `serena` CLI command `serena activate_project --path ${{ github.workspace }}` to initialize the workspace and verify it succeeds (do NOT use bash to run go commands - use the serena CLI)
- After initialization, use `serena find_symbol --name_path <symbol>` to search for symbols and verify that at least 3 symbols are found in the results
4. **Playwright CLI Testing**: Use the `playwright` CLI command (`playwright browser_navigate --url https://github.com`) to navigate to <https://github.com> and verify the page title contains "GitHub" (do NOT try to install playwright - use the `playwright` CLI command)
5. **Web Fetch Testing**: Use the web-fetch tool to fetch https://github.com and verify the response contains "GitHub" (do NOT use bash or playwright for this test - use the web-fetch tool directly)
6. **File Writing Testing**: Create a test file `/tmp/gh-aw/agent/smoke-test-copilot-${{ github.run_id }}.txt` with content "Smoke test passed for Copilot at $(date)" (create the directory if it doesn't exist)
7. **Bash Tool Testing**: Execute bash commands to verify file creation was successful (use `cat` to read the file back)
Expand All @@ -141,7 +142,7 @@ strict: false
9. **Build gh-aw**: Run `GOCACHE=/tmp/go-cache GOMODCACHE=/tmp/go-mod make build` to verify the agent can successfully build the gh-aw project (both caches must be set to /tmp because the default cache locations are not writable). If the command fails, mark this test as ❌ and report the failure.
10. **Discussion Creation Testing**: Use the `create_discussion` safe-output tool to create a discussion in the announcements category titled "copilot was here" with the label "ai-generated"
11. **Workflow Dispatch Testing**: Use the `dispatch_workflow` safe output tool to trigger the `haiku-printer` workflow with a haiku as the message input. Create an original, creative haiku about software testing or automation.
12. **PR Review Testing**: Review the diff of the current pull request. Leave 1-2 inline `create_pull_request_review_comment` comments on specific lines, then call `submit_pull_request_review` with a brief body summarizing your review and event `COMMENT`. To test `reply_to_pull_request_review_comment`: use the `pull_request_read` tool (with `method: "get_review_comments"` and `pullNumber: ${{ github.event.pull_request.number }}`) to fetch the PR's existing review comments, then reply to the most recent one using `reply_to_pull_request_review_comment` with its actual numeric `id` as the `comment_id`. Note: `create_pull_request_review_comment` does not return a `comment_id` — you must fetch existing comment IDs from the GitHub API. If the PR has no existing review comments, skip the reply sub-test.
12. **PR Review Testing**: Review the diff of the current pull request. Leave 1-2 inline `create_pull_request_review_comment` comments on specific lines, then call `submit_pull_request_review` with a brief body summarizing your review and event `COMMENT`. To test `reply_to_pull_request_review_comment`: use the `github` CLI command (`github pull_request_read --method get_review_comments --owner <owner> --repo <repo> --pullNumber ${{ github.event.pull_request.number }}`) to fetch the PR's existing review comments, then reply to the most recent one using `reply_to_pull_request_review_comment` with its actual numeric `id` as the `comment_id`. Note: `create_pull_request_review_comment` does not return a `comment_id` — you must fetch existing comment IDs via the `github` CLI. If the PR has no existing review comments, skip the reply sub-test.

## Output

Expand Down
Loading
Loading