Add setup-local-sdk skill for global.json paths feature by jfversluis · Pull Request #506 · dotnet/skills

jfversluis · 2026-04-08T13:02:47Z

Summary

Adds a new skill that guides users through installing a .NET SDK into a project-local directory using the global.json paths feature (new in .NET 10).

Files

File	Description
`plugins/dotnet/skills/setup-local-sdk/SKILL.md`	250-line skill with 12-step workflow
`tests/dotnet/setup-local-sdk/eval.yaml`	5 eval scenarios
`.github/CODEOWNERS`	Added entry for `@jfversluis` and `@redth`

What the skill covers

Verifying .NET 10+ host prerequisite
Installing a prerelease/specific SDK with dotnet-install scripts
Configuring global.json with paths and $host$
Installing workloads (MAUI, wasm-tools) on the local SDK
Cross-platform team install scripts
Updating .gitignore
Verification and cleanup guidance

Key design decisions

Workload commands always use ./.dotnet/dotnet rather than the system dotnet. Testing revealed that global.json paths routes SDK resolution correctly for build/run/test, but workload metadata is stored relative to the host's dotnet root, not the resolved SDK root (dotnet/sdk#49825).
No Aspire workload references since Aspire 9+ is NuGet package-based and no longer requires a workload.
.NET 10+ is a hard requirement -- no fallback guidance for older hosts.
All instructions tested against real .NET 11 preview.2 on macOS.

Adds a skill that guides users through installing a .NET SDK into a project-local directory using the global.json paths feature (.NET 10+). Includes: - 12-step workflow: verify host, install SDK, configure global.json, gitignore, workloads, team scripts, verification - MAUI and wasm-tools workload support - Cross-platform install scripts (bash + PowerShell) - 7 eval scenarios covering basic setup, exact version, team scripts, troubleshooting, incompatible host, existing .dotnet/, and MAUI workload - CODEOWNERS entry for @jfversluis and @Redth Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds a new .NET skill (setup-local-sdk) under the dotnet plugin to guide users through installing a project-local SDK via global.json paths (requires .NET 10+ host), along with evaluation scenarios and ownership routing.

Changes:

Added plugins/dotnet/skills/setup-local-sdk/SKILL.md with a 12-step workflow for local SDK installs (including workloads and team scripts).
Added tests/dotnet/setup-local-sdk/eval.yaml with 7 evaluation scenarios for the skill.
Updated .github/CODEOWNERS to assign owners for the new skill and tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File	Description
plugins/dotnet/skills/setup-local-sdk/SKILL.md	New skill documentation/workflow for project-local SDK installs via `global.json paths`, including workload and script guidance.
tests/dotnet/setup-local-sdk/eval.yaml	New eval coverage for the skill’s expected guidance/behavior.
.github/CODEOWNERS	Adds owners for the new skill and its tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

plugins/dotnet/skills/setup-local-sdk/SKILL.md

tests/dotnet/setup-local-sdk/eval.yaml

jfversluis · 2026-04-08T13:06:53Z

/evaluate

- Add Windows PowerShell equivalents for version check (Step 6) and workload list commands - Fix rollForward description: latestFeature rolls across feature bands, not just patches - Add global.json backup in team install scripts before overwriting - Fix eval scenario: provide explicit host version (9.0.306) in prompt so the incompatible-host test is deterministic Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

plugins/dotnet/skills/setup-local-sdk/SKILL.md

tests/dotnet/setup-local-sdk/eval.yaml

SKILL.md: - Windows -Version flag for exact installs (vs --version on bash) - Workload note includes both ./.dotnet/dotnet and .\.dotnet\dotnet.exe - Cleanup includes Windows Remove-Item equivalent - Revert/delete instructions include both OS forms eval.yaml: - Remove brittle --version assertion; use rubric for version flag check - Increase all scenario timeouts from 120s to 180s (many were timing out) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

jfversluis · 2026-04-08T13:32:28Z

/evaluate

239 lines, -52%): - Rewrite description: intent-focused USE FOR/DO NOT USE FOR pattern with activation keywords (MAUI, existing, testing, team) - Remove personas table, checkpoint markers, verbose notes - Condense all sections while preserving complete workflow 5 entries) - Remove redundant Validation section 5 scenarios): - Drop 'Handle existing' (handled naturally by Step 4) - Drop 'Verify SDK resolution' (covered by basic setup rubric) - Add expect_tools: ['bash'] to actionable scenarios - Reduce rubric items to 3-4 per scenario - Incompatible host scenario: 60s timeout (quick response) Expected improvements: - Token usage: -50% (skill is half the size) - Activation: USE FOR keywords match all prompts - Variance: fewer scenarios = less variability - Eval time: -40% (fewer scenarios, shorter timeouts) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

jfversluis · 2026-04-08T13:59:23Z

/evaluate

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

plugins/dotnet/skills/setup-local-sdk/SKILL.md

tests/dotnet/setup-local-sdk/eval.yaml

github-actions · 2026-04-08T14:06:56Z

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
setup-local-sdk	Basic local SDK setup with .NET 11 preview	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: skill / ✅ setup-local-sdk; tools: skill, create	—	❌ [1]
setup-local-sdk	Install a specific SDK version locally	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: skill / ✅ setup-local-sdk; tools: skill, create	—	✅ [2]
setup-local-sdk	Set up local SDK with MAUI workload	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: skill / ✅ setup-local-sdk; tools: skill, create	—	❌ [3]
setup-local-sdk	Create team install scripts	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: skill / ✅ setup-local-sdk; tools: skill, create	—	❌ [4]
setup-local-sdk	Detect incompatible .NET host version	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: skill, report_intent	—	❌ [5]

[1] (Isolated) Quality unchanged but weighted score is -1.7% due to: tokens (24374 → 28707), tool calls (3 → 4)
[2] ⚠️ High run-to-run variance (CV=8.50) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=18.33) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -0.0% due to: efficiency metrics
[4] ⚠️ High run-to-run variance (CV=6.58) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -0.2% due to: tokens (18193 → 23535)
[5] ⚠️ High run-to-run variance (CV=6.36) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -0.1% due to: efficiency metrics

⏰ timeout — run(s) hit the (60s, 180s) scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output (increase via timeout in eval.yaml)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

- MINGW/MSYS/CYGWIN treated as bash-capable (Git Bash), not PowerShell - Remove hardcoded 'Install directory' input row - Make allowPrerelease conditional on preview installs - Make errorMessage conditional on team scripts being created - Add PowerShell equivalent for .gitignore update - Add assertion to incompatible host eval scenario Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

jfversluis · 2026-04-08T14:15:04Z

/evaluate

jfversluis · 2026-04-08T14:20:41Z

Closing this fork-based PR and reopening from an upstream branch. The eval judges can't access secrets from fork PRs, causing all quality scores to default to 3.0/5. Same commits, new PR from dotnet/skills:skills/setup-local-sdk.

github-actions · 2026-04-08T14:22:30Z

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
setup-local-sdk	Basic local SDK setup with .NET 11 preview	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: report_intent, skill / ✅ setup-local-sdk; tools: report_intent, skill, bash, create	—	❌ [1]
setup-local-sdk	Install a specific SDK version locally	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: skill / ✅ setup-local-sdk; tools: skill, create	—	✅ [2]
setup-local-sdk	Set up local SDK with MAUI workload	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: report_intent, skill / ✅ setup-local-sdk; tools: skill, report_intent, bash, create	—	❌ [3]
setup-local-sdk	Create team install scripts	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: skill / ✅ setup-local-sdk; tools: report_intent, skill, bash, create	—	❌ [4]
setup-local-sdk	Detect incompatible .NET host version	3.0/5 ⏰ → 3.0/5 ⏰	✅ setup-local-sdk; tools: skill, report_intent	—	❌ [5]

[1] ⚠️ High run-to-run variance (CV=0.67) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -7.5% due to: tokens (30 → 8307), tool calls (0 → 1)
[2] ⚠️ High run-to-run variance (CV=2.58) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=3.46) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -5.0% due to: tokens (6045 → 14220)
[4] ⚠️ High run-to-run variance (CV=0.79) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -5.0% due to: tokens (6086 → 22329)
[5] ⚠️ High run-to-run variance (CV=4.63) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -0.2% due to: efficiency metrics

⏰ timeout — run(s) hit the (60s, 180s) scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output (increase via timeout in eval.yaml)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

jfversluis requested review from dbreshears and timheuer as code owners April 8, 2026 13:02

Copilot AI review requested due to automatic review settings April 8, 2026 13:02

Copilot started reviewing on behalf of jfversluis April 8, 2026 13:03 View session

jfversluis force-pushed the skills/setup-local-sdk branch from 5e7f56d to a027331 Compare April 8, 2026 13:05

Copilot AI reviewed Apr 8, 2026

View reviewed changes

jfversluis and others added 2 commits April 8, 2026 15:09

Trim SKILL.md to 500 lines, condense notes

47df259

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 8, 2026 13:11

Copilot started reviewing on behalf of jfversluis April 8, 2026 13:11 View session

This comment was marked as outdated.

Sign in to view

Copilot AI reviewed Apr 8, 2026

View reviewed changes

github-actions bot added a commit that referenced this pull request Apr 8, 2026

Update session data (PR #506)

f06f5a7

This comment was marked as outdated.

Sign in to view

Copilot AI review requested due to automatic review settings April 8, 2026 13:59

Copilot started reviewing on behalf of jfversluis April 8, 2026 13:59 View session

Copilot AI reviewed Apr 8, 2026

View reviewed changes

jfversluis closed this Apr 8, 2026

jfversluis mentioned this pull request Apr 8, 2026

Add setup-local-sdk skill for global.json paths feature #508

Open

Conversation

jfversluis commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Files

What the skill covers

Key design decisions

Related

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jfversluis commented Apr 8, 2026

Uh oh!

This comment was marked as outdated.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jfversluis commented Apr 8, 2026

Uh oh!

This comment was marked as outdated.

jfversluis commented Apr 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 8, 2026

Skill Validation Results

Uh oh!

jfversluis commented Apr 8, 2026

Uh oh!

jfversluis commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Skill Validation Results

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jfversluis commented Apr 8, 2026 •

edited

Loading