Add Agentic skills: draft-issue, review-blog-post, review-pull-request

[vitorvasc]

Summary

Contributes to #9397.

Introduces 3 agentic skills under .claude/skills/ to assist contributors working on this repo:

• draft-issue - draft issues against the real .github/ISSUE_TEMPLATE/ templates
• review-blog-post - review blog PRs/drafts for frontmatter, conventions, gh-url-hash, and OTel terminology
• review-pull-request - review PRs against CI checks, CLA/approval workflow, refcache, and content conventions

All content introduced in this PR is an initial proposal. Iteration on the scope, prompts, and conventions are expected before merging. Feedback is always welcome :)

Usage

/review-pull-request

Arguments:

/review-pull-request <PR number or URL>

Examples:

/review-pull-request 9614
/review-pull-request https://github.com/open-telemetry/opentelemetry.io/pull/9614

/review-blog-post

Arguments:

/review-blog-post <blog post path or PR number>

Examples:

/review-blog-post content/en/blog/2026/my-new-post/index.md
/review-blog-post 9580

/draft-issue

Arguments:

/draft-issue <description of issue to draft>

Examples:

/draft-issue the Kubernetes getting-started page references a deprecated flag
/draft-issue feature: add a copy-to-clipboard button on code samples
/draft-issue blog: proposal for a post about exemplars in Prometheus

[theletterf]

Interesting! How are arguments parsed?

[vitorvasc]

Interesting! How are arguments parsed?

@theletterf, Claude interprets them semantically, so I haven't been explicitly documenting that in the instructions. It's worked fine so far, but given our recent threads on Slack and @chalin's experience while using Copilot to interpret agent skills, I think it's worth being explicit about the expected arguments.

I'll update the skills to make it clearer. :)

[Copilot]

Pull request overview

Adds an initial set of contributor-focused “agentic skills” and supporting assets under .claude/ to help draft issues, review PRs/blog posts, and batch-triage GitHub issues for the OpenTelemetry website repo (and optionally other repos), including a triage sub-agent, schemas, and a frontmatter validation hook.

Changes:

• Introduces 4 skills: otel-triage, otel-pr-review, otel-issue-draft, otel-blog-review.
• Adds an otel-issue-triager sub-agent plus triage profile/state JSON schemas and repo-specific triage profiles.
• Adds a .claude PreToolUse hook intended to validate blog frontmatter on Write/Edit operations.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
.claude/skills/otel-triage/SKILL.md	Defines the batch issue triage workflow, report/state format, and profile merge behavior.
.claude/skills/otel-pr-review/SKILL.md	Documents PR review workflow and CI check semantics for opentelemetry.io.
.claude/skills/otel-issue-draft/SKILL.md	Guides drafting issues aligned to repo templates and label taxonomy.
.claude/skills/otel-blog-review/SKILL.md	Documents blog post review rules (frontmatter, gh-url-hash, terminology, etc.).
.claude/agents/otel-issue-triager.md	Sub-agent spec for per-issue deep triage analysis and dossier generation.
.claude/schemas/triage-state.schema.json	JSON schema for triage run state tracking.
.claude/schemas/triage-profiles.schema.json	JSON schema for triage profiles (repo config + evaluation criteria).
.claude/data/opentelemetry-website.yml	Repo profile for open-telemetry/opentelemetry.io (keywords, taxonomy, templates).
.claude/data/bloomberg-mentorship.yml	Evaluation profile for Bloomberg mentorship suitability.
.claude/hooks/hooks.json	Registers a pre-tool hook for Write/Edit actions.
.claude/hooks/frontmatter-check.sh	Hook script to validate blog post frontmatter and heading rules.

[Copilot]

The H1 check scans the entire body with a simple grep '^# ', which will also match shell comments or other lines inside fenced code blocks (common in blog posts), causing false failures. Consider tracking fenced-code state (``` / ~~~) and only flagging # headings outside code fences, or using a markdown-aware parser.

Suggested change

	# Check for H1 headings in content (after frontmatter)
	BODY=$(echo "$CONTENT" | awk 'BEGIN{n=0} /^---$/{n++; next} n>=2{print}')
	if echo "$BODY" | grep -qE '^# [^#]'; then
	# Check for H1 headings in content (after frontmatter), ignoring fenced code blocks
	BODY=$(echo "$CONTENT" | awk 'BEGIN{n=0} /^---$/{n++; next} n>=2{print}')
	if echo "$BODY" | awk '
	BEGIN {
	in_fence = 0
	fence_char = ""
	}
	/^[[:space:]]*```/ {
	if (!in_fence) {
	in_fence = 1
	fence_char = "`"
	next
	}
	if (fence_char == "`") {
	in_fence = 0
	fence_char = ""
	next
	}
	}
	/^[[:space:]]*~~~/ {
	if (!in_fence) {
	in_fence = 1
	fence_char = "~"
	next
	}
	if (fence_char == "~") {
	in_fence = 0
	fence_char = ""
	next
	}
	}
	!in_fence && /^# [^#]/ {
	found = 1
	exit
	}
	END {
	exit found ? 0 : 1
	}
	'; then

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: otelbot[bot] / name: otelbot (b07f64c, fc2d197)
• ✅ login: vitorvasc / name: Vitor Vasconcellos (0195f15, 0ab1e1d, 495920f, 8342181, 15de35e, 1bce79b, 2262678, 26800c7, 28b0e8b, 2eae4e7, 36cfce0, 3ce688a, 3d6c8fa, 3eab61e, 49f1ec3, 4ac1d03, 4ec2f17, 5d2b59a, 6e7dc34, 73a8ed7, 756422a, 7628ae3, 7a5af3a, 7b92013, 7de665b, 8480fbd, 89a0ae3, 8e43588, 94b5733, 9540d6b, 984b474, 9dab0fb, a447d96, b8fcbdc, bceb1b3, bcecd90, c07ba12, d4f1337, d8cd6a3, e4c2b71, e6bfe03, eb7f3cd, f51b3a1, f614530)

[chalin]

Thanks for the updates. My main concern is that it feels like non-DRY docs to have so much content under the references subfolders, but I could be wrong. WDYT?

Included below is a review from Claude, with some focus on DRY-ness.

Review: Quality, Consistency, DRYness, Test Coverage

Overall assessment

The branch has evolved materially since my earlier reviews. Notably:

• The bash hook was rewritten as Node (scripts/validate/front-matter-check/index.mjs) with a pure validate() function and unit tests.
• triage-issue was decomposed into an orchestrator SKILL.md + 6 references/*.md files (progressive disclosure).
• review-pull-request got the same treatment with 6 references/*.md files.
• content/en/site/skills/_index.md now documents both skills and the new hook.

This is a big improvement. Most of my prior suggestions (progressive disclosure, collocating the hook as code+tests, self-consistency with existing conventions like test:local-tools) have been addressed. The issues below are what remains.

---

🔴 Blocking / correctness issues

1. triage-profiles.schema.json — verdict_labels schema is internally inconsistent

"verdict_labels": {
  "required": ["recommended", "maybe", "not_suitable"],
  "additionalProperties": false,
  "properties": { "recommended": { … } }   // ← only `recommended` defined
}

With additionalProperties: false, the three required keys maybe and not_suitable are simultaneously required and disallowed. No value can validate — any call in triage-issue Phase 0.3 ("validate against schema, abort on failure") will always fail.

2. bloomberg-mentorship.yml is missing two required verdict_labels

verdict_labels:
  recommended: 'mentorship:bloomberg'

Even after fixing (1), this file lacks maybe and not_suitable. Consequence: when a reviewer runs /triage-issue --profile bloomberg-mentorship, profile load fails (or the skill silently skips validation, which is also a bug).

Fix: either mark only recommended as required in the schema, or add sensible defaults/empty strings for maybe / not_suitable in the YAML.

---

🟡 DRY / mutual consistency issues

3. Label taxonomy duplicated between draft-issue/SKILL.md and .claude/data/opentelemetry-website.yml

The draft-issue skill maintains a handwritten label taxonomy in prose. The opentelemetry-website.yml profile maintains a structured label taxonomy. They've already drifted:

Label	In draft-issue SKILL.md	In opentelemetry-website.yml
analytics+observability, IA, metadata-quality, dependencies, blocked	✅	❌
contribfest, mentorship:bloomberg	✅	❌
docs:mobile, docs:blog, docs:registry, docs:javascript, docs:vendor-list	✅	❌
sig:collector:refactor	✅	❌
triage:rejected, triage:rejected:duplicate, etc.	✅	❌

triage-issue/references/label-taxonomy.md explicitly says draft-issue is the source of truth, but the YAML schema says labels must come from PROFILE.repo.label_taxonomy. Pick one:

• Recommended: make the YAML the source of truth; have draft-issue/SKILL.md point to it (keep the "PR-only labels" warning inline since that's editorial guidance, not a label list).

4. Action tokens / confidence tiers / close-reason mapping duplicated

Three files all carry the same tables with explicit "mirror when editing" notes:

• .claude/agents/otel-issue-triager.md (488 lines, embeds everything)
• .claude/skills/triage-issue/references/analysis-checklist.md
• .claude/skills/triage-issue/references/gh-commands.md

The subagent file is the only version actually executed by the subagent at runtime; the orchestrator references are read by the human/main agent. You cannot trivially remove the duplication, but you can:

• Refactor otel-issue-triager.md the same way triage-issue was refactored — move the checklist / tables into triage-issue/references/*.md and have the subagent file Read those files at runtime instead of inlining them. That makes the reference files truly the single source of truth.

5. Comment templates exist in three places

• .claude/data/opentelemetry-website.yml → comment_templates: (5 templates)
• .claude/skills/triage-issue/references/comment-templates.md (fallback variants)
• .claude/agents/otel-issue-triager.md → ## Comment Templates (fallback variants, again)

The last two are near-identical fallbacks. Pick one and reference it from the other.

6. Staleness tiers table duplicated verbatim

Same 4-row table (Critical / High / Medium / Low with day thresholds) in:

• .claude/agents/otel-issue-triager.md §3
• .claude/skills/triage-issue/references/analysis-checklist.md §3

---

🟡 Documentation self-consistency nits

7. _index.md says "covered by *.test.mjs files" (plural) — only one exists today

Pure logic lives in `index.mjs` and is covered by `*.test.mjs` files in the same
folder (`npm run test:local-tools` to run them).

The phrasing copies content/en/site/build/ci-workflows.md (which covers multiple tools), but here there's a single index.test.mjs. Acceptable as-is (the glob still works for future additions), but could read "is covered by index.test.mjs" for accuracy.

8. The "Pure logic lives in …" paragraph is under ## Hooks but is only about the one hook

Visually it reads as a generic section-level note. If you add a second hook later without tests, this paragraph becomes misleading. Consider moving it into the bullet for the blog front-matter hook.

9. .claude/hooks/hooks.json reaches outside the plugin directory

"command": "printf '%s' \"$TOOL_INPUT\" | node ${PLUGIN_DIR}/../scripts/validate/front-matter-check/index.mjs"

${PLUGIN_DIR}/.. traverses out of .claude/ and relies on the plugin being embedded in this repo (i.e., not installed as a stand-alone Claude plugin). This is intentional per the content/en/site/skills/_index.md split — hook config in .claude/, hook source under scripts/validate/. Worth a one-line comment in hooks.json (or a note in the site docs) so future maintainers don't "fix" the path.

10. .claude/agents/otel-issue-triager.md frontmatter name: OpenTelemetry Issue Triager

Noted in the earlier review — still the case on this branch. Rename to otel-issue-triager (matching file name and kebab-case convention) to avoid confusion with skill naming rules.

---

🟢 Test coverage — mostly good, a few gaps

scripts/validate/front-matter-check/index.test.mjs has 9 focused tests exercising validate() directly. Solid.

Not covered (suggest adding):

Gap	Why it matters
content vs new_string fallback (Edit vs Write tool)	This logic is in main() not validate(), so the behavior Edit tool calls depend on is untested. Could be covered by refactoring the precedence into a helper and testing it, or by spawning the script in a subprocess test.
CRLF line endings	lines[0] !== '---' would be false ('---\r'); the hook silently no-ops on Windows-style input.
Missing closing --- (unterminated frontmatter)	Current code returns [] (silent pass). Confirm this is intended and lock it in with a test.
H1-looking text inside a fenced code block	/^# [^#]/m would match # comment inside a ```bash block and falsely flag. Non-trivial to fix without a real Markdown parser, but at minimum document the limitation or test that it's accepted as a known limitation.
Author with surrounding single quotes '[Jane](https://…)'	The AUTHOR_LINK_RE allows it, but it's not in the tests, and it's the idiomatic form in the repo per review-blog-post/SKILL.md.
YAML block scalar variants >+, `	,
Schema validation tests	No tests validate bloomberg-mentorship.yml or opentelemetry-website.yml against triage-profiles.schema.json. A 10-line test with ajv would have caught issues 1 & 2 above.

Minor code nits:

• valueOf() uses replace(/^['"]|['"]$/g, '') which strips quotes independently — "foo' becomes foo. Fine for well-formed input, but imprecise.
• find() uses startsWith('${key}:'), so a literal line like title: x (indented) is skipped — acceptable, but worth a comment.
• The // Run only when invoked directly guard import.meta.url === \file://${process.argv[1]}`will break on Windows (paths use backslashes, nofile://prefix inargv[1]`). Since the repo is Unix-first, not a blocker.

---

✅ What's done well on this branch

• Pure-function architecture of index.mjs makes testing trivial and matches the existing scripts/gh/specs/pick-branch/ convention — good DRY-in-conventions.
• triage-issue progressive disclosure is nicely executed: each references/*.md starts with a clear "When to read:" callout.
• review-pull-request references/ split shrunk the SKILL.md from 454 → 205 lines while preserving all substantive detail.
• Cross-skill links (labels.md → process-rules.md#stale-handling) keep the references network navigable.
• _index.md site docs are consistent with the rest of content/en/site/ (same phrasing patterns as ci-workflows.md).
• Refcache updates include all new publicly linked paths (hooks.json, scripts/validate/…, each SKILL.md).
• Test file naming (index.test.mjs) matches test:local-tools glob without any package.json change.

---

Actionable summary

Blocking

1. Fix triage-profiles.schema.json: either drop maybe/not_suitable from verdict_labels.required, or add them to properties. The current schema rejects every input.
2. Fix bloomberg-mentorship.yml: add maybe and not_suitable entries under verdict_labels (or leave them empty if allowed by the fixed schema).

DRY / consistency (pick one canonical source)

3. Label taxonomy: consolidate — likely make opentelemetry-website.yml the source and have draft-issue/SKILL.md defer to it; then audit the ~15 labels currently only in one of the two.
4. Refactor .claude/agents/otel-issue-triager.md to read the same triage-issue/references/*.md files the orchestrator uses — eliminating the duplicated action tokens, confidence tiers, staleness table, close-reason mapping, and comment templates.
5. Delete either triage-issue/references/comment-templates.md or the equivalent section in otel-issue-triager.md.

Test coverage

6. Add an ajv-based schema validation test that loads every .claude/data/*.yml against triage-profiles.schema.json. This catches issues like (2) in CI.
7. Add front-matter-check tests for: CRLF line endings, missing closing ---, Edit-tool new_string path, single-quoted single-line author, H1 inside fenced code block (either fix or document as known limitation).

Docs nits

8. In content/en/site/skills/_index.md, move the "Pure logic lives in index.mjs …" paragraph under the blog front-matter hook bullet (it's per-hook, not section-wide). Consider singular "index.test.mjs".
9. Add a brief comment in .claude/hooks/hooks.json (or adjacent docs) explaining the ${PLUGIN_DIR}/../scripts/validate/... traversal is intentional for this monorepo layout.
10. Rename .claude/agents/otel-issue-triager.md frontmatter name: value from OpenTelemetry Issue Triager → otel-issue-triager.

Items 1, 2, and 6 are the only hard blockers — everything else is improvement.

[vitorvasc]

/fix:format

[otelbot-docs]

✅ fix:format applied successfully in run 25069384208.

[chalin]

TY @vitorvasc! ✨
Let's give this a spin, and incrementally improve. I think you've removed the triage skill (if so update the opening comment). Looking forward to seeing that land as well in a followup PR, if that's what you choose.

[vitorvasc]

@chalin - that's correct! Just updated the title and description, and I'll raise the follow-up PR for the triaging skill as soon as I get back to it. :)

Let me know if you run into any issues with the skills in this PR.