Skip to content

docs: add supersede-older-reviews, review safe outputs, version baseline #789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
PureWeen merged 2 commits into from
Apr 28, 2026
+76 −10
Merged

docs: add supersede-older-reviews, review safe outputs, version baseline #789

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ff15524
docs: add supersede-older-reviews, review safe outputs, version baseline
PureWeen Apr 28, 2026
49a5ff8
docs: comprehensive v0.68.4–v0.71.1 feature audit
PureWeen Apr 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 32 additions & 3 deletions .claude/skills/gh-aw-guide/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ description: >-

This skill provides a complete reference for building, securing, and maintaining GitHub Agentic Workflows. It covers the gh-aw platform's architecture, security model, and all available features.

> **Version baseline:** This guide targets **v0.68.3** (the default version installed with `gh extension install github/gh-aw`). Features from later versions are marked with their minimum version (e.g., `(v0.70.0+)`). When using a feature marked with a version tag, verify your compiler version with `gh aw --version` and upgrade if needed: `gh extension install github/gh-aw --pin <version>`.

## Quick Start

gh-aw workflows are authored as `.md` files with YAML frontmatter, compiled to `.lock.yml` via `gh aw compile`. The lock file is auto-generated — **never edit it manually**.
Expand Down Expand Up @@ -75,10 +77,13 @@ gh aw upgrade # Upgrade gh-aw CLI extension
| Custom post-processing jobs for agent output | `safe-outputs.jobs:` custom jobs with MCP tool access | [Custom Safe Outputs](https://github.github.com/gh-aw/reference/custom-safe-outputs/) |
| Wrapping GitHub Actions as agent-callable tools | `safe-outputs.actions:` action wrappers | [Custom Safe Outputs](https://github.github.com/gh-aw/reference/custom-safe-outputs/) |
| Triggering CI on agent-created PRs | `github-token-for-extra-empty-commit:` on `create-pull-request` | [Triggering CI](https://github.github.com/gh-aw/reference/triggering-ci/) |
| No guard against agent approving PRs | `allowed-events: [COMMENT]` on `submit-pull-request-review` (prefer over `[COMMENT, REQUEST_CHANGES]` to avoid stale blocking reviews) | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) |
| No guard against agent approving PRs | `allowed-events: [COMMENT]` on `submit-pull-request-review`; or `[COMMENT, REQUEST_CHANGES]` with `supersede-older-reviews: true` to auto-dismiss stale blocking reviews | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs-pull-requests/) |
| Stale blocking reviews from previous `/review` runs | `supersede-older-reviews: true` on `submit-pull-request-review` — dismisses older same-workflow `REQUEST_CHANGES` reviews after posting replacement | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs-pull-requests/) |
| Merging PRs via shell `gh pr merge` in post-steps | Use `push-to-pull-request-branch` + branch protection auto-merge, or `dispatch-workflow` to trigger a merge workflow | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) |
| Manually updating existing bot comments (delete + repost) | `hide-older-comments: true` on `add-comment` — collapses previous comments before posting new | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) |
| Configuring the GitHub CLI proxy mode | `tools.github.mode: gh-proxy` — official config; old `cli-proxy` feature flag is deprecated | [Engines](https://github.github.com/gh-aw/reference/engines/) |
| Replying to inline review comments manually | `reply-to-pull-request-review-comment` safe output — threads replies under existing review comments | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs-pull-requests/) |
| Resolving review threads manually | `resolve-pull-request-review-thread` safe output — marks review threads as resolved | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs-pull-requests/) |
| Configuring the GitHub CLI proxy mode | `tools.github.mode: gh-proxy` (v0.70.0+) — official config; old `cli-proxy` feature flag is deprecated | [Engines](https://github.github.com/gh-aw/reference/engines/) |
| `slash_command:` without `events:` filter (subscribes to ALL comment events) | `events: [pull_request_comment]` or `events: [issue_comment]` | [Command Triggers](https://github.github.com/gh-aw/reference/command-triggers/) |
| `cancel-in-progress: true` on `slash_command:` workflows | `cancel-in-progress: false` — non-matching events cancel in-progress agent runs | [Concurrency](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/using-concurrency) |
| Using `pull_request` trigger for agentic workflows | `slash_command:` or `schedule` — `pull_request` causes the "Approve and run" gate for ALL workflows | [Triggers](https://github.github.com/gh-aw/reference/triggers/) |
Expand Down Expand Up @@ -119,6 +124,27 @@ safe-outputs:
target: "*" # Required for workflow_dispatch (no triggering PR context)
```

> **`reply_to_id`** — `add_comment` supports a `reply_to_id` parameter to thread replies in discussion threads. On `pull_request_review_comment` triggers (v0.70.0+), `add_comment` automatically replies in the review thread instead of posting at the PR level.

### Submit PR Reviews

```yaml
safe-outputs:
submit-pull-request-review:
max: 1
allowed-events: [COMMENT] # Blocks APPROVE; non-blocking
# allowed-events: [COMMENT, REQUEST_CHANGES] # Blocking reviews
# supersede-older-reviews: true # Auto-dismiss stale blocking reviews from same workflow
footer: "if-body" # Omit footer on approval-only reviews
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Minor · 2/3 consensus

The inline comment # Omit footer on approval-only reviews is misleading: the active config on the line above (allowed-events: [COMMENT]) blocks APPROVE events entirely, so "approval-only reviews" are unreachable in this configuration.

footer: "if-body" omits the AI-generated footer when the review body is empty — it has nothing to do with APPROVE events.

Suggestion: # Omit AI footer when review body is empty

create-pull-request-review-comment:
max: 30 # Inline annotations on diff lines
side: "RIGHT"
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Minor · 2/3 consensus (disputed)

side: "RIGHT" is shown as a YAML-level safe-outputs: config, but one reviewer found no source .md files using it (suggesting it may be a per-call MCP tool parameter only), while another found it present in compiled lock files (suggesting the compiler does accept it).

Worth verifying this is a valid compile-time config rather than only a per-call parameter. If side is only meaningful at call time, removing it from this example avoids confusion.

reply-to-pull-request-review-comment:
max: 10 # Thread replies under existing review comments
resolve-pull-request-review-thread:
max: 10 # Mark review threads as resolved
```

### Concurrency

Include all trigger-specific PR number sources. **Use `cancel-in-progress: false` for `slash_command:` workflows** — a non-matching event (ordinary comment) in the same concurrency group can cancel an in-progress matching run (the actual `/command`), killing the agent mid-execution:
Expand Down Expand Up @@ -257,10 +283,13 @@ safe-outputs:
submit-pull-request-review:
# COMMENT-only: no stale blocking reviews, safe for iterative /review re-runs
allowed-events: [COMMENT]
# Or allow REQUEST_CHANGES for stronger merge-gating (but stale reviews persist until manually dismissed):
# Or allow REQUEST_CHANGES for stronger merge-gating with auto-dismiss of stale reviews:
# allowed-events: [COMMENT, REQUEST_CHANGES]
# supersede-older-reviews: true # Dismiss older blocking reviews from same workflow after posting replacement
```

> **`supersede-older-reviews: true`** — When using `REQUEST_CHANGES`, set this to automatically dismiss older blocking reviews from the same workflow after posting a replacement. This solves the stale-review problem: without it, a `REQUEST_CHANGES` review persists even after the author fixes everything and re-runs `/review`, because gh-aw has no `dismiss-pull-request-review` safe output and the compiler rejects `pull-requests: write`. With `supersede-older-reviews`, the new review replaces the old one (best-effort). This makes `[COMMENT, REQUEST_CHANGES]` a viable option alongside `[COMMENT]`-only.

**3. Integrity filtering** — controls what content the agent can **see** (vs. `roles:` which controls who can **trigger**). The MCP gateway filters content by author trust level:

| Level | Who qualifies |
Expand Down
16 changes: 13 additions & 3 deletions .claude/skills/gh-aw-guide/references/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,16 +343,26 @@ Safe outputs enforce security through separation: agents run read-only and reque
- `expires: 14` — Auto-close after 14 days (same-repo only)
- `excluded-files: ["**/*.lock"]` — Strip files from the patch entirely
- `github-token-for-extra-empty-commit:` — Push empty commit with separate token to trigger CI
- `protected-files: fallback-to-issue` — Create issue instead of failing when protected files modified; optionally add `fallback-labels:` to tag that issue
- `protected-files: fallback-to-issue` — Create issue instead of failing when protected files modified; optionally add `fallback-labels:` to tag that issue (v0.70.0+)
- `base-branch: "vnext"` — Target non-default branch
- `allowed-base-branches: [main, "release/*"]` — Allow agent to override base branch at runtime (glob patterns)
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Version-gating omission — will cause compile failure on v0.68.3 baseline

This bullet lists allowed-base-branches with no version tag, but line 385 (Limitations table, same file) says:

allowed-base-branches compile validation — gh aw compile incorrectly reported ... as unknown field — Fixed in v0.70.0

The skill declares a v0.68.3 baseline (SKILL.md line 17). An agent or user targeting v0.68.3 who adds allowed-base-branches will get a compile error. Every other post-baseline feature is tagged ((v0.70.0+) on gh-proxy, fallback-labels), but this one is not.

Suggested fix: Add (v0.70.0+) — e.g.:

- `allowed-base-branches: [main, "release/*"]` (v0.70.0+) — Allow agent to override base branch at runtime (glob patterns)

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Minor · 3/3 consensus

allowed-base-branches has no version tag. Per the version baseline note added in this PR, features introduced after v0.68.3 should be tagged (e.g., (v0.70.0+)). Other options in this same diff — protected-files: fallback-to-issue at line 346 — were correctly tagged.

If allowed-base-branches was added after v0.68.3, add the appropriate version tag. The same applies to preserve-branch-name, fallback-as-issue, team-reviewers, and max below if they're also post-baseline features.

- `auto-close-issue: false` — Don't add `Fixes #N` to PR description
- `allowed-events: [COMMENT]` — On `submit-pull-request-review`, blocks agent from approving PRs (bypasses branch protection). **Always use this** for review workflows.
- **Stale review limitation**: Prefer `allowed-events: [COMMENT]` unless you need the "Changes requested" badge. `REQUEST_CHANGES` reviews from `github-actions[bot]` cannot be dismissed by the agent (no `dismiss-pull-request-review` safe output, `pull-requests: write` rejected by compiler). A stale blocking review persists until a human dismisses it manually.
- `preserve-branch-name: true` — Omit random hex suffix from branch name (fails if branch exists)
- `fallback-as-issue: false` — Disable issue fallback when PR creation fails (default: true)
- `team-reviewers: [platform-reviewers]` — Request team reviews
- `max: 3` — Multiple PRs per run (each on its own branch)

**`submit-pull-request-review` notable options:**
- `allowed-events: [COMMENT]` — Blocks APPROVE; non-blocking reviews. **Always use this** unless you need merge-gating.
- `allowed-events: [COMMENT, REQUEST_CHANGES]` — Blocking reviews. Pair with `supersede-older-reviews: true`.
- `supersede-older-reviews: true` — Auto-dismiss older blocking reviews from the same workflow after posting replacement. Solves the stale-review problem without needing `dismiss-pull-request-review` (which doesn't exist as a safe output).
Comment on lines +355 to +358
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Internal contradiction — Limitations table (L375) and Troubleshooting table (L417) still say stale reviews "can't be dismissed"

This new section says supersede-older-reviews: true "Solves the stale-review problem." But two pre-existing tables in this same file were not updated and now contradict it:

  • L375 (Limitations): "Agent reviews with REQUEST_CHANGES block PRs and can't be dismissed" → Workaround: "Use allowed-events: [COMMENT]"
  • L417 (Troubleshooting): "Agent posted REQUEST_CHANGES but can't dismiss it" → Fix: "Switch to allowed-events: [COMMENT]"

An agent consulting this skill will get opposite guidance depending on which section it reads. Since context windows often include partial files, this creates a real risk of contradictory output.

Suggested fix: Update L375 workaround to: "Use supersede-older-reviews: true to auto-dismiss stale reviews, or switch to allowed-events: [COMMENT]". Make an analogous update to L417.

- `footer: "if-body"` — Omit AI footer on approval-only reviews (cleaner)
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Minor · 2/3 consensus (same root cause as SKILL.md line 138)

"Omit AI footer on approval-only reviews (cleaner)" — same misleading description. footer: "if-body" controls whether the footer is appended when the review body is empty, not specifically for approval events.

Suggestion: - \footer: "if-body"` — Omit AI footer when review body is empty (cleaner)`


**`add-comment` notable options:**
- `hide-older-comments: true` — Collapse previous comments from same workflow
- `max: N` — Limit comments per run (default: 1)
- `target: "*"` — Required for `workflow_dispatch` (no triggering PR context)
- `reply_to_id` — Thread replies in discussion threads
- **PR review thread routing** (v0.70.0+): On `pull_request_review_comment` triggers, `add_comment` now replies directly in the review thread instead of posting at the PR level.

---
Expand Down
Loading