Skip to content

Add changelog docs check gh-aw workflow#3282

Draft
annabellscha wants to merge 11 commits into
mainfrom
codex/changelog-docs-check
Draft

Add changelog docs check gh-aw workflow#3282
annabellscha wants to merge 11 commits into
mainfrom
codex/changelog-docs-check

Conversation

@annabellscha

@annabellscha annabellscha commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a daily/manual gh-aw workflow that reviews recent langfuse product changes against docs and changelog coverage
  • add a shared safe-output Slack job that posts via SLACK_WEBHOOK_URL and appends the Annabell feedback footer
  • commit the generated gh-aw lockfiles and workflow artifacts

Testing

  • gh aw compile --approve

Notes

  • manual workflow testing can happen after merge because workflow_dispatch only appears once the workflow exists on the default branch
  • the Slack incoming webhook should be configured to post to #lf-team-engineering

Greptile Summary

This PR adds a daily/manual GitHub Agentic Workflow (gh-aw) that uses Claude Code to scan recent merged PRs and releases in langfuse/langfuse against changelog and docs coverage in langfuse/langfuse-docs, posting a structured Slack digest via an incoming webhook. It also ships the auto-generated maintenance workflow, compiled lock file, SHA-pinned actions lock, and .gitattributes marking the lock files as generated artifacts.

  • New changelog-docs-check.md + compiled lock — read-only Claude Code agent, scoped to two langfuse repos via CROSS_REPO_PAT, firewall-restricted egress, and a single Slack safe-output per run. All actions and container images are digest-pinned.
  • Shared slack-dm-notify.md — safe-output job that posts to a Slack incoming webhook and appends a feedback footer; includes staged-mode preview, correct ok-response validation, and duplicate-footer guard.
  • agentics-maintenance.yml — standard gh-aw maintenance workflow for expiring entities, cache cleanup, and manual operations, with per-job fork guards and minimal permission scopes.

Confidence Score: 4/5

Safe to merge — the change adds a new read-only automated workflow with no write access to either product repo, all secrets are correctly scoped, and the generated lock file is properly SHA-pinned throughout.

The workflow logic is sound and the security posture is well-considered. The 72-hour lookback window on a daily schedule will produce three consecutive Slack alerts for any uncovered product change, which may become noisy over time. The actions/github-script@v8 pin in the source file is inconsistent with the @v9 used in the rest of the compiled output. Neither issue blocks correctness but both are worth addressing before the workflow generates its first real output.

changelog-docs-check.md for the review window duration, and shared/slack-dm-notify.md for the github-script version.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Cron as Schedule (0 7 * * *)
    participant Act as activation job
    participant Ag as agent job
    participant ClaudeCode as Claude Code CLI
    participant GH as GitHub MCP (langfuse/langfuse + langfuse-docs)
    participant SO as safeoutputs MCP
    participant SlackJob as slack-channel-notify (safe-output)
    participant Slack as Slack Incoming Webhook

    Cron->>Act: trigger daily
    Act->>Act: validate secrets, check lock file, build prompt
    Act->>Ag: pass activation artifact
    Ag->>ClaudeCode: run with MCP tools
    ClaudeCode->>GH: list recent PRs / releases in langfuse/langfuse (72h)
    GH-->>ClaudeCode: product changes
    ClaudeCode->>GH: inspect content/changelog/ in langfuse-docs
    GH-->>ClaudeCode: existing coverage
    ClaudeCode->>SO: slack_channel_notify(message)
    SO-->>Ag: write safe output JSONL
    Ag->>SlackJob: read safeoutputs.jsonl
    SlackJob->>SlackJob: append Annabell footer
    SlackJob->>Slack: POST webhook JSON payload
    Slack-->>SlackJob: "ok"
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Cron as Schedule (0 7 * * *)
    participant Act as activation job
    participant Ag as agent job
    participant ClaudeCode as Claude Code CLI
    participant GH as GitHub MCP (langfuse/langfuse + langfuse-docs)
    participant SO as safeoutputs MCP
    participant SlackJob as slack-channel-notify (safe-output)
    participant Slack as Slack Incoming Webhook

    Cron->>Act: trigger daily
    Act->>Act: validate secrets, check lock file, build prompt
    Act->>Ag: pass activation artifact
    Ag->>ClaudeCode: run with MCP tools
    ClaudeCode->>GH: list recent PRs / releases in langfuse/langfuse (72h)
    GH-->>ClaudeCode: product changes
    ClaudeCode->>GH: inspect content/changelog/ in langfuse-docs
    GH-->>ClaudeCode: existing coverage
    ClaudeCode->>SO: slack_channel_notify(message)
    SO-->>Ag: write safe output JSONL
    Ag->>SlackJob: read safeoutputs.jsonl
    SlackJob->>SlackJob: append Annabell footer
    SlackJob->>Slack: POST webhook JSON payload
    Slack-->>SlackJob: "ok"
Loading
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
.github/workflows/changelog-docs-check.md:73-75
**72-hour window causes repeated Slack alerts**

The scheduled cron runs once per day (`0 7 * * *`), but the review window is set to 72 hours. Any product change that genuinely lacks docs coverage will appear in three consecutive daily Slack messages before it ages out of the window. A window of 25–27 hours captures all changes without meaningful overlap between runs, keeping the channel signal-to-noise ratio high.

### Issue 2 of 2
.github/workflows/shared/slack-dm-notify.md:18-19
The source file pins `actions/github-script` at `@v8`, but the gh-aw compiler uses `@v9.0.0` (SHA `3a2844b7e9c422d3c10d287c895573f7108da1b3`) for every step it generates in the lock file. This mismatch means the slack-channel-notify safe-output job runs on the older major version. Updating the source to `@v9` would keep the lock file uniform and avoid inheriting any v8-era bugs.

```suggestion
        - name: Send Slack message
          uses: actions/github-script@v9
```

Reviews (1): Last reviewed commit: "Add temporary push trigger for workflow ..." | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

@annabellscha
annabellscha requested a review from a team as a code owner July 10, 2026 11:55
@vercel

vercel Bot commented Jul 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
langfuse-docs Error Error Jul 10, 2026 2:52pm

Request Review

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. docs labels Jul 10, 2026
@github-actions

Copy link
Copy Markdown

@claude review

Comment on lines +73 to +75

## Review window

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 72-hour window causes repeated Slack alerts

The scheduled cron runs once per day (0 7 * * *), but the review window is set to 72 hours. Any product change that genuinely lacks docs coverage will appear in three consecutive daily Slack messages before it ages out of the window. A window of 25–27 hours captures all changes without meaningful overlap between runs, keeping the channel signal-to-noise ratio high.

Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/changelog-docs-check.md
Line: 73-75

Comment:
**72-hour window causes repeated Slack alerts**

The scheduled cron runs once per day (`0 7 * * *`), but the review window is set to 72 hours. Any product change that genuinely lacks docs coverage will appear in three consecutive daily Slack messages before it ages out of the window. A window of 25–27 hours captures all changes without meaningful overlap between runs, keeping the channel signal-to-noise ratio high.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant