ci: open automation PRs with GITHUB_TOKEN instead of Docs Bot (#1198)

* ci: open automation PRs with GITHUB_TOKEN instead of Docs Bot

Drop the GitHub App token helper and generate llms.txt in the toolkit
docs workflow so automation PRs no longer depend on a second workflow
starting from the bot push.

Co-authored-by: Anirudh Kamath <kamath@users.noreply.github.com>

* fix: avoid Biome noTemplateCurlyInString in workflow test

The assertion for github.token used a GitHub Actions expression inside a
regular string, which Ultracite flagged as a leftover template placeholder.

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Anirudh Kamath <kamath@users.noreply.github.com>

Author: kamath

.github/actions/app-token/action.yml

@@ -7,13 +7,15 @@ This workflow regenerates toolkit JSON and opens a PR with the changes. It can b
 1. Builds the toolkit docs generator.
 2. Generates toolkit JSON in `toolkit-docs-generator/data/toolkits` using the experience API's public tool catalog, which serves tool definitions and toolkit branding in one anonymous read.
 3. Syncs integrations sidebar navigation from the generated JSON.
-4. Creates or updates a PR on the stable `automation/toolkit-docs` branch if any files changed. Later runs overwrite that open PR with the latest generated docs.
+4. Regenerates `public/llms.txt` so the automation PR does not depend on a second workflow.
+5. Creates or updates a PR on the stable `automation/toolkit-docs` branch if any files changed. Later runs overwrite that open PR with the latest generated docs.
 
 ## Inputs and secrets
 
 Required secrets:
 
 - `ANTHROPIC_API_KEY`
+- `OPENAI_API_KEY` (for `llms.txt` summaries)
 
 Optional secrets:
 

.github/workflows/generate-toolkit-docs.md

@@ -3,7 +3,11 @@ name: Generate toolkit docs
 # Flow:
 # 1) Generate toolkit JSON into toolkit-docs-generator/data/toolkits
 # 2) Sync integrations sidebar _meta.tsx from toolkit-docs-generator/data/toolkits
-# 3) Create or update a PR if changes were produced
+# 3) Regenerate public/llms.txt so the automation PR is self-contained
+# 4) Create or update a PR if changes were produced
+#
+# Opens the PR with GITHUB_TOKEN. That does not start other workflows, so this
+# job generates llms.txt here instead of depending on llmstxt.yml to fire.
 
 on:
   repository_dispatch:
@@ -97,24 +101,20 @@ jobs:
       - name: Sync toolkit sidebar navigation
         run: pnpm exec tsx toolkit-docs-generator/scripts/sync-toolkit-sidebar.ts --remove-empty-sections=false --verbose
 
-      # GITHUB_TOKEN PRs do not start other workflows. Docs Bot is a GitHub
-      # App, so the opened automation PR still runs Test.
-      - name: Get app token
-        id: app-token
-        uses: ./.github/actions/app-token
-        with:
-          app-id: ${{ secrets.DOCS_BOT_CLIENT_ID }}
-          private-key: ${{ secrets.DOCS_BOT_PRIVATE_KEY }}
+      # GITHUB_TOKEN PRs do not start llmstxt.yml, so generate the file here.
+      # The toolkit generator CLI still does not write llms.txt.
+      - name: Generate llms.txt
+        run: pnpm llmstxt
+        env:
+          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
 
       - name: Create pull request
         id: cpr
         uses: peter-evans/create-pull-request@v7
         env:
           HUSKY: 0
         with:
-          token: ${{ steps.app-token.outputs.token }}
-          author: ${{ steps.app-token.outputs.committer }}
-          committer: ${{ steps.app-token.outputs.committer }}
+          token: ${{ github.token }}
           commit-message: "[AUTO] Adding MCP Servers docs update"
           title: "[AUTO] Adding MCP Servers docs update"
           body: |
@@ -141,7 +141,7 @@ jobs:
             echo "::warning::Could not request review on PR #${{ steps.cpr.outputs.pull-request-number }}: $(cat review-error.log)"
           fi
         env:
-          GH_TOKEN: ${{ steps.app-token.outputs.token }}
+          GH_TOKEN: ${{ github.token }}
 
       - name: Upload generation report
         if: always()
@@ -224,7 +224,7 @@ jobs:
 
           if [ "$GENERATION_SUCCEEDED" = "true" ]; then
             headline=":warning: Toolkit docs generated, but the workflow failed afterward"
-            detail="Generation succeeded, so the toolkit JSON is fine. A later workflow step failed — inspect the run to determine whether sidebar sync, PR creation, artifact upload, or Slack notification needs attention."
+            detail="Generation succeeded, so the toolkit JSON is fine. A later workflow step failed — inspect the run to determine whether sidebar sync, llms.txt generation, PR creation, artifact upload, or Slack notification needs attention."
           else
             headline=":rotating_light: Toolkit docs generation failed"
             detail="The failed run contains the exact file path and validation error."

.github/workflows/generate-toolkit-docs.yml

@@ -37,10 +37,6 @@ jobs:
         uses: actions/checkout@v4
         with:
           ref: ${{ github.event.pull_request.head.ref || github.ref }}
-          # No credentials to persist: the app token is minted further down,
-          # once there is something to push. The commit step below supplies it
-          # to git explicitly.
-          persist-credentials: false
 
       - name: Install dependencies
         run: npm install -g pnpm
@@ -62,38 +58,26 @@ jobs:
             echo "has_changes=false" >> $GITHUB_OUTPUT
           fi
 
-      # GITHUB_TOKEN pushes and PRs do not start other workflows. Docs Bot
-      # is a GitHub App, so Test still runs on the new SHA.
-      - name: Get app token
-        if: steps.check-changes.outputs.has_changes == 'true'
-        id: app-token
-        uses: ./.github/actions/app-token
-        with:
-          app-id: ${{ secrets.DOCS_BOT_CLIENT_ID }}
-          private-key: ${{ secrets.DOCS_BOT_PRIVATE_KEY }}
-
+      # GITHUB_TOKEN pushes do not start other workflows. That is acceptable
+      # here: Test already ran on the author's commit, and llms.txt is a
+      # generated artifact.
       - name: Commit changes to PR
         if: steps.check-changes.outputs.has_changes == 'true' && github.event_name == 'pull_request'
         env:
-          GH_APP_TOKEN: ${{ steps.app-token.outputs.token }}
-          USER_NAME: ${{ steps.app-token.outputs.user-name }}
-          USER_EMAIL: ${{ steps.app-token.outputs.user-email }}
           HEAD_REF: ${{ github.event.pull_request.head.ref }}
         run: |
-          git config user.name "$USER_NAME"
-          git config user.email "$USER_EMAIL"
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
           git add public/llms.txt
           git commit -m "🤖 Regenerate LLMs.txt"
-          git push "https://x-access-token:${GH_APP_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" "HEAD:${HEAD_REF}"
+          git push origin "HEAD:${HEAD_REF}"
 
       - name: Create Pull Request (for scheduled/manual runs)
         if: steps.check-changes.outputs.has_changes == 'true' && github.event_name != 'pull_request'
         id: cpr
         uses: peter-evans/create-pull-request@v7
         with:
-          token: ${{ steps.app-token.outputs.token }}
-          author: ${{ steps.app-token.outputs.committer }}
-          committer: ${{ steps.app-token.outputs.committer }}
+          token: ${{ github.token }}
           commit-message: Regenerate LLMs.txt and related files
           branch: auto-update-llms-txt
           delete-branch: true
@@ -107,11 +91,11 @@ jobs:
         continue-on-error: true
         run: gh pr edit ${{ steps.cpr.outputs.pull-request-number }} --add-reviewer ArcadeAI/engineering-tools-and-dx
         env:
-          GH_TOKEN: ${{ steps.app-token.outputs.token }}
+          GH_TOKEN: ${{ github.token }}
 
       - name: Enable Pull Request Automerge
         if: steps.check-changes.outputs.has_changes == 'true' && github.event_name != 'pull_request' && steps.cpr.outputs.pull-request-number != ''
         continue-on-error: true
         run: gh pr merge --squash --auto ${{ steps.cpr.outputs.pull-request-number }}
         env:
-          GH_TOKEN: ${{ steps.app-token.outputs.token }}
+          GH_TOKEN: ${{ github.token }}

.github/workflows/llmstxt.yml

@@ -47,16 +47,8 @@ jobs:
           pnpm vitest run toolkit-docs-generator/tests/scripts/sync-toolkit-sidebar.test.ts toolkit-docs-generator/tests/sources/oauth-provider-resolver.test.ts
           pnpm run build
 
-      # GITHUB_TOKEN PRs do not start other workflows. Docs Bot is a GitHub
-      # App, so the opened automation PR still runs Test.
-      - name: Get app token
-        if: steps.check-changes.outputs.has_changes == 'true'
-        id: app-token
-        uses: ./.github/actions/app-token
-        with:
-          app-id: ${{ secrets.DOCS_BOT_CLIENT_ID }}
-          private-key: ${{ secrets.DOCS_BOT_PRIVATE_KEY }}
-
+      # Compatibility tests and a production build run above, so this workflow
+      # does not depend on Test starting on the opened PR.
       - name: Create pull request
         if: steps.check-changes.outputs.has_changes == 'true'
         id: cpr
@@ -65,9 +57,7 @@ jobs:
           HUSKY: 0
           SKIP_HUSKY: 1
         with:
-          token: ${{ steps.app-token.outputs.token }}
-          author: ${{ steps.app-token.outputs.committer }}
-          committer: ${{ steps.app-token.outputs.committer }}
+          token: ${{ github.token }}
           commit-message: "chore: update @arcadeai/design-system to latest"
           title: "chore: update @arcadeai/design-system to latest"
           body: |
@@ -89,4 +79,4 @@ jobs:
         continue-on-error: true
         run: gh pr edit ${{ steps.cpr.outputs.pull-request-number }} --add-reviewer ArcadeAI/engineering-tools-and-dx
         env:
-          GH_TOKEN: ${{ steps.app-token.outputs.token }}
+          GH_TOKEN: ${{ github.token }}

.github/workflows/update-design-system-dependency.yml

@@ -1,4 +1,4 @@
-import { readFileSync } from "node:fs";
+import { existsSync, readFileSync } from "node:fs";
 import { join } from "node:path";
 import { expect, test } from "vitest";
 
@@ -12,22 +12,21 @@ const AUTOMATION_WORKFLOWS = [
 ] as const;
 
 for (const name of AUTOMATION_WORKFLOWS) {
-  test(`${name} authenticates PR writes with Docs Bot, not GITHUB_TOKEN`, () => {
+  test(`${name} opens PRs with GITHUB_TOKEN, not Docs Bot`, () => {
     const yaml = readWorkflow(name);
 
-    expect(yaml).toContain("uses: ./.github/actions/app-token");
-    expect(yaml).toContain("secrets.DOCS_BOT_CLIENT_ID");
-    expect(yaml).toContain("secrets.DOCS_BOT_PRIVATE_KEY");
-    expect(yaml).toContain("steps.app-token.outputs.token");
-    expect(yaml).not.toContain("secrets.GITHUB_TOKEN");
+    expect(yaml).not.toContain("uses: ./.github/actions/app-token");
+    expect(yaml).not.toContain("DOCS_BOT_CLIENT_ID");
+    expect(yaml).not.toContain("DOCS_BOT_PRIVATE_KEY");
+    expect(yaml).not.toContain("create-github-app-token");
+    expect(yaml).toContain("github.token");
   });
 }
 
-test("app-token composite action mints a GitHub App installation token", () => {
-  const action = readFileSync(
-    join(process.cwd(), ".github", "actions", "app-token", "action.yml"),
-    "utf-8"
-  );
-
-  expect(action).toContain("actions/create-github-app-token@v2");
+test("docs automation no longer ships a GitHub App token action", () => {
+  expect(
+    existsSync(
+      join(process.cwd(), ".github", "actions", "app-token", "action.yml")
+    )
+  ).toBe(false);
 });

tests/docs-automation-tokens.test.ts

@@ -37,9 +37,8 @@ flowchart TD
 
     json --> verify["Verify output"]
     verify --> sidebar["Sync sidebar _meta.tsx"]
-    sidebar --> pr["Pull request on<br/>automation/toolkit-docs"]
-    pr --> llms["llms.txt workflow<br/>regenerates public/llms.txt"]
-    llms --> pr
+    sidebar --> llms["pnpm llmstxt<br/>regenerates public/llms.txt"]
+    llms --> pr["Pull request on<br/>automation/toolkit-docs"]
     pr -->|"a human merges"| main["main"]
     main --> vercel["Vercel: next build"]
     vercel --> pages["Static toolkit pages"]
@@ -64,9 +63,10 @@ files live in this repository and merge like any other change, so a prose edit
 reaches the site through the next generation run rather than through a manual
 edit of the generated JSON.
 
-Two things the diagram deliberately shows as separate. A second workflow
-regenerates `llms.txt` when the data files change, so the generator never writes
-it. The sidebar sync writes navigation only, and never touches toolkit JSON.
+Two things the diagram deliberately shows as separate. The same workflow
+regenerates `llms.txt` after sidebar sync, so the generator never writes it and
+the automation PR does not depend on a second workflow starting. The sidebar sync
+writes navigation only, and never touches toolkit JSON.
 
 ## Core components
 

toolkit-docs-generator/ARCHITECTURE.md

@@ -48,11 +48,12 @@ test("porter workflow generates docs and opens a PR", () => {
   expect(workflowContents).toContain("HUSKY: 0");
   expect(workflowContents).toContain("[AUTO] Adding MCP Servers docs update");
   expect(workflowContents).toContain("pull-requests: write");
-  // GITHUB_TOKEN PRs do not start other workflows, so Test would never run
-  // on the automation PR. Docs Bot is a GitHub App and avoids that restriction.
-  expect(workflowContents).toContain("uses: ./.github/actions/app-token");
-  expect(workflowContents).toContain("steps.app-token.outputs.token");
-  expect(workflowContents).not.toContain("secrets.GITHUB_TOKEN");
+  // GITHUB_TOKEN PRs do not start other workflows. Generate llms.txt here
+  // instead of depending on llmstxt.yml, and open the PR with github.token.
+  expect(workflowContents).toContain("pnpm llmstxt");
+  expect(workflowContents).toContain(`token: ${"${"}{ github.token }}`);
+  expect(workflowContents).not.toContain("uses: ./.github/actions/app-token");
+  expect(workflowContents).not.toContain("DOCS_BOT");
 });
 
 test("porter workflow does not build the docs generator before running it", () => {