Skip to content
Translate Docs

Translate Docs #1247

Sign in to view logs

Translate Docs

Translate Docs #1247

Workflow file for this run

.github/workflows/translate_docs.yml at ffcb3be
name: Translate Docs
on:
push:
branches: [main]
paths:
- 'content/docs/**'
# Exclude generated translations of any locale. A `*.<locale>.mdx` or
# `meta.<locale>.json` file is bot output, never a translation source, so it
# must not retrigger this workflow. Keep both two-letter and regional locale
# patterns here, matching TARGET_LOCALES in lib/i18n/config.ts.
- '!content/docs/**/*.[a-z][a-z].mdx'
- '!content/docs/**/*.[a-z][a-z]-[A-Z][A-Z].mdx'
- '!content/docs/**/meta.[a-z][a-z].json'
- '!content/docs/**/meta.[a-z][a-z]-[A-Z][A-Z].json'
# Convergence sweep. A push only translates after a content change, and adding a
# new language doesn't touch content/docs at all — so a big batch (several new
# locales and/or long files) can't fully land in the single push-triggered run
# under the flex-tier rate limit. This run writes generated docs to
# TRANSLATE_BRANCH and keeps translation memory in the Actions cache, so main stays
# free of partial progress commits and cache files.
#
# Every 6 hours. Content pushes trigger this workflow directly, so the sweep is
# only a catch-up for batches that could not finish in one run. A sweep with
# nothing to do is genuinely free in API terms — every block is a cache hit — so
# it is not worth a skip mechanism to avoid a few minutes of CI. An earlier
# revision fingerprinted the inputs to skip idle sweeps; it saved no spend and
# kept finding new ways to skip work that had not actually been published.
schedule:
- cron: '23 */6 * * *'
workflow_dispatch:
inputs:
force:
description: 'Retranslate everything (ignore the cache)'
type: boolean
default: false
permissions:
contents: write
env:
TRANSLATE_BRANCH: automation/docs-translations
# Durable home for the translation memory. A GitHub Actions cache is evictable
# (7-day idle, or LRU once the repo passes 10 GB), and losing it means paying to
# re-translate all ~135k blocks across 13 locales from scratch. This ref is a
# single force-pushed orphan commit — no history growth — and it is deliberately
# NOT the translations branch, so the memory can never ride a docs PR into main.
MEMORY_BRANCH: automation/i18n-memory
concurrency:
group: translate-docs
cancel-in-progress: false
jobs:
translate:
runs-on: ubuntu-latest
environment: Production
steps:
- uses: actions/checkout@v4
with:
# Full history so the long-lived translation branch can merge current
# main before each convergence pass.
fetch-depth: 0
- name: Prepare translation branch
id: prepare
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch origin "+refs/heads/main:refs/remotes/origin/main"
if git fetch origin "+refs/heads/${TRANSLATE_BRANCH}:refs/remotes/origin/${TRANSLATE_BRANCH}" 2>/dev/null; then
git switch -c "$TRANSLATE_BRANCH" "origin/$TRANSLATE_BRANCH"
# main wins conflicting hunks: it also edits locale files directly (the
# config version sync rewrites every librechat.yaml snippet), and those
# edits would otherwise conflict with the branch's translations of the
# same lines and stop every run. Non-conflicting translations are kept.
git merge -X theirs --no-edit origin/main
else
git switch -c "$TRANSLATE_BRANCH" origin/main
fi
# Layer 1 (fallback): the Actions cache. Restore/save split rather than
# actions/cache — the key is unique per run, so the combined action re-uploaded
# the whole memory (~4 MB) even on sweeps that translated nothing, churning the
# repo's shared 10 GB cache budget. Only a run that did work saves now.
- name: Restore translation memory
uses: actions/cache/restore@v4
with:
path: content/.i18n-cache
key: translate-docs-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
translate-docs-
# Layer 2 (durable): MEMORY_BRANCH, unioned into whatever layer 1 restored
# rather than overwriting it. The Actions cache is saved before the branch is
# pushed, so a push that fails or is cancelled leaves the cache holding blocks
# the branch does not have; clobbering would throw away work already paid for.
# Entries are content-hash keyed, so a union is always safe — a stale entry is
# dropped by the script's own prune on the next full run.
- name: Restore translation memory from the memory branch
run: |
# No --depth: the ref is a single parentless commit, so a shallow fetch
# downloads nothing less but does leave a .git/shallow boundary behind in
# this checkout, which can make the later branch push a shallow update.
if ! git fetch origin "+refs/heads/${MEMORY_BRANCH}:refs/remotes/origin/${MEMORY_BRANCH}" 2>/dev/null; then
echo "no ${MEMORY_BRANCH} yet; using the Actions cache for this run"
exit 0
fi
snapshot=$(mktemp -d)
git archive "origin/${MEMORY_BRANCH}" | tar -x -C "$snapshot"
mkdir -p content/.i18n-cache
for src in "$snapshot"/*.json; do
[ -e "$src" ] || continue
name=$(basename "$src")
# state.json belonged to the removed skip check. Older caches may still
# carry one; it is not memory, so keep it off the branch.
[ "$name" = "state.json" ] && continue
target="content/.i18n-cache/$name"
# Both operands must be JSON objects. A truncated or non-object file makes
# jq exit nonzero, which fails this step under `bash -e` — and because
# caches are immutable and the prefix restore keeps selecting the newest
# one, every later sweep would fail here too. TM.load() already recovers
# from a corrupt memory file by discarding it; merging must not take that
# recovery away.
if ! jq -e 'type == "object"' "$src" >/dev/null 2>&1; then
echo "::warning::ignoring malformed $name on ${MEMORY_BRANCH}"
continue
fi
if [ ! -f "$target" ]; then
cp "$src" "$target"
elif ! jq -e 'type == "object"' "$target" >/dev/null 2>&1; then
echo "::warning::replacing malformed cached $name with the branch snapshot"
cp "$src" "$target"
else
# `.[1] * .[0]` — the restored cache ($target) wins collisions, not the
# branch snapshot. jq's `*` gives the right operand precedence, which
# would let an older branch value overwrite a newer cached one: a stale
# filefail counter replacing the current one buys another paid pass over
# a page that should already have quarantined.
jq -s '.[1] * .[0]' "$target" "$src" > "$target.merged"
mv "$target.merged" "$target"
fi
done
rm -rf "$snapshot"
echo "translation memory merged from ${MEMORY_BRANCH}"
- name: Seed translation memory from git history
run: |
if [ -d content/.i18n-cache ]; then
exit 0
fi
# First run after removing cache files from main: seed from the parent of
# the cache-removal commit, then unstage it so the files stay out of git.
seed_commit=$(git rev-list -n 1 HEAD -- content/.i18n-cache || true)
if [ -n "$seed_commit" ]; then
git checkout "${seed_commit}^" -- content/.i18n-cache || true
git reset -- content/.i18n-cache || true
fi
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 24
cache: pnpm
- run: pnpm install --frozen-lockfile
- name: Translate docs
env:
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
OPENROUTER_TRANSLATE_MODEL: ${{ secrets.OPENROUTER_TRANSLATE_MODEL }}
run: |
FLAGS=""
if [ "${{ inputs.force }}" = "true" ]; then
FLAGS="--force"
fi
npx tsx scripts/translate.ts $FLAGS
# `!cancelled()` so a translate step that failed part-way still commits the
# pages it did finish; gated on the prepare step so a failure in checkout or
# branch preparation stops here instead, when `content/docs` is not in a known
# state.
- name: Commit translations to branch
if: ${{ !cancelled() && steps.prepare.conclusion == 'success' }}
run: |
git add content/docs
if git diff --cached --quiet; then
echo "No translation changes to commit"
else
# [skip ci] keeps the long-lived translation branch from running CI on
# every half-hourly convergence commit. Main only changes when this branch
# is intentionally merged.
git commit --no-verify -m "chore: update docs translations [skip ci]"
fi
if git show-ref --verify --quiet "refs/remotes/origin/${TRANSLATE_BRANCH}"; then
base="origin/${TRANSLATE_BRANCH}"
else
base="origin/main"
fi
ahead=$(git rev-list --count "${base}..HEAD")
if [ "$ahead" -eq 0 ]; then
echo "Translation branch already up to date"
exit 0
fi
pushed=false
for attempt in 1 2 3 4 5; do
if git push origin HEAD:"$TRANSLATE_BRANCH"; then
pushed=true
break
fi
echo "push attempt $attempt rejected (remote advanced); merging latest branch/main"
git fetch origin "+refs/heads/main:refs/remotes/origin/main"
git fetch origin "+refs/heads/${TRANSLATE_BRANCH}:refs/remotes/origin/${TRANSLATE_BRANCH}" || true
if git show-ref --verify --quiet "refs/remotes/origin/${TRANSLATE_BRANCH}"; then
git merge --no-edit "origin/${TRANSLATE_BRANCH}"
fi
git merge -X theirs --no-edit origin/main
sleep 5
done
if [ "$pushed" != "true" ]; then
echo "::error::failed to push translations to $TRANSLATE_BRANCH after 5 attempts"
exit 1
fi
# Both layers run after any translate attempt, even a failed one: the blocks
# that did translate are already in the memory and must not be paid for twice.
- name: Save translation memory
if: ${{ !cancelled() && steps.prepare.conclusion == 'success' }}
uses: actions/cache/save@v4
with:
path: content/.i18n-cache
key: translate-docs-${{ github.run_id }}-${{ github.run_attempt }}
# A parentless commit built with a throwaway index and force-pushed over the
# ref, so the branch is always exactly one commit and never accumulates
# history. Plumbing rather than `git worktree add --orphan` (git 2.42+) so it
# does not depend on the runner's git version, and a temporary GIT_INDEX_FILE
# so this checkout's index and working tree are untouched. Runs are serialized
# by the job's concurrency group, so the force-push cannot race another run.
#
# Carries the per-locale memory only. The control files stay in the Actions
# cache: they are a statement about what this pipeline published, and letting a
# branch snapshot restore them could resurrect a stale "nothing pending".
#
# Runs on idle sweeps too, not just working ones. If a previous run translated,
# saved its cache, and then failed this push, the next sweep sees a matching
# fingerprint with nothing pending and skips — so gating this on `skip` would
# leave the durable branch stale until unrelated work happened to come along,
# and an eviction in that window means re-buying every block. Comparing trees
# first makes the step idempotent, so an idle run repairs the branch and a
# branch already in sync costs one rev-parse.
- name: Push translation memory to the memory branch
if: ${{ !cancelled() && steps.prepare.conclusion == 'success' }}
run: |
snapshot=$(mktemp -d)
for src in content/.i18n-cache/*.json; do
[ -e "$src" ] || continue
name=$(basename "$src")
[ "$name" = "state.json" ] && continue
cp "$src" "$snapshot/$name"
done
if [ -z "$(ls -A "$snapshot")" ]; then
echo "no translation memory to persist"
rm -rf "$snapshot"
exit 0
fi
index=$(mktemp -u)
GIT_INDEX_FILE="$index" git --work-tree="$snapshot" add -A
tree=$(GIT_INDEX_FILE="$index" git write-tree)
rm -f "$index"
rm -rf "$snapshot"
if [ "$(git rev-parse --verify --quiet "origin/${MEMORY_BRANCH}^{tree}" || true)" = "$tree" ]; then
echo "${MEMORY_BRANCH} already holds this memory"
exit 0
fi
commit=$(git commit-tree "$tree" -m "chore: docs translation memory [skip ci]")
git push --force origin "${commit}:refs/heads/${MEMORY_BRANCH}"
echo "translation memory persisted to ${MEMORY_BRANCH} as ${commit}"