Skip to content

AGENTS.md

Latest commit

 

History

History
270 lines (206 loc) · 19.1 KB

AGENTS.md

File metadata and controls

270 lines (206 loc) · 19.1 KB
Raw

ADT Studio

ADT Studio is a desktop-first application for automated book production — extracting content from PDFs, processing through LLM pipelines, and generating formatted output bundles.

Tech Stack

  • Monorepo: pnpm workspaces
  • Backend: Hono (HTTP server), node-sqlite3-wasm (pure WASM SQLite), Zod
  • Frontend: React + Vite SPA, TanStack (Router, Query, Table, Form), Tailwind CSS
  • Desktop: Electron (electron-vite + electron-builder; API server bundled into the main process)
  • Language: TypeScript (strict mode)
  • Testing: Vitest

6 Core Principles (Non-Negotiable)

  1. Book-Level Storage — All book data lives in one directory (zippable, shareable). Never store book data elsewhere.
  2. Entity-Level Versioning — NEVER overwrite entities. Always create new versions. Users must be able to roll back.
  3. LLM-Level Caching — Cache at the LLM call level. Hash ordered inputs for cache keys. Reruns are fast if params unchanged.
  4. Maximum Transparency — All LLM calls, prompts, and responses must be user-inspectable. No black boxes.
  5. Minimize Dependencies — Avoid new deps. Flat files > database when sufficient. In-memory queues > external services.
  6. Pure JS/TS Over Native — Always prefer pure JS/WASM libraries over native C/C++ bindings (e.g., node-sqlite3-wasm over better-sqlite3).

Operating rules

Use the SDD process to select a lane, and follow its review and merge requirements for every PR. Resolve an uncertain lane at triage before implementation.

  1. Load the stack first. Start from this file and follow the context map. For spec-lane work the approved spec is required session input.
  2. One task, one issue, one PR. If the branch grows past its issue, stop, split, open the second issue.
  3. Search before opening. Check open PRs and issues for the same change.
  4. Declare honestly. The PR body lists what was verified (commands, results) and what was not.
  5. Never merge your own agent's work unreviewed. The person who prompted the agent is the author, not the reviewer.
  6. Respect invariants mechanically. Read docs/INVARIANTS.md before touching entities, IDs, staleness or layering; run pnpm lint:invariants (planned; see docs/INVARIANTS.md).
  7. Update the map you used. A doc found wrong during the session is fixed in the same PR.
  8. Experiments stay in the experiment lane. A branch that outgrew its issue becomes a finding + spec, not a bigger PR.

Context map — read before you start

If the task touches… Read first
an entity, save, regenerate docs/ARCHITECTURE.md#book-directory-structure (an #entities section is planned), docs/INVARIANTS.md
staleness, re-render, cache docs/specs/SPEC-0001-per-section-staleness.md (in review — arrives with PR #879)
sectioning, reading order docs/specs/SPEC-0003-sectioning-modes.md (planned), #834
a prompt under prompts/ docs/PROMPTS.md (planned; prompt contracts), the prompt validator
the reader (adt-runtime) docs/ARCHITECTURE.md#runtime (planned), docs/SECURITY_MODEL.md (planned)
exports or publishing docs/PRODUCT.md#registry (planned), docs/specs/SPEC-0006-publishing (planned)
anything user-facing docs/PRODUCT.md#critical-path (planned)

Architecture

packages/          # Shared libraries (@adt/* workspace packages)
  types/           # Zod schemas — ALL types defined here
  pipeline/        # Extraction & generation — pure functions only
  llm/             # LLM client, prompts, caching, cost tracking
  pdf/             # PDF extraction
  output/          # Bundle packaging

apps/              # Application tier
  api/             # Hono HTTP server
  studio/          # React SPA (Vite)
  desktop/         # Electron desktop wrapper

templates/         # Layout templates
config/            # Global configuration
docs/              # Documentation (guidelines, architecture)

Layer rule: studio/desktop → (HTTP only) → api → (direct imports) → packages/* Frontend MUST NOT import from packages directly. All data flows through the API.

Exception: @adt/types may be imported by studio for the shared PIPELINE definition and derived constants (stage/step names, ordering). No business logic — only type-level and constant data.

Pipeline Architecture

The pipeline uses a two-level DAG model defined in a single source of truth: packages/types/src/pipeline.ts.

Terminology

  • Stage — A high-level grouping visible in the UI (e.g., Extract, Storyboard, Quizzes). Stages have inter-stage dependencies forming a DAG.
  • Step — An atomic processing operation within a stage (e.g., image-filtering, page-sectioning). Steps have intra-stage dependencies. Steps within the same stage can run in parallel if their dependencies are met.

Single Source of Truth

The PIPELINE constant in @adt/types defines all stages, their steps, labels, and dependency graphs. Everything else is derived:

  • CLI progress bars — pre-created from PIPELINE
  • API step runner — stage ordering and step groupings from STAGE_ORDER
  • UI sidebar, cards, indicators — all derived from PIPELINE
  • DAG execution engine — reads PIPELINE to build the execution graph

Never hardcode stage/step ordering, names, or groupings outside of PIPELINE. If you need a new derived lookup, add it to packages/types/src/pipeline.ts alongside the existing ones (STAGE_ORDER, STEP_TO_STAGE, STAGE_BY_NAME, ALL_STEP_NAMES).

Docker

Three build targets in Dockerfile:

Target Description Used by
api Node.js API server only docker-compose.yml (multi-container)
studio nginx serving the built SPA docker-compose.yml (multi-container)
app Combined single-image (API + nginx) Release CI → ghcr.io/unicef/adt-studio 
# Multi-container — local testing (two separate services)
docker compose up --build

# Single-image — same image end users download
docker build --target app -t adt-studio .
docker run -p 8080:80 -v ./books:/app/books adt-studio

Runtime env vars and volumes:

Env var Default Override
BOOKS_DIR /app/books -v ./books:/app/books
PROMPTS_DIR /app/prompts -v ./prompts:/app/prompts
CONFIG_PATH /app/config.yaml -v ./config.yaml:/app/config.yaml:ro
FONTS_CACHE_DIR <BOOKS_DIR>/.fonts-cache Global Google Fonts cache shared across books (persists in the books volume by default)
STYLEGUIDES_DIR <BOOKS_DIR>/.styleguides User-uploaded global style guides. LLM-generated guides live under each book's styleguides/ directory so project exports remain self-contained. Bundled presets remain read-only.
PORT 3001 Internal only — nginx proxies to this

TEMPLATES_DIR trap: The Dockerfile and docker-compose.yml set TEMPLATES_DIR=/app/templates but the application never reads this env var. Templates dir is always derived from path.join(path.dirname(PROMPTS_DIR), "templates"). To use a custom templates directory, mount it as a sibling of prompts/ — i.e. override PROMPTS_DIR and keep templates/ next to it.

Release pipeline: pushing a v* tag triggers .github/workflows/release.yml which builds the app target and pushes to ghcr.io/unicef/adt-studio:latest and ghcr.io/unicef/adt-studio:<tag>. A ready-to-use docker-compose.yml (generated from docker/compose-release.yml.template) is uploaded as a release asset.

Key Docker files:

  • Dockerfile — multi-stage build (base → deps → build → api / studio / app)
  • docker-compose.yml — local dev/testing (multi-container)
  • docker/nginx.conf — nginx config for the studio stage (proxies to http://api:3001)
  • docker/nginx-single.conf — nginx config for the app stage (proxies to http://127.0.0.1:3001)
  • docker/entrypoint.sh — starts API + nginx in the app stage, health-checks API before nginx starts
  • docker/compose-release.yml.template — template for the release asset

External packages in Docker: jsdom, esbuild, tailwindcss, postcss, and playwright cannot be bundled by esbuild because they read data files relative to their own __dirname. They are installed into apps/api/dist/node_modules/ by the Dockerfile build stage via npm. If a new package exhibits the same pattern (ENOENT error pointing to a path under /app/apps/), add it to both the external array in apps/api/scripts/bundle-server.mjs and the npm install step in the Dockerfile.

The claude-agent provider ships no dependency at all. @anthropic-ai/claude-agent-sdk is deliberately not a dependency — its per-platform optional packages each carry a ~250 MB claude executable, and the ADT only needs the CLI's headless --print --input-format stream-json --output-format stream-json contract (packages/llm/src/providers/claude-agent/cli.ts, flag floor MINIMUM_CLAUDE_CLI_VERSION). The provider resolves the executable from the machine — CLAUDE_AGENT_EXECUTABLE override first, then a Claude Code installation found on PATH or in known install locations (~/.local/bin; on Windows only claude.exe, never .cmd shims) — reusing that machine's claude login when no API key is set. Like the codex provider, it reports "cli-not-found" wherever no Claude Code installation exists (containers included). Model discovery without an API key serves the CLI's stable family aliases (sonnet, opus, haiku) since the CLI has no model-listing command.

The codex provider ships no dependency at all. It spawns the Codex CLI already on the machine via packages/llm/src/providers/codex/cli.ts — CODEX_EXECUTABLE first, then codex on PATH, then the common install dirs (~/.local/bin, /opt/homebrew/bin, /usr/local/bin) and, on macOS, the CLI bundled inside the ChatGPT desktop app (/Applications/ChatGPT.app/Contents/Resources/codex), so GUI-launched desktop builds with a minimal PATH still find it — reusing that machine's codex login. Studio can run that login itself (Settings → Providers → OpenAI → Codex CLI → "Sign in with ChatGPT") by spawning the CLI's own codex login (packages/llm/src/providers/codex/login.ts, exposed as the optional cliLogin port and the /providers/:id/cli-login routes): the CLI opens the browser and serves the OAuth callback on localhost itself, so the browser must run on the same machine as the API (true for the desktop app and local dev); the ADT only relays the sign-in URL as a fallback link and never reads the tokens. The device-code variant (--device-auth) is deliberately not used — ChatGPT accounts have it disabled by default. A CODEX_API_KEY in the server environment is treated as the configured credential and outranks that login. @openai/codex-sdk is deliberately not a dependency — it carries its own copy of the CLI (~410 MB per platform), and the ADT only needs the codex exec --json contract. So nothing has to be added to the external array, the Dockerfile npm step, or install-server-runtime.mjs: the provider simply reports "Codex CLI not found" wherever no CLI is installed (containers included), exactly like the keyless auth it depends on.

Commands

pnpm install       # Install dependencies
pnpm dev           # Run API + Studio dev servers
pnpm test          # Run tests (builds first via pretest)
pnpm typecheck     # TypeScript strict check (tsc --build; excludes apps/adt-runtime)
pnpm lint          # Lint apps/studio — the only package with an eslint config
pnpm build         # Build all packages
pnpm pipeline doctor [books-dir]   # Read-only compatibility check of existing books (schema, locks, ownership, duplicate ids, re-keyed catalog ids)

ESLint is configured for apps/studio only (apps/studio/eslint.config.js), so pnpm lint delegates to pnpm --filter @adt/studio lint — the same command the CI i18n job runs. apps/adt-runtime is not linted; it is covered by pnpm test and its own pnpm --filter @adt/runtime typecheck.

Desktop Development

Prerequisites: Node.js 20+ and pnpm. No Rust, no platform native toolchains required for app code (electron-builder pulls platform-specific signing/packaging tools as needed).

# Dev mode — electron-vite drives main/preload/renderer with HMR
pnpm dev:desktop

# Production build — bundles API server, Studio SPA, and Electron app, then packages with electron-builder
pnpm build:desktop                       # all-in-one
pnpm --filter @adt/desktop build:unpack  # unpacked dir, no installer
pnpm --filter @adt/desktop build:win     # Windows NSIS installer
pnpm --filter @adt/desktop build:mac     # macOS DMG
pnpm --filter @adt/desktop build:linux   # Linux AppImage

How the desktop app runs the API

The API server is bundled by esbuild into a single ESM file (apps/api/dist-electron/api-server.mjs, plus required WASM assets) and copied into the Electron output. The Electron main process boots the API in-process on a free local port and the renderer process loads the Studio SPA. The frontend detects the Electron environment via the preload-injected window.electronAPI and points its API calls at the local API port.

Key files:

  • apps/api/scripts/bundle-electron-server.mjs — esbuild bundle of the API + WASM copy
  • apps/api/scripts/install-server-runtime.mjs — installs runtime-only deps (jsdom, playwright, etc.) next to the bundle so esbuild externals resolve at runtime
  • apps/desktop/electron.vite.config.ts — electron-vite config (main / preload / renderer)
  • apps/desktop/electron-builder.js — packaging config (appId, targets, signing, extraResources)
  • apps/desktop/src/main/index.ts — main process entry; spawns the API and creates windows
  • apps/desktop/src/main/api/index.ts — API lifecycle (port selection, startup, shutdown)
  • apps/desktop/src/preload/index.ts — exposes window.electronAPI to the renderer
  • apps/studio/src/api/client.ts — Electron base URL detection

Releasing

Releases are cut from branches, never by pushing a hand-made tag. The Release workflow calculates the next version from the existing git tags, builds and signs the desktop installers, publishes the Docker image, then creates the tag and GitHub Release itself.

  • develop ships beta releases (beta, beta-minor, beta-major).
  • main ships stable releases (patch, minor, major).

Trigger it from Actions → Release → Run workflow (pick the branch and a version increment) or by pushing a commit whose subject is a release type (e.g. RELEASE: beta on develop, RELEASE: patch on main).

Never create a v* tag by hand. Version numbers are derived from the tags by scripts/calculate-release-version.mjs, so a manually pushed tag corrupts every future version calculation. The v* tag namespace is protected so only the release automation (via RELEASE_PAT) can create tags — see docs/RELEASING.md.

See docs/RELEASING.md for the full branch, staging, and version-calculation flow.

Internationalization (i18n)

The Studio app uses Lingui v5 for i18n. All user-visible text in apps/studio/ must be translated to all supported locales: en, pt-BR, es, fr, sq.

Rules

  • Every user-visible string must be wrapped in a Lingui macro — no raw string literals in JSX or component output
    • In React components: const { t } = useLingui() → t\Your string``
    • In JSX content: <Trans>Your string</Trans>
    • In non-React code (utils, constants): msg\Your string`+i18n._()` at runtime
  • After adding or changing any string, run pnpm --filter @adt/studio extract to update all .po catalog files and commit them alongside the code change
  • All locales must be fully translated — no empty msgstr entries in es.po, pt-BR.po, fr.po, or sq.po
  • CI enforces both rules automatically via the i18n job in .github/workflows/ci.yml

Available locales

Defined in apps/studio/src/i18n/locales.ts (single source of truth for locale metadata) and mirrored in apps/studio/lingui.config.ts (required separately by the Lingui CLI):

  • en — English (source locale)
  • pt-BR — Portuguese (Brazil)
  • es — Spanish
  • fr — French
  • sq — Albanian (Kosovo)

ESLint — hardcoded string detection

The lingui/no-unlocalized-strings rule in apps/studio/eslint.config.js flags raw strings that should be wrapped in a macro. It has an ignoreNames list for prop names whose values are never user-visible (e.g. variant, className, href).

When adding a new prop or variable name that holds a non-translatable value (e.g. a new component prop like iconName or queryKey):

  1. Check if it's already in ignoreNames in eslint.config.js
  2. If not, decide: is this value ever shown to the user as text?
    • No (it's a key, identifier, or config value) → add it to ignoreNames
    • Yes (it's displayed as UI text) → wrap the value in the appropriate macro instead
  3. After updating ignoreNames, regenerate the suppressions baseline: 
    cd apps/studio
npx eslint src --suppressions-location ./eslint-suppressions.json --prune-suppressions
npx eslint src --suppress-all --suppressions-location ./eslint-suppressions.json

Adding a new language

See docs/I18N_ADD_LANGUAGE.md.

Key Rules

  • All types defined as Zod schemas in packages/types/, infer TS types with z.infer<>
  • All API calls from frontend go through apps/studio/src/api/client.ts + TanStack Query
  • Styling: Tailwind utility classes only — no CSS modules, no styled-components
  • Keep React component files focused. If a file grows multiple substantial components, split it into a folder with a primary component file plus focused child components, hooks, and helpers (for example ComponentName/ComponentName.tsx, ComponentName/Child.tsx, ComponentName/helpers.ts).
  • Server state: TanStack Query — no Redux, Zustand, or global stores; useState for UI-only state
  • Routing: TanStack Router (type-safe), Forms: TanStack Form, Tables: TanStack Table
  • Pipeline functions must be pure (no side effects, all deps as params)
  • All user input validated with Zod (API layer)
  • API keys: header-based (X-OpenAI-Key), never logged, never in URLs
  • File paths: always validate against base directory (path traversal prevention)
  • SQL: parameterized queries only

Full Guidelines

For complete coding standards, security requirements, patterns, and anti-patterns, see docs/GUIDELINES.md. For technology decisions and reasoning, see docs/DECISIONS.md. For per-role responsibilities when assigning focused agent tasks (Code Reviewer, Pipeline Developer, Frontend Developer), see docs/AGENT_ROLES.md.