diff --git a/hindsight-docs/docs-integrations/coding-agents.md b/hindsight-docs/docs-integrations/coding-agents.md index 044d10a3bd..0bfa933dcb 100644 --- a/hindsight-docs/docs-integrations/coding-agents.md +++ b/hindsight-docs/docs-integrations/coding-agents.md @@ -501,7 +501,7 @@ hook by Codex...), so one shared config serves several agents side by side: | `autoReflect` | `true` | inject a one-time reflect synthesis on the session's **first prompt**. `false` = tool-only reflect: nothing is injected; the agent searches knowledge pages first and reflects only when they are too shallow | | `pageRefreshEveryTurns` | `10` | refetch the knowledge pages and re-inject the page roster + tool guide every N user turns | | `pageTriggerType` | `"auto-refresh"` | when NEW knowledge pages refresh, i.e. what keeping them current costs — `"auto-refresh"` after every consolidation that produced new material, `"cron"` on `pageTriggerCron` only, `"manual"` never on their own. Auto-refresh is the most current and the most expensive: one synthesis per page per consolidation. Maps to the page's `trigger.refresh_after_consolidation` in the Hindsight API (`true` for auto-refresh, `false` for manual) | -| `pageTriggerCron` | — | schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. `"0 3 * * *"`. Sets the page's `trigger.refresh_cron`, which the API treats as mutually exclusive with `refresh_after_consolidation`; a scheduled refresh is skipped when nothing changed | +| `pageTriggerCron` | — | schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. `"0 3 * * *"`. Sets the page's `trigger.refresh_cron`, which the API treats as mutually exclusive with `refresh_after_consolidation`; a scheduled refresh is skipped when nothing changed. Write a field as `H` to give each page its own value there — see **Spreading refreshes with `H`** below | | `autoSeed` | `true` | SessionStart: auto-seed a cold repo's bank from git history | | `seedLimit` | `300` | auto-seed: most-recent-N-commits cap | | `codebaseSurvey` | `true` | SessionStart: headless survey of a cold repo's structure, run under the current harness's own CLI (claude/codex/antigravity/opencode), falling back to any available agent | @@ -521,6 +521,30 @@ belongs to the server: Hindsight creates a knowledge page with a delta refresh ( page instead of rebuilding it) that doesn't reflect over sibling pages, and these settings merge over those defaults rather than replacing them. +### Spreading refreshes with `H` + +One `pageTriggerCron` is shared by every page in every repo you point this plugin at. So a literal +`"0 3 * * *"` does not schedule _a_ refresh at 03:00 — it schedules **all** of them at 03:00, five +pages per bank, on the same worker pool that serves retain. A session ingesting at 03:0x queues +behind the pile, and moving the hour just moves the pile. + +Write a field as `H` and it is replaced, per page, by a value hashed from the bank id and the page +name. Each page gets its own slot, the same slot on every run: + +| `pageTriggerCron` | what each page gets | +| ------------------ | ------------------------------------------------------ | +| `"H H * * *"` | once a day, at its own minute and hour | +| `"H * * * *"` | once an hour, at its own minute | +| `"H 3 * * *"` | daily at 03:MM — spread inside the hour you chose | +| `"H H(0-5) * * *"` | daily, spread across 00:00–05:59 only | +| `"0 3 * * *"` | no `H`, no hashing — exactly what it says, all at once | + +`H` is [Jenkins' syntax](https://www.jenkins.io/doc/book/pipeline/syntax/#cron-syntax) for the same +problem. It never reaches the API: the plugin resolves it to an ordinary cron expression +(`"41 17 * * *"`) when it creates the page, so the schedule you see in the control plane is a plain +one you can edit. Hashing spreads pages out, it does not partition them — two pages can still land +on the same minute, just not all of them. + **These settings apply to pages created from here on.** Changing them does not migrate the pages a repo already has: a page keeps the trigger it was created with, so a bank seeded before you set `"manual"` keeps refreshing on every consolidation. To move an existing page, change its trigger diff --git a/hindsight-integrations/coding-agents/README.md b/hindsight-integrations/coding-agents/README.md index 22569c2a61..37c26f9647 100644 --- a/hindsight-integrations/coding-agents/README.md +++ b/hindsight-integrations/coding-agents/README.md @@ -504,7 +504,7 @@ hook by Codex...), so one shared config serves several agents side by side: | `autoReflect` | `true` | inject a one-time reflect synthesis on the session's **first prompt**. `false` = tool-only reflect: nothing is injected; the agent searches knowledge pages first and reflects only when they are too shallow | | `pageRefreshEveryTurns` | `10` | refetch the knowledge pages and re-inject the page roster + tool guide every N user turns | | `pageTriggerType` | `"auto-refresh"` | when NEW knowledge pages refresh, i.e. what keeping them current costs — `"auto-refresh"` after every consolidation that produced new material, `"cron"` on `pageTriggerCron` only, `"manual"` never on their own. Auto-refresh is the most current and the most expensive: one synthesis per page per consolidation. Maps to the page's `trigger.refresh_after_consolidation` in the Hindsight API (`true` for auto-refresh, `false` for manual) | -| `pageTriggerCron` | — | schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. `"0 3 * * *"`. Sets the page's `trigger.refresh_cron`, which the API treats as mutually exclusive with `refresh_after_consolidation`; a scheduled refresh is skipped when nothing changed | +| `pageTriggerCron` | — | schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. `"0 3 * * *"`. Sets the page's `trigger.refresh_cron`, which the API treats as mutually exclusive with `refresh_after_consolidation`; a scheduled refresh is skipped when nothing changed. Write a field as `H` to give each page its own value there — see **Spreading refreshes with `H`** below | | `autoSeed` | `true` | SessionStart: auto-seed a cold repo's bank from git history | | `seedLimit` | `300` | auto-seed: most-recent-N-commits cap | | `codebaseSurvey` | `true` | SessionStart: headless survey of a cold repo's structure, run under the current harness's own CLI (claude/codex/antigravity/opencode), falling back to any available agent | @@ -524,6 +524,30 @@ belongs to the server: Hindsight creates a knowledge page with a delta refresh ( page instead of rebuilding it) that doesn't reflect over sibling pages, and these settings merge over those defaults rather than replacing them. +### Spreading refreshes with `H` + +One `pageTriggerCron` is shared by every page in every repo you point this plugin at. So a literal +`"0 3 * * *"` does not schedule _a_ refresh at 03:00 — it schedules **all** of them at 03:00, five +pages per bank, on the same worker pool that serves retain. A session ingesting at 03:0x queues +behind the pile, and moving the hour just moves the pile. + +Write a field as `H` and it is replaced, per page, by a value hashed from the bank id and the page +name. Each page gets its own slot, the same slot on every run: + +| `pageTriggerCron` | what each page gets | +| ------------------ | ------------------------------------------------------ | +| `"H H * * *"` | once a day, at its own minute and hour | +| `"H * * * *"` | once an hour, at its own minute | +| `"H 3 * * *"` | daily at 03:MM — spread inside the hour you chose | +| `"H H(0-5) * * *"` | daily, spread across 00:00–05:59 only | +| `"0 3 * * *"` | no `H`, no hashing — exactly what it says, all at once | + +`H` is [Jenkins' syntax](https://www.jenkins.io/doc/book/pipeline/syntax/#cron-syntax) for the same +problem. It never reaches the API: the plugin resolves it to an ordinary cron expression +(`"41 17 * * *"`) when it creates the page, so the schedule you see in the control plane is a plain +one you can edit. Hashing spreads pages out, it does not partition them — two pages can still land +on the same minute, just not all of them. + **These settings apply to pages created from here on.** Changing them does not migrate the pages a repo already has: a page keeps the trigger it was created with, so a bank seeded before you set `"manual"` keeps refreshing on every consolidation. To move an existing page, change its trigger diff --git a/hindsight-integrations/coding-agents/skill/SKILL.md b/hindsight-integrations/coding-agents/skill/SKILL.md index 93add576b4..27badba742 100644 --- a/hindsight-integrations/coding-agents/skill/SKILL.md +++ b/hindsight-integrations/coding-agents/skill/SKILL.md @@ -210,7 +210,7 @@ hook by Codex...), so one shared config serves several agents side by side: | `autoReflect` | `true` | inject a one-time reflect synthesis on the session's **first prompt**. `false` = tool-only reflect: nothing is injected; the agent searches knowledge pages first and reflects only when they are too shallow | | `pageRefreshEveryTurns` | `10` | refetch the knowledge pages and re-inject the page roster + tool guide every N user turns | | `pageTriggerType` | `"auto-refresh"` | when NEW knowledge pages refresh, i.e. what keeping them current costs — `"auto-refresh"` after every consolidation that produced new material, `"cron"` on `pageTriggerCron` only, `"manual"` never on their own. Auto-refresh is the most current and the most expensive: one synthesis per page per consolidation. Maps to the page's `trigger.refresh_after_consolidation` in the Hindsight API (`true` for auto-refresh, `false` for manual) | -| `pageTriggerCron` | — | schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. `"0 3 * * *"`. Sets the page's `trigger.refresh_cron`, which the API treats as mutually exclusive with `refresh_after_consolidation`; a scheduled refresh is skipped when nothing changed | +| `pageTriggerCron` | — | schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. `"0 3 * * *"`. Sets the page's `trigger.refresh_cron`, which the API treats as mutually exclusive with `refresh_after_consolidation`; a scheduled refresh is skipped when nothing changed. Write a field as `H` to give each page its own value there — see **Spreading refreshes with `H`** below | | `autoSeed` | `true` | SessionStart: auto-seed a cold repo's bank from git history | | `seedLimit` | `300` | auto-seed: most-recent-N-commits cap | | `codebaseSurvey` | `true` | SessionStart: headless survey of a cold repo's structure, run under the current harness's own CLI (claude/codex/antigravity/opencode), falling back to any available agent | @@ -230,6 +230,30 @@ belongs to the server: Hindsight creates a knowledge page with a delta refresh ( page instead of rebuilding it) that doesn't reflect over sibling pages, and these settings merge over those defaults rather than replacing them. +### Spreading refreshes with `H` + +One `pageTriggerCron` is shared by every page in every repo you point this plugin at. So a literal +`"0 3 * * *"` does not schedule _a_ refresh at 03:00 — it schedules **all** of them at 03:00, five +pages per bank, on the same worker pool that serves retain. A session ingesting at 03:0x queues +behind the pile, and moving the hour just moves the pile. + +Write a field as `H` and it is replaced, per page, by a value hashed from the bank id and the page +name. Each page gets its own slot, the same slot on every run: + +| `pageTriggerCron` | what each page gets | +| ------------------ | ------------------------------------------------------ | +| `"H H * * *"` | once a day, at its own minute and hour | +| `"H * * * *"` | once an hour, at its own minute | +| `"H 3 * * *"` | daily at 03:MM — spread inside the hour you chose | +| `"H H(0-5) * * *"` | daily, spread across 00:00–05:59 only | +| `"0 3 * * *"` | no `H`, no hashing — exactly what it says, all at once | + +`H` is [Jenkins' syntax](https://www.jenkins.io/doc/book/pipeline/syntax/#cron-syntax) for the same +problem. It never reaches the API: the plugin resolves it to an ordinary cron expression +(`"41 17 * * *"`) when it creates the page, so the schedule you see in the control plane is a plain +one you can edit. Hashing spreads pages out, it does not partition them — two pages can still land +on the same minute, just not all of them. + **These settings apply to pages created from here on.** Changing them does not migrate the pages a repo already has: a page keeps the trigger it was created with, so a bank seeded before you set `"manual"` keeps refreshing on every consolidation. To move an existing page, change its trigger diff --git a/hindsight-integrations/coding-agents/src/core/config.ts b/hindsight-integrations/coding-agents/src/core/config.ts index cf9e0866e9..4cfa32f2f3 100644 --- a/hindsight-integrations/coding-agents/src/core/config.ts +++ b/hindsight-integrations/coding-agents/src/core/config.ts @@ -18,6 +18,7 @@ import { DEFAULT_SEED_LIMIT } from "./seed"; import { isOptedIn } from "./bank"; import { log } from "./log"; import { DEFAULT_OBSERVATION_SCOPES, type ObservationScopes } from "./hindsight"; +import { isHashedCron, parseHashedCron } from "./missions"; /** Default config-file path: ~/.hindsight/coding-agent.json */ export // HINDSIGHT_CONFIG joins the two env exceptions (diag/log files): it points at THE config file, @@ -114,7 +115,10 @@ export interface RawConfig { * consolidation, which adds up fast across auto-surveyed repos (#3506). Existing pages keep the * trigger they were created with — this changes what NEW pages get. */ pageTriggerType?: "auto-refresh" | "cron" | "manual"; - /** Schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. "0 3 * * *". */ + /** Schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. "0 3 * * *". + * A field written `H` ("0 3 * * *" -> "H H * * *") is replaced per page by a value hashed from + * bank + page name, so pages spread across the period instead of all firing on the one minute + * this shared setting names. See `expandCronHash` in core/missions.ts. */ pageTriggerCron?: string; autoSeed?: boolean; // SessionStart: auto-seed a cold repo's bank from git history (default true) seedLimit?: number; // SessionStart auto-seed: most-recent-N-commits cap (default 300) @@ -230,15 +234,26 @@ export interface Config { * `"cron"` without a `pageTriggerCron` is a broken config, not a request to stop refreshing: the * API rejects a cron trigger with no expression, which would fail page creation outright. Fall * back to the default and say so — a user who wants pages to stop refreshing writes "manual". + * + * A malformed `H` is refused here for the same reason and not one step later: `expandCronHash` + * leaves an expression it cannot read alone, so an unchecked `"H(9-3) * * * *"` would reach the + * server verbatim and fail page creation with a cron parse error naming syntax this package + * invented. Ordinary cron syntax stays unvalidated — the server owns that, and duplicating its + * parser here would only disagree with it. */ function resolvePageTriggerType(raw: RawConfig): "auto-refresh" | "cron" | "manual" { if (raw.pageTriggerType === "manual") return "manual"; if (raw.pageTriggerType === "cron") { - if (raw.pageTriggerCron?.trim()) return "cron"; + const cron = raw.pageTriggerCron?.trim(); + if (cron && (!isHashedCron(cron) || parseHashedCron(cron))) return "cron"; log.warn( "config", - 'pageTriggerType "cron" needs pageTriggerCron (UTC 5-field, e.g. "0 3 * * *") — ' + - 'falling back to "auto-refresh"' + cron + ? `pageTriggerCron ${JSON.stringify(cron)} has a malformed hashed field — ` + + 'write `H` or `H(-)` within the field\'s own range, e.g. "H H(0-5) * * *" — ' + + 'falling back to "auto-refresh"' + : 'pageTriggerType "cron" needs pageTriggerCron (UTC 5-field, e.g. "H H * * *") — ' + + 'falling back to "auto-refresh"' ); } return "auto-refresh"; diff --git a/hindsight-integrations/coding-agents/src/core/hindsight.pages.test.ts b/hindsight-integrations/coding-agents/src/core/hindsight.pages.test.ts index 1db08fdc31..ec7925b971 100644 --- a/hindsight-integrations/coding-agents/src/core/hindsight.pages.test.ts +++ b/hindsight-integrations/coding-agents/src/core/hindsight.pages.test.ts @@ -402,6 +402,51 @@ describe("HindsightClient.seedPages", () => { } }); + /** + * The whole point of `H`: one `pageTriggerCron` is copied into every repo's config, so a literal + * expression puts all five pages of every bank on the same minute, competing with retain on the + * same worker pool. Resolution happens HERE, at page creation, because that is where the page's + * identity exists. + */ + it("gives each page its own slot when the cron asks to be hashed", async () => { + const seed = async (bank: string) => { + const calls: any[] = []; + stubFetchRouted(calls, [ + { match: (m, u) => m === "GET" && u.endsWith("/knowledge-base/tree"), json: { roots: [] } }, + ]); + const c = new HindsightClient({ apiUrl: "http://x", bank }); + await c.seedPages( + buildPageTrigger(resolveConfig({ pageTriggerType: "cron", pageTriggerCron: "H H * * *" })) + ); + return calls + .filter((k) => k.method === "POST" && k.url.endsWith("/knowledge-base/pages")) + .map((k) => k.body.trigger.refresh_cron as string); + }; + + const a = await seed("repo-a"); + expect(a).toHaveLength(PAGES.length); + // Resolved to plain cron — `H` is this package's syntax and the server would reject it. + for (const cron of a) expect(cron).toMatch(/^\d+ \d+ \* \* \*$/); + expect(new Set(a).size).toBe(PAGES.length); + // Stable across runs, and different in another bank seeded from the same config. + expect(await seed("repo-a")).toEqual(a); + expect(await seed("repo-b")).not.toEqual(a); + }); + + it("sends a cron with no H exactly as configured", async () => { + const calls: any[] = []; + stubFetchRouted(calls, [ + { match: (m, u) => m === "GET" && u.endsWith("/knowledge-base/tree"), json: { roots: [] } }, + ]); + const c = new HindsightClient({ apiUrl: "http://x", bank: "repo-a" }); + await c.seedPages( + buildPageTrigger(resolveConfig({ pageTriggerType: "cron", pageTriggerCron: "0 3 * * *" })) + ); + for (const post of calls.filter((k) => k.method === "POST")) { + expect(post.body.trigger.refresh_cron).toBe("0 3 * * *"); + } + }); + it("falls back to the bank id when no project is supplied, never an unscoped query", async () => { const calls: any[] = []; stubFetchRouted(calls, [ @@ -564,6 +609,44 @@ describe("HindsightClient.captureInitiative", () => { expect(item.context).toContain(`[[page:${result.page_id}]]`); }); + /** An initiative page is one of these pages, so it is staggered on the same terms — seeded by + * its own title rather than a taxonomy name. */ + it("hashes an initiative page's own schedule", async () => { + const capture = async (title: string) => { + const calls: any[] = []; + stubFetchRouted(calls, [ + { match: (m, u) => m === "GET" && u.endsWith("/knowledge-base/tree"), json: { roots: [] } }, + { + match: (m, u) => m === "POST" && u.endsWith("/knowledge-base/folders"), + json: { id: "folder-abc" }, + }, + { + match: (m, u) => m === "POST" && u.endsWith("/knowledge-base/pages"), + json: { page_id: "pg" }, + }, + { + match: (m, u) => m === "POST" && u.endsWith("/memories"), + json: { operation_id: "op-1" }, + }, + ]); + const c = new HindsightClient({ apiUrl: "http://x", bank: "repo-a" }); + await c.captureInitiative({ + title, + summary: "…", + pageTrigger: buildPageTrigger( + resolveConfig({ pageTriggerType: "cron", pageTriggerCron: "H H * * *" }) + ), + }); + return calls.find((k) => k.method === "POST" && k.url.endsWith("/knowledge-base/pages"))!.body + .trigger.refresh_cron as string; + }; + + const one = await capture("Retry backoff for the uploader"); + expect(one).toMatch(/^\d+ \d+ \* \* \*$/); + expect(await capture("Retry backoff for the uploader")).toBe(one); + expect(await capture("Typo-tolerant tag matching")).not.toBe(one); + }); + it("enhancement (relatesToPageId): NO page POST; marker names the existing page id", async () => { const calls: any[] = []; stubFetchRouted(calls, [ diff --git a/hindsight-integrations/coding-agents/src/core/hindsight.ts b/hindsight-integrations/coding-agents/src/core/hindsight.ts index 99dfbbe781..f286e059d2 100644 --- a/hindsight-integrations/coding-agents/src/core/hindsight.ts +++ b/hindsight-integrations/coding-agents/src/core/hindsight.ts @@ -12,6 +12,7 @@ import { PAGE_MAX_TOKENS, pagesFor, type PageTrigger, + pageTriggerFor, } from "./missions"; import { pool, semverGte, sleep } from "./util"; import type { RetainStamp } from "./retain-stamp"; @@ -652,7 +653,9 @@ export class HindsightClient { source_query: page.source_query, tags: page.tags, max_tokens: PAGE_MAX_TOKENS, - trigger: pageTrigger, + // Resolved HERE, not in `buildPageTrigger`: a hashed cron (`H`) needs the page's identity, + // and one trigger is built per session for all of them. + trigger: pageTriggerFor(pageTrigger, this.bank, page.name), }; if (!hit) { // 409 = another deepen run seeded this name between our tree read and this POST. That is @@ -751,7 +754,7 @@ export class HindsightClient { source_query: `Summarize the "${args.title}" initiative: what is being built or changed and why, and its current state — drawn from the project's memory.`, parent_id: folderId, tags: ["knowledge:feature-work"], - trigger: args.pageTrigger ?? buildPageTrigger(), + trigger: pageTriggerFor(args.pageTrigger ?? buildPageTrigger(), this.bank, args.title), }); try { const j = (await r.json()) as { page_id?: string; id?: string }; diff --git a/hindsight-integrations/coding-agents/src/core/missions.test.ts b/hindsight-integrations/coding-agents/src/core/missions.test.ts index e39fb7adc0..41505e29bd 100644 --- a/hindsight-integrations/coding-agents/src/core/missions.test.ts +++ b/hindsight-integrations/coding-agents/src/core/missions.test.ts @@ -5,7 +5,9 @@ import { buildPageTrigger, CODING_BANK_TEMPLATE, codingBankManifest, + expandCronHash, KNOWLEDGE_LABELS, + pageTriggerFor, PAGE_FACT_TYPES, REFLECT_MISSION, RETAIN_STRATEGIES, @@ -74,6 +76,93 @@ describe("buildPageTrigger", () => { }); }); +/** + * One `pageTriggerCron` is shared by every page in every bank running this plugin, so a literal + * expression schedules ALL of them on the one minute it names — a pile of LLM syntheses on the + * worker pool that also serves retain. `H` is Jenkins' answer: hash the field per page. + */ +describe("hashed cron fields", () => { + const cron = (raw: string, bank: string, page: string) => + pageTriggerFor( + buildPageTrigger(resolveConfig({ pageTriggerType: "cron", pageTriggerCron: raw })), + bank, + page + ).refresh_cron; + + it("leaves an expression without H exactly as written", () => { + expect(cron("0 3 * * *", "repo-a", "Component map")).toBe("0 3 * * *"); + expect(cron("0 3 * * *", "repo-b", "Core concepts")).toBe("0 3 * * *"); + }); + + it("resolves H to an ordinary cron expression the server can parse", () => { + // `H` never leaves this package — `refresh_cron` is standard 5-field cron server-side. + expect(cron("H H * * *", "repo-a", "Component map")).toMatch( + /^(?:[0-9]|[1-5][0-9]) (?:[0-9]|1[0-9]|2[0-3]) \* \* \*$/ + ); + expect(cron("H * * * *", "repo-a", "Component map")).toMatch( + /^(?:[0-9]|[1-5][0-9]) \* \* \* \*$/ + ); + }); + + it("keeps the fields the operator wrote and hashes only the H", () => { + // "spread within 03:00" — the hour is a decision, the minute is not. + const daily = cron("H 3 * * *", "repo-a", "Component map"); + expect(daily).toMatch(/^\d+ 3 \* \* \*$/); + // A range bounds where the hash may land: spread across the night only. + const hours = new Set( + Array.from({ length: 60 }, (_, i) => + Number(cron("H H(0-5) * * *", `repo-${i}`, "Component map")!.split(" ")[1]) + ) + ); + expect(Math.min(...hours)).toBeGreaterThanOrEqual(0); + expect(Math.max(...hours)).toBeLessThanOrEqual(5); + expect(hours.size).toBeGreaterThan(1); + }); + + it("gives each page its own slot, stably", () => { + const one = cron("H H * * *", "repo-a", "Component map"); + // Stable: a page keeps its slot across runs, machines and releases, or every session would + // reschedule it (and the seed PATCH would report drift forever). + expect(cron("H H * * *", "repo-a", "Component map")).toBe(one); + expect(cron("H H * * *", "repo-a", "Core concepts")).not.toBe(one); + expect(cron("H H * * *", "repo-b", "Component map")).not.toBe(one); + }); + + it("spreads a shared config across banks instead of piling them on one minute", () => { + const slots = Array.from({ length: 50 }, (_, i) => + cron("H H * * *", `repo-${i}`, "Component map") + ); + // The whole point: 50 banks copying the same setting do not collide. Hashing distributes + // approximately — it does not partition — so a couple of collisions are expected, not a bug. + expect(new Set(slots).size).toBeGreaterThan(45); + }); + + it("does not derive minute and hour from the same number", () => { + // Hashing the field index alongside the seed is what keeps "H H * * *" worth 1440 slots + // rather than 60 correlated ones. + const minutes = new Set(); + const hours = new Set(); + for (let i = 0; i < 200; i++) { + const [m, h] = cron("H H * * *", `repo-${i}`, "Component map")!.split(" "); + minutes.add(Number(m)); + hours.add(Number(h)); + } + expect(minutes.size).toBeGreaterThan(40); + expect(hours.size).toBe(24); + }); + + it("leaves a malformed expression for the server to reject", () => { + // Rewriting it here would invent a schedule nobody asked for; resolveConfig refuses it first. + expect(expandCronHash("H(9-3) * * * *", "seed")).toBe("H(9-3) * * * *"); + expect(expandCronHash("H H", "seed")).toBe("H H"); + }); + + it("passes a trigger with no cron through untouched", () => { + const auto = buildPageTrigger(resolveConfig({})); + expect(pageTriggerFor(auto, "repo-a", "Component map")).toBe(auto); + }); +}); + describe("page trigger config resolution", () => { it("keeps today's behaviour when nothing is configured", () => { expect(resolveConfig({}).pageTriggerType).toBe("auto-refresh"); @@ -90,6 +179,27 @@ describe("page trigger config resolution", () => { ); }); + /** + * `expandCronHash` leaves an expression it cannot read alone, so an unchecked malformed `H` + * would reach the server verbatim and fail page creation with a parse error naming syntax this + * package invented. Only the H fields are checked — ordinary cron syntax is the server's. + */ + it("falls back to auto-refresh on a malformed hashed field", () => { + for (const bad of ["H(9-3) * * * *", "H(0-99) * * * *", "H H", "Hx * * * *"]) { + expect(resolveConfig({ pageTriggerType: "cron", pageTriggerCron: bad }).pageTriggerType).toBe( + "auto-refresh" + ); + } + }); + + it("accepts a well-formed hashed cron", () => { + for (const good of ["H H * * *", "H * * * *", "H 3 * * *", "H H(0-5) * * *"]) { + const cfg = resolveConfig({ pageTriggerType: "cron", pageTriggerCron: good }); + expect(cfg.pageTriggerType).toBe("cron"); + expect(cfg.pageTriggerCron).toBe(good); + } + }); + it("ignores a value that is not one of the three types", () => { expect(resolveConfig({ pageTriggerType: "whenever" as never }).pageTriggerType).toBe( "auto-refresh" diff --git a/hindsight-integrations/coding-agents/src/core/missions.ts b/hindsight-integrations/coding-agents/src/core/missions.ts index 1321a86d16..233ca01db5 100644 --- a/hindsight-integrations/coding-agents/src/core/missions.ts +++ b/hindsight-integrations/coding-agents/src/core/missions.ts @@ -6,6 +6,8 @@ * every harness adapter. */ +import { createHash } from "node:crypto"; + // ── retain missions (git vs chat need different extraction) ───────────────────── export const GIT_MISSION = "You are ingesting a single git commit: its message and its full diff. Extract the concrete " + @@ -324,6 +326,127 @@ export interface PageTriggerConfig { pageTriggerCron?: string; } +// ── hashed cron fields (`H`) ─────────────────────────────────────────────────── +/** + * A cron field written `H` means "pick a value in this field's range by hashing the page", so + * every page gets its OWN stable slot instead of the one the config literally names. + * + * One `pageTriggerCron` is shared by every page in every bank running this plugin — it ships as a + * single documented example and is copied verbatim. A literal `"0 3 * * *"` therefore does not + * schedule a refresh at 03:00; it schedules ALL of them at 03:00, on the worker pool that also + * serves retain, so a session ingesting at 03:0x queues behind ~5 page syntheses per bank that + * happened to share the one minute the docs suggested. Moving the hour moves the pile. + * + * `H` is Jenkins' syntax for exactly this problem, borrowed rather than invented because it is + * already recognisable, and it composes with the rest of the expression instead of replacing it: + * + * "H H * * *" once a day, at this page's own minute and hour + * "H * * * *" once an hour, at this page's own minute + * "H 3 * * *" daily at 03:MM — spread within the hour the operator chose + * "H H(0-5) * * *" daily, spread across the night only + * "0 3 * * *" unchanged: no `H`, no hashing, exactly what it says + * + * The alternative — one enum member per period (`daily-staggered`, then `hourly-staggered`, then + * whatever is asked for next) — spells the schedule in the type name, so every new period is a new + * config value, a new branch, and a new row of docs. Spreading is a property of the SCHEDULE, so it + * belongs in the expression. + * + * `H` never leaves this package: `expandCronHash` resolves it to an ordinary 5-field expression + * before the trigger is sent, because `refresh_cron` is parsed server-side as standard cron. + */ +const CRON_FIELD_RANGES: readonly (readonly [number, number])[] = [ + [0, 59], // minute + [0, 23], // hour + [1, 31], // day of month + [1, 12], // month + [0, 6], // day of week +]; + +const HASHED_FIELD = /^H(?:\((\d+)-(\d+)\))?$/; + +/** + * Does this expression ask for hashing at all? Plain crons take every path below unchanged. + * + * Any field STARTING with `H` counts, not just a well-formed one: no standard cron field begins + * with `H` (values are digits, `*`, `,`, `-`, `/`, and the JAN-DEC/SUN-SAT names), so `"Hx"` is a + * typo in this package's syntax rather than something the server was going to accept. Claiming it + * here is what gets it reported as a malformed hashed field instead of an opaque cron parse error. + */ +export function isHashedCron(cron: string): boolean { + return /(^|\s)H/.test(cron); +} + +/** + * The five fields of `cron` when every `H` in it is well-formed, else `undefined`. + * + * Only the `H` fields are checked. The rest are the server's to validate, as they already are — + * this package does not own cron syntax, only the extension it adds to it. + */ +export function parseHashedCron(cron: string): string[] | undefined { + const fields = cron.trim().split(/\s+/); + if (fields.length !== CRON_FIELD_RANGES.length) return undefined; + for (const [i, field] of fields.entries()) { + if (!field.startsWith("H")) continue; + const m = HASHED_FIELD.exec(field); + if (!m) return undefined; + if (m[1] === undefined) continue; + const [lo, hi] = [Number(m[1]), Number(m[2])]; + const [min, max] = CRON_FIELD_RANGES[i]; + if (lo > hi || lo < min || hi > max) return undefined; + } + return fields; +} + +/** + * `seed`'s own value in `[lo, hi]` — stable across machines, processes and releases. + * + * The field index is hashed alongside the seed so `H H * * *` does not derive its minute and its + * hour from one number: the two would move together across pages, collapsing the 1440 daily slots + * the expression offers back towards 60. + */ +function hashedValue(seed: string, field: number, lo: number, hi: number): number { + const digest = createHash("sha256").update(`${seed}\u0000${field}`).digest(); + return lo + (digest.readUInt32BE(0) % (hi - lo + 1)); +} + +/** + * `cron` with each `H` replaced by `seed`'s own value for that field — an ordinary cron expression. + * + * Returns the input untouched when it holds no `H`, and when an `H` in it is malformed: a bad + * expression is reported by the server that parses crons, not silently rewritten into a valid one + * that runs at a time nobody asked for. `resolvePageTriggerType` rejects it before it gets here. + */ +export function expandCronHash(cron: string, seed: string): string { + if (!isHashedCron(cron)) return cron; + const fields = parseHashedCron(cron); + if (!fields) return cron; + return fields + .map((field, i) => { + const m = HASHED_FIELD.exec(field); + if (!m) return field; + const [lo, hi] = + m[1] === undefined ? CRON_FIELD_RANGES[i] : ([Number(m[1]), Number(m[2])] as const); + return String(hashedValue(seed, i, lo, hi)); + }) + .join(" "); +} + +/** + * `trigger` as it should be sent for ONE page, resolving any `H` against that page's identity. + * + * Applied where a page is created rather than where the trigger is built, because that is the only + * place the identity exists: `buildPageTrigger` runs once per session for all of them. + * + * The seed is bank + page name — the pair that identifies a page across runs — so a page keeps its + * slot for as long as it keeps its name, and two banks seeded from the same config land on + * different ones. Hashing distributes; it does not partition, so two pages CAN still collide. + */ +export function pageTriggerFor(trigger: PageTrigger, bank: string, page: string): PageTrigger { + const cron = trigger.refresh_cron; + if (!cron || !isHashedCron(cron)) return trigger; + return { ...trigger, refresh_cron: expandCronHash(cron, `${bank}\u0000${page}`) }; +} + /** * How this project's pages keep themselves current. * diff --git a/skills/hindsight-docs/references/sdks/integrations/coding-agents.md b/skills/hindsight-docs/references/sdks/integrations/coding-agents.md index a24105dd97..41d947449a 100644 --- a/skills/hindsight-docs/references/sdks/integrations/coding-agents.md +++ b/skills/hindsight-docs/references/sdks/integrations/coding-agents.md @@ -496,7 +496,7 @@ hook by Codex...), so one shared config serves several agents side by side: | `autoReflect` | `true` | inject a one-time reflect synthesis on the session's **first prompt**. `false` = tool-only reflect: nothing is injected; the agent searches knowledge pages first and reflects only when they are too shallow | | `pageRefreshEveryTurns` | `10` | refetch the knowledge pages and re-inject the page roster + tool guide every N user turns | | `pageTriggerType` | `"auto-refresh"` | when NEW knowledge pages refresh, i.e. what keeping them current costs — `"auto-refresh"` after every consolidation that produced new material, `"cron"` on `pageTriggerCron` only, `"manual"` never on their own. Auto-refresh is the most current and the most expensive: one synthesis per page per consolidation. Maps to the page's `trigger.refresh_after_consolidation` in the Hindsight API (`true` for auto-refresh, `false` for manual) | -| `pageTriggerCron` | — | schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. `"0 3 * * *"`. Sets the page's `trigger.refresh_cron`, which the API treats as mutually exclusive with `refresh_after_consolidation`; a scheduled refresh is skipped when nothing changed | +| `pageTriggerCron` | — | schedule for `pageTriggerType: "cron"` — UTC, standard 5-field cron, e.g. `"0 3 * * *"`. Sets the page's `trigger.refresh_cron`, which the API treats as mutually exclusive with `refresh_after_consolidation`; a scheduled refresh is skipped when nothing changed. Write a field as `H` to give each page its own value there — see **Spreading refreshes with `H`** below | | `autoSeed` | `true` | SessionStart: auto-seed a cold repo's bank from git history | | `seedLimit` | `300` | auto-seed: most-recent-N-commits cap | | `codebaseSurvey` | `true` | SessionStart: headless survey of a cold repo's structure, run under the current harness's own CLI (claude/codex/antigravity/opencode), falling back to any available agent | @@ -516,6 +516,30 @@ belongs to the server: Hindsight creates a knowledge page with a delta refresh ( page instead of rebuilding it) that doesn't reflect over sibling pages, and these settings merge over those defaults rather than replacing them. +### Spreading refreshes with `H` + +One `pageTriggerCron` is shared by every page in every repo you point this plugin at. So a literal +`"0 3 * * *"` does not schedule _a_ refresh at 03:00 — it schedules **all** of them at 03:00, five +pages per bank, on the same worker pool that serves retain. A session ingesting at 03:0x queues +behind the pile, and moving the hour just moves the pile. + +Write a field as `H` and it is replaced, per page, by a value hashed from the bank id and the page +name. Each page gets its own slot, the same slot on every run: + +| `pageTriggerCron` | what each page gets | +| ------------------ | ------------------------------------------------------ | +| `"H H * * *"` | once a day, at its own minute and hour | +| `"H * * * *"` | once an hour, at its own minute | +| `"H 3 * * *"` | daily at 03:MM — spread inside the hour you chose | +| `"H H(0-5) * * *"` | daily, spread across 00:00–05:59 only | +| `"0 3 * * *"` | no `H`, no hashing — exactly what it says, all at once | + +`H` is [Jenkins' syntax](https://www.jenkins.io/doc/book/pipeline/syntax/#cron-syntax) for the same +problem. It never reaches the API: the plugin resolves it to an ordinary cron expression +(`"41 17 * * *"`) when it creates the page, so the schedule you see in the control plane is a plain +one you can edit. Hashing spreads pages out, it does not partition them — two pages can still land +on the same minute, just not all of them. + **These settings apply to pages created from here on.** Changing them does not migrate the pages a repo already has: a page keeps the trigger it was created with, so a bank seeded before you set `"manual"` keeps refreshing on every consolidation. To move an existing page, change its trigger