feat: AI summarize with extensible subjects, trace context, and security hardening by alex-fedotyev · Pull Request #2108 · hyperdxio/hyperdx

alex-fedotyev · 2026-04-11T02:49:52Z

Summary

Replaces the April Fools Easter egg AI Summarize feature with a real LLM-powered summarization system designed to scale to new subjects (alerts, incidents, metrics) without schema churn.

What's in this PR

Backend (`packages/api`)

POST /ai/summarize — subject-registry endpoint accepting { kind, content, tone?, messages? }
- kind is an enum — event | pattern | alert. New kinds are added by registering a system prompt in aiSummarize.ts.
- messages is an optional conversation history (capped length + size). Not wired to UI yet — ships with the shape fixed so future follow-up-question flows don't need an API change.
- System prompts warn the model that severity labels can be misleading and instruct it to cross-check against body/attributes.
Prompt injection defense — user content is passed through redactSecrets() (scrubs password=, token=, Bearer ..., JWTs) and wrapped in <data>...</data> tags. The system prompt explicitly tells the model to treat anything inside <data> as data, not instructions.
Rate limiting — 30 requests/min/user (in-memory; middleware/rateLimit.ts) to protect the shared LLM API budget from runaway callers.
.env.development.local override — new scripts/env-local-preload.js so API dev scripts respect a gitignored override file the way Next.js does.

Frontend (`packages/app`)

Subject registry — aiSummarize/subjects.ts + per-subject formatters (eventSubject.ts, patternSubject.ts). Adding a new surface (e.g. alerts) = define a subject + render <AISummaryPanel> bound to useAISummarizeState({ subject, input }).
useAISummarizeState hook — consolidates state machine previously duplicated across both button components. Handles real-AI vs easter-egg branch, tone picker, dismiss, regenerate, input-change abort, and lazy trace-context fetch.
Shared classifiers (classifiers.ts) — isErrorEvent, isWarnEvent, normalizeSeverity that cross-check multiple signals (severity, status code, HTTP code, exception, body regex). Used to decide what to prioritize in trace context. Raw severity/body always go to the model unchanged so it can draw its own conclusion.
Trace context enrichment — when summarizing a span in a trace, the client builds a compact context (total spans, error count, span groups with count/errors/sum/p50 durations, up to 10 error spans with exception details) and appends to the prompt. Capped at 4KB. Fetched lazily — only when user clicks Summarize, not on panel open.
Markdown output — AI replies render via react-markdown with **bold** highlights for key details and `code` for values. Easter egg mode remains plain italic text.
Tone/style picker — ?smart=true URL flag enables a compact tone selector (Detective Noir / Nature Documentary / Shakespearean Drama) that persists in localStorage and auto-regenerates on change. Tone values are enum-validated server-side; no freeform prompt injection.

Extensibility notes

The architecture is designed so the following changes don't require rethinking shared code:

Adding a new subject (e.g. alerts): define ALERT_SUBJECT with a formatAlertContent, register a prompt in the backend's SUBJECT_PROMPTS map, render <AISummaryPanel> bound to the hook. No changes to the hook or panel.
Follow-up questions / conversation: API already accepts messages: ConversationMessage[]. Future UI sends user's follow-up plus prior turns; no server change needed.
New tone / redaction patterns: add entries to the respective registries — no refactor.

Security

Enum-only schema on the wire; freeform prompt text cannot come from the client.
<data> tag delimiters + explicit instruction to ignore embedded instructions.
Secret redaction before content hits the LLM (password, token, Bearer, JWT).
Per-user rate limit (30/min).
db.statement is NOT included in trace context — redacted even post-regex, and span body usually has enough signal.

Test coverage

55 client tests covering: classifiers, trace context builder (truncation, error grouping, non-string attributes), format functions, button components, the shared hook, and the panel.
20 API tests covering: schema validation, per-kind prompt selection, redaction, <data> wrapping, conversation/messages mode, and error handling.
All integration tests pass (make dev-int FILE=aiSummarize).

How to test locally

Set ANTHROPIC_API_KEY in packages/api/.env.development.local (or AI_API_KEY + AI_PROVIDER=anthropic).
yarn dev
Click a log row → open side panel → click Summarize. Output renders as formatted markdown.
Add ?smart=true to the URL to see the tone picker; changing it regenerates.
Without an API key, the Easter egg fallback still shows themed summaries (Apr 1–30 window).

References

Linear: HDX-3992

changeset-bot · 2026-04-11T02:49:59Z

⚠️ No Changeset found

Latest commit: bcfae6a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

vercel · 2026-04-11T02:49:59Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ready	Preview, Comment	Apr 21, 2026 3:16am

github-actions · 2026-04-11T02:54:49Z

E2E Test Results

✅ All tests passed • 145 passed • 3 skipped • 1087s

Status	Count
✅ Passed	145
❌ Failed	0
⚠️ Flaky	5
⏭️ Skipped	3

Tests ran across 4 shards in parallel.

View full report →

- Add POST /ai/summarize endpoint that uses the configured LLM to generate concise, actionable summaries for individual events and patterns - Add useAISummarize hook in the frontend to call the new endpoint - Update AISummarizeButton and AISummarizePatternButton to use real AI when aiAssistantEnabled is true, falling back to the Easter egg themes when no AI provider is configured - Update AISummaryPanel to support both real AI and Easter egg display modes (no info popover / dismiss for real AI, no italic theme label) HDX-3992 Co-authored-by: Alex Fedotyev <alex-fedotyev@users.noreply.github.com>

- Export formatEventContent/formatPatternContent for testability - Always show 'Don't show' link (both real AI and easter egg modes) - Real AI: visible always (not gated by easter egg dates) - Easter egg: still uses dismissEasterEgg() for localStorage persistence - AISummaryPanel: show 'Don't show' link in collapsed state for easy dismissal, remove April Fools popover in real AI mode Tests (42 new): - formatEventContent: 9 tests covering all field extraction paths - formatPatternContent: 3 tests for pattern/samples formatting - AISummarizeButton: 10 tests (real AI, fake AI, dismiss, error, toggle) - AISummarizePatternButton: 5 tests (visibility, AI/fallback, dismiss) - AISummaryPanel: 8 tests (dismiss, error, theme label, real vs fake) - POST /ai/summarize: 7 tests (validation, prompts, error handling) Co-authored-by: Alex Fedotyev <alex-fedotyev@users.noreply.github.com>

The Pattern type requires an 'id' field. Added it to all test fixtures to fix the CI TypeScript check. Co-authored-by: Alex Fedotyev <alex-fedotyev@users.noreply.github.com>

SampleLog requires __hdx_timestamp alongside __hdx_pattern_field. Added the missing field and used proper Pattern typing throughout. Co-authored-by: Alex Fedotyev <alex-fedotyev@users.noreply.github.com>

…own output - Enrich trace summaries with full trace context (span groups with count/sum/p50 durations, error spans with details) - Add tone/style picker (noir, attenborough, shakespeare) gated behind ?smart=true — persisted in localStorage, auto-regenerates on change - Render AI output as markdown with highlighted key details - Improve prompts: terse for healthy events, focused for errors - Enrich pattern summaries with sample attributes - Add env-local-preload.js so .env.development.local overrides work - Fix react-markdown ESM mock for Jest

- Make trace span fetch lazy (only when user clicks Summarize) - Filter __hdx_ internal keys from pattern sample attributes - Add bounds clamp to percentile calculation - Cap trace context output at 4KB to stay within content limits - Fix stale test assertions for rewritten prompts - Remove stale comment in AISummaryPanel

useEventsAroundFocus runs getConfig in a useMemo even when enabled=false. When the real trace source hasn't loaded yet, passing undefined crashes on source.kind access. Provide a minimal stub TTraceSource so the useMemo produces a harmless no-op config without throwing.

github-actions · 2026-04-21T02:11:23Z

Knip - Unused Code Analysis

🔴 24 issues found

Unused exports (12)

packages/app/src/components/aiSummarize/index.ts:dismissEasterEgg
packages/app/src/components/aiSummarize/index.ts:getSavedTone
packages/app/src/components/aiSummarize/index.ts:isEasterEggVisible
packages/app/src/components/aiSummarize/index.ts:isSmartMode
packages/app/src/components/aiSummarize/index.ts:saveTone
packages/app/src/components/aiSummarize/index.ts:TONE_OPTIONS
packages/app/src/components/aiSummarize/index.ts:buildTraceContext
packages/app/src/components/aiSummarize/index.ts:isErrorEvent
packages/app/src/components/aiSummarize/index.ts:isWarnEvent
packages/app/src/components/aiSummarize/index.ts:normalizeSeverity
packages/app/src/components/aiSummarize/index.ts:formatEventContent
packages/app/src/components/aiSummarize/index.ts:formatPatternContent

Unused exported types (12)

packages/app/src/components/aiSummarize/index.ts:AISummarizeTone
packages/app/src/components/aiSummarize/index.ts:Theme
packages/app/src/components/aiSummarize/index.ts:TraceSpan
packages/app/src/components/aiSummarize/index.ts:ClassifierSignals
packages/app/src/components/aiSummarize/index.ts:SummarySubject
packages/app/src/components/aiSummarize/index.ts:SummarySubjectKind
packages/app/src/components/aiSummarize/index.ts:UseAISummarizeStateResult
packages/api/src/routers/api/aiSummarize.ts:SummarizeBody
packages/app/src/hooks/ai.ts:SummarizeKind
packages/app/src/hooks/ai.ts:AISummarizeTone
packages/app/src/hooks/ai.ts:ConversationMessage
packages/app/src/components/aiSummarize/useAISummarizeState.ts:TraceSpan

Knip finds unused files, dependencies, and exports in your codebase.
Run yarn knip locally to see full details.

…marize Architectural refactor so adding new subjects (alerts, metric anomalies) or follow-up conversation flows doesn't require rethinking shared code. Backend - Subject-keyed prompt registry in routers/api/aiSummarize.ts - API schema: { kind, content, tone?, messages? } — messages shape fixed now so future conversation UI doesn't need a schema change - Per-user in-memory rate limiter (30/min) via new middleware/rateLimit.ts - Prompt injection defense: redactSecrets() scrubs password/token/Bearer/JWT patterns, content wrapped in <data>...</data> tags, system prompt tells the model to treat anything inside as data not instructions - Prompts explicitly note severity labels can be misleading Frontend - Shared useAISummarizeState hook deduplicates state from both buttons - Subject definitions (EVENT_SUBJECT, PATTERN_SUBJECT) in dedicated files - Shared classifiers (isErrorEvent, isWarnEvent, normalizeSeverity) that cross-check severity + statusCode + HTTP + body regex instead of trusting severity alone - traceContext.ts uses classifiers, handles non-string SpanAttributes safely, drops db.statement to avoid credential leaks - Button components trimmed from ~250 lines each to ~80 Tests - +45 new tests: classifiers, traceContext edge cases, API redaction, delimiter wrapping, alert kind, conversation/messages mode

- Add rate-limit middleware tests (6 cases: under/over limit, window reset, per-user isolation, per-limiter isolation, error payload shape) - Document messages[] trust boundary in aiSummarize schema — caller can claim any assistant role, acceptable for single-shot, requires server-side state when a follow-up UI is built - Extend secret redaction to JSON-shape ("key":"value") and HTTP-header shape (X-Api-Key: value) + tests - Add supportsTraceContext gate tests — pattern subject and event subjects without a traceId must never enable the span fetch - Dedupe coerce-attribute helpers into formatHelpers.ts; used by both subject formatters and traceContext - Simplify TraceAttributeValue to `unknown` (the other union members were absorbed anyway) - Comment why abort-reset uses queueMicrotask instead of setTimeout(0) (jest.useFakeTimers would otherwise freeze the reset)

alex-fedotyev · 2026-05-06T02:31:05Z

Superseding this draft. I'm decomposing HDX-3992 into a stacked PR series so each piece reviews on its own.

Stack base: #2188 (redactSecrets util), already open with all checks green.

Stack on top:

PR A: backend POST /ai/summarize for event and pattern kinds. Drops the custom rate-limit middleware in favor of express-rate-limit (already in packages/api). Drops the alert kind and messages array, no UI consumer for either, will add when the consumer ships.
PR B: frontend wiring (subject registry, useAISummarizeState hook, button components, plain-text rendering).
PR C: trace context enrichment (per-span aggregation, 4 KB cap, lazy-fetched on click).
PR D: tone picker + URL flag, closes HDX-3992.

Keeping this branch around for cherry-picks, but closing the PR so it doesn't sit in the queue.

…2188) ## Summary Adds a reusable best-effort secret redactor at `packages/api/src/utils/redactSecrets.ts`. Internal-only utility; no consumer in this PR. The next AI-summarize PR (HDX-3992 split) imports it; future LLM endpoints that ingest observability data should also. The file header codifies the design rule for HyperDX AI endpoints: > Any LLM input derived from observability data passes through `redactSecrets` before leaving the API process. User-authored prose (the chart-builder assistant where the user types their own question) does NOT, because redacting the user's own input would strip exactly what they meant to ask. ## Patterns covered | name | shape | |------|-------| | `pem` | `-----BEGIN ... PRIVATE KEY-----` blocks (RSA, EC, DSA, OPENSSH, PKCS#8) | | `basic-auth-url` | `https://user:pass@host` | | `key-value` | `password=...`, `api_key=...`, `token=...`, etc. | | `json-quoted` | `{"password":"..."}` and similar | | `http-header` | `X-Api-Key:`, `X-Auth-Token:`, `Api-Key:` | | `bearer` | `Authorization: Bearer ...` | | `basic` | `Authorization: Basic ...` | | `jwt` | `eyJ...` three-segment base64url | | `aws-access-key` | `AKIA[16]`, `ASIA[16]` | | `slack-token` | `xox[a-z]-...` | | `github-token` | `ghp_`, `gho_`, `ghu_`, `ghs_`, `ghr_` | ## Known gaps (deferred to follow-ups) - URL percent-encoded values (beyond what query-string parsing already catches) - Vendor-specific tokens not listed above (Stripe, Twilio, Datadog) - Generic high-entropy hex blobs (too many false positives without surrounding context) ## Why this is its own PR Splits cleanly from the larger AI summarize work (HDX-3992 / #2108). Lands as a small, isolated, test-heavy change so review is fast and the util is in place before downstream consumers arrive. ## Tests 38 cases in `packages/api/src/utils/__tests__/redactSecrets.test.ts` covering: each pattern with a positive case, a "looks similar but isn't" negative, and at least one multi-secret payload. Pattern-coverage assertion exposes the registry shape so future additions get a compile-time signal. ``` yarn jest src/utils/__tests__/redactSecrets.test.ts # Test Suites: 1 passed, 1 total # Tests: 38 passed, 38 total ``` All neighboring api utils tests still pass (8 suites, 61 tests). ## No user-facing change The util is internal API code with no production consumer in this PR. No changeset. ## References - Linear: HDX-3992 - Related (do-not-merge while we split): #2108

vercel Bot deployed to Preview April 11, 2026 02:51 View deployment

vercel Bot deployed to Preview April 11, 2026 02:56 View deployment

vercel Bot deployed to Preview April 11, 2026 03:03 View deployment

vercel Bot deployed to Preview April 15, 2026 03:06 View deployment

vercel Bot deployed to Preview April 15, 2026 03:23 View deployment

cursoragent and others added 4 commits April 20, 2026 19:01

fix: add required 'id' field to Pattern test fixtures

9289c3e

The Pattern type requires an 'id' field. Added it to all test fixtures to fix the CI TypeScript check. Co-authored-by: Alex Fedotyev <alex-fedotyev@users.noreply.github.com>

fix: use correct Pattern/SampleLog types in test fixtures

ff76d1a

SampleLog requires __hdx_timestamp alongside __hdx_pattern_field. Added the missing field and used proper Pattern typing throughout. Co-authored-by: Alex Fedotyev <alex-fedotyev@users.noreply.github.com>

vercel Bot deployed to Preview April 21, 2026 02:02 View deployment

alex-fedotyev added 3 commits April 20, 2026 19:02

alex-fedotyev force-pushed the cursor/ai-summarize-real-2405 branch from 351795a to 57086e8 Compare April 21, 2026 02:10

vercel Bot deployed to Preview April 21, 2026 02:13 View deployment

vercel Bot deployed to Preview April 21, 2026 03:01 View deployment

alex-fedotyev changed the title ~~feat: implement true AI callbacks to summarize log/trace/pattern~~ feat: AI summarize with extensible subjects, trace context, and security hardening Apr 21, 2026

vercel Bot deployed to Preview April 21, 2026 03:16 View deployment

alex-fedotyev mentioned this pull request May 4, 2026

feat(api): redactSecrets util for LLM input from observability data #2188

Merged

alex-fedotyev closed this May 6, 2026

alex-fedotyev mentioned this pull request May 6, 2026

feat(api): POST /ai/summarize endpoint (HDX-3992) #2206

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: AI summarize with extensible subjects, trace context, and security hardening#2108

feat: AI summarize with extensible subjects, trace context, and security hardening#2108
alex-fedotyev wants to merge 9 commits into
mainfrom
cursor/ai-summarize-real-2405

alex-fedotyev commented Apr 11, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Apr 11, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Apr 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 21, 2026 •

edited

Loading

Uh oh!

alex-fedotyev commented May 6, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

alex-fedotyev commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's in this PR

Backend (packages/api)

Frontend (packages/app)

Extensibility notes

Security

Test coverage

How to test locally

References

Uh oh!

changeset-bot Bot commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

vercel Bot commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Test Results

Uh oh!

github-actions Bot commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Knip - Unused Code Analysis

Unused exports (12)

Unused exported types (12)

Uh oh!

alex-fedotyev commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alex-fedotyev commented Apr 11, 2026 •

edited

Loading

Backend (`packages/api`)

Frontend (`packages/app`)

changeset-bot Bot commented Apr 11, 2026 •

edited

Loading

vercel Bot commented Apr 11, 2026 •

edited

Loading

github-actions Bot commented Apr 11, 2026 •

edited

Loading

github-actions Bot commented Apr 21, 2026 •

edited

Loading

alex-fedotyev commented May 6, 2026 •

edited

Loading