docs(self-hosting): add v3 to v4 upgrade guide draft#3246
Conversation
Adds the self-hosted Langfuse v3 to v4 migration guide, structured as three independent steps (infrastructure, server upgrade with write-mode choice, data-model migration) with backfill and retention-based rollover options. Inline TODOs mark items to verify against the codebase before publishing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
@claude review |
AkioNuernberger
left a comment
There was a problem hiding this comment.
Herculean effort and excited for this step. Looks great overall.
Few minor things:
- Clear deadlines to prevent Parkinson's law: Add info on how long we are planning to support dual writes and the legacy tables. Users need this information to plan accordingly or to convince people to invest time in the upgrade.
- Would add a sub-step in Step 3 Move to the new data model after 3.1 Upgrade your producers to compatible SDKs for completeness. Add: Upgrade your API consumers/exports to new APIs: Link to the APIs listed above.
- Might make sense to add that another steps to migrate the evals
- For the V4 events (observations) view, would it make sense to set the default view to root spans only and nudge the user in product to add select more observations if they want to deep dive? I feel like this would a user friendly way to not overwhelm the users with all the different observations. (not sure how this would behave in installations without a rootspan tho)
|
Also, please let us know before you are going to publish this → I would like to coordinate with @jannikmaierhoefer to send out communication to the enterprise customers. |
annabellscha
left a comment
There was a problem hiding this comment.
minor comments, never deployed or migrated langfuse, so hard to judge
Add /docs/api-migration with per-endpoint v3->v4 REST mapping (stable anchors for programmatic use) and a self-contained Legacy v3 API reference page. Move the legacy metrics v1 / daily metrics sections out of metrics-api.mdx, point feature pages at the new pages, convert the human-in-the-loop guide to v2 observations, repoint inbound links, changelog canonicals, and the daily-metrics redirect. Includes fixes from two adversarial review rounds and bot reviews: corrected legacy observation limits, events_only 404 claims scoped to the gated read endpoints (dataset-run reads excepted), parseIoAsJson deprecation, dataset-scoped experiments example. Merge after #3246 (self-hosted v3->v4 upgrade guide): the upgrade-guide links and the Cloud-only callout reconciliation depend on it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
As per DM, @marliessophie to add context on how to upgrade trace-level evals for self-hosters. |
There was a problem hiding this comment.
No specific comments, looks nice! One thing to keep in mind: these changes will probably interfere with @maxdeichmann's PR here and there: #3330
…-to-v4 # Conflicts: # content/docs/api-and-data-platform/features/export-to-blob-storage.mdx # content/docs/api-and-data-platform/features/observations-api.mdx # content/docs/evaluation/evaluation-methods/llm-as-a-judge.mdx # content/docs/metrics/features/metrics-api.mdx # content/docs/v4.mdx # content/faq/all/explore-observations-in-v4.mdx # content/integrations/analytics/mixpanel.mdx # content/integrations/analytics/posthog.mdx # content/integrations/native/opentelemetry/index.mdx
Revert branch changes to files that conflict with main or are rewritten by the incoming v4-release docs PR (#3330). The v3-to-v4 upgrade guide and its supporting self-hosting pages are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add the Langfuse v4 minimums/recommendations (ClickHouse >= 25.12, 26.4 recommended; PostgreSQL 16 recommended; Redis 7.2 recommended) to the infrastructure pages, linking to the v3-to-v4 upgrade guide. Also update one changelog card title from Fast Preview to Langfuse v4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the self-hosted Langfuse v3 to v4 migration guide, structured as three independent steps (infrastructure, server upgrade with write-mode choice, data-model migration) with backfill and retention-based rollover options. Inline TODOs mark items to verify against the codebase before publishing.
Greptile Summary
This PR introduces the self-hosted Langfuse v3-to-v4 migration guide and updates all existing docs to replace the "Fast Preview" label with "Langfuse v4," reflecting the feature's general availability for self-hosted deployments.
upgrade-v3-to-v4.mdx): documents a three-step, independent upgrade path \u2014 infrastructure (ClickHouse version gate), server upgrade with a choice betweenlegacy,dual, andevents_onlywrite modes, and a data-model cutover via automated backfill or retention-based rollover.Confidence Score: 3/5
The terminology sweep is safe; the new migration guide has two broken links that should be fixed before publishing.
The bulk of the PR is a mechanical rename across docs files — low risk and internally consistent. The new upgrade guide is detailed and well-structured, but the intro callout links to the wrong GitHub Discussion (12518 vs the correct 14157), and the Support section uses a relative /issues path that resolves to the docs host rather than GitHub. Both affect navigation-critical content aimed at operators performing an irreversible migration.
content/self-hosting/upgrade/upgrade-guides/upgrade-v3-to-v4.mdx — the intro callout GitHub Discussion link and the Support section /issues URL should both be corrected before publishing.
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A([Start: Existing Langfuse v3 deployment]) --> B[Step 1: Upgrade ClickHouse to 25.12+ / 26.4 recommended] B --> C[Step 2: Upgrade server to Langfuse v4] C --> D{Choose write mode} D -->|events_only default| E[events_only: Writes only to new events tables. Historic backfill starts automatically] D -->|dual| F[dual: Writes to old AND new tables. UI toggle and v2 APIs available] D -->|legacy| G[legacy: Full v3 behavior retained. No writes to new tables] G --> H[Switch to dual write when ready] H --> F E --> DONE([Migration complete]) F --> I[Step 3: Upgrade SDKs - Python v4 / JS-TS v5 / OTEL] I --> J{Historic data strategy} J -->|Option A: Backfill| K[Enable backfill. Newest-first, gradual. Requires 3x disk headroom] J -->|Option B: Retention rollover| L[Wait one full retention window while dual-writing] K --> M[Cut over to events_only. Remove migration env vars] L --> M M --> N[Optional: Truncate old tables to reclaim ClickHouse disk] N --> DONE%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% flowchart TD A([Start: Existing Langfuse v3 deployment]) --> B[Step 1: Upgrade ClickHouse to 25.12+ / 26.4 recommended] B --> C[Step 2: Upgrade server to Langfuse v4] C --> D{Choose write mode} D -->|events_only default| E[events_only: Writes only to new events tables. Historic backfill starts automatically] D -->|dual| F[dual: Writes to old AND new tables. UI toggle and v2 APIs available] D -->|legacy| G[legacy: Full v3 behavior retained. No writes to new tables] G --> H[Switch to dual write when ready] H --> F E --> DONE([Migration complete]) F --> I[Step 3: Upgrade SDKs - Python v4 / JS-TS v5 / OTEL] I --> J{Historic data strategy} J -->|Option A: Backfill| K[Enable backfill. Newest-first, gradual. Requires 3x disk headroom] J -->|Option B: Retention rollover| L[Wait one full retention window while dual-writing] K --> M[Cut over to events_only. Remove migration env vars] L --> M M --> N[Optional: Truncate old tables to reclaim ClickHouse disk] N --> DONEPrompt To Fix All With AI
Reviews (1): Last reviewed commit: "Merge branch 'main' into docs/upgrade-gu..." | Re-trigger Greptile