docs: clarify propagate_attributes scope, trace.list lightweight views, and ingestion lag#3316
Conversation
…s, and ingestion lag Close three documentation gaps that repeatedly cause debugging sessions: - Sessions/Users/Instrumentation: document exact propagate_attributes scope (forward-only: currently active observation + observations started while the context is active; detached roots stay unstamped) and where the values land (trace-level fields via api.trace.get; observation objects do not expose session_id/user_id). - Query via SDKs: document that trace.list returns lightweight views with observation/score IDs only, with a list-then-get pagination example for usage/cost aggregation. - Query via SDKs + Scores: document async ingestion (flush guarantees delivery, not read visibility; typically 15-30s, no SLA) and add a bounded retry-with-backoff pattern for get-after-write (404 NotFoundError). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dosubot
Bot
added
the
size:L
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
@claude review |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| langfuse.flush() # ensure delivery before polling | ||
| trace = get_trace_with_retry(trace_id, timeout=60) |
There was a problem hiding this comment.
trace_id undefined in usage example
The concluding call-site snippet uses trace_id as if it were already in scope, but the variable is never assigned anywhere in the snippet. Readers who copy these two lines verbatim will get a NameError at runtime. A short stub assignment or a # replace with your trace ID comment before the call would make the snippet self-contained.
Prompt To Fix With AI
This is a comment left during a code review.
Path: content/docs/api-and-data-platform/features/query-via-sdk.mdx
Line: 286-287
Comment:
**`trace_id` undefined in usage example**
The concluding call-site snippet uses `trace_id` as if it were already in scope, but the variable is never assigned anywhere in the snippet. Readers who copy these two lines verbatim will get a `NameError` at runtime. A short stub assignment or a `# replace with your trace ID` comment before the call would make the snippet self-contained.
How can I resolve this? If you propose a fix, please make it concise.| ### Traces: `list` returns lightweight views [#traces-list-vs-get] | ||
|
|
||
| `langfuse.api.trace.list(...)` returns **lightweight trace views**: the `observations` and `scores` fields contain only **IDs (strings)**, not full objects. Trace-level aggregates (`total_cost`, `latency`) are included, but per-observation usage and cost details are not. To get full observation objects, call `langfuse.api.trace.get(trace_id)` per trace (or prefer `observations.get_many(trace_id=...)` above). | ||
|
|
||
| List-then-get pagination example for usage/cost aggregation: | ||
|
|
||
| ```python | ||
| from collections import defaultdict | ||
| from datetime import datetime, timedelta, timezone | ||
|
|
||
| usage_by_model = defaultdict(int) | ||
| from_timestamp = datetime.now(timezone.utc) - timedelta(days=1) | ||
| page = 1 | ||
|
|
||
| while True: | ||
| traces = langfuse.api.trace.list( | ||
| from_timestamp=from_timestamp, | ||
| page=page, | ||
| limit=50, | ||
| ) | ||
| for trace_view in traces.data: | ||
| # trace_view.observations is a list of ID strings only | ||
| full_trace = langfuse.api.trace.get(trace_view.id) | ||
| for obs in full_trace.observations: # full objects incl. usage/cost | ||
| if obs.usage_details: | ||
| model = obs.model or "unknown" | ||
| usage_by_model[model] += obs.usage_details.get("total", 0) | ||
| if page >= traces.meta.total_pages: | ||
| break | ||
| page += 1 | ||
| ``` | ||
|
|
||
| If you only need trace-level totals, `trace.list` alone is enough (`trace_view.total_cost`, `trace_view.latency`) — no per-trace `get` required. For large-scale aggregation, prefer the [Metrics API v2](/docs/metrics/features/metrics-api#v2) over paginating row-level data. |
There was a problem hiding this comment.
The new "Traces: list returns lightweight views" section — including the pagination example and the note about total_cost/latency versus per-observation fields — lives entirely inside the Python SDK tab. The JS/TS SDK tab does not mention that trace.list returns ID-only observations/scores fields, even though the same REST endpoint underlies both clients. Users working with @langfuse/client who hit this behaviour have no documentation pointer to guide them.
Prompt To Fix With AI
This is a comment left during a code review.
Path: content/docs/api-and-data-platform/features/query-via-sdk.mdx
Line: 75-107
Comment:
**Lightweight-view behaviour only documented for Python tab**
The new "Traces: `list` returns lightweight views" section — including the pagination example and the note about `total_cost`/`latency` versus per-observation fields — lives entirely inside the Python SDK tab. The JS/TS SDK tab does not mention that `trace.list` returns ID-only `observations`/`scores` fields, even though the same REST endpoint underlies both clients. Users working with `@langfuse/client` who hit this behaviour have no documentation pointer to guide them.
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
|
||
| - **Covered**: the observation that is currently active when you enter the propagation context, and every observation started while the context is active (any nesting depth, any type). | ||
| - **Not covered**: observations started before entering the context. In the Python SDK, a detached root created with `start_observation()` (which does not become the active span) stays unstamped if you enter `propagate_attributes` afterwards — start it inside the context, or use `start_as_current_observation` for the root and propagate within it. | ||
|
|
||
| Where the value lands: each covered observation carries `session.id` in the exported OTel span data, and after ingestion `sessionId` surfaces as a **trace-level field**. When verifying programmatically, read it from the trace (`langfuse.api.trace.get(trace_id).session_id`) — observation objects returned by the public API do not expose a `session_id` field, so asserting on fetched child observations will always fail. |
There was a problem hiding this comment.
🔴 The new callouts in sessions.mdx, users.mdx, and instrumentation.mdx claim observation objects from "the public API" never expose sessionId/userId, but this contradicts observations-api.mdx: the v2 Observations API's basic field group — returned by default and requested by the observations.get_many(..., fields="core,basic,usage") call recommended one paragraph above — explicitly returns userId and sessionId per observation (its sample response shows "sessionId": "support-chat-session"). The claim is only true for the legacy trace.get(trace_id).observations nested view and should be scoped accordingly in all three files rather than stated as a blanket property of "the public API."
Extended reasoning...
The bug: All three edited pages add the same unqualified claim — e.g. sessions.mdx: observation objects returned by the public API do not expose a session_id field, so asserting on fetched child observations will always fail. The equivalent sentence appears in users.mdx (user_id) and instrumentation.mdx (these fields = both). As written, this reads as a blanket statement about every public-API path for fetching observations.
Why it's wrong: content/docs/api-and-data-platform/features/observations-api.mdx (unchanged by this PR, so it's the existing source of truth) documents the v2 Observations API's field-group system. Its field-group table lists the basic group as including userId, sessionId per observation, and states that core + basic are returned by default if no fields param is given. The sample JSON response in that same doc shows a populated, per-observation "sessionId": "support-chat-session" (and "userId": ""). So the v2 endpoint plainly does expose these fields on observation objects.
The triggering code path: This isn't a hypothetical edge case — it's the exact path the docs themselves recommend. One paragraph above the disputed callout, query-via-sdk.mdx (edited by this same PR) shows:
observations = langfuse.api.observations.get_many(
trace_id="abcdef1234",
type="GENERATION",
limit=100,
fields="core,basic,usage"
)That fields="core,basic,usage" call requests exactly the basic group that carries userId/sessionId. A reader who follows this recommendation, then reads the new callout a page later, is told they "will always fail" to find session_id/user_id on the very objects that just returned it.
Step-by-step proof:
- Reader calls
langfuse.api.observations.get_many(trace_id=..., fields="core,basic,usage")as instructed in query-via-sdk.mdx. - Per observations-api.mdx's field-group table, the
basicgroup returnsuserIdandsessionIdon each observation in the response. - Per observations-api.mdx's sample response, a real observation payload contains
"userId": ""and"sessionId": "support-chat-session"— populated values, not nulls/omissions. - Reader then goes to sessions.mdx (or users.mdx / instrumentation.mdx) and reads: "observation objects returned by the public API do not expose a
session_idfield, so asserting on fetched child observations will always fail." - This directly contradicts what the reader just observed in step 3 — the field is right there, non-empty, on the response object.
Why nothing else catches this: observations-api.mdx wasn't touched by this PR, so there's no cross-reference or lint that would flag the new prose in sessions.mdx/users.mdx/instrumentation.mdx as inconsistent with it. The claim is likely true for the older, distinct trace.get(trace_id).observations nested view (a different, legacy object shape), which is presumably what the PR author had in mind — but the text says "the public API" without that qualification, so it reads as universal.
Impact: Actively wrong debugging guidance. A user trying to verify that sessionId/userId propagation worked, following the docs' own recommended v2 endpoint, would be told by this new text that verification is impossible on observations and to only trust the trace-level field — when in fact the observation-level field is directly readable and would have caught propagation bugs the callout is trying to help with.
Fix: In each of the three locations, scope the claim to the specific object it's actually true for — e.g. "the legacy trace.get(trace_id).observations nested objects do not expose session_id/user_id" — rather than "observation objects returned by the public API." Alternatively, drop the blanket claim and simply recommend verifying via trace.get(trace_id).session_id as the more convenient/canonical check, without asserting the observation-level field categorically doesn't exist.
Addressing the one refutation on file: A refutation for the instrumentation.mdx instance (bug_005) argued it's a duplicate of the sessions.mdx instance (bug_001) and should be dropped as redundant. That refutation is about the bug tracker's dedup mechanics, not about whether the underlying claim is correct — it doesn't dispute the factual error, it just argues one location's report is redundant with another's. Since this synthesized report (merged_bug_001) explicitly covers all three occurrences (sessions.mdx ~line 198, users.mdx ~line 187, instrumentation.mdx ~line 541) as a single finding, that concern is resolved: all three locations are called out here rather than filed as separate near-duplicate reports.
| ### Traces: `list` returns lightweight views [#traces-list-vs-get] | ||
|
|
||
| `langfuse.api.trace.list(...)` returns **lightweight trace views**: the `observations` and `scores` fields contain only **IDs (strings)**, not full objects. Trace-level aggregates (`total_cost`, `latency`) are included, but per-observation usage and cost details are not. To get full observation objects, call `langfuse.api.trace.get(trace_id)` per trace (or prefer `observations.get_many(trace_id=...)` above). | ||
|
|
||
| List-then-get pagination example for usage/cost aggregation: |
There was a problem hiding this comment.
🟡 The new 'Traces: list returns lightweight views' section (documenting that trace.list's observations/scores fields are IDs-only, plus a list-then-get pagination example) was added only to the Python SDK tab in query-via-sdk.mdx; the JS/TS SDK tab jumps straight from Observations to Metrics with no equivalent guidance, even though this is a property of the shared Public API that applies identically to langfuse.api.trace.list in @langfuse/client. Consider mirroring the section (with a TS list-then-get example) into the JS/TS tab so those readers get the same warning.
Extended reasoning...
The bug: this PR adds a new ### Traces: list returns lightweight views [#traces-list-vs-get] section to content/docs/api-and-data-platform/features/query-via-sdk.mdx, but it is inserted only inside <Tab title="Python SDK">. The parallel <Tab title="JS/TS SDK"> block goes directly from ### Observations to ### Metrics (then ### Other resources) with no equivalent section and no TS list-then-get example.
Why this matters: the behavior being documented — that trace.list returns lightweight trace views where observations/scores are ID strings rather than full objects, while trace-level aggregates (total_cost, latency) are still included — is a property of the shared Public API. The api namespace on both SDKs is generated from the same OpenAPI spec, so langfuse.api.trace.list(...) in @langfuse/client exhibits the exact same lightweight-view behavior as langfuse.api.trace.list(...) in the Python SDK. There's nothing Python-specific about the gotcha.
Proof of the gap: walk the JS/TS tab as a reader would. Starting at ### Observations (JS/TS), the next heading is ### Metrics, then ### Other resources. A JS/TS developer who wants to aggregate per-observation usage/cost by paginating langfuse.api.trace.list(...) gets no warning that traces.data[i].observations is a list of ID strings, not objects with usageDetails/cost. They would write code assuming full objects, get IDs instead, and hit exactly the confusing failure mode this PR's own description says it set out to close ("agent-behavior testing showed cause real debugging sessions").
Why existing code doesn't prevent it: this is docs-only, so there's no compiler/lint check enforcing tab parity. The PR's own precedent shows parity was achievable and intended elsewhere — the new 'Handling ingestion lag' section further down the same file explicitly provides both a Python and a JS/TS tab (with a TS getTraceWithRetry example paralleling the Python get_trace_with_retry). That makes the omission in the trace.list section read as an oversight rather than deliberate scoping to Python only.
Impact: purely a documentation completeness gap — nothing renders incorrectly, no build/link check fails, and the Python content that does exist is accurate. But it undercuts the stated goal of the PR for exactly the audience (JS/TS SDK users) who would otherwise hit the same footgun.
Suggested fix: add a mirrored ### Traces: list returns lightweight views section to the JS/TS tab, with a langfuse.api.trace.list-based list-then-get TS example analogous to the Python one (fetching traces.data, noting observations/scores are ID arrays, and calling langfuse.api.trace.get(traceId) for full objects), plus the same pointer to observations.getMany/Metrics API v2 for scale.
| ## Scope and ordering of `sessionId` propagation [#scope-and-ordering] | ||
|
|
||
| Propagation is forward-only and follows the active execution context: | ||
|
|
||
| - **Covered**: the observation that is currently active when you enter the propagation context, and every observation started while the context is active (any nesting depth, any type). | ||
| - **Not covered**: observations started before entering the context. In the Python SDK, a detached root created with `start_observation()` (which does not become the active span) stays unstamped if you enter `propagate_attributes` afterwards — start it inside the context, or use `start_as_current_observation` for the root and propagate within it. | ||
|
|
||
| Where the value lands: each covered observation carries `session.id` in the exported OTel span data, and after ingestion `sessionId` surfaces as a **trace-level field**. When verifying programmatically, read it from the trace (`langfuse.api.trace.get(trace_id).session_id`) — observation objects returned by the public API do not expose a `session_id` field, so asserting on fetched child observations will always fail. |
There was a problem hiding this comment.
🟡 The new 'Scope and ordering' sections in sessions.mdx and users.mdx are near-verbatim duplicates (only sessionId/userId swapped), and instrumentation.mdx adds a third paragraph restating the same rules — duplicating the exact pattern that PropagationRestrictionsCallout.tsx was built to avoid across these same three pages. Consider consolidating into that shared component (e.g. via a scope/ordering prop) so future corrections to this behavior only need to be made once.
Extended reasoning...
The PR introduces a new "Scope and ordering" section on sessions.mdx (lines 191-198) and an essentially identical section on users.mdx (lines 180-187). Comparing the two: the "Covered"/"Not covered" bullets are 100% identical text, and the "Where the value lands" paragraph differs only by swapping sessionId/userId, session.id/user.id, and session_id/user_id. A third, more condensed paragraph restating the same claims (forward-only propagation, currently-active + started-while-active coverage, detached start_observation() roots staying unstamped, trace-level field, observations not exposing the field) is also added inline in instrumentation.mdx (~line 541).
This is exactly the situation the repo already solved once: components/PropagationRestrictionsCallout.tsx exists specifically to avoid duplicating propagation-restriction prose across these same three files (sessions.mdx, users.mdx, instrumentation.mdx all import and render it today, parameterized only by an attributes array). The new content follows the pre-existing hand-duplication pattern that the callout component was introduced to eliminate, instead of extending that same component (e.g., adding a sessionId/userId-aware scope prop, or a small sibling shared component) to cover the "scope and ordering" rule as well.
CLAUDE.md's review guidelines explicitly call this out: "If blocks of text or code are largely repeated on multiple documentation pages, suggest consolidating them in components-mdx to improve maintainability and consistency." This is a repo-instruction-driven check, not a subjective style opinion, so it is being surfaced even though it doesn't break anything at merge time.
Concrete duplication proof:
sessions.mdx: "Covered: the observation that is currently active when you enter the propagation context, and every observation started while the context is active (any nesting depth, any type)." / "Not covered: observations started before entering the context. In the Python SDK, a detached root created withstart_observation()(which does not become the active span) stays unstamped if you enterpropagate_attributesafterwards — start it inside the context, or usestart_as_current_observationfor the root and propagate within it."users.mdx: the exact same two sentences, verbatim, character-for-character.- Only the header (
sessionIdvsuserId) and the final "Where the value lands" sentence (session.id/session_idvsuser.id/user_id) differ between the two pages. instrumentation.mdxrestates the same four claims (forward-only, active+later coverage, detached-root exception, trace-level field/API gap) in one merged paragraph.
Impact: nothing breaks today — the pages render correctly and the content is accurate. But because the three copies are hand-maintained prose rather than one parameterized source, any future correction to this behavior (e.g., if SDK semantics change, or to fix the trace/observation field wording) now requires editing three separate places, and the copies can silently drift out of sync — which is a real maintainability cost given how close in wording (but not byte-identical) they already are.
Suggested fix: extend PropagationRestrictionsCallout.tsx (or add a small sibling component in components/ or components-mdx/) to accept the entity name (sessionId/userId) and render the scope/ordering rule once, then reference/import it from all three pages (and reuse the same source for the instrumentation.mdx summary paragraph, or link out to the Sessions/Users sections as the PR already does for further detail).
1 participant
What changed
Closes three documentation gaps that agent-behavior testing showed cause real debugging sessions. All claims were verified against the Python SDK source and unit tests (companion docstring PR in langfuse-python to follow).
1.
propagate_attributesscope semanticsstart_observation()root started before entering the context stays unstamped. Documents where the values land:sessionId/userIdare trace-level fields — verify viaapi.trace.get(trace_id), since observation objects returned by the public API don't expose these fields (asserting on fetched children always fails).PropagationRestrictionsCallout: same scope rule stated where all propagation docs converge.2.
trace.listreturns lightweight viewstrace.listitems carry observation/score IDs (strings), not full objects, while trace-level aggregates (total_cost,latency) are included. Adds a list-then-get pagination example for per-observation usage/cost aggregation, with pointers toobservations.get_manyand Metrics API v2 for scale.3. Ingestion lag (get-after-write)
flush()guarantees delivery not read visibility, no fixed SLA, can lag longer under parallel load) plus a new "Handling ingestion lag" section with bounded retry-with-backoff snippets (Pythonlangfuse.api.NotFoundError+ JS/TS), including the trap that a successfultrace.getdoesn't imply all observations have arrived yet.Reviewer notes
langfuse-python: the span processor stamps propagated attributes on every span started inside the context (not root-only); the confusion in the field comes from the API surface (observations carry no session/user fields) and detached roots.langfuse.api.NotFoundError— there is noLangfuseNotFoundErrorin the Python SDK.pnpm run format; content-only changes plus one shared component tweak, no build config touched.🤖 Generated with Claude Code
Greptile Summary
This PR closes three documentation gaps in Langfuse's observability and SDK query docs: it clarifies the forward-only scope of
propagate_attributes, explains thattrace.listreturns lightweight views (IDs, not full objects), and adds a bounded retry-with-backoff pattern for ingestion lag.PropagationRestrictionsCallout.tsx,sessions.mdx,users.mdx, andinstrumentation.mdxnow consistently document that only the currently active observation and observations started while the context is active receive propagated attributes, plus the trace-level-field caveat forsessionId/userIdverification.trace.listlightweight views: New Python-only section documenting ID-onlyobservations/scoresfields and a list-then-get pagination example; the JS/TS tab lacks an equivalent note.#ingestion-lagsection with bounded exponential-backoff retry snippets for both Python (langfuse.api.NotFoundError) and JS/TS, plus a short callout inscores-via-sdk.mdx.Confidence Score: 4/5
Documentation-only changes with no build config or code logic touched; safe to merge with minor polish.
The propagation-scope additions and the ingestion-lag retry pattern are accurate and internally consistent across the six changed files. The only gaps found are:
trace_idappears undefined in the Python retry usage snippet (a copy-paste hazard), and thetrace.listlightweight-view behaviour is documented for Python only while the JS/TS tab is silent on it.content/docs/api-and-data-platform/features/query-via-sdk.mdx — both the undefined
trace_idin the usage example and the absent JS/TS lightweight-view note live here.Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant App participant SDK participant API participant Store App->>SDK: trace/observation created SDK->>API: flush() — delivers payload API-->>SDK: 202 Accepted Note over API,Store: async ingestion pipeline (15–30s typical) API->>Store: write to storage App->>SDK: trace.get(trace_id) [immediate] SDK->>API: "GET /trace/{id}" API-->>SDK: 404 NotFoundError Note over App,SDK: retry with exponential backoff App->>SDK: trace.get(trace_id) [after delay] SDK->>API: "GET /trace/{id}" API-->>SDK: 200 trace (may still be missing observations) Note over App: check observation completeness separately%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant App participant SDK participant API participant Store App->>SDK: trace/observation created SDK->>API: flush() — delivers payload API-->>SDK: 202 Accepted Note over API,Store: async ingestion pipeline (15–30s typical) API->>Store: write to storage App->>SDK: trace.get(trace_id) [immediate] SDK->>API: "GET /trace/{id}" API-->>SDK: 404 NotFoundError Note over App,SDK: retry with exponential backoff App->>SDK: trace.get(trace_id) [after delay] SDK->>API: "GET /trace/{id}" API-->>SDK: 200 trace (may still be missing observations) Note over App: check observation completeness separatelyPrompt To Fix All With AI
Reviews (1): Last reviewed commit: "docs: clarify propagate_attributes scope..." | Re-trigger Greptile