-
Notifications
You must be signed in to change notification settings - Fork 263
docs: clarify propagate_attributes scope, trace.list lightweight views, and ingestion lag #3316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,7 +18,7 @@ | |
|
|
||
| <Callout type="info"> | ||
|
|
||
| New data is typically available for querying within 15-30 seconds of ingestion, though processing times may vary at times. Please visit [status.langfuse.com](https://status.langfuse.com) if you encounter any issues. | ||
| Ingestion is **asynchronous**: SDK `flush()` only guarantees delivery to the API, not read visibility. New data is typically available for querying within 15-30 seconds of ingestion, though processing times may vary — under heavy or parallel load, visibility can lag longer (there is no fixed SLA). Reading a trace immediately after writing it may return a 404. See [Handling ingestion lag](#ingestion-lag) for the recommended retry pattern, and visit [status.langfuse.com](https://status.langfuse.com) if you encounter persistent issues. | ||
|
|
||
| </Callout> | ||
|
|
||
|
|
@@ -72,6 +72,40 @@ | |
|
|
||
| Use `trace_id` to retrieve the observations that belong to a single trace. Use `parent_observation_id` in the response to reconstruct an observation tree when needed. | ||
|
|
||
| ### Traces: `list` returns lightweight views [#traces-list-vs-get] | ||
|
|
||
| `langfuse.api.trace.list(...)` returns **lightweight trace views**: the `observations` and `scores` fields contain only **IDs (strings)**, not full objects. Trace-level aggregates (`total_cost`, `latency`) are included, but per-observation usage and cost details are not. To get full observation objects, call `langfuse.api.trace.get(trace_id)` per trace (or prefer `observations.get_many(trace_id=...)` above). | ||
|
|
||
| List-then-get pagination example for usage/cost aggregation: | ||
|
Check warning on line 79 in content/docs/api-and-data-platform/features/query-via-sdk.mdx
|
||
|
|
||
| ```python | ||
| from collections import defaultdict | ||
| from datetime import datetime, timedelta, timezone | ||
|
|
||
| usage_by_model = defaultdict(int) | ||
| from_timestamp = datetime.now(timezone.utc) - timedelta(days=1) | ||
| page = 1 | ||
|
|
||
| while True: | ||
| traces = langfuse.api.trace.list( | ||
| from_timestamp=from_timestamp, | ||
| page=page, | ||
| limit=50, | ||
| ) | ||
| for trace_view in traces.data: | ||
| # trace_view.observations is a list of ID strings only | ||
| full_trace = langfuse.api.trace.get(trace_view.id) | ||
| for obs in full_trace.observations: # full objects incl. usage/cost | ||
| if obs.usage_details: | ||
| model = obs.model or "unknown" | ||
| usage_by_model[model] += obs.usage_details.get("total", 0) | ||
| if page >= traces.meta.total_pages: | ||
| break | ||
| page += 1 | ||
| ``` | ||
|
|
||
| If you only need trace-level totals, `trace.list` alone is enough (`trace_view.total_cost`, `trace_view.latency`) — no per-trace `get` required. For large-scale aggregation, prefer the [Metrics API v2](/docs/metrics/features/metrics-api#v2) over paginating row-level data. | ||
|
Comment on lines
+75
to
+107
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The new "Traces: Prompt To Fix With AIThis is a comment left during a code review.
Path: content/docs/api-and-data-platform/features/query-via-sdk.mdx
Line: 75-107
Comment:
**Lightweight-view behaviour only documented for Python tab**
The new "Traces: `list` returns lightweight views" section — including the pagination example and the note about `total_cost`/`latency` versus per-observation fields — lives entirely inside the Python SDK tab. The JS/TS SDK tab does not mention that `trace.list` returns ID-only `observations`/`scores` fields, even though the same REST endpoint underlies both clients. Users working with `@langfuse/client` who hit this behaviour have no documentation pointer to guide them.
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! |
||
|
|
||
| ### Metrics | ||
|
|
||
| For the full query schema, supported dimensions, filters, and examples, see the [Metrics API v2 documentation](/docs/metrics/features/metrics-api#v2). | ||
|
|
@@ -217,6 +251,73 @@ | |
| </Tab> | ||
| </LangTabs> | ||
|
|
||
| ## Handling ingestion lag (get-after-write) [#ingestion-lag] | ||
|
|
||
| Trace ingestion is asynchronous. `flush()` in the SDKs guarantees that data was **delivered** to the Langfuse API, not that it is **readable** yet. Reading right after writing — e.g. `trace.get(trace_id)` after a flush, fetching a score you just created, or fetching dataset run items right after an experiment — may raise a 404 (`langfuse.api.NotFoundError` in Python) or return incomplete data until processing completes. Typical visibility is within 15-30 seconds; under heavy or parallel load it can take longer, and there is no fixed SLA. | ||
|
|
||
| Instead of a fixed `time.sleep()`, use bounded retries with backoff: | ||
|
|
||
| <LangTabs items={["Python SDK", "JS/TS SDK"]}> | ||
| <Tab title="Python SDK"> | ||
|
|
||
| ```python | ||
| import time | ||
|
|
||
| from langfuse import get_client | ||
| from langfuse.api import NotFoundError | ||
|
|
||
| langfuse = get_client() | ||
|
|
||
|
|
||
| def get_trace_with_retry(trace_id: str, *, timeout: float = 60.0): | ||
| """Fetch a trace, retrying on 404 until ingestion completes.""" | ||
| deadline = time.monotonic() + timeout | ||
| delay = 1.0 | ||
| while True: | ||
| try: | ||
| return langfuse.api.trace.get(trace_id) | ||
| except NotFoundError: | ||
| if time.monotonic() >= deadline: | ||
| raise | ||
| time.sleep(delay) | ||
| delay = min(delay * 2, 10.0) # exponential backoff, capped | ||
|
|
||
|
|
||
| langfuse.flush() # ensure delivery before polling | ||
| trace = get_trace_with_retry(trace_id, timeout=60) | ||
|
Comment on lines
+286
to
+287
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The concluding call-site snippet uses Prompt To Fix With AIThis is a comment left during a code review.
Path: content/docs/api-and-data-platform/features/query-via-sdk.mdx
Line: 286-287
Comment:
**`trace_id` undefined in usage example**
The concluding call-site snippet uses `trace_id` as if it were already in scope, but the variable is never assigned anywhere in the snippet. Readers who copy these two lines verbatim will get a `NameError` at runtime. A short stub assignment or a `# replace with your trace ID` comment before the call would make the snippet self-contained.
How can I resolve this? If you propose a fix, please make it concise. |
||
| ``` | ||
|
|
||
| Note: a successful `trace.get` means the trace record exists — observations and scores of that trace may still be arriving. If you depend on completeness (e.g. counting observations or summing costs), additionally check that the expected observations are present before proceeding. | ||
|
|
||
| </Tab> | ||
| <Tab title="JS/TS SDK"> | ||
|
|
||
| ```ts | ||
| import { LangfuseClient } from "@langfuse/client"; | ||
|
|
||
| const langfuse = new LangfuseClient(); | ||
|
|
||
| async function getTraceWithRetry(traceId: string, timeoutMs = 60_000) { | ||
| const deadline = Date.now() + timeoutMs; | ||
| let delay = 1_000; | ||
| while (true) { | ||
| try { | ||
| return await langfuse.api.trace.get(traceId); | ||
| } catch (err: any) { | ||
| const is404 = err?.statusCode === 404; | ||
| if (!is404 || Date.now() >= deadline) throw err; | ||
| await new Promise((r) => setTimeout(r, delay)); | ||
| delay = Math.min(delay * 2, 10_000); // exponential backoff, capped | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| </Tab> | ||
| </LangTabs> | ||
|
|
||
| The same pattern applies to any read-after-write on ingested entities: traces, observations, scores, and dataset run items. Entities created via synchronous endpoints (prompts, datasets, dataset items) are readable immediately and do not need retries. | ||
|
|
||
| ## Related Resources | ||
|
|
||
| - To move existing trace or observation reads to v2, see [Observations API v2](/docs/api-and-data-platform/features/observations-api#v2). | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -188,6 +188,15 @@ | |
|
|
||
| <PropagationRestrictionsCallout attributes={["sessionId"]} /> | ||
|
|
||
| ## Scope and ordering of `sessionId` propagation [#scope-and-ordering] | ||
|
|
||
| Propagation is forward-only and follows the active execution context: | ||
|
|
||
| - **Covered**: the observation that is currently active when you enter the propagation context, and every observation started while the context is active (any nesting depth, any type). | ||
| - **Not covered**: observations started before entering the context. In the Python SDK, a detached root created with `start_observation()` (which does not become the active span) stays unstamped if you enter `propagate_attributes` afterwards — start it inside the context, or use `start_as_current_observation` for the root and propagate within it. | ||
|
|
||
| Where the value lands: each covered observation carries `session.id` in the exported OTel span data, and after ingestion `sessionId` surfaces as a **trace-level field**. When verifying programmatically, read it from the trace (`langfuse.api.trace.get(trace_id).session_id`) — observation objects returned by the public API do not expose a `session_id` field, so asserting on fetched child observations will always fail. | ||
|
Check failure on line 198 in content/docs/observability/features/sessions.mdx
|
||
|
Comment on lines
+194
to
+198
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 The new callouts in sessions.mdx, users.mdx, and instrumentation.mdx claim observation objects from "the public API" never expose Extended reasoning...The bug: All three edited pages add the same unqualified claim — e.g. sessions.mdx: Why it's wrong: The triggering code path: This isn't a hypothetical edge case — it's the exact path the docs themselves recommend. One paragraph above the disputed callout, observations = langfuse.api.observations.get_many(
trace_id="abcdef1234",
type="GENERATION",
limit=100,
fields="core,basic,usage"
)That Step-by-step proof:
Why nothing else catches this: observations-api.mdx wasn't touched by this PR, so there's no cross-reference or lint that would flag the new prose in sessions.mdx/users.mdx/instrumentation.mdx as inconsistent with it. The claim is likely true for the older, distinct Impact: Actively wrong debugging guidance. A user trying to verify that Fix: In each of the three locations, scope the claim to the specific object it's actually true for — e.g. "the legacy Addressing the one refutation on file: A refutation for the instrumentation.mdx instance (bug_005) argued it's a duplicate of the sessions.mdx instance (bug_001) and should be dropped as redundant. That refutation is about the bug tracker's dedup mechanics, not about whether the underlying claim is correct — it doesn't dispute the factual error, it just argues one location's report is redundant with another's. Since this synthesized report (merged_bug_001) explicitly covers all three occurrences (sessions.mdx ~line 198, users.mdx ~line 187, instrumentation.mdx ~line 541) as a single finding, that concern is resolved: all three locations are called out here rather than filed as separate near-duplicate reports.
Comment on lines
+191
to
+198
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 The new 'Scope and ordering' sections in sessions.mdx and users.mdx are near-verbatim duplicates (only sessionId/userId swapped), and instrumentation.mdx adds a third paragraph restating the same rules — duplicating the exact pattern that PropagationRestrictionsCallout.tsx was built to avoid across these same three pages. Consider consolidating into that shared component (e.g. via a scope/ordering prop) so future corrections to this behavior only need to be made once. Extended reasoning...The PR introduces a new "Scope and ordering" section on This is exactly the situation the repo already solved once: CLAUDE.md's review guidelines explicitly call this out: "If blocks of text or code are largely repeated on multiple documentation pages, suggest consolidating them in Concrete duplication proof:
Impact: nothing breaks today — the pages render correctly and the content is accurate. But because the three copies are hand-maintained prose rather than one parameterized source, any future correction to this behavior (e.g., if SDK semantics change, or to fix the trace/observation field wording) now requires editing three separate places, and the copies can silently drift out of sync — which is a real maintainability cost given how close in wording (but not byte-identical) they already are. Suggested fix: extend |
||
|
|
||
| ## Example | ||
|
|
||
| Try this feature using the public [example project](/docs/demo). | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 The new 'Traces: list returns lightweight views' section (documenting that trace.list's observations/scores fields are IDs-only, plus a list-then-get pagination example) was added only to the Python SDK tab in query-via-sdk.mdx; the JS/TS SDK tab jumps straight from Observations to Metrics with no equivalent guidance, even though this is a property of the shared Public API that applies identically to
langfuse.api.trace.listin@langfuse/client. Consider mirroring the section (with a TS list-then-get example) into the JS/TS tab so those readers get the same warning.Extended reasoning...
The bug: this PR adds a new
### Traces:listreturns lightweight views [#traces-list-vs-get]section tocontent/docs/api-and-data-platform/features/query-via-sdk.mdx, but it is inserted only inside<Tab title="Python SDK">. The parallel<Tab title="JS/TS SDK">block goes directly from### Observationsto### Metrics(then### Other resources) with no equivalent section and no TS list-then-get example.Why this matters: the behavior being documented — that
trace.listreturns lightweight trace views whereobservations/scoresare ID strings rather than full objects, while trace-level aggregates (total_cost,latency) are still included — is a property of the shared Public API. Theapinamespace on both SDKs is generated from the same OpenAPI spec, solangfuse.api.trace.list(...)in@langfuse/clientexhibits the exact same lightweight-view behavior aslangfuse.api.trace.list(...)in the Python SDK. There's nothing Python-specific about the gotcha.Proof of the gap: walk the JS/TS tab as a reader would. Starting at
### Observations(JS/TS), the next heading is### Metrics, then### Other resources. A JS/TS developer who wants to aggregate per-observation usage/cost by paginatinglangfuse.api.trace.list(...)gets no warning thattraces.data[i].observationsis a list of ID strings, not objects withusageDetails/cost. They would write code assuming full objects, get IDs instead, and hit exactly the confusing failure mode this PR's own description says it set out to close ("agent-behavior testing showed cause real debugging sessions").Why existing code doesn't prevent it: this is docs-only, so there's no compiler/lint check enforcing tab parity. The PR's own precedent shows parity was achievable and intended elsewhere — the new 'Handling ingestion lag' section further down the same file explicitly provides both a Python and a JS/TS tab (with a TS
getTraceWithRetryexample paralleling the Pythonget_trace_with_retry). That makes the omission in thetrace.listsection read as an oversight rather than deliberate scoping to Python only.Impact: purely a documentation completeness gap — nothing renders incorrectly, no build/link check fails, and the Python content that does exist is accurate. But it undercuts the stated goal of the PR for exactly the audience (JS/TS SDK users) who would otherwise hit the same footgun.
Suggested fix: add a mirrored
### Traces:listreturns lightweight viewssection to the JS/TS tab, with alangfuse.api.trace.list-based list-then-get TS example analogous to the Python one (fetchingtraces.data, notingobservations/scoresare ID arrays, and callinglangfuse.api.trace.get(traceId)for full objects), plus the same pointer toobservations.getMany/Metrics API v2 for scale.