From 3fd5857e36eb141cd75d6ddaf3d65adea0893372 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jannik=20Maierh=C3=B6fer?= Date: Thu, 16 Jul 2026 16:50:59 +0200 Subject: [PATCH] docs: clarify propagate_attributes scope, trace.list lightweight views, and ingestion lag Close three documentation gaps that repeatedly cause debugging sessions: - Sessions/Users/Instrumentation: document exact propagate_attributes scope (forward-only: currently active observation + observations started while the context is active; detached roots stay unstamped) and where the values land (trace-level fields via api.trace.get; observation objects do not expose session_id/user_id). - Query via SDKs: document that trace.list returns lightweight views with observation/score IDs only, with a list-then-get pagination example for usage/cost aggregation. - Query via SDKs + Scores: document async ingestion (flush guarantees delivery, not read visibility; typically 15-30s, no SLA) and add a bounded retry-with-backoff pattern for get-after-write (404 NotFoundError). Co-Authored-By: Claude Fable 5 --- components/PropagationRestrictionsCallout.tsx | 7 +- .../features/query-via-sdk.mdx | 103 +++++++++++++++++- .../evaluation-methods/scores-via-sdk.mdx | 6 + .../docs/observability/features/sessions.mdx | 9 ++ content/docs/observability/features/users.mdx | 9 ++ .../observability/sdk/instrumentation.mdx | 2 + 6 files changed, 133 insertions(+), 3 deletions(-) diff --git a/components/PropagationRestrictionsCallout.tsx b/components/PropagationRestrictionsCallout.tsx index 17287dd806..261e490a4c 100644 --- a/components/PropagationRestrictionsCallout.tsx +++ b/components/PropagationRestrictionsCallout.tsx @@ -71,8 +71,11 @@ export function PropagationRestrictionsCallout({ )}
  • Call early in your trace to ensure all observations - are covered. This way you make sure that all Metrics in Langfuse are - accurate. + are covered. Only the currently active observation{" "} + and observations started while the context is active{" "} + receive the attributes — observations started earlier are not + retroactively updated. This way you make sure that all Metrics in + Langfuse are accurate.
  • Invalid values are dropped with a warning
    • diff --git a/content/docs/api-and-data-platform/features/query-via-sdk.mdx b/content/docs/api-and-data-platform/features/query-via-sdk.mdx index f014e51209..b0b93f09b4 100644 --- a/content/docs/api-and-data-platform/features/query-via-sdk.mdx +++ b/content/docs/api-and-data-platform/features/query-via-sdk.mdx @@ -18,7 +18,7 @@ If you are new to Langfuse, we recommend familiarizing yourself with the [Langfu -New data is typically available for querying within 15-30 seconds of ingestion, though processing times may vary at times. Please visit [status.langfuse.com](https://status.langfuse.com) if you encounter any issues. +Ingestion is **asynchronous**: SDK `flush()` only guarantees delivery to the API, not read visibility. New data is typically available for querying within 15-30 seconds of ingestion, though processing times may vary — under heavy or parallel load, visibility can lag longer (there is no fixed SLA). Reading a trace immediately after writing it may return a 404. See [Handling ingestion lag](#ingestion-lag) for the recommended retry pattern, and visit [status.langfuse.com](https://status.langfuse.com) if you encounter persistent issues. @@ -72,6 +72,40 @@ observations = langfuse.api.observations.get_many( Use `trace_id` to retrieve the observations that belong to a single trace. Use `parent_observation_id` in the response to reconstruct an observation tree when needed. +### Traces: `list` returns lightweight views [#traces-list-vs-get] + +`langfuse.api.trace.list(...)` returns **lightweight trace views**: the `observations` and `scores` fields contain only **IDs (strings)**, not full objects. Trace-level aggregates (`total_cost`, `latency`) are included, but per-observation usage and cost details are not. To get full observation objects, call `langfuse.api.trace.get(trace_id)` per trace (or prefer `observations.get_many(trace_id=...)` above). + +List-then-get pagination example for usage/cost aggregation: + +```python +from collections import defaultdict +from datetime import datetime, timedelta, timezone + +usage_by_model = defaultdict(int) +from_timestamp = datetime.now(timezone.utc) - timedelta(days=1) +page = 1 + +while True: + traces = langfuse.api.trace.list( + from_timestamp=from_timestamp, + page=page, + limit=50, + ) + for trace_view in traces.data: + # trace_view.observations is a list of ID strings only + full_trace = langfuse.api.trace.get(trace_view.id) + for obs in full_trace.observations: # full objects incl. usage/cost + if obs.usage_details: + model = obs.model or "unknown" + usage_by_model[model] += obs.usage_details.get("total", 0) + if page >= traces.meta.total_pages: + break + page += 1 +``` + +If you only need trace-level totals, `trace.list` alone is enough (`trace_view.total_cost`, `trace_view.latency`) — no per-trace `get` required. For large-scale aggregation, prefer the [Metrics API v2](/docs/metrics/features/metrics-api#v2) over paginating row-level data. + ### Metrics For the full query schema, supported dimensions, filters, and examples, see the [Metrics API v2 documentation](/docs/metrics/features/metrics-api#v2). @@ -217,6 +251,73 @@ Explore more entities via Intellisense on `langfuse.api`. +## Handling ingestion lag (get-after-write) [#ingestion-lag] + +Trace ingestion is asynchronous. `flush()` in the SDKs guarantees that data was **delivered** to the Langfuse API, not that it is **readable** yet. Reading right after writing — e.g. `trace.get(trace_id)` after a flush, fetching a score you just created, or fetching dataset run items right after an experiment — may raise a 404 (`langfuse.api.NotFoundError` in Python) or return incomplete data until processing completes. Typical visibility is within 15-30 seconds; under heavy or parallel load it can take longer, and there is no fixed SLA. + +Instead of a fixed `time.sleep()`, use bounded retries with backoff: + + + + +```python +import time + +from langfuse import get_client +from langfuse.api import NotFoundError + +langfuse = get_client() + + +def get_trace_with_retry(trace_id: str, *, timeout: float = 60.0): + """Fetch a trace, retrying on 404 until ingestion completes.""" + deadline = time.monotonic() + timeout + delay = 1.0 + while True: + try: + return langfuse.api.trace.get(trace_id) + except NotFoundError: + if time.monotonic() >= deadline: + raise + time.sleep(delay) + delay = min(delay * 2, 10.0) # exponential backoff, capped + + +langfuse.flush() # ensure delivery before polling +trace = get_trace_with_retry(trace_id, timeout=60) +``` + +Note: a successful `trace.get` means the trace record exists — observations and scores of that trace may still be arriving. If you depend on completeness (e.g. counting observations or summing costs), additionally check that the expected observations are present before proceeding. + + + + +```ts +import { LangfuseClient } from "@langfuse/client"; + +const langfuse = new LangfuseClient(); + +async function getTraceWithRetry(traceId: string, timeoutMs = 60_000) { + const deadline = Date.now() + timeoutMs; + let delay = 1_000; + while (true) { + try { + return await langfuse.api.trace.get(traceId); + } catch (err: any) { + const is404 = err?.statusCode === 404; + if (!is404 || Date.now() >= deadline) throw err; + await new Promise((r) => setTimeout(r, delay)); + delay = Math.min(delay * 2, 10_000); // exponential backoff, capped + } + } +} +``` + + + + +The same pattern applies to any read-after-write on ingested entities: traces, observations, scores, and dataset run items. Entities created via synchronous endpoints (prompts, datasets, dataset items) are readable immediately and do not need retries. + ## Related Resources - To move existing trace or observation reads to v2, see [Observations API v2](/docs/api-and-data-platform/features/observations-api#v2). diff --git a/content/docs/evaluation/evaluation-methods/scores-via-sdk.mdx b/content/docs/evaluation/evaluation-methods/scores-via-sdk.mdx index 686cfe529b..5c33ce380d 100644 --- a/content/docs/evaluation/evaluation-methods/scores-via-sdk.mdx +++ b/content/docs/evaluation/evaluation-methods/scores-via-sdk.mdx @@ -22,6 +22,12 @@ You can add scores via the Langfuse SDKs or API. Scores can take one of four dat If a score is ingested manually using a `trace_id` to link the score to a trace, it is not necessary to wait until the trace has been created. The score will show up in the scores table and will be linked to the trace once the trace with the same `trace_id` is created. + + +Score ingestion is asynchronous. A score created via SDK is queued, sent on the next flush, and processed server-side — it is typically queryable via the API/UI within 15-30 seconds, potentially longer under load. If you fetch a score right after creating it, retry with bounded backoff. See [Handling ingestion lag](/docs/api-and-data-platform/features/query-via-sdk#ingestion-lag). + + + Here are examples by `Score` data types. For trace and observation scores, `trace_id`/`traceId` is required and `observation_id`/`observationId` is optional. If you attach a score to an observation, always provide both the observation ID and the corresponding trace ID. diff --git a/content/docs/observability/features/sessions.mdx b/content/docs/observability/features/sessions.mdx index 3ede8b5354..5256f3e1ae 100644 --- a/content/docs/observability/features/sessions.mdx +++ b/content/docs/observability/features/sessions.mdx @@ -188,6 +188,15 @@ The [Flowise Integration](/docs/flowise) automatically maps the Flowise chatId t +## Scope and ordering of `sessionId` propagation [#scope-and-ordering] + +Propagation is forward-only and follows the active execution context: + +- **Covered**: the observation that is currently active when you enter the propagation context, and every observation started while the context is active (any nesting depth, any type). +- **Not covered**: observations started before entering the context. In the Python SDK, a detached root created with `start_observation()` (which does not become the active span) stays unstamped if you enter `propagate_attributes` afterwards — start it inside the context, or use `start_as_current_observation` for the root and propagate within it. + +Where the value lands: each covered observation carries `session.id` in the exported OTel span data, and after ingestion `sessionId` surfaces as a **trace-level field**. When verifying programmatically, read it from the trace (`langfuse.api.trace.get(trace_id).session_id`) — observation objects returned by the public API do not expose a `session_id` field, so asserting on fetched child observations will always fail. + ## Example Try this feature using the public [example project](/docs/demo). diff --git a/content/docs/observability/features/users.mdx b/content/docs/observability/features/users.mdx index cd5ff6ac0f..10de5a45c2 100644 --- a/content/docs/observability/features/users.mdx +++ b/content/docs/observability/features/users.mdx @@ -177,6 +177,15 @@ await startActiveObservation("langchain-call", async () => { +## Scope and ordering of `userId` propagation [#scope-and-ordering] + +Propagation is forward-only and follows the active execution context: + +- **Covered**: the observation that is currently active when you enter the propagation context, and every observation started while the context is active (any nesting depth, any type). +- **Not covered**: observations started before entering the context. In the Python SDK, a detached root created with `start_observation()` (which does not become the active span) stays unstamped if you enter `propagate_attributes` afterwards — start it inside the context, or use `start_as_current_observation` for the root and propagate within it. + +Where the value lands: each covered observation carries `user.id` in the exported OTel span data, and after ingestion `userId` surfaces as a **trace-level field**. When verifying programmatically, read it from the trace (`langfuse.api.trace.get(trace_id).user_id`) — observation objects returned by the public API do not expose a `user_id` field, so asserting on fetched child observations will always fail. + ## View all users The user list provides an overview of all users that have been tracked by Langfuse. It makes it simple to segment by overall token usage, number of traces, and user feedback. diff --git a/content/docs/observability/sdk/instrumentation.mdx b/content/docs/observability/sdk/instrumentation.mdx index 6755ed63cb..385e3fb61b 100644 --- a/content/docs/observability/sdk/instrumentation.mdx +++ b/content/docs/observability/sdk/instrumentation.mdx @@ -538,6 +538,8 @@ await startActiveObservation("user-workflow", async () => { +**Scope**: propagation is forward-only. The attributes are applied to the observation that is currently active when the context is entered, and to every observation started while the context is active. Observations started earlier are not retroactively updated — in the Python SDK, a detached span from `start_observation()` (never the active span) stays unstamped if you enter `propagate_attributes` after starting it. After ingestion, `userId` and `sessionId` surface as trace-level fields (read them via `api.trace.get(trace_id)`); observation objects returned by the public API do not expose these fields. See [Sessions](/docs/observability/features/sessions#scope-and-ordering) and [Users](/docs/observability/features/users#scope-and-ordering) for details. + ### Cross-service propagation