From 08153ed74522feec71edd59fa61c633274e7c068 Mon Sep 17 00:00:00 2001 From: CoderSufiyan Date: Wed, 8 Jul 2026 19:35:53 +0530 Subject: [PATCH 1/2] docs: add StatsD configuration details for worker scaling --- .../self-hosting/configuration/scaling.mdx | 24 ++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/content/self-hosting/configuration/scaling.mdx b/content/self-hosting/configuration/scaling.mdx index ca02b2270c..57d980d8f1 100644 --- a/content/self-hosting/configuration/scaling.mdx +++ b/content/self-hosting/configuration/scaling.mdx @@ -30,8 +30,30 @@ On very high loads, it may become necessary to apply additional settings that in For most environments, we recommend to scale the worker containers by their CPU load as this is a straightforward metric to measure. A load above 50% for a 2 CPU container is an indicator that the instance is saturated and that the throughput should increase by adding more containers. -In addition, the Langfuse worker also publishes queue length metrics via statsd that can be used to scale the worker containers. +In addition, the Langfuse worker also publishes queue length metrics via StatsD (DogStatsD format) that can be used to scale the worker containers. `langfuse.queue.ingestion.length` is the main metric that we use to make scaling decisions. + +#### Configuring StatsD + +Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/dd_libraries/nodejs/) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables: + +| Environment Variable | Default | Description | +| ----------------------- | ------------- | ------------------------------------------ | +| `DD_AGENT_HOST` | `localhost` | StatsD agent hostname or IP | +| `DD_DOGSTATSD_PORT` | `8125` | DogStatsD server port | +| `DD_ENV` | — | Environment tag applied to all metrics | +| `DD_SERVICE` | — | Service name tag applied to all metrics | + +These variables apply to both the web and worker containers. + +Langfuse-specific queue metric configuration (worker container only): + +| Environment Variable | Default | Description | +| ---------------------------------------------- | -------- | -------------------------------------------------------------- | +| `LANGFUSE_QUEUE_METRICS_ENABLED` | `true` | Master toggle for publishing queue metrics via StatsD | +| `LANGFUSE_QUEUE_METRICS_INTERVAL_MS` | `1000` | Poll interval for queue depth metrics (minimum 100ms) | +| `LANGFUSE_QUEUE_METRICS_SAMPLE_RATE` | `0.3` | Probability for recording sharded queue depth metrics (0-1) | + The queue metrics can also be published to AWS CloudWatch by setting `ENABLE_AWS_CLOUDWATCH_METRIC_PUBLISHING=true` to configure auto-scalers based on AWS metrics. ### Reducing ClickHouse reads within the ingestion processing From 40bd87e613f9000f33e097c5e0692acdf6c2f19a Mon Sep 17 00:00:00 2001 From: CoderSufiyan Date: Wed, 8 Jul 2026 19:40:44 +0530 Subject: [PATCH 2/2] fix: update dd-trace link to DogStatsD docs and fix DD_AGENT_HOST description --- content/self-hosting/configuration/scaling.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/self-hosting/configuration/scaling.mdx b/content/self-hosting/configuration/scaling.mdx index 57d980d8f1..afc8fbd652 100644 --- a/content/self-hosting/configuration/scaling.mdx +++ b/content/self-hosting/configuration/scaling.mdx @@ -35,11 +35,11 @@ In addition, the Langfuse worker also publishes queue length metrics via StatsD #### Configuring StatsD -Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/dd_libraries/nodejs/) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables: +Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/developers/dogstatsd/?tab=nodejs) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables: | Environment Variable | Default | Description | | ----------------------- | ------------- | ------------------------------------------ | -| `DD_AGENT_HOST` | `localhost` | StatsD agent hostname or IP | +| `DD_AGENT_HOST` | `localhost` | DataDog Agent hostname or IP (must run a DogStatsD listener) | | `DD_DOGSTATSD_PORT` | `8125` | DogStatsD server port | | `DD_ENV` | — | Environment tag applied to all metrics | | `DD_SERVICE` | — | Service name tag applied to all metrics |