From dcdb92792609762d7d40da19ec1e31bd81382500 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Wed, 14 Oct 2026 19:04:27 +0200 Subject: [PATCH 01/19] Prometheus: define more precisely the conversion of resource attributes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- .../prometheus_and_openmetrics.md | 116 ++++++++++++++---- 1 file changed, 94 insertions(+), 22 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index ff853dcf85a..48f1891ee69 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -29,6 +29,7 @@ aliases: * [Dropped Types](#dropped-types) * [Exemplars](#exemplars) * [Instrumentation Scope](#instrumentation-scope) + * [Metric Attributes](#metric-attributes) * [Resource Attributes](#resource-attributes) - [OTLP Metric points to Prometheus](#otlp-metric-points-to-prometheus) * [Metric Metadata](#metric-metadata-1) @@ -40,7 +41,7 @@ aliases: + [Histograms as Prometheus NHCB](#histograms-as-prometheus-nhcb) * [Exponential Histograms](#exponential-histograms) * [Summaries](#summaries-1) - * [Metric Attributes](#metric-attributes) + * [Metric Attributes](#metric-attributes-1) * [Exemplar Conversion](#exemplar-conversion) * [Resource Attributes](#resource-attributes-1) @@ -368,42 +369,113 @@ MUST be assigned an instrumentation scope identifying the entity performing the translation from Prometheus to OpenTelemetry (e.g. the collector's Prometheus receiver). +### Metric Attributes + +**Status**: [Development](../document-status.md) + +Labels of metrics MUST be copied to data point +attributes, with the exception of: + +- `job` and `instance` labels MUST NOT be copied to data point attributes, +- `otel_scope_*` labels are excluded due to rules around instrumentation scope, +- labels of the `target` info metric are implicitly excluded as `target` info +metric samples are not turned into data points. + ### Resource Attributes **Status**: [Development](../document-status.md) When scraping a Prometheus endpoint, resource attributes MUST be added to the scraped metrics to distinguish them from metrics from other Prometheus -endpoints. In particular, `service.name` and `service.instance.id`, are needed -to ensure Prometheus exporters can disambiguate metrics using -[`job` and `instance` labels](https://prometheus.io/docs/concepts/jobs_instances/) -as [described below](#resource-attributes-1). +endpoints. + +NOTE(krajo): this only talks about scraping, but similar rules apply to remote +write receiver which uses `target` info metric. There are some differences +though. + +TODO(krajo): deal with the remote write receiver use case. + +In Prometheus metrics the `job` and `instance` are considered identifying, +therefore a new resource MUST be created for each `job` and `instance` pair +encountered in the metrics labels, except for the `target` info metric. + +NOTE(krajo): both the prometheusreceiver and prometheusremotewritereceiver +work this way, thus both scrape and remote write can carry multiple resources +with the `job` and `instance` being the join key between regular metrics and +the `target` info metric. + +If there is no `job` label on a metric and the metrics are being scraped, +then the `job` label MUST be considered to be equal to the `job` label of the +scrape target. During scrape, the `job` label of the target may be equal to the scrape +job name or customized by relabel rules. + +If there is no `job` label on a metric and there is no `job` label for the +scrape target, the scrape MUST fail. Note that this is different from the +Prometheus behavior that still scrapes the metrics without `job`. + +If there is no `instance` label and the metrics are being scraped, the +`instance` label MUST be considered to be equal to the `instance` label of the +scrape target. Note that the `instance` label of the target may be equal to the +`__address__` of the target or customized by relabel rules. + +For each resource the `service.name` attribute MUST be set to the +`service.name` label in `target` info metric, if that label exists in the +`target` info metric with the same `job` and `instance` key as the resource. +Otherwise `service.name` MUST be set to the value of the `job` label. + +NOTE(krajo): when an SDK uses default translation, then `service.name` is +translated into `service_name`, thus the `job` is copied into `service.name` +and `service_name` is also kept as a resource attribute. +This doesn't happen if there is no translation. + +For each resource the `service.instance.id` attribute MUST be set to the +`service.instance.id` label in `target` info metric, if that label exists in the +`target` info metric with the same `job` and `instance` key as the resource. +Otherwise `service.instance.id` MUST be set to the value of the `instance` +label. + +NOTE(krajo): when an SDK uses default translation, then `service.instance.id` is +translated into `service_instance_id`, thus the `instance` is copied into +`service.instance.id` and `service_instance_id` is also kept as a resource +attribute. +This doesn't happen if there is no translation. -The following attributes MUST be associated with scraped metrics as resource -attributes, and MUST NOT be added as metric attributes: +The following attributes SHOULD be associated with scraped metrics as resource +attributes, and MUST NOT be added as metric +attributes: | OTLP Resource Attribute | Description | | ----------------------- | ----------- | -| `service.name` | The configured name of the service that the target belongs to | -| `service.instance.id` | A unique identifier of the target. By default, it should be the `:` of the scraped URL | +| `server.address` | The `` portion of the `instance` unless it is localhost, a loopback address, or an unspecified address | +| `server.port` | The `` portion of the `instance` | +| `url.scheme` | `http` or `https` from the `__scheme__` label on the target | -The following attributes SHOULD be associated with scraped metrics as resource -attributes, and MUST NOT be added as metric attributes: +NOTE(krajo): the receiver sets empty string for `server.port` if it cannot be +deduced, but perheaps this is a bug? -| OTLP Resource Attribute | Description | -| ----------------------- | ----------- | -| `server.address` | The `` portion of the target's URL that was scraped | -| `server.port` | The `` portion of the target's URL that was scraped | -| `url.scheme` | `http` or `https` | +NOTE(krajo): there is a bug in the receiver whereby it ignores the relabelling +of the `__scheme__`, so `url.scheme` always reflects the pre-relabelling state. + +Further attributes (such as from k8s service discovery metadata labels) MAY be +added to the resource attributes. Labels from the `target` info metric take +precedence over all derived resource attributes. + +To summarize, `service.name` and `service.instance.id`, are needed +to ensure Prometheus exporters can disambiguate metrics using +[`job` and `instance` labels](https://prometheus.io/docs/concepts/jobs_instances/) +as [described below](#resource-attributes-1), therefore this specification +ensures that `service.name` and `service.instance.id` are always defined. In addition to the attributes above, the -[target](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) +[`target`](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) info metric is used to supply additional resource attributes. If present, -the `target` info metric MUST be dropped from the batch of metrics, and all labels from -the `target` info metric MUST be converted to resource attributes -attached to all other metrics which are part of the scrape. By default, label -keys and values MUST NOT be altered (such as replacing `_` with `.` characters -in keys). +the `target` info metric MUST be dropped from the batch of metrics, and all labels +except `job`, `instance` and `__name__` from the `target` info metric MUST be +converted to resource attributes attached to all other metrics which are part of +the same resource identified by `job` and `instance`. + +By default, label keys and values of the `target` info metric MUST NOT be +altered (such as replacing `_` with `.` characters in keys). ## OTLP Metric points to Prometheus From e89b7d4d844bcedc70741a1363a9e37178f7d3f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Thu, 15 Oct 2026 13:38:33 +0200 Subject: [PATCH 02/19] add notes on the OTLP to Prometheus side MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- .../compatibility/prometheus_and_openmetrics.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 48f1891ee69..8e15926e67b 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -846,6 +846,15 @@ attributes MUST be combined as `/`, or `` if namespace is empty, to form the `job` metric label. The `service.instance.id` attribute, if present, MUST be converted to the `instance` label; otherwise, `instance` should be added with an empty value. + +NOTE(krajo): implicitly: existing `job` data point attribute would be +overwritten, however it is kept if there is no `service.name`. + +NOTE(krajo): both prometheusremotewriteexporter and the Prometheus OTLP endpoint +keep track of existing `target` info metrics by the full label set, not just +`job` and `instance` , so theoretically you can end up with two `target` info +metric that have the same `job` and `instance`, but different attributes. + Other resource attributes SHOULD be converted to a [target](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) info metric, or MUST be dropped. The `target` metric is an info-typed metric From 6709a74caae6408e231faa22f987e292ff4bbecf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 19 Oct 2026 11:26:12 +0200 Subject: [PATCH 03/19] Exclude __name__, le and quantile from data point attributes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- .../compatibility/prometheus_and_openmetrics.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 8e15926e67b..c504ddfdd1b 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -373,12 +373,17 @@ Prometheus receiver). **Status**: [Development](../document-status.md) -Labels of metrics MUST be copied to data point -attributes, with the exception of: +Labels of metrics MUST be copied to data point attributes, except for the +following labels, which MUST NOT be copied: -- `job` and `instance` labels MUST NOT be copied to data point attributes, -- `otel_scope_*` labels are excluded due to rules around instrumentation scope, -- labels of the `target` info metric are implicitly excluded as `target` info +- `__name__`, +- `job` and `instance`, +- the `le` label of classic Prometheus histogram buckets, +- the `quantile` label of Prometheus summaries, +- labels with the `otel_scope_` prefix, which are handled according to the + [Instrumentation Scope](#instrumentation-scope) rules. + +Labels of the `target` info metric are implicitly excluded, as `target` info metric samples are not turned into data points. ### Resource Attributes From 933ed0122568f61ca7ffb241428de4bc7f66f9cd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 19 Oct 2026 12:25:34 +0200 Subject: [PATCH 04/19] Move inline notes to PR review comments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- .../prometheus_and_openmetrics.md | 28 ------------------- 1 file changed, 28 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index c504ddfdd1b..7b280fb34dd 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -394,21 +394,10 @@ When scraping a Prometheus endpoint, resource attributes MUST be added to the scraped metrics to distinguish them from metrics from other Prometheus endpoints. -NOTE(krajo): this only talks about scraping, but similar rules apply to remote -write receiver which uses `target` info metric. There are some differences -though. - -TODO(krajo): deal with the remote write receiver use case. - In Prometheus metrics the `job` and `instance` are considered identifying, therefore a new resource MUST be created for each `job` and `instance` pair encountered in the metrics labels, except for the `target` info metric. -NOTE(krajo): both the prometheusreceiver and prometheusremotewritereceiver -work this way, thus both scrape and remote write can carry multiple resources -with the `job` and `instance` being the join key between regular metrics and -the `target` info metric. - If there is no `job` label on a metric and the metrics are being scraped, then the `job` label MUST be considered to be equal to the `job` label of the scrape target. During scrape, the `job` label of the target may be equal to the scrape @@ -428,23 +417,12 @@ For each resource the `service.name` attribute MUST be set to the `target` info metric with the same `job` and `instance` key as the resource. Otherwise `service.name` MUST be set to the value of the `job` label. -NOTE(krajo): when an SDK uses default translation, then `service.name` is -translated into `service_name`, thus the `job` is copied into `service.name` -and `service_name` is also kept as a resource attribute. -This doesn't happen if there is no translation. - For each resource the `service.instance.id` attribute MUST be set to the `service.instance.id` label in `target` info metric, if that label exists in the `target` info metric with the same `job` and `instance` key as the resource. Otherwise `service.instance.id` MUST be set to the value of the `instance` label. -NOTE(krajo): when an SDK uses default translation, then `service.instance.id` is -translated into `service_instance_id`, thus the `instance` is copied into -`service.instance.id` and `service_instance_id` is also kept as a resource -attribute. -This doesn't happen if there is no translation. - The following attributes SHOULD be associated with scraped metrics as resource attributes, and MUST NOT be added as metric attributes: @@ -455,12 +433,6 @@ attributes: | `server.port` | The `` portion of the `instance` | | `url.scheme` | `http` or `https` from the `__scheme__` label on the target | -NOTE(krajo): the receiver sets empty string for `server.port` if it cannot be -deduced, but perheaps this is a bug? - -NOTE(krajo): there is a bug in the receiver whereby it ignores the relabelling -of the `__scheme__`, so `url.scheme` always reflects the pre-relabelling state. - Further attributes (such as from k8s service discovery metadata labels) MAY be added to the resource attributes. Labels from the `target` info metric take precedence over all derived resource attributes. From 80b971210664ad116d02add2f11181afd7b9da2e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 19 Oct 2026 12:28:14 +0200 Subject: [PATCH 05/19] Fix K8s spelling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- specification/compatibility/prometheus_and_openmetrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 7b280fb34dd..fe5acab5255 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -433,7 +433,7 @@ attributes: | `server.port` | The `` portion of the `instance` | | `url.scheme` | `http` or `https` from the `__scheme__` label on the target | -Further attributes (such as from k8s service discovery metadata labels) MAY be +Further attributes (such as from K8s service discovery metadata labels) MAY be added to the resource attributes. Labels from the `target` info metric take precedence over all derived resource attributes. From a119e892cc7fdaa57cc848f6cfba50fedf7f105e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 19 Oct 2026 13:56:39 +0200 Subject: [PATCH 06/19] Remove remaining inline notes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- specification/compatibility/prometheus_and_openmetrics.md | 8 -------- 1 file changed, 8 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index fe5acab5255..6ef956fe07b 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -824,14 +824,6 @@ attributes MUST be combined as `/`, or `service.instance.id` attribute, if present, MUST be converted to the `instance` label; otherwise, `instance` should be added with an empty value. -NOTE(krajo): implicitly: existing `job` data point attribute would be -overwritten, however it is kept if there is no `service.name`. - -NOTE(krajo): both prometheusremotewriteexporter and the Prometheus OTLP endpoint -keep track of existing `target` info metrics by the full label set, not just -`job` and `instance` , so theoretically you can end up with two `target` info -metric that have the same `job` and `instance`, but different attributes. - Other resource attributes SHOULD be converted to a [target](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) info metric, or MUST be dropped. The `target` metric is an info-typed metric From 1cb54d4252ec9eba2ae2acb9d6534830519ed7fe Mon Sep 17 00:00:00 2001 From: George Krajcsovits Date: Mon, 19 Oct 2026 14:10:33 +0200 Subject: [PATCH 07/19] Apply suggestion from @krajorama --- specification/compatibility/prometheus_and_openmetrics.md | 1 - 1 file changed, 1 deletion(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 6ef956fe07b..3eee06a878a 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -823,7 +823,6 @@ attributes MUST be combined as `/`, or `` if namespace is empty, to form the `job` metric label. The `service.instance.id` attribute, if present, MUST be converted to the `instance` label; otherwise, `instance` should be added with an empty value. - Other resource attributes SHOULD be converted to a [target](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) info metric, or MUST be dropped. The `target` metric is an info-typed metric From 2377f294783ca5dbd6e4ee76390e3a1b31a4164d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 19 Oct 2026 14:12:32 +0200 Subject: [PATCH 08/19] Add changelog entry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 96d43c13896..a9bf06c7378 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,6 +52,9 @@ release. ([#5213](https://github.com/open-telemetry/opentelemetry-specification/pull/5213)) - Stabilize Prometheus Info metric transformation. ([#5211](https://github.com/open-telemetry/opentelemetry-specification/pull/5211)) +- Clarify the conversion of Prometheus labels and the `target` info metric to + OTLP resources, data point attributes and resource attributes. + ([#5356](https://github.com/open-telemetry/opentelemetry-specification/pull/5356)) ### SDK Configuration From 86ff3b934f81b09037fdb0094e8983e301b187fb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 26 Oct 2026 10:06:54 +0100 Subject: [PATCH 09/19] Rename Prometheus to OTLP metric attributes section to Metric Labels MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Restores the #metric-attributes anchor for the OTLP to Prometheus section, which is referenced from within the document and from the spec compliance matrix. Signed-off-by: György Krajcsovits --- specification/compatibility/prometheus_and_openmetrics.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 3eee06a878a..064dffef5bd 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -29,7 +29,7 @@ aliases: * [Dropped Types](#dropped-types) * [Exemplars](#exemplars) * [Instrumentation Scope](#instrumentation-scope) - * [Metric Attributes](#metric-attributes) + * [Metric Labels](#metric-labels) * [Resource Attributes](#resource-attributes) - [OTLP Metric points to Prometheus](#otlp-metric-points-to-prometheus) * [Metric Metadata](#metric-metadata-1) @@ -41,7 +41,7 @@ aliases: + [Histograms as Prometheus NHCB](#histograms-as-prometheus-nhcb) * [Exponential Histograms](#exponential-histograms) * [Summaries](#summaries-1) - * [Metric Attributes](#metric-attributes-1) + * [Metric Attributes](#metric-attributes) * [Exemplar Conversion](#exemplar-conversion) * [Resource Attributes](#resource-attributes-1) @@ -369,7 +369,7 @@ MUST be assigned an instrumentation scope identifying the entity performing the translation from Prometheus to OpenTelemetry (e.g. the collector's Prometheus receiver). -### Metric Attributes +### Metric Labels **Status**: [Development](../document-status.md) From 5c438ee76926b32ffccdb0afef65adb3edfe0749 Mon Sep 17 00:00:00 2001 From: George Krajcsovits Date: Mon, 26 Oct 2026 10:08:13 +0100 Subject: [PATCH 10/19] Apply suggestion from @krajorama --- specification/compatibility/prometheus_and_openmetrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 064dffef5bd..41abf953013 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -394,7 +394,7 @@ When scraping a Prometheus endpoint, resource attributes MUST be added to the scraped metrics to distinguish them from metrics from other Prometheus endpoints. -In Prometheus metrics the `job` and `instance` are considered identifying, +In Prometheus metrics the `job` and `instance` are used as join keys between metrics and the `target` info metric. therefore a new resource MUST be created for each `job` and `instance` pair encountered in the metrics labels, except for the `target` info metric. From 528d2d601fd2eef2c42736089ccc510d9f78230e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Tue, 27 Oct 2026 09:03:32 +0100 Subject: [PATCH 11/19] Simplify job and instance fallback rules for scraped metrics MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reference the Prometheus jobs and instances documentation and describe the fallback to the scrape target's job and instance labels in a single rule, matching the Prometheus receiver behavior. Signed-off-by: György Krajcsovits --- .../prometheus_and_openmetrics.md | 29 ++++++++----------- 1 file changed, 12 insertions(+), 17 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 41abf953013..cd615133c56 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -394,23 +394,18 @@ When scraping a Prometheus endpoint, resource attributes MUST be added to the scraped metrics to distinguish them from metrics from other Prometheus endpoints. -In Prometheus metrics the `job` and `instance` are used as join keys between metrics and the `target` info metric. -therefore a new resource MUST be created for each `job` and `instance` pair -encountered in the metrics labels, except for the `target` info metric. - -If there is no `job` label on a metric and the metrics are being scraped, -then the `job` label MUST be considered to be equal to the `job` label of the -scrape target. During scrape, the `job` label of the target may be equal to the scrape -job name or customized by relabel rules. - -If there is no `job` label on a metric and there is no `job` label for the -scrape target, the scrape MUST fail. Note that this is different from the -Prometheus behavior that still scrapes the metrics without `job`. - -If there is no `instance` label and the metrics are being scraped, the -`instance` label MUST be considered to be equal to the `instance` label of the -scrape target. Note that the `instance` label of the target may be equal to the -`__address__` of the target or customized by relabel rules. +In Prometheus metrics, the `job` and `instance` labels are used as join keys +between metrics and the `target` info metric. Therefore a new resource MUST be +created for each `job` and `instance` pair encountered in the metrics labels, +except for the `target` info metric. + +The `job` and `instance` labels are attached by Prometheus to each scraped +metric as described in [Jobs and instances](https://prometheus.io/docs/concepts/jobs_instances/). +If the `job` or `instance` label of a metric is missing or empty, for example +due to `metric_relabel_configs`, it MUST be set to the corresponding label of +the scrape target, after +[relabeling](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config). +If either label is still missing or empty, the scrape MUST fail. For each resource the `service.name` attribute MUST be set to the `service.name` label in `target` info metric, if that label exists in the From 264eda50c6eeabd66862c266d2905aec8ee83582 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Tue, 27 Oct 2026 09:24:44 +0100 Subject: [PATCH 12/19] Clarify which metrics receive target info resource attributes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- specification/compatibility/prometheus_and_openmetrics.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index cd615133c56..4730e20eb77 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -443,8 +443,8 @@ In addition to the attributes above, the info metric is used to supply additional resource attributes. If present, the `target` info metric MUST be dropped from the batch of metrics, and all labels except `job`, `instance` and `__name__` from the `target` info metric MUST be -converted to resource attributes attached to all other metrics which are part of -the same resource identified by `job` and `instance`. +converted to resource attributes attached to all other metrics which share the +same `job` and `instance` as the `target` info metric. By default, label keys and values of the `target` info metric MUST NOT be altered (such as replacing `_` with `.` characters in keys). From 7cdfcf6f19be6ab9c6017a5f6d238d7302567b7b Mon Sep 17 00:00:00 2001 From: George Krajcsovits Date: Mon, 2 Nov 2026 10:50:45 +0100 Subject: [PATCH 13/19] Apply suggestion from @aknuds1 Co-authored-by: Arve Knudsen --- specification/compatibility/prometheus_and_openmetrics.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 4730e20eb77..5ee87f2dd1d 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -378,6 +378,7 @@ following labels, which MUST NOT be copied: - `__name__`, - `job` and `instance`, +- labels with empty values, - the `le` label of classic Prometheus histogram buckets, - the `quantile` label of Prometheus summaries, - labels with the `otel_scope_` prefix, which are handled according to the From 713c587bae6eacb97813e7229d0cbc7ebe2e4f18 Mon Sep 17 00:00:00 2001 From: George Krajcsovits Date: Mon, 2 Nov 2026 10:51:17 +0100 Subject: [PATCH 14/19] Apply suggestion from @krajorama --- specification/compatibility/prometheus_and_openmetrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 5ee87f2dd1d..bcd3edb9591 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -403,7 +403,7 @@ except for the `target` info metric. The `job` and `instance` labels are attached by Prometheus to each scraped metric as described in [Jobs and instances](https://prometheus.io/docs/concepts/jobs_instances/). If the `job` or `instance` label of a metric is missing or empty, for example -due to `metric_relabel_configs`, it MUST be set to the corresponding label of +due to `metric_relabel_configs`, the missing label MUST be set to the corresponding label of the scrape target, after [relabeling](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config). If either label is still missing or empty, the scrape MUST fail. From 7a9daddc6c996372fd7b4d1fa825a71809ca9cf3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 2 Nov 2026 12:12:01 +0100 Subject: [PATCH 15/19] Split Prometheus to OTLP resource attributes into pull and push rules MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Move the target info rules to the common part of the Resource Attributes section and add separate subsections for scraped metrics and metrics received over Prometheus Remote Write. The push rules describe how service.name, service.namespace and service.instance.id are derived from the job and instance labels, matching the Prometheus Remote Write receiver behavior. Clarify that url.scheme is taken from the scrape target before relabeling. Signed-off-by: György Krajcsovits --- .../prometheus_and_openmetrics.md | 81 ++++++++++++++----- 1 file changed, 60 insertions(+), 21 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index bcd3edb9591..afe3af70168 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -31,6 +31,8 @@ aliases: * [Instrumentation Scope](#instrumentation-scope) * [Metric Labels](#metric-labels) * [Resource Attributes](#resource-attributes) + + [Scraping metrics (pull)](#scraping-metrics-pull) + + [Remote Write metrics (push)](#remote-write-metrics-push) - [OTLP Metric points to Prometheus](#otlp-metric-points-to-prometheus) * [Metric Metadata](#metric-metadata-1) * [Instrumentation Scope](#instrumentation-scope-1) @@ -391,15 +393,40 @@ metric samples are not turned into data points. **Status**: [Development](../document-status.md) -When scraping a Prometheus endpoint, resource attributes MUST be added to the -scraped metrics to distinguish them from metrics from other Prometheus -endpoints. - In Prometheus metrics, the `job` and `instance` labels are used as join keys between metrics and the `target` info metric. Therefore a new resource MUST be created for each `job` and `instance` pair encountered in the metrics labels, except for the `target` info metric. +The `service.name` and `service.instance.id` resource +attributes are needed to ensure Prometheus exporters can disambiguate metrics +using [`job` and `instance` labels](https://prometheus.io/docs/concepts/jobs_instances/) +as [described below](#resource-attributes-1), therefore this specification +aims to ensure that `service.name` and `service.instance.id` are defined +whenever `job` and `instance` are available. + +In addition to the attributes described below, the +[`target`](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) + info metric is used to supply additional resource attributes. If present, +the `target` info metric MUST be dropped from the batch of metrics, and all labels +except `job`, `instance` and `__name__` from the `target` info metric MUST be +converted to resource attributes attached to all other metrics which share the +same `job` and `instance` as the `target` info metric. + +Labels from the `target` info metric take precedence over all derived resource +attributes below. + +By default, label keys and values of the `target` info metric MUST NOT be +altered (such as replacing `_` with `.` characters in keys). + +#### Scraping metrics (pull) + +**Status**: [Development](../document-status.md) + +When scraping a Prometheus endpoint, resource attributes MUST be added to the +scraped metrics to distinguish them from metrics from other Prometheus +endpoints. + The `job` and `instance` labels are attached by Prometheus to each scraped metric as described in [Jobs and instances](https://prometheus.io/docs/concepts/jobs_instances/). If the `job` or `instance` label of a metric is missing or empty, for example @@ -427,28 +454,40 @@ attributes: | ----------------------- | ----------- | | `server.address` | The `` portion of the `instance` unless it is localhost, a loopback address, or an unspecified address | | `server.port` | The `` portion of the `instance` | -| `url.scheme` | `http` or `https` from the `__scheme__` label on the target | +| `url.scheme` | `http` or `https` from the `__scheme__` label on the scrape target before relabelling | Further attributes (such as from K8s service discovery metadata labels) MAY be -added to the resource attributes. Labels from the `target` info metric take -precedence over all derived resource attributes. +added to the resource attributes. -To summarize, `service.name` and `service.instance.id`, are needed -to ensure Prometheus exporters can disambiguate metrics using -[`job` and `instance` labels](https://prometheus.io/docs/concepts/jobs_instances/) -as [described below](#resource-attributes-1), therefore this specification -ensures that `service.name` and `service.instance.id` are always defined. +#### Remote Write metrics (push) -In addition to the attributes above, the -[`target`](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) - info metric is used to supply additional resource attributes. If present, -the `target` info metric MUST be dropped from the batch of metrics, and all labels -except `job`, `instance` and `__name__` from the `target` info metric MUST be -converted to resource attributes attached to all other metrics which share the -same `job` and `instance` as the `target` info metric. +**Status**: [Development](../document-status.md) -By default, label keys and values of the `target` info metric MUST NOT be -altered (such as replacing `_` with `.` characters in keys). +When receiving metrics over Prometheus Remote Write, a missing `job` or +`instance` label MUST be treated as an empty strings when creating a new +resource for the metrics. + +For each resource the `service.name` attribute MUST be set to the +`service.name` label in `target` info metric, if that label exists in the +`target` info metric with the same `job` and `instance` key as the resource. +Otherwise, if the `job` is not the empty string, then `service.name` MUST be +set from the `job` label as follows: + +- if `job` can be split into two strings along exactly one slash `/`, + then `service.name` MUST be set to the string after the `/`. +- otherwise `service.name` MUST be set to equal `job`. + +For each resource the `service.namespace` attribute MUST be set to the +`service.namespace` label in `target` info metric, if that label exists in the +`target` info metric with the same `job` and `instance` key as the resource. +Otherwise, if the `job` is not the empty string and has exactly one `/`, then +`service.namespace` MUST be set to the string before the `/`. + +For each resource the `service.instance.id` attribute MUST be set to the +`service.instance.id` label in `target` info metric, if that label exists in the +`target` info metric with the same `job` and `instance` key as the resource. +Otherwise, if `instance` is not the empty string, then `service.instance.id` +MUST be set to equal the `instance` label. ## OTLP Metric points to Prometheus From 3eed842762ae924ea6211d323f6eaff78e12d03b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 9 Nov 2026 08:50:10 +0100 Subject: [PATCH 16/19] move the changelog entry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- CHANGELOG.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 393b95fb9cc..efc50ad9ff3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -32,6 +32,10 @@ release. ### Compatibility +- Clarify the conversion of Prometheus labels and the `target` info metric to + OTLP resources, data point attributes and resource attributes. + ([#5356](https://github.com/open-telemetry/opentelemetry-specification/pull/5356)) + ### SDK Configuration ### Supplementary Guidelines @@ -98,9 +102,6 @@ release. ([#5213](https://github.com/open-telemetry/opentelemetry-specification/pull/5213)) - Stabilize Prometheus Info metric transformation. ([#5211](https://github.com/open-telemetry/opentelemetry-specification/pull/5211)) -- Clarify the conversion of Prometheus labels and the `target` info metric to - OTLP resources, data point attributes and resource attributes. - ([#5356](https://github.com/open-telemetry/opentelemetry-specification/pull/5356)) ### SDK Configuration From eebb6a89e437c4ffa08ce62c4eb9572adebaee0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 9 Nov 2026 09:03:12 +0100 Subject: [PATCH 17/19] update the server.address, server.port, url.scheme mapping MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Due to https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/52570 Signed-off-by: György Krajcsovits --- .../compatibility/prometheus_and_openmetrics.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index c700fda12e5..1aed3fa2cd2 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -447,14 +447,13 @@ Otherwise `service.instance.id` MUST be set to the value of the `instance` label. The following attributes SHOULD be associated with scraped metrics as resource -attributes, and MUST NOT be added as metric -attributes: +attributes, and MUST NOT be added as metric attributes: | OTLP Resource Attribute | Description | | ----------------------- | ----------- | -| `server.address` | The `` portion of the `instance` unless it is localhost, a loopback address, or an unspecified address | -| `server.port` | The `` portion of the `instance` | -| `url.scheme` | `http` or `https` from the `__scheme__` label on the scrape target before relabelling | +| `server.address` | The `` portion of the `__address__` label of the scrape target after relabeling, unless it is localhost, a loopback address, or an unspecified address | +| `server.port` | The `` portion of the `__address__` label of the scrape target after relabeling, if present | +| `url.scheme` | `http` or `https` from the `__scheme__` label of the scrape target after relabeling | Further attributes (such as from K8s service discovery metadata labels) MAY be added to the resource attributes. From 401d5cc7146cfd177933d2ef98a15d271c92f8dc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 9 Nov 2026 09:12:29 +0100 Subject: [PATCH 18/19] clarify text around empty string keys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- specification/compatibility/prometheus_and_openmetrics.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index 1aed3fa2cd2..f8692dde7f7 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -463,8 +463,9 @@ added to the resource attributes. **Status**: [Development](../document-status.md) When receiving metrics over Prometheus Remote Write, a missing `job` or -`instance` label MUST be treated as an empty strings when creating a new -resource for the metrics. +`instance` label MUST be treated as the empty string. Metrics MUST be grouped +into one resource per distinct `job` and `instance` pair, including pairs +where one or both values are the empty string. For each resource the `service.name` attribute MUST be set to the `service.name` label in `target` info metric, if that label exists in the From 6c6e8509e66e28574f8c2a6601aedce98e7e7fb8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Mon, 9 Nov 2026 09:16:34 +0100 Subject: [PATCH 19/19] Update the changelog to include the change as well explicitly. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- CHANGELOG.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2516d2d32b4..9254c008446 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,7 +62,13 @@ release. Prometheus `target` info metric along with all other resource attributes. ([#5368](https://github.com/open-telemetry/opentelemetry-specification/pull/5368)) - Clarify the conversion of Prometheus labels and the `target` info metric to - OTLP resources, data point attributes and resource attributes. + OTLP resources, data point attributes and resource attributes, and the + differences between scrape and Remote Write. + ([#5356](https://github.com/open-telemetry/opentelemetry-specification/pull/5356)) +- Clarify that `server.address`, `server.port` and `url.scheme` of scraped + metrics are derived from the scrape target's `__address__` and `__scheme__` + labels after relabeling, and that `server.address` is omitted for localhost, + loopback and unspecified addresses. ([#5356](https://github.com/open-telemetry/opentelemetry-specification/pull/5356)) ### SDK Configuration