feat(metrics): add optional id and properties fields to metrics output by not4s · Pull Request #5983 · firecracker-microvm/firecracker

not4s · 2026-06-19T16:32:10Z

Changes

Add two optional, independent fields to the metrics configuration (PUT /metrics and the metrics block of a config file), each emitted as a top-level key on every metrics line and each off by default:

emit_id (boolean): emits the microVM instance id (the value passed to the jailer --id) under a top-level id field.
properties (map of string to string): emits operator-defined key-value pairs under a top-level properties field.

When neither is configured, the metrics line is byte-for-byte unchanged, so existing consumers are unaffected.

Example line with both enabled:

{"utc_timestamp_ms": 1739000000000, "id": "my-instance",
"properties": {"bundle_id": "fn-abc", "customer_id": "1234"},
"api_server": {"...": 0}}

Reason

Operators that collect metrics from many microVMs into a single destination (for example, a log agent that globs every VM's metrics file into one log stream) cannot tell which microVM a given line came from, because the lines carry no identifier. The original feature request asked for arbitrary key-value tagging to attribute lines to a VM; subsequent discussion clarified that the primary need is simply a stable, authoritative identifier, with operator tags as an enrichment.

Design decisions

id as a separate top-level field, not a key inside properties. An earlier idea was to auto-populate the instance id inside the properties map. A separate id field was chosen instead, for several reasons:
- The instance id is owned and known by Firecracker; the properties are arbitrary data supplied by operators. Keeping them separate preserves the distinction and avoids a reserved-key collision if an operator also wants a key named id.
- An operator can want only the id, only custom tags, or both.
- Consumers querying by VM identity get a top-level field.
Two independent opt-in switches: Both default to off so the default line shape is unchanged (backwards compatibility). emit_id is a boolean toggle modeled on the existing LoggerConfig show_level / show_log_origin toggles.
No CLI flag for properties: Structured data is supplied through the API or a config file, not the command line. This matches convention that CLI arguments are scalars, paths, and booleans, with structured input provided by file (as MMDS content already does). --metrics-path continues to configure only the path.
Omitted when unset, not emitted empty: When a field is not configured its key is absent entirely.
Snapshot behavior unchanged: Metrics configuration is not part of the snapshot and is reconfigured on a restored microVM; this change adds nothing to the snapshot surface.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

This functionality cannot be added in rust-vmm.

codecov · 2026-06-19T16:37:17Z

Codecov Report

❌ Patch coverage is 92.45283% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.12%. Comparing base (e04e55f) to head (bd6f9f0).

Files with missing lines	Patch %	Lines
src/firecracker/src/main.rs	0.00%	2 Missing ⚠️
src/vmm/src/logger/metrics.rs	97.50%	1 Missing ⚠️
src/vmm/src/vmm_config/metrics.rs	90.90%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5983      +/-   ##
==========================================
+ Coverage   83.11%   83.12%   +0.01%     
==========================================
  Files         277      277              
  Lines       30207    30259      +52     
==========================================
+ Hits        25106    25154      +48     
- Misses       5101     5105       +4

Flag	Coverage Δ
5.10-m5n.metal	`83.42% <92.45%> (+0.01%)`	⬆️
5.10-m6a.metal	`82.78% <92.45%> (+0.01%)`	⬆️
5.10-m6g.metal	`80.11% <92.45%> (+0.02%)`	⬆️
5.10-m6i.metal	`83.43% <92.45%> (+0.01%)`	⬆️
5.10-m7a.metal-48xl	`82.77% <92.45%> (+0.01%)`	⬆️
5.10-m7g.metal	`80.11% <92.45%> (+0.02%)`	⬆️
5.10-m7i.metal-24xl	`83.40% <92.45%> (+0.01%)`	⬆️
5.10-m7i.metal-48xl	`83.39% <92.45%> (+<0.01%)`	⬆️
5.10-m8g.metal-24xl	`80.11% <92.45%> (+0.02%)`	⬆️
5.10-m8g.metal-48xl	`80.11% <92.45%> (+0.02%)`	⬆️
5.10-m8i.metal-48xl	`83.40% <92.45%> (+0.01%)`	⬆️
5.10-m8i.metal-96xl	`83.40% <92.45%> (+0.01%)`	⬆️
6.1-m5n.metal	`83.45% <92.45%> (+0.01%)`	⬆️
6.1-m6a.metal	`82.81% <92.45%> (+0.02%)`	⬆️
6.1-m6g.metal	`80.11% <92.45%> (+0.02%)`	⬆️
6.1-m6i.metal	`83.45% <92.45%> (+0.01%)`	⬆️
6.1-m7a.metal-48xl	`82.80% <92.45%> (+0.01%)`	⬆️
6.1-m7g.metal	`80.11% <92.45%> (+0.02%)`	⬆️
6.1-m7i.metal-24xl	`83.46% <92.45%> (+0.02%)`	⬆️
6.1-m7i.metal-48xl	`83.47% <92.45%> (+0.02%)`	⬆️
6.1-m8g.metal-24xl	`80.10% <92.45%> (+0.01%)`	⬆️
6.1-m8g.metal-48xl	`80.11% <92.45%> (+0.02%)`	⬆️
6.1-m8i.metal-48xl	`83.47% <92.45%> (+0.01%)`	⬆️
6.1-m8i.metal-96xl	`83.46% <92.45%> (+0.01%)`	⬆️
6.18-m5n.metal	`83.45% <92.45%> (+0.01%)`	⬆️
6.18-m6a.metal	`82.81% <92.45%> (+0.01%)`	⬆️
6.18-m6g.metal	`80.11% <92.45%> (+0.02%)`	⬆️
6.18-m6i.metal	`83.45% <92.45%> (+0.01%)`	⬆️
6.18-m7a.metal-48xl	`82.80% <92.45%> (+0.01%)`	⬆️
6.18-m7g.metal	`80.11% <92.45%> (+0.02%)`	⬆️
6.18-m7i.metal-24xl	`83.46% <92.45%> (+<0.01%)`	⬆️
6.18-m7i.metal-48xl	`83.46% <92.45%> (+0.01%)`	⬆️
6.18-m8g.metal-24xl	`80.11% <92.45%> (+0.02%)`	⬆️
6.18-m8g.metal-48xl	`80.11% <92.45%> (+0.02%)`	⬆️
6.18-m8i.metal-48xl	`83.46% <92.45%> (+0.01%)`	⬆️
6.18-m8i.metal-96xl	`83.47% <92.45%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ShadowCurse · 2026-07-01T10:35:15Z

+/// Maximum number of metrics properties that can be configured.
+const MAX_PROPERTIES: usize = 10;
+/// Maximum length (in bytes) of a metrics property key.
+const MAX_KEY_LEN: usize = 64;
+/// Maximum length (in bytes) of a metrics property value.
+const MAX_VALUE_LEN: usize = 512;


Maybe I don't remember, but was there a decision to put these restrictions? imho these look very arbitrary and I don't see a lot of value in them.

I agree that they are a bit arbitrary, but have communicated with mancio and Nemo on this. Would you say even the cap on the number of properties doesn't have a lot of value? Although the strength for the case is a bit different, I do see that there are cases where we validate input from operator input such as MAX_SUPPORTED_VCPUS or CMDLINE_MAX_SIZE

ShadowCurse · 2026-07-01T10:42:44Z

 /// Configures the metrics as described in `metrics_cfg`.
 pub fn init_metrics(metrics_cfg: MetricsConfig) -> Result<(), MetricsConfigError> {
+    if let Some(properties) = &metrics_cfg.properties {
+        validate_properties(properties)?;


you can move this validate to the lower if check, moves all properties code in one place

ShadowCurse · 2026-07-01T11:13:50Z

+    /// Enables emission of the `id` field. Returns an error if already enabled.
+    pub fn enable(&self) -> Result<(), MetricsError> {
+        if self.enabled.swap(true, Ordering::Relaxed) {
+            return Err(MetricsError::AlreadyInitialized);


Is this really an error to enable id multiple times?

You're right, technically speaking this is an AtomicBool not a OnceLock so isn't really an error. But was thinking we might want to be consistent with the other types since the AtomicBool here is intended to be set once.

Allow operators to attach custom key-value properties to the metrics config, via PUT /metrics or the boot config file. When set, the properties are emitted under a top-level "properties" field on every metrics line; when omitted, output is unchanged for backwards compatibility. Properties are set once at config time (pre-boot) and fixed for the VM's lifetime, matching how metrics are configured today. They are bounded (max 10 keys, 64-byte keys, 512-byte values) to keep metrics lines from bloating. This lets operators tag a microVM's metrics with their own context, such as the customer or bundle running in it. Signed-off-by: Jay Chung <jaehoc@amazon.com>

The metrics-line validator builds a strict jsonschema (additionalProperties: False), so the optional top-level "properties" map emitted by configurable metrics properties would be rejected as an unexpected key. Register "properties" in the schema as an optional object with string values, without marking it required. Real metrics stay strictly validated; the key is tolerated when present and ignored when absent. Signed-off-by: Jay Chung <jaehoc@amazon.com>

Add an "Optional metrics fields" section describing the operator-defined `properties` map: how to set it via the API or a config file, its size bounds, and example output. Also document it in the swagger Metrics definition. Signed-off-by: Jay Chung <jaehoc@amazon.com>

Operators shipping metrics from many microVMs to a shared sink (e.g. one CloudWatch log group) cannot tell which VM a line came from, since lines carry no identifier. Add an opt-in `emit_id` field to the metrics configuration that emits the microVM instance id (the jailer `--id`) under a top-level `id` field on every metrics line. Keep it independent of the custom `properties` map: either can be enabled without the other. Both default to off, so the line shape is unchanged for existing consumers. Signed-off-by: Jay Chung <jaehoc@amazon.com>

Tolerate the top-level `id` field as an optional string in the metrics schema validator, mirroring the existing `properties` tolerance, while keeping every real metric strictly validated. Signed-off-by: Jay Chung <jaehoc@amazon.com>

Add the `id` field to the "Optional metrics fields" docs and the swagger Metrics definition: how `emit_id` enables it, and example output showing it alongside the properties. Signed-off-by: Jay Chung <jaehoc@amazon.com>

Record the two new opt-in metrics fields, `emit_id` and `properties`, under the Unreleased section as a single entry for the PR. Signed-off-by: Jay Chung <jaehoc@amazon.com>

Cover the emitted metrics line shape, which the unit tests cannot reach since they exercise the serializer in isolation rather than a running microVM. Assert each combination of the two opt-in fields: neither set, `emit_id` alone, `properties` alone, and both together, checking that the `id` and `properties` keys appear (or not) and carry the configured values. Signed-off-by: Jay Chung <jaehoc@amazon.com>

not4s force-pushed the feat/metrics-properties branch from 8cdc240 to e5233da Compare June 19, 2026 16:32

not4s force-pushed the feat/metrics-properties branch 2 times, most recently from aaebe7b to dee5388 Compare June 22, 2026 01:16

not4s added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Jun 22, 2026

not4s marked this pull request as ready for review June 22, 2026 09:21

JackThomson2 reviewed Jun 29, 2026

View reviewed changes

Comment thread src/vmm/src/logger/metrics.rs Outdated

JackThomson2 reviewed Jun 29, 2026

View reviewed changes

Comment thread src/vmm/src/logger/metrics.rs Outdated

not4s force-pushed the feat/metrics-properties branch from dee5388 to c5ba4ef Compare June 29, 2026 16:30

JamesC1305 reviewed Jun 29, 2026

View reviewed changes

Comment thread src/vmm/src/logger/metrics.rs

Comment thread docs/metrics.md Outdated

Comment thread docs/metrics.md

Comment thread src/vmm/src/logger/metrics.rs Outdated

Comment thread docs/metrics.md

Comment thread src/vmm/src/logger/metrics.rs

not4s force-pushed the feat/metrics-properties branch 2 times, most recently from 4afe981 to ec7e4cc Compare June 30, 2026 15:00

not4s requested a review from JackThomson2 June 30, 2026 15:30

ShadowCurse reviewed Jul 1, 2026

View reviewed changes

not4s force-pushed the feat/metrics-properties branch 2 times, most recently from b45143b to f5cdd70 Compare July 1, 2026 14:50

not4s added 8 commits July 1, 2026 16:03

test(metrics): allow optional id in validation

5dfcc20

Tolerate the top-level `id` field as an optional string in the metrics schema validator, mirroring the existing `properties` tolerance, while keeping every real metric strictly validated. Signed-off-by: Jay Chung <jaehoc@amazon.com>

docs(metrics): document the metrics id field

28cfdbf

Add the `id` field to the "Optional metrics fields" docs and the swagger Metrics definition: how `emit_id` enables it, and example output showing it alongside the properties. Signed-off-by: Jay Chung <jaehoc@amazon.com>

docs(changelog): add entries for metrics id and properties

bee958b

Record the two new opt-in metrics fields, `emit_id` and `properties`, under the Unreleased section as a single entry for the PR. Signed-off-by: Jay Chung <jaehoc@amazon.com>

not4s force-pushed the feat/metrics-properties branch from f5cdd70 to bd6f9f0 Compare July 1, 2026 15:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(metrics): add optional id and properties fields to metrics output#5983

feat(metrics): add optional id and properties fields to metrics output#5983
not4s wants to merge 8 commits into
firecracker-microvm:mainfrom
not4s:feat/metrics-properties

not4s commented Jun 19, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jun 19, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ShadowCurse Jul 1, 2026

Uh oh!

not4s Jul 1, 2026 •

edited

Loading

Uh oh!

ShadowCurse Jul 1, 2026

Uh oh!

ShadowCurse Jul 1, 2026

Uh oh!

not4s Jul 1, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

not4s commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Reason

Design decisions

License Acceptance

PR Checklist

Uh oh!

codecov Bot commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ShadowCurse Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

not4s Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShadowCurse Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

ShadowCurse Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

not4s Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

not4s commented Jun 19, 2026 •

edited

Loading

codecov Bot commented Jun 19, 2026 •

edited

Loading

not4s Jul 1, 2026 •

edited

Loading

not4s Jul 1, 2026 •

edited

Loading