Skip to content

fix: preserve requested DBN metadata version and align v3 output - #20

Open
vbmithr wants to merge 1 commit into
NimbleMarkets:mainfrom
deepmarker:fix/metadata-v3-writer
Open

vbmithr wants to merge 1 commit into
NimbleMarkets:mainfrom
deepmarker:fix/metadata-v3-writer

Conversation

@vbmithr

@vbmithr vbmithr commented Oct 8, 2026

Copy link
Copy Markdown

Problem

Metadata.Write routes every non-v1 version to writeV2, which hardcodes the DBN\x02 prefix. Asking for v3 therefore silently produces v2 metadata, including when the following records use v3 layouts (notably statistics). Unsupported versions are silently downgraded too.

Change

  • Preserve the requested version for v1, v2 and v3; reject unsupported versions before writing.
  • Add the trailing metadata padding required for v3's eight-byte alignment, and include it in the encoded metadata length.
  • Preserve existing v1/v2 layouts and document that record layouts must match the metadata version.
  • Add tests for all three supported versions, all eight v3 alignment residues, the first record boundary, and invalid versions.

Validation

  • GOWORK=off go test . ./live passed, including the existing Ginkgo suite.
  • Application interoperability tests passed with Databento's dbn CLI 0.71.0 for trades, MBO, MBP1, MBP10, OHLCV and statistics. The statistics fixture uses a quantity above 32 bits to exercise the v3 layout.
  • V3 version bytes, declared metadata lengths and record decoding were checked independently of dbn-go's scanner.

The alignment follows Databento's encoder.

Write the requested v1, v2 or v3 metadata version and reject unsupported
versions before emitting bytes. Pad v3 metadata to an eight-byte
boundary without changing the v1/v2 layouts. Add version, alignment and
record boundary regressions and document version-matched records. Core
and live package tests pass; Databento’s CLI decodes v3 output across
six schemas.

Co-Authored-By: GPT-6 <noreply@openai.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant