Skip to content

fix(etl): persist DDEX rights metadata on track create - #517

Merged
rickyrombo merged 1 commit into
mainfrom
mjp-etl-track-create-ddex-fields
Aug 17, 2026
Merged

fix(etl): persist DDEX rights metadata on track create#517
rickyrombo merged 1 commit into
mainfrom
mjp-etl-track-create-ddex-fields

Conversation

@rickyrombo

Copy link
Copy Markdown
Contributor

What

mergeTrackFromMetadata (the update path) has read these eight fields for a
while. The create INSERT never listed them:

artists, resource_contributors, indirect_resource_contributors,
rights_controller, copyright_line, producer_copyright_line,
parental_warning_type, is_original_available

A track that arrives complete in a single Create silently loses all eight.

Why the migration hits it every time

Every genesis-migration track is exactly that shape — one Create, never
updated. The writer selects and emits these fields
(cmd/genesis-writer/entities_track.go), the transaction carries them, and the
row lands with NULLs.

Verified end to end on track 1810289900:

source tracks.rights_controller     {"name": "Michael Kumar", "roles": ["RightsController"]}
migration tx data.rights_controller {"name": "Michael Kumar", "roles": ["RightsController"]}   <- writer did its job
ETL tracks.rights_controller        NULL                                                      <- dropped here

Measured loss (2026-08-07 snapshot)

column rows lost
is_original_available 28,827
producer_copyright_line 1,459
parental_warning_type 1,459
artists 1,447
rights_controller 1,404
indirect_resource_contributors 782
copyright_line 782
resource_contributors 686

Test

TestTrackCreate_PersistsDdexRightsMetadata uses metadata shaped as
genesis-writer emits it. Reverting the fix fails it on all four assertions:

artists is empty; the create INSERT dropped it
rights_controller is empty; the create INSERT dropped it
parental_warning_type = <nil>, want "Explicit"
is_original_available = false, want true

🤖 Generated with Claude Code

mergeTrackFromMetadata (the update path) has read artists,
resource_contributors, indirect_resource_contributors, rights_controller,
copyright_line, producer_copyright_line, parental_warning_type and
is_original_available for a while. The create INSERT never listed them, so a
track that arrives complete in a single Create silently loses all eight.

That is exactly the shape of every genesis-migration track: the writer selects
and emits these fields (cmd/genesis-writer/entities_track.go), the transaction
carries them, and the row lands with NULLs. Verified end to end on track
1810289900 -- data.rights_controller = {"name":"Michael Kumar",...} in the
migration tx, NULL in the tracks table.

Measured against the 2026-08-07 snapshot, the migration dropped:

  artists                          1,447
  rights_controller                1,404
  producer_copyright_line          1,459
  parental_warning_type            1,459
  is_original_available           28,827
  indirect_resource_contributors     782
  copyright_line                     782
  resource_contributors              686

The new test fails without this change on all four assertions.
@rickyrombo
rickyrombo merged commit 88b83b3 into main Aug 17, 2026
5 checks passed
@rickyrombo
rickyrombo deleted the mjp-etl-track-create-ddex-fields branch August 17, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant