Skip to content

Core: Enforce that v4 manifests do not contain POSITION_DELETES#16386

Closed
yadavay-amzn wants to merge 1 commit into
apache:mainfrom
yadavay-amzn:fix/16340-v4-no-position-deletes
Closed

Core: Enforce that v4 manifests do not contain POSITION_DELETES#16386
yadavay-amzn wants to merge 1 commit into
apache:mainfrom
yadavay-amzn:fix/16340-v4-no-position-deletes

Conversation

@yadavay-amzn
Copy link
Copy Markdown
Contributor

Fixes #16340.

Follow-up from PR #16100 discussion.

Problem

The Variant v4 format uses deletion vectors instead of positional delete files. TrackedFileStruct currently accepts POSITION_DELETES content type without validation, and V4Metadata includes fields (REFERENCED_DATA_FILE, CONTENT_OFFSET, CONTENT_SIZE) that are only relevant to positional deletes.

Fix

  1. TrackedFileStruct: Add validation in internalSet to reject FileContent.POSITION_DELETES with a clear error message.
  2. V4Metadata: Remove positional-delete-specific fields from the file type schema and remove POSITION_DELETES handling from DataFileWrapper.get().

Testing

  • Added testPositionDeletesRejected verifying IllegalArgumentException when attempting to set POSITION_DELETES content type.
  • Updated testAllFileContentTypesSupported to skip POSITION_DELETES for v4.
  • All existing manifest tests pass.

@github-actions github-actions Bot added the core label May 17, 2026
@yadavay-amzn
Copy link
Copy Markdown
Contributor Author

Closing -- the issue was resolved separately. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enforce that v4 manifests do not contain POSITION_DELETES

1 participant