feat: add runtime per-base store params for dataset reads#6422
Open
feat: add runtime per-base store params for dataset reads#6422
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
westonpace
approved these changes
Apr 8, 2026
Member
westonpace
left a comment
There was a problem hiding this comment.
This seems to be ok as is but I have a few questions.
Is it possible to set an option for the default base? Or would you need to override that option in every base?
Can you mask an option to hide it from the bases? For example, maybe the default storage options has a proxy_url but you don't want to use that in the non-default bases?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds runtime-only per-base object store params for dataset reads, keyed by
BasePath.path, so callers can open different registered bases with different storage options without extendingBasePathin the manifest format. The builder now normalizes per-base storage options intoObjectStoreParamsonce at load time, andDataset::object_store_for_baseuses those params when resolving stores.This follows the current direction from discussion #6307 to keep provider-specific configuration in runtime state instead of the table format.