Pluggable page spilling API for the Parquet ArrowWriter (PageStore)#10020

Open

adriangb wants to merge 7 commits into

apache:mainfrom

pydantic:parquet-page-spill

Commits on Jun 2, 2026

parquet: add pluggable PageStore for ArrowWriter page buffering

adriangb
and
claude
committed
parquet: stream column chunks out of the PageStore at splice

adriangb
and
claude
committed
parquet: make PageKey constructible + prove the memory bound with a spilling backend

adriangb
and
claude
committed
parquet: spill dictionary-column data pages too (close accumulation point #2)

adriangb
and
claude
committed
parquet: gate page_spill_memory integration test on the arrow feature

adriangb
and
claude
committed
parquet: drop dhat from the page-spill memory test

adriangb
and
claude
committed
Update parquet/src/column/page_store.rs

adriangb
and
alamb
authored