Iceberg unit tests, support Iceberg + nonhive catalogs, Iceberg Kryo Serializer#993

Merged

pengyu-hou merged 40 commits intoairbnb:mainfrom

abbywh:iceberg_unit_tests

Jun 10, 2025

Commits on May 13, 2025

staging changes for testing iceberg
Abby Whittier
committed
timeboxing test changes
Abby Whittier
committed

Commits on May 14, 2025

bootstrapping spark test, 1/3 working on FormatTest
Abby Whittier
committed

Commits on May 15, 2025

got most tests working except droppartitions and the new dateint test
Abby Whittier
committed
cleaning some local changes
Abby Whittier
committed
reverting some local changes
Abby Whittier
committed
formatting
Abby Whittier
committed
more silly local hacks
Abby Whittier
committed
Merge branch 'main' into iceberg_unit_tests
abbywh
authored

Commits on May 16, 2025

fixed the constant derby flake
Abby Whittier
committed
refactored to match deltalake
Abby Whittier
committed
added Iceberg Kryo Serializer
Abby Whittier
committed
scalafmt
Abby Whittier
committed
iceberg circleci integration
Abby Whittier
committed
fixing typo
Abby Whittier
committed
giving circleci a dependency
Abby Whittier
committed
removing env file
Abby Whittier
committed
moving integration test to spark_embedded
Abby Whittier
committed
figured out why delta lake was on 2.13, need it for spark 3.2
Abby Whittier
committed
typo
Abby Whittier
committed
scalafmt
Abby Whittier
committed
skipping the flink parts since it doesn't compile to 2.13.6
Abby Whittier
committed
including TableUtilsTest as well in CI
Abby Whittier
committed
sperating table utils and format for seperate jvms
Abby Whittier
committed
typo
Abby Whittier
committed
corrected behavior for long partitions
Abby Whittier
committed
Merge branch 'main' into iceberg_unit_tests
abbywh
authored

Commits on May 17, 2025

eventeventlongds test, more kryo registration
Abby Whittier
committed
iceberg drop partitions
Abby Whittier
committed
long partition testing
Abby Whittier
committed
Merge branch 'main' into iceberg_unit_tests
abbywh
authored

Commits on May 19, 2025

unskipping fixed tests
Abby Whittier
committed
changing test schema
Abby Whittier
committed
updating drop partitions to be schemaless
Abby Whittier
committed

Commits on May 21, 2025

found bug during CI testing
Abby Whittier
committed

Commits on Jun 7, 2025

Apply suggestions from code review

abbywh
and
pengyu-hou
authored
formatting
Abby Whittier
committed
propping name refactor
Abby Whittier
committed
fixing some typos
Abby Whittier
committed

Commits on Jun 8, 2025

Merge branch 'main' into iceberg_unit_tests
abbywh
authored