Issues

• Discover Hive partitions like Spark: allow partition columns stored in the files, and infer their types

  enhancement
  apache/datafusion-ballista#2555 · Filter by author andygrove opened on Oct 7, 2026

• ShuffleWriterExec doesn't compact Utf8View columns, so small results can exceed the 16 MiB gRPC limit

  bug
  apache/datafusion-ballista#2554 · Filter by author andygrove opened on Oct 7, 2026

• Scan tasks are skewed when Parquet statistics prune most files: file groups are built before pruning

  apache/datafusion-ballista#2551 · Filter by author andygrove opened on Oct 6, 2026

• issues found when publishing crates for 55.0.0

  bug
  apache/datafusion-ballista#2550 · Filter by author andygrove opened on Oct 6, 2026
  1

• Job hangs after losing one executor with AQE: stale tasks can't be cancelled in pull-staged mode, then "Can't find active stage to update stage outputs"

  apache/datafusion-ballista#2539 · Filter by author andygrove opened on Oct 4, 2026

• Executor keeps a session's first S3 settings after SET s3.* changes them

  bug
  apache/datafusion-ballista#2533 · Filter by author andygrove opened on Oct 4, 2026

• Proposal: pluggable S3 credential providers

  enhancement
  apache/datafusion-ballista#2532 · Filter by author andygrove opened on Oct 4, 2026
  

• Adopt DataFusion MemoryDriftTracker after upgrading DataFusion

  apache/datafusion-ballista#2531 · Filter by author andygrove opened on Oct 3, 2026

• Support rolling upgrades without an outage window

  enhancement
  apache/datafusion-ballista#2525 · Filter by author andygrove opened on Oct 2, 2026
  1

• Sort-shuffle writer compacts view columns even when they're already dense

  apache/datafusion-ballista#2521 · Filter by author NoahKusaba opened on Oct 1, 2026

• Performance review: shuffle, executor, scheduler and client findings (tracking)

  apache/datafusion-ballista#2520 · Filter by author NoahKusaba opened on Oct 1, 2026

• Coalesce small batches in the passthrough shuffle writer

  apache/datafusion-ballista#2517 · Filter by author NoahKusaba opened on Oct 1, 2026

• perf: shuffle writers use one blocking thread per output partition and compress outside the vcore budget

  apache/datafusion-ballista#2509 · Filter by author NoahKusaba opened on Sep 29, 2026
  1

  

• Discussion: move the expensive CI jobs to a merge queue, like Comet

  development-process
  apache/datafusion-ballista#2500 · Filter by author andygrove opened on Sep 27, 2026

• Proposal: Decoupled Result Service for Ballista

  apache/datafusion-ballista#2484 · Filter by author villebro opened on Sep 23, 2026
  1

• AQE replan can swap a broadcast exchange onto the probe side of a CollectLeft join, pinning the join stage to one task

  apache/datafusion-ballista#2454 · Filter by author NoahKusaba opened on Sep 14, 2026
  1

  

• Design note: composing multi-partition tasks with plan-level AQE in DataFusion

  apache/datafusion-ballista#2445 · Filter by author NoahKusaba opened on Sep 12, 2026

• Emit the final (post-AQE) physical plan to the event log

  apache/datafusion-ballista#2422 · Filter by author andygrove opened on Sep 3, 2026

• CI: Python client tests do not run on Rust changes and do not test the working tree

  enhancement
  apache/datafusion-ballista#2372 · Filter by author andygrove opened on Aug 25, 2026

• Python client silently forces the static planner (pinned to ballista 54.0.0 defaults)

  bug
  apache/datafusion-ballista#2371 · Filter by author andygrove opened on Aug 25, 2026
  4

• TPCH benchmark fails with Failed to open partition file at "/var/folders/..../data.arrow"

  bug
  apache/datafusion-ballista#2367 · Filter by author martin-g opened on Aug 25, 2026
  3

  

• Graceful executor shutdown terminates active Flight streams

  apache/datafusion-ballista#2357 · Filter by author DanilSerd opened on Aug 23, 2026

• [EPIC] Add public APIs required for scheduler high availability

  apache/datafusion-ballista#2349 · Filter by author phillipleblanc opened on Aug 20, 2026
  4

• Document HA scheduler

  documentationenhancement
  apache/datafusion-ballista#2347 · Filter by author milenkovicm opened on Aug 19, 2026
  2

• Allow a stage to start on partially complete inputs instead of waiting for the whole upstream stage

  enhancementperformance
  apache/datafusion-ballista#2320 · Filter by author andygrove opened on Aug 16, 2026