All pull requests

Pull requests

• feat: write top-level Iceberg unknown columns natively

  area:Icebergarea:writerenhancementrun-all-spark-profiles
  #7100·yunchipang opened Oct 18, 2026·

  ·
  8

• fix: handle NullType struct fields in JVM columnar shuffle

  area:shufflebug
  #7099·jadenfix opened Oct 17, 2026·

  ·
  1

• fix: point the spill limit error at spark.comet.maxTempDirectorySize

  area:ffibug
  #7098·ump45nose opened Oct 17, 2026·

  ·
  1

• fix: make fair_unified's per-consumer limit bind when tasks run concurrently

  area:Deltaarea:Icebergarea:memoryarea:writerbugrun-all-spark-profiles
  #7096·andygrove opened Oct 17, 2026Member·

  ·
  2

• fix: scope fallback reasons to the current conversion

  area:expressionsbug
  #7095·sam-1112 opened Oct 17, 2026Contributor·

  ·
  1

• perf: load the next Parquet file's footer and page index while the current file is read

  area:scanenhancementperformance
  #7094·andygrove opened Oct 17, 2026Member·

  ·
  9

• docs: consolidate Parquet and Iceberg documentation into one page each

  area:Icebergdocumentation
  #7090·andygrove opened Oct 17, 2026Member·

  ·
  9

• fix: reject TIMESTAMP(NANOS) Parquet columns read as timestamp

  area:Icebergarea:scanbug
  #7089·ump45nose opened Oct 17, 2026·

  ·

• fix: match Spark nullness for missing nested Parquet fields

  area:Icebergarea:scanbugrun-iceberg-testsrun-spark-3.5-testsrun-spark-4.1-tests
  #7085·rich7420 opened Oct 17, 2026Contributor·

  ·
  1

  1

• fix: hash NullType and ANSI interval values natively

  area:expressionsbug
  #7084·ump45nose opened Oct 17, 2026·

  ·
  1

  1

• perf: share Parquet footer and page index metadata across the tasks of a stage

  area:scanenhancementperformancerun-spark-4.1-tests
  #7082·andygrove opened Oct 16, 2026Member·

  ·
  4

• fix: preserve Parquet root field IDs after nested pruning

  area:scanbugrun-all-spark-profiles
  #7081·LinSimon-901101 opened Oct 16, 2026Contributor·

  ·
  1

  1

• perf: create a native plan's spill directories on its first spill

  enhancementperformancerun-spark-4.1-tests
  #7066·andygrove opened Oct 16, 2026Member·

  ·
  1

  19

• perf: write every partition of a JVM shuffle spill in one native call

  area:memoryarea:shuffleenhancementperformancerun-benchmark-checkrun-spark-4.1-tests
  #7065·andygrove opened Oct 16, 2026Member·

  ·

  · 1.2.0
  1

  4

• perf: start bypass shuffle partition writers on small pages that grow

  area:shuffleenhancementperformancerun-spark-4.1-tests
  #7059·andygrove opened Oct 16, 2026Member·

  ·

  · 1.2.0
  1

  2

• feat: spilling shuffled hash join (research draft)

  area:joinsenhancementrun-spark-4.1-tests
  #7057·andygrove opened Oct 16, 2026Member·

  ·

• refactor: share filter and projection metrics wrapper

  enhancement
  #7056·rich7420 opened Oct 16, 2026Contributor·

  ·
  1

• test: re-enable the NullType Parquet read tests that #4199 skipped

  enhancementrun-spark-4.1-testsrun-spark-4.2-teststest
  #7054·andygrove opened Oct 16, 2026Member·

  ·

• fix: write the Spark version into the footer of natively written Parquet files

  area:writerbug
  #7053·andygrove opened Oct 16, 2026Member·

  ·

  · 1.2.0
  1

  7

• build(deps): bump actions/setup-python from 6 to 7

  area:cidependencies
  #7046·dependabot[bot] opened Oct 16, 2026Bot·

  ·

• fix: fall back to Spark when Iceberg reports an output ordering

  area:Icebergarea:scanbug
  #7043·parthchandra opened Oct 15, 2026Contributor·

  ·
  6

• fix: keep each native panic's backtrace with its own thread

  area:ffibug
  #7042·viirya opened Oct 15, 2026Member·

  ·
  1

  3

• feat: bump iceberg-rust past apache/iceberg-rust#3255 and narrow the nested schema evolution fallback

  area:Icebergarea:scanarea:writerenhancement
  #7028·andygrove opened Oct 15, 2026Member·

  ·

  · 1.2.0
  2

• fix: fall back for Iceberg write settings that iceberg-java rejects

  area:Icebergarea:writerbug
  #7027·andygrove opened Oct 15, 2026Member·

  ·
  1

• fix: decline native scans whose S3 configuration diverges from Hadoop's

  area:scanbug
  #7026·kazuyukitanimura opened Oct 15, 2026Contributor·

  ·
  2