Commits

Commit history

1. Commits on Oct 16, 2026

   • [SPARK-60238][PYTHON] Migrate stateful streaming UDF eval types to the EvalTypeHandler pipeline

     Show description for f39493b
     Yicong-Huang

     committed
     f39493bView commit details

     Copy full SHA for f39493b

     View code at this pointBrowse repository at this point
2. Commits on Oct 14, 2026

   • [SPARK-60220][PYTHON] Add ASV benchmarks for SQL_TABLE_UDF eval type

     Show description for bcc9019
     Yicong-Huang

     committed
     bcc9019View commit details

     Copy full SHA for bcc9019

     View code at this pointBrowse repository at this point
3. Commits on Sep 24, 2026

   • [SPARK-59755][PYTHON] Add ASV benchmarks for Arrow incremental aggregate and window UDF eval types

     Show description for c7080fc
     Yicong-Huang

     committed
     c7080fcView commit details

     Copy full SHA for c7080fc

     View code at this pointBrowse repository at this point
4. Commits on Sep 18, 2026

   • [SPARK-59455][PYTHON][FOLLOWUP] Clarify private Arrow-to-pandas array helpers

     Show description for 92033bf
     

     Spenserrrr

     authored and

     zhengruifeng

     committed
     92033bfView commit details

     Copy full SHA for 92033bf

     View code at this pointBrowse repository at this point
5. Commits on Sep 14, 2026

   • [SPARK-59455][PYTHON] Move Arrow-to-pandas batch conversion into ArrowToPandasConversion

     Show description for 3033357
     

     Spenserrrr

     authored and

     Yicong-Huang

     committed
     3033357View commit details

     Copy full SHA for 3033357

     View code at this pointBrowse repository at this point
6. Commits on Sep 4, 2026

   • [SPARK-59219][PYTHON] Avoid copying the Series to name it in convert_numpy

     Show description for 7b60b3e
     

     Spenserrrr

     authored and

     zhengruifeng

     committed
     7b60b3eView commit details

     Copy full SHA for 7b60b3e

     View code at this pointBrowse repository at this point
7. Commits on Aug 27, 2026

   • [SPARK-59060][PYTHON] Add missing cpuAmount to the benchmark mock task context

     Show description for c5eebb3
     viirya

     committed
     c5eebb3View commit details

     Copy full SHA for c5eebb3

     View code at this pointBrowse repository at this point
8. Commits on Aug 25, 2026

   • [SPARK-58986][PYTHON] Sort import for python files

     Show description for c95d969
     gaogaotiantian

     committed
     c95d969View commit details

     Copy full SHA for c95d969

     View code at this pointBrowse repository at this point
9. Commits on Aug 21, 2026

   • [SPARK-58699][PYTHON] Short-circuit identity conversion for string and binary in LocalDataToArrowConversion

     Show description for 5748826
     Yicong-Huang

     committed
     5748826View commit details

     Copy full SHA for 5748826

     View code at this pointBrowse repository at this point
10. Commits on Jul 24, 2026

    • [SPARK-58297][PYTHON] Remove TransformWithStateInPySparkRowSerializer

      Show description for fd3d03c
      Yicong-Huang

      committed
      fd3d03cView commit details

      Copy full SHA for fd3d03c

      View code at this pointBrowse repository at this point
11. Commits on Jul 23, 2026

    • [SPARK-58157][PYTHON] Remove TransformWithStateInPySparkRowInitStateSerializer

      Show description for 93c3367
      

      Yicong-Huang

      authored and

      uros-b

      committed
      93c3367View commit details

      Copy full SHA for 93c3367

      View code at this pointBrowse repository at this point
12. Commits on Jul 22, 2026

    • [SPARK-58243][PYTHON] Remove unused ApplyInPandasWithStateSerializer

      Show description for 600e361
      Yicong-Huang

      committed
      600e361View commit details

      Copy full SHA for 600e361

      View code at this pointBrowse repository at this point
13. Commits on Jul 21, 2026

    • [SPARK-58225][PYTHON] Delete unused TransformWithStateInPandasSerializer

      Show description for e3eda67
      Yicong-Huang

      committed
      e3eda67View commit details

      Copy full SHA for e3eda67

      View code at this pointBrowse repository at this point
14. Commits on Jul 20, 2026

    • [SPARK-58190][PYTHON] Delete unused TransformWithStateInPandasInitStateSerializer

      Show description for 2695dcd
      Yicong-Huang

      committed
      2695dcdView commit details

      Copy full SHA for 2695dcd

      View code at this pointBrowse repository at this point
15. Commits on Jul 17, 2026

    • [SPARK-58130][PYTHON] Refactor legacy SQL_ARROW_TABLE_UDF pandas path and remove ArrowStreamPandasUDTFSerializer

      Show description for 3dc79a4
      

      Yicong-Huang

      authored and

      zhengruifeng

      committed
      3dc79a4View commit details

      Copy full SHA for 3dc79a4

      View code at this pointBrowse repository at this point
16. Commits on Jul 15, 2026

    • [SPARK-58080][PYTHON][TEST] Add ASV microbenchmark for SQL_GROUPED_MAP_PANDAS_UDF_WITH_STATE

      Show description for 4bea4ad
      Yicong-Huang

      committed
      4bea4adView commit details

      Copy full SHA for 4bea4ad

      View code at this pointBrowse repository at this point
17. Commits on Jul 14, 2026

    • [SPARK-58067][PYTHON][TEST] Add ASV microbenchmark for SQL_TRANSFORM_WITH_STATE_PYTHON_ROW_INIT_STATE_UDF

      Show description for 51d887d
      Yicong-Huang

      committed
      51d887dView commit details

      Copy full SHA for 51d887d

      View code at this pointBrowse repository at this point

    • [SPARK-57903][PYTHON][TEST] Add ASV microbenchmark for SQL_TRANSFORM_WITH_STATE_PYTHON_ROW_UDF

      Show description for 488b92b
      Yicong-Huang

      committed
      488b92bView commit details

      Copy full SHA for 488b92b

      View code at this pointBrowse repository at this point
18. Commits on Jul 11, 2026

    • [SPARK-58024][PYTHON] Convert Arrow struct and map columns to Python rows in bulk

      Show description for 9c6e57b
      viirya

      committed
      9c6e57bView commit details

      Copy full SHA for 9c6e57b

      View code at this pointBrowse repository at this point
19. Commits on Jul 10, 2026

    • [SPARK-58019][PYTHON] Convert Arrow list columns to Python rows in bulk

      Show description for 484342a
      viirya

      committed
      484342aView commit details

      Copy full SHA for 484342a

      View code at this pointBrowse repository at this point
20. Commits on Jul 2, 2026

    • [SPARK-57704][PYTHON][TEST] Add ASV microbenchmark for SQL_TRANSFORM_WITH_STATE_PANDAS_INIT_STATE_UDF

      Show description for 332025b
      Yicong-Huang

      committed
      332025bView commit details

      Copy full SHA for 332025b

      View code at this pointBrowse repository at this point
21. Commits on Jun 25, 2026

    • [SPARK-56642][SQL] Add pipelined JVM-Python UDF data transfer

      Show description for 59d4fa2
      viirya

      committed
      59d4fa2View commit details

      Copy full SHA for 59d4fa2

      View code at this pointBrowse repository at this point
22. Commits on Jun 24, 2026

    • [SPARK-57645][PYTHON][TEST] Add ASV microbenchmark for SQL_GROUPED_AGG_PANDAS_ITER_UDF

      Show description for 4b682c6
      Yicong-Huang

      committed
      4b682c6View commit details

      Copy full SHA for 4b682c6

      View code at this pointBrowse repository at this point
23. Commits on Jun 15, 2026

    • [SPARK-57423][PYTHON][TEST] Share base mixin across MapInBatch Arrow/Pandas siblings

      Show description for 0802a5a
      viirya

      committed
      0802a5aView commit details

      Copy full SHA for 0802a5a

      View code at this pointBrowse repository at this point
24. Commits on Jun 12, 2026

    • [SPARK-57414][PYTHON][TEST] Unify classmethod and eval type conventions across bench_eval_type.py mixins

      Show description for f9d6a6f
      viirya

      committed
      f9d6a6fView commit details

      Copy full SHA for f9d6a6f

      View code at this pointBrowse repository at this point
25. Commits on Jun 11, 2026

    • [SPARK-57020][PYTHON][TEST] Add ASV microbenchmark for SQL_TRANSFORM_WITH_STATE_PANDAS_UDF

      Show description for 8cced6f
      Yicong-Huang

      committed
      8cced6fView commit details

      Copy full SHA for 8cced6f

      View code at this pointBrowse repository at this point
26. Commits on May 29, 2026

    • [SPARK-57138][PYTHON][TEST] Share base mixin across Window and Cogroup Arrow/Pandas siblings

      Show description for 481f8a9
      viirya

      committed
      481f8a9View commit details

      Copy full SHA for 481f8a9

      View code at this pointBrowse repository at this point
27. Commits on May 28, 2026

    • [SPARK-57137][PYTHON][TEST] Share base mixin across Arrow/Pandas siblings

      Show description for 40e9a14
      viirya

      committed
      40e9a14View commit details

      Copy full SHA for 40e9a14

      View code at this pointBrowse repository at this point

    • [SPARK-55724][PYTHON][TEST][FOLLOWUP] Unify wide_values to wide_cols in bench

      Show description for b76b6a8
      viirya

      committed
      b76b6a8View commit details

      Copy full SHA for b76b6a8

      View code at this pointBrowse repository at this point

    • [SPARK-55754][PYTHON][TEST][FOLLOWUP] Fix pure_ints type mismatch in bench

      Show description for fc5abd6
      viirya

      committed
      fc5abd6View commit details

      Copy full SHA for fc5abd6

      View code at this pointBrowse repository at this point

    • [SPARK-55934][PYTHON][TEST][FOLLOWUP] Fix MAP_ARROW_ITER bench UDF return type

      Show description for 70469a2
      viirya

      committed
      70469a2View commit details

      Copy full SHA for 70469a2

      View code at this pointBrowse repository at this point

    • [SPARK-56120][PYTHON][TEST][FOLLOWUP] Make _WindowAggArrowBenchMixin scenarios lazy

      Show description for 7da33f3
      viirya

      committed
      7da33f3View commit details

      Copy full SHA for 7da33f3

      View code at this pointBrowse repository at this point
28. Commits on May 27, 2026

    • [SPARK-56324] Introducing message-based communication to Spark -> PySpark communication channel

      Show description for 28904a3
      

      sven-weber-db

      authored and

      dtenedor

      committed
      28904a3View commit details

      Copy full SHA for 28904a3

      View code at this pointBrowse repository at this point
29. Commits on May 13, 2026

    • [SPARK-56837][PYTHON][TESTS] Pass ArrowBatchedUDF benchmark input_type via EvalConf

      Show description for f40eccd
      

      Yicong-Huang

      authored and

      zhengruifeng

      committed
      f40eccdView commit details

      Copy full SHA for f40eccd

      View code at this pointBrowse repository at this point
30. Commits on May 12, 2026

    • [SPARK-56717][PYTHON][TESTS] Add ASV microbenchmark for SQL_ARROW_TABLE_UDF

      Show description for 74934af
      

      Yicong-Huang

      authored and

      zhengruifeng

      committed
      74934afView commit details

      Copy full SHA for 74934af

      View code at this pointBrowse repository at this point