Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Aug 6, 2023

• [SPARK-44629][PYTHON][DOCS] Publish PySpark Test Guidelines webpage

  Show description for a640373
  

  asl3

  authored and

  HyukjinKwon

  committed
  a640373

  Copy full SHA for a640373

Commits on Aug 4, 2023

• [SPARK-43562][SPARK-43870][PS] Remove APIs from `DataFrame` and `Series`

  Show description for 678f472
  

  itholic

  authored and

  HyukjinKwon

  committed
  678f472

  Copy full SHA for 678f472

Commits on Jul 28, 2023

• [SPARK-44548][PYTHON] Add support for pandas-on-Spark DataFrame assertDataFrameEqual

  Show description for 7c1ad5b
  

  asl3

  authored and

  HyukjinKwon

  committed
  7c1ad5b

  Copy full SHA for 7c1ad5b

Commits on Jul 25, 2023

• [SPARK-44509][PYTHON][CONNECT] Add job cancellation API set in Spark Connect Python client

  Show description for 5ee462d
  HyukjinKwon

  committed
  5ee462d

  Copy full SHA for 5ee462d

Commits on Jul 20, 2023

• [SPARK-44481][CONNECT][PYTHON] Make pyspark.sql.is_remote an API

  Show description for 11e2e42
  HyukjinKwon

  committed
  11e2e42

  Copy full SHA for 11e2e42

Commits on Jul 19, 2023

• [SPARK-44401][PYTHON][DOCS] Arrow Python UDF Use Guide

  Show description for bdea4c5
  

  xinrong-meng

  authored and

  HyukjinKwon

  committed
  bdea4c5

  Copy full SHA for bdea4c5

Commits on Jul 16, 2023

• [SPARK-44264][PYTHON][ML][FOLLOW-UP] Clean Up Deepspeed Code

  Show description for 001e93d
  

  mathewjacob1002

  authored and

  HyukjinKwon

  committed
  001e93d

  Copy full SHA for 001e93d

• [SPARK-44216][PYTHON] Make assertSchemaEqual API public

  Show description for a00a32a
  

  asl3

  authored and

  HyukjinKwon

  committed
  a00a32a

  Copy full SHA for a00a32a

Commits on Jul 14, 2023

• [SPARK-44416][CONNECT][BUILD] Upgrade buf to v1.24.0

  Show description for 5f8d41b
  zhengruifeng

  committed
  5f8d41b

  Copy full SHA for 5f8d41b

• [SPARK-44222][BUILD][PYTHON] Upgrade grpc to 1.56.0 with lower/upperbound

  Show description for de59caa
  HyukjinKwon

  committed
  de59caa

  Copy full SHA for de59caa

• Revert "[SPARK-44222][BUILD][PYTHON] Upgrade `grpc` to 1.56.0"

  Show description for a47bde2
  HyukjinKwon

  committed
  a47bde2

  Copy full SHA for a47bde2

Commits on Jul 12, 2023

• [SPARK-43974][CONNECT][BUILD] Upgrade buf to v1.23.1

  Show description for d9248e8
  

  panbingkun

  authored and

  zhengruifeng

  committed
  d9248e8

  Copy full SHA for d9248e8

Commits on Jul 11, 2023

• [SPARK-43974][CONNECT][BUILD] Revert "[] Upgrade buf to v1.23.1"

  Show description for d7bc6f5
  

  panbingkun

  authored and

  zhengruifeng

  committed
  d7bc6f5

  Copy full SHA for d7bc6f5

• [SPARK-43974][CONNECT][BUILD] Upgrade buf to v1.23.1

  Show description for b0b12cf
  

  panbingkun

  authored and

  zhengruifeng

  committed
  b0b12cf

  Copy full SHA for b0b12cf

Commits on Jul 10, 2023

• [SPARK-44131][SQL] Add call_function and deprecate call_udf for Scala API

  Show description for 46dd3aa
  

  beliefer

  authored and

  zhengruifeng

  committed
  46dd3aa

  Copy full SHA for 46dd3aa

• [SPARK-44329][CONNECT][PYTHON] Add hll_sketch_agg, hll_union_agg, to_varchar, try_aes_decrypt to Scala and Python

  Show description for ea6bacf
  

  panbingkun

  authored and

  zhengruifeng

  committed
  ea6bacf

  Copy full SHA for ea6bacf

• [SPARK-44331][CONNECT][PYTHON] Add bitmap functions to Scala and Python

  Show description for 32d170f
  

  panbingkun

  authored and

  zhengruifeng

  committed
  32d170f

  Copy full SHA for 32d170f

Commits on Jul 8, 2023

• [SPARK-44061][PYTHON] Add assertDataFrameEqual util function

  Show description for e4a356a
  

  asl3

  authored and

  HyukjinKwon

  committed
  e4a356a

  Copy full SHA for e4a356a

Commits on Jul 4, 2023

• [SPARK-44194][PYTHON][CORE] Add JobTag APIs to PySpark SparkContext

  Show description for 4fcecfe
  HyukjinKwon

  committed
  4fcecfe

  Copy full SHA for 4fcecfe

Commits on Jun 30, 2023

• [SPARK-43926][CONNECT][PYTHON] Add array_agg, array_size, cardinality, count_min_sketch,mask,named_struct,json_* to Scala and Python

  Show description for de8ec74
  

  2 peopleauthored and

  zhengruifeng

  committed
  de8ec74

  Copy full SHA for de8ec74

Commits on Jun 28, 2023

• [SPARK-44222][BUILD][PYTHON] Upgrade `grpc` to 1.56.0

  Show description for f26bdb7
  dongjoon-hyun

  committed
  f26bdb7

  Copy full SHA for f26bdb7

• [SPARK-44182][DOCS] Use Spark version variables in Python and Spark Connect installation docs

  Show description for f00de6f
  

  dongjoon-hyun

  authored and

  yaooqinn

  committed
  f00de6f

  Copy full SHA for f00de6f

Commits on Jun 26, 2023

• [SPARK-44183][PYTHON] Increate `PyArrow` minimum version to 4.0.0

  Show description for cb9f6bf
  dongjoon-hyun

  committed
  cb9f6bf

  Copy full SHA for cb9f6bf

• [SPARK-43924][CONNECT][PYTHON] Add misc functions to Scala and Python

  Show description for 6a0ae19
  

  panbingkun

  authored and

  zhengruifeng

  committed
  6a0ae19

  Copy full SHA for 6a0ae19

• [SPARK-44184][PYTHON][DOCS] Remove a wrong doc about `ARROW_PRE_0_15_IPC_FORMAT`

  Show description for 00e7c08
  dongjoon-hyun

  committed
  00e7c08

  Copy full SHA for 00e7c08

Commits on Jun 23, 2023

• [SPARK-44135][PYTHON][CONNECT][DOCS] Document Spark Connect only API in PySpark

  Show description for 8eda2d8
  HyukjinKwon

  committed
  8eda2d8

  Copy full SHA for 8eda2d8

Commits on Jun 20, 2023

• [SPARK-43939][CONNECT][PYTHON] Add try_* functions to Scala and Python

  Show description for 68b3005
  

  panbingkun

  authored and

  zhengruifeng

  committed
  68b3005

  Copy full SHA for 68b3005

• [SPARK-43929][SPARK-44073][SQL][PYTHON][CONNECT][FOLLOWUP] Add extract, date_part, datepart to Scala, Python and Connect API

  Show description for 8dc0286
  

  beliefer

  authored and

  zhengruifeng

  committed
  8dc0286

  Copy full SHA for 8dc0286

• [SPARK-44073][SQL][PYTHON][CONNECT] Add date time functions to Scala, Python and Connect - part 2

  Show description for 7a8f2ec
  

  beliefer

  authored and

  zhengruifeng

  committed
  7a8f2ec

  Copy full SHA for 7a8f2ec

Commits on Jun 19, 2023

• [SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Python - part 1

  Show description for 476c58e
  

  panbingkun

  authored and

  zhengruifeng

  committed
  476c58e

  Copy full SHA for 476c58e

• [SPARK-43929][SQL][PYTHON][CONNECT] Add date time functions to Scala, Python and Connect API - part 1

  Show description for 25a14c3
  

  beliefer

  authored and

  zhengruifeng

  committed
  25a14c3

  Copy full SHA for 25a14c3

Commits on Jun 18, 2023

• [SPARK-43944][CONNECT][PYTHON] Add string functions to Scala and Python - part 2

  Show description for 4503efc
  

  panbingkun

  authored and

  zhengruifeng

  committed
  4503efc

  Copy full SHA for 4503efc

Commits on Jun 17, 2023

• [SPARK-43928][SQL][PYTHON][CONNECT] Add bit operations to Scala, Python and Connect API

  Show description for 02beb50
  

  beliefer

  authored and

  zhengruifeng

  committed
  02beb50

  Copy full SHA for 02beb50

Commits on Jun 16, 2023

• [SPARK-43925][SQL][PYTHON][CONNECT] Add some, bool_or,bool_and,every to Scala, Python and Connect

  Show description for 1b12094
  

  beliefer

  authored and

  zhengruifeng

  committed
  1b12094

  Copy full SHA for 1b12094

• [SPARK-43937][CONNECT][PYTHON] Add ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and Python

  Show description for 06b2b1a
  

  panbingkun

  authored and

  zhengruifeng

  committed
  06b2b1a

  Copy full SHA for 06b2b1a