You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2025/07/29 23:02:39 UTC
(spark) branch master updated: [SPARK-52992][PYTHON][DOCS] Restore API reference of `pandas_udf`
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 42553b94e368 [SPARK-52992][PYTHON][DOCS] Restore API reference of `pandas_udf`
42553b94e368 is described below
commit 42553b94e3682be207cb0e64e633ae483bdb8e90
Author: Ruifeng Zheng <ru...@apache.org>
AuthorDate: Wed Jul 30 08:02:28 2025 +0900
[SPARK-52992][PYTHON][DOCS] Restore API reference of `pandas_udf`
### What changes were proposed in this pull request?
Restore API reference of `pandas_udf`
### Why are the changes needed?
it was mistakenly deleted in 4.1, see
- 4.1: https://apache.github.io/spark/api/python/reference/pyspark.sql/api/pyspark.sql.functions.pandas_udf.html?highlight=pandas_udf#pyspark.sql.functions.pandas_udf
- 4.0: https://spark.apache.org/docs/4.0.0/api/python/reference/pyspark.sql/api/pyspark.sql.functions.pandas_udf.html
### Does this PR introduce _any_ user-facing change?
No, 4.1 is not released yet
### How was this patch tested?
CI, and will double check the generated documents
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #51699 from zhengruifeng/restore_pandas_udf_doc.
Authored-by: Ruifeng Zheng <ru...@apache.org>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
python/pyspark/sql/pandas/functions.py | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/python/pyspark/sql/pandas/functions.py b/python/pyspark/sql/pandas/functions.py
index ed4466f31ea4..b1861efd2ca8 100644
--- a/python/pyspark/sql/pandas/functions.py
+++ b/python/pyspark/sql/pandas/functions.py
@@ -56,20 +56,11 @@ def arrow_udf(f=None, returnType=None, functionType=None):
def pandas_udf(f=None, returnType=None, functionType=None):
- return vectorized_udf(f, returnType, functionType, "pandas")
-
-
-def vectorized_udf(
- f=None,
- returnType=None,
- functionType=None,
- kind: str = "pandas",
-):
"""
- Creates a vectorized user defined function.
+ Creates a pandas user defined function.
Pandas UDFs are user defined functions that are executed by Spark using Arrow to transfer
- data and Pandas to work with the data, which allows vectorized operations. A Pandas UDF
+ data and Pandas to work with the data, which allows pandas operations. A Pandas UDF
is defined using the `pandas_udf` as a decorator or to wrap the function, and no
additional configuration is required. A Pandas UDF behaves as a regular PySpark function
API in general.
@@ -392,6 +383,15 @@ def vectorized_udf(
# Note: Python 3.11.9, Pandas 2.2.3 and PyArrow 17.0.0 are used.
# Note: Timezone is KST.
# Note: 'X' means it throws an exception during the conversion.
+ return vectorized_udf(f, returnType, functionType, "pandas")
+
+
+def vectorized_udf(
+ f=None,
+ returnType=None,
+ functionType=None,
+ kind: str = "pandas",
+):
require_minimum_pandas_version()
require_minimum_pyarrow_version()
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org