You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Ben Hurdelhey (Jira)" <ji...@apache.org> on 2025/08/22 16:14:00 UTC
[jira] [Updated] (SPARK-53355) Capture input / output type behavior of pyspark udfs
[ https://issues.apache.org/jira/browse/SPARK-53355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ben Hurdelhey updated SPARK-53355:
----------------------------------
Description:
* add pandas_udf coverage script from [https://github.com/apache/spark/pull/51635] and link where appropriate. We already have one for python udfs in type_coercion_udf.py ([https://issues.apache.org/jira/projects/SPARK/issues/SPARK-52952?filter=allissues])
* consider automatically running these in CI and comparing their results to stored files to catch
* consider adding more tests/tables for how input types get converted to python types
was:
* add pandas_udf coverage script and link where appropriate. We already have one for python udfs (https://issues.apache.org/jira/projects/SPARK/issues/SPARK-52952?filter=allissues)
* consider automatically running these in CI and comparing their results to stored files to catch
* consider adding more tests/tables for how input types get converted to python types
> Capture input / output type behavior of pyspark udfs
> ------------------------------------------------------
>
> Key: SPARK-53355
> URL: https://issues.apache.org/jira/browse/SPARK-53355
> Project: Spark
> Issue Type: Task
> Components: PySpark
> Affects Versions: 4.1.0
> Reporter: Ben Hurdelhey
> Priority: Major
>
> * add pandas_udf coverage script from [https://github.com/apache/spark/pull/51635] and link where appropriate. We already have one for python udfs in type_coercion_udf.py ([https://issues.apache.org/jira/projects/SPARK/issues/SPARK-52952?filter=allissues])
> * consider automatically running these in CI and comparing their results to stored files to catch
> * consider adding more tests/tables for how input types get converted to python types
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org