You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Burak Yavuz (JIRA)" <ji...@apache.org> on 2016/11/29 20:55:59 UTC
[jira] [Updated] (SPARK-18634) Corruption and Correctness issues
with exploding Python UDFs
[ https://issues.apache.org/jira/browse/SPARK-18634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Burak Yavuz updated SPARK-18634:
--------------------------------
Summary: Corruption and Correctness issues with exploding Python UDFs (was: Issues with exploding Python UDFs)
> Corruption and Correctness issues with exploding Python UDFs
> ------------------------------------------------------------
>
> Key: SPARK-18634
> URL: https://issues.apache.org/jira/browse/SPARK-18634
> Project: Spark
> Issue Type: Bug
> Components: PySpark, SQL
> Affects Versions: 2.0.2, 2.1.0
> Reporter: Burak Yavuz
>
> There are some weird issues with exploding Python UDFs in SparkSQL.
> There are 2 cases where based on the DataType of the exploded column, the result can be flat out wrong, or corrupt. Seems like something bad is happening when telling Tungsten the schema of the rows during or after applying the UDF.
> Please check the attached notebook for reproduction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org