You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:35:35 UTC

[jira] [Resolved] (SPARK-15589) Anaylze simple PySpark closures and generate SQL expressions

     [ https://issues.apache.org/jira/browse/SPARK-15589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-15589.
----------------------------------
    Resolution: Incomplete

> Anaylze simple PySpark closures and generate SQL expressions
> ------------------------------------------------------------
>
>                 Key: SPARK-15589
>                 URL: https://issues.apache.org/jira/browse/SPARK-15589
>             Project: Spark
>          Issue Type: Improvement
>          Components: PySpark, SQL
>            Reporter: holdenk
>            Priority: Major
>              Labels: bulk-closed
>
> Similar to SPARK-14083 we can try introspecting simple Python functions and see if we can generate an equivalent SQL expression. This would result in an even greater performance increase for PySpark users than Scala users as not only would they benefit from better codegen, it would also avoid substantial serialization cost.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org