[jira] [Resolved] (SPARK-26182) Cost increases when optimizing scalaUDF

["bupt_ljy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-26182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

bupt_ljy resolved SPARK-26182.
------------------------------
    Resolution: Invalid

> Cost increases when optimizing scalaUDF
> ---------------------------------------
>
>                 Key: SPARK-26182
>                 URL: https://issues.apache.org/jira/browse/SPARK-26182
>             Project: Spark
>          Issue Type: Improvement
>          Components: Optimizer
>    Affects Versions: 2.4.0
>            Reporter: bupt_ljy
>            Priority: Major
>
> Let's assume that we have a udf called splitUDF which outputs a map data.
>  The SQL
> {code:java}
> select
>     g['a'], g['b']
> from
>    ( select splitUDF(x) as g from table) tbl
> {code}
> will be optimized to the same logical plan of
> {code:java}
> select splitUDF(x)['a'], splitUDF(x)['b'] from table
> {code}
> which means that the splitUDF is executed twice instead of once.
> The optimization is from CollapseProject. 
>  I'm not sure whether this is a bug or not. Please tell me if I was wrong about this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org