You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Xiao Li (JIRA)" <ji...@apache.org> on 2018/09/20 17:04:00 UTC

[jira] [Resolved] (SPARK-25450) PushProjectThroughUnion rule uses the same exprId for project expressions in each Union child, causing mistakes in constant propagation

     [ https://issues.apache.org/jira/browse/SPARK-25450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xiao Li resolved SPARK-25450.
-----------------------------
       Resolution: Fixed
         Assignee: Maryann Xue
    Fix Version/s: 2.4.0
                   2.3.3

> PushProjectThroughUnion rule uses the same exprId for project expressions in each Union child, causing mistakes in constant propagation
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-25450
>                 URL: https://issues.apache.org/jira/browse/SPARK-25450
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.3.1
>            Reporter: Maryann Xue
>            Assignee: Maryann Xue
>            Priority: Major
>             Fix For: 2.3.3, 2.4.0
>
>
> The problem was cause by the PushProjectThroughUnion rule, which, when creating new Project for each child of Union, uses the same exprId for expressions of the same position. This is wrong because, for each child of Union, the expressions are all independent, and it can lead to a wrong result if other rules like FoldablePropagation kicks in, taking two different expressions as the same.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org