[jira] [Resolved] (SPARK-41539) stats and constraints in LogicalRDD may not be in sync with output attributes

["Jungtaek Lim (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-41539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jungtaek Lim resolved SPARK-41539.
----------------------------------
    Fix Version/s: 3.4.0
       Resolution: Fixed

Issue resolved by pull request 39082
[https://github.com/apache/spark/pull/39082]

> stats and constraints in LogicalRDD may not be in sync with output attributes
> -----------------------------------------------------------------------------
>
>                 Key: SPARK-41539
>                 URL: https://issues.apache.org/jira/browse/SPARK-41539
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.4.0
>            Reporter: Jungtaek Lim
>            Assignee: Jungtaek Lim
>            Priority: Major
>             Fix For: 3.4.0
>
>
> We encountered the case where the output of logical plan and optimized plan were different in LogicalRDD (the difference was exprId for the case), led the situation that stats and constraints are out of sync with output attributes, eventually failed the query.
> We should remap stats and constraints based on the output of logical plan, assuming that the output of logical plan and optimized plan are "slightly" different (e.g. exprId) but "semantically" same.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org