You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Asif (Jira)" <ji...@apache.org> on 2023/10/25 05:00:00 UTC

[jira] [Created] (SPARK-45658) Canonicalization of DynamicPruningSubquery is broken

Asif created SPARK-45658:
----------------------------

             Summary: Canonicalization of DynamicPruningSubquery is broken
                 Key: SPARK-45658
                 URL: https://issues.apache.org/jira/browse/SPARK-45658
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.5.0, 3.5.1
            Reporter: Asif


The canonicalization of (buildKeys: Seq[Expression]) in the class DynamicPruningSubquery is broken, as the buildKeys are canonicalized just by calling 
buildKeys.map(_.canonicalized)
The  above would result in incorrect canonicalization as it would not be normalizing the exprIds
The fix is to use the buildQuery : LogicalPlan's output to normalize the buildKeys expression
as given below, using the standard approach.

buildKeys.map(QueryPlan.normalizeExpressions(_, buildQuery.output)),

Will be filing a PR and bug test for the same.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org