[jira] [Resolved] (SPARK-32521) WithFields Expression should not be foldable

["Xiao Li (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-32521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xiao Li resolved SPARK-32521.
-----------------------------
    Fix Version/s: 3.1.0
       Resolution: Fixed

> WithFields Expression should not be foldable
> --------------------------------------------
>
>                 Key: SPARK-32521
>                 URL: https://issues.apache.org/jira/browse/SPARK-32521
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: fqaiser94
>            Priority: Major
>             Fix For: 3.1.0
>
>
> The following query currently fails on master brach: 
> {code:scala}
> sql("SELECT named_struct('a', 1, 'b', 2) a")
> .select($"a".withField("c", lit(3)).as("a"))
> .show(false)
> // java.lang.UnsupportedOperationException: Cannot evaluate expression: with_fields(named_struct(a, 1, b, 2), c, 3)
> {code}
> This happens because the Catalyst optimizer tries to statically evaluate the {{WithFields Expression}} (via the {{ConstantFolding}} rule), however it cannot do so because {{WithFields Expression}} is {{Unevaluable}}. 
> The likely solution here is to change {{WithFields Expression}} so that it is not {{foldable}} under any circumstance. This should solve the issue because then Catalyst will no longer attempt to statically evaluate {{WithFields Expression}} anymore. 
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org