[jira] [Assigned] (SPARK-41441) Allow Generate with no required child output to host outer references

["Apache Spark (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-41441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-41441:
------------------------------------

    Assignee: Apache Spark

> Allow Generate with no required child output to host outer references
> ---------------------------------------------------------------------
>
>                 Key: SPARK-41441
>                 URL: https://issues.apache.org/jira/browse/SPARK-41441
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 3.4.0
>            Reporter: Allison Wang
>            Assignee: Apache Spark
>            Priority: Major
>
> Currently, in CheckAnalysis, Spark disallows Generate to host any outer references when it's required child output is not empty. But when the child output is empty, it can host outer references, which DecorrelateInnerQuery does not handle.
> For example,
> {code:java}
> select * from t, lateral (select explode(array(c1, c2))){code}
> This throws an internal error :
> {code:java}
> Caused by: java.lang.AssertionError: assertion failed: Correlated column is not allowed in Generate explode(array(outer(c1#219), outer(c2#220))), false, [col#221] +- OneRowRelation{code}
>  We should support Generate to host outer references when its required child output is empty.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org