[jira] [Created] (SPARK-41441) Allow Generate with no required child output to host outer references

["Allison Wang (Jira)" <ji...@apache.org>]

Allison Wang created SPARK-41441:
------------------------------------

             Summary: Allow Generate with no required child output to host outer references
                 Key: SPARK-41441
                 URL: https://issues.apache.org/jira/browse/SPARK-41441
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 3.4.0
            Reporter: Allison Wang


Currently, in CheckAnalysis, Spark disallows Generate to host any outer references when it's required child output is not empty. But when the child output is empty, it can host outer references, which DecorrelateInnerQuery does not handle.

For example,
{code:java}
select * from t, lateral (select explode(array(c1, c2))){code}
This throws an internal error :
{code:java}
Caused by: java.lang.AssertionError: assertion failed: Correlated column is not allowed in Generate explode(array(outer(c1#219), outer(c2#220))), false, [col#221] +- OneRowRelation{code}
 We should support Generate to host outer references when its required child output is empty.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org