[jira] [Commented] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

["Bruce Robbins (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-43718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725122#comment-17725122 ] 

Bruce Robbins commented on SPARK-43718:
---------------------------------------

I think I have a handle on this. I will submit in a PR in the coming days.

> References to a specific side's key in a USING join can have wrong nullability
> ------------------------------------------------------------------------------
>
>                 Key: SPARK-43718
>                 URL: https://issues.apache.org/jira/browse/SPARK-43718
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.5.0
>            Reporter: Bruce Robbins
>            Priority: Major
>              Labels: correctness
>
> Assume this data:
> {noformat}
> create or replace temp view t1 as values (1), (2), (3) as (c1);
> create or replace temp view t2 as values (2), (3), (4) as (c1);
> {noformat}
> The following query produces incorrect results:
> {noformat}
> spark-sql (default)> select explode(array(t1.c1, t2.c1)) as x1
> from t1
> full outer join t2
> using (c1);
> 1
> -1      <== should be null
> 2
> 2
> 3
> 3
> -1      <== should be null
> 4
> Time taken: 0.663 seconds, Fetched 8 row(s)
> spark-sql (default)> 
> {noformat}
> Similar issues occur with right outer join and left outer join.
> {{t1.c1}} and {{t2.c1}} have the wrong nullability at the time the array is resolved, so the array's {{containsNull}} value is incorrect.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org