You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yuming Wang (Jira)" <ji...@apache.org> on 2023/07/11 03:22:00 UTC
[jira] [Assigned] (SPARK-44251) Potential for incorrect results or NPE when full outer USING join has null key value
[ https://issues.apache.org/jira/browse/SPARK-44251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuming Wang reassigned SPARK-44251:
-----------------------------------
Assignee: Bruce Robbins
> Potential for incorrect results or NPE when full outer USING join has null key value
> ------------------------------------------------------------------------------------
>
> Key: SPARK-44251
> URL: https://issues.apache.org/jira/browse/SPARK-44251
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.3.2, 3.4.1, 3.5.0
> Reporter: Bruce Robbins
> Assignee: Bruce Robbins
> Priority: Major
> Labels: correctness
>
> The following query produces incorrect results:
> {noformat}
> create or replace temp view v1 as values (1, 2), (null, 7) as (c1, c2);
> create or replace temp view v2 as values (2, 3) as (c1, c2);
> select explode(array(c1)) as x
> from v1
> full outer join v2
> using (c1);
> -1 <== should be null
> 1
> 2
> {noformat}
> The following query fails with a {{NullPointerException}}:
> {noformat}
> create or replace temp view v1 as values ('1', 2), (null, 7) as (c1, c2);
> create or replace temp view v2 as values ('2', 3) as (c1, c2);
> select explode(array(c1)) as x
> from v1
> full outer join v2
> using (c1);
> 23/06/25 17:06:39 ERROR Executor: Exception in task 0.0 in stage 14.0 (TID 11)
> java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.UnsafeWriter.write(UnsafeWriter.java:110)
> at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage3.generate_doConsume_0$(Unknown Source)
> at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage3.smj_consumeFullOuterJoinRow_0$(Unknown Source)
> at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage3.wholestagecodegen_findNextJoinRows_0$(Unknown Source)
> at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage3.processNext(Unknown Source)
> at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
> ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org