[jira] [Created] (SPARK-22984) Fix incorrect bitmap copying and offset shifting in GenerateUnsafeRowJoiner

["Josh Rosen (JIRA)" <ji...@apache.org>]

Josh Rosen created SPARK-22984:
----------------------------------

             Summary: Fix incorrect bitmap copying and offset shifting in GenerateUnsafeRowJoiner
                 Key: SPARK-22984
                 URL: https://issues.apache.org/jira/browse/SPARK-22984
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.2.0, 2.1.0, 2.0.0, 1.6.0, 1.5.0, 2.3.0
            Reporter: Josh Rosen
            Assignee: Josh Rosen
            Priority: Critical


The following query returns an incorrect answer:

{code}
set spark.sql.autoBroadcastJoinThreshold=-1;

create table a as select * from values 1;
create table b as select * from values 2;

SELECT
t3.col1,
t1.col1
FROM a t1
CROSS JOIN b t2
CROSS JOIN b t3
{code}

This should return the row {{2, 1}} but instead it returns {{null, 1}}. If you permute the order of the columns in the select statement or the order of the joins then it returns a valid answer (i.e. one without incorrect NULLs).

This turns out to be due to two longstanding bugs in GenerateUnsafeRowJoiner, which I'll describe in more detail in my PR.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org