[jira] [Created] (DRILL-1988) join returned maps are all empty

["Chun Chang (JIRA)" <ji...@apache.org>]

Chun Chang created DRILL-1988:
---------------------------------

             Summary: join returned maps are all empty
                 Key: DRILL-1988
                 URL: https://issues.apache.org/jira/browse/DRILL-1988
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 0.8.0
            Reporter: Chun Chang
            Assignee: Chris Westin


#Fri Jan 09 20:39:31 EST 2015
git.commit.id.abbrev=487d98e

For complex json type, a join query returned all maps with empty value. The actual data has empty maps for some rows, but mostly with value. Data can be downloaded from:

https://s3.amazonaws.com/apache-drill/files/complex.json.gz

{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select a.id, a.soa[3].str, b.soa[3].str, a.ooa[1].fl from `complex.json` a inner join `complex.json` b on a.soa[3].str=b.soa[3].str order by a.id limit 10;
+------------+------------+------------+------------+
|     id     |   EXPR$1   |   EXPR$2   |   EXPR$3   |
+------------+------------+------------+------------+
| 1          | here is a string at row 1 | here is a string at row 1 | {}         |
| 2          | here is a string at row 2 | here is a string at row 2 | {}         |
| 3          | here is a string at row 3 | here is a string at row 3 | {}         |
| 4          | here is a string at row 4 | here is a string at row 4 | {}         |
| 5          | here is a string at row 5 | here is a string at row 5 | {}         |
| 6          | here is a string at row 6 | here is a string at row 6 | {}         |
| 7          | here is a string at row 7 | here is a string at row 7 | {}         |
| 8          | here is a string at row 8 | here is a string at row 8 | {}         |
| 9          | here is a string at row 9 | here is a string at row 9 | {}         |
| 10         | here is a string at row 10 | here is a string at row 10 | {}         |
+------------+------------+------------+------------+
{code}

As you can see from the following query, maps is not empty for most of the row IDs.

{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select a.id, a.ooa[1].fl from `complex.json` a limit 10;
+------------+------------+
|     id     |   EXPR$1   |
+------------+------------+
| 1          | {"f1":1.6789,"f2":54331.0} |
| 2          | {}         |
| 3          | {"f1":3.6789,"f2":54351.0} |
| 4          | {"f1":4.6789,"f2":54361.0} |
| 5          | {"f1":5.6789,"f2":54371.0} |
| 6          | {}         |
| 7          | {"f1":7.6789,"f2":54391.0} |
| 8          | {}         |
| 9          | {"f1":9.6789,"f2":54411.0} |
| 10         | {"f1":10.6789,"f2":54421.0} |
+------------+------------+
{code}

physical plan:

{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ> explain plan for select a.id, a.soa[3].str, b.soa[3].str, a.ooa[1].fl from `complex.json` a inner join `complex.json` b on a.soa[3].str=b.soa[3].str order by a.id limit 10;
+------------+------------+
|    text    |    json    |
+------------+------------+
| 00-00    Screen
00-01      Project(id=[$0], EXPR$1=[$1], EXPR$2=[$2], EXPR$3=[$3])
00-02        SelectionVectorRemover
00-03          Limit(fetch=[10])
00-04            SingleMergeExchange(sort0=[0 ASC])
01-01              SelectionVectorRemover
01-02                TopN(limit=[10])
01-03                  HashToRandomExchange(dist0=[[$0]])
02-01                    Project(id=[$0], EXPR$1=[$2], EXPR$2=[$5], EXPR$3=[$3])
02-02                      HashJoin(condition=[=($1, $4)], joinType=[inner])
02-04                        HashToRandomExchange(dist0=[[$1]])
03-01                          Project(id=[$2], $f4=[ITEM(ITEM($1, 3), 'str')], ITEM=[ITEM(ITEM($1, 3), 'str')], ITEM3=[ITEM(ITEM($0, 1), 'fl')])
03-02                            Scan(groupscan=[EasyGroupScan [selectionRoot=/drill/testdata/complex_type/json/complex.json, numFiles=1, columns=[`id`, `soa`[3].`str`, `ooa`[1].`fl`], files=[maprfs:/drill/testdata/complex_type/json/complex.json]]])
02-03                        Project($f40=[$0], ITEM0=[$1])
02-05                          HashToRandomExchange(dist0=[[$0]])
04-01                            Project($f4=[ITEM(ITEM($0, 3), 'str')], ITEM=[ITEM(ITEM($0, 3), 'str')])
04-02                              Scan(groupscan=[EasyGroupScan [selectionRoot=/drill/testdata/complex_type/json/complex.json, numFiles=1, columns=[`soa`[3].`str`], files=[maprfs:/drill/testdata/complex_type/json/complex.json]]])
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)