You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Chun Chang (JIRA)" <ji...@apache.org> on 2015/03/31 02:59:52 UTC
[jira] [Commented] (DRILL-1988) join returned maps are all empty
[ https://issues.apache.org/jira/browse/DRILL-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387735#comment-14387735 ]
Chun Chang commented on DRILL-1988:
-----------------------------------
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select * from sys.version;
+------------+----------------+-------------+-------------+------------+
| commit_id | commit_message | commit_time | build_email | build_time |
+------------+----------------+-------------+-------------+------------+
| 462e50ce9c4b829c2a4bafdeb9763bfba677c726 | DRILL-2575: FragmentExecutor.cancel() blasts through state transitions regardless of current state | 25.03.2015 @ 21:11:23 PDT
1 row selected (0.054 seconds)
0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select a.id, a.soa[3].str, b.soa[3].str, a.ooa[1].fl from `complex.json` a inner join `complex.json` b on a.soa[3].str=b.soa[3].str order by a.id limit 10;
+------------+------------+------------+------------+
| id | EXPR$1 | EXPR$2 | EXPR$3 |
+------------+------------+------------+------------+
| 1 | here is a string at row 1 | here is a string at row 1 | {"f1":1.6789,"f2":54331.0} |
| 2 | here is a string at row 2 | here is a string at row 2 | {} |
| 3 | here is a string at row 3 | here is a string at row 3 | {"f1":3.6789,"f2":54351.0} |
| 4 | here is a string at row 4 | here is a string at row 4 | {"f1":4.6789,"f2":54361.0} |
| 5 | here is a string at row 5 | here is a string at row 5 | {"f1":5.6789,"f2":54371.0} |
| 6 | here is a string at row 6 | here is a string at row 6 | {} |
| 7 | here is a string at row 7 | here is a string at row 7 | {"f1":7.6789,"f2":54391.0} |
| 8 | here is a string at row 8 | here is a string at row 8 | {} |
| 9 | here is a string at row 9 | here is a string at row 9 | {"f1":9.6789,"f2":54411.0} |
| 10 | here is a string at row 10 | here is a string at row 10 | {"f1":10.6789,"f2":54421.0} |
+------------+------------+------------+------------+
{code}
> join returned maps are all empty
> --------------------------------
>
> Key: DRILL-1988
> URL: https://issues.apache.org/jira/browse/DRILL-1988
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
> Affects Versions: 0.8.0
> Reporter: Chun Chang
> Assignee: Mehant Baid
> Priority: Critical
> Attachments: DRILL-1988.patch
>
>
> #Fri Jan 09 20:39:31 EST 2015
> git.commit.id.abbrev=487d98e
> For complex json type, a join query returned all maps with empty value. The actual data has empty maps for some rows, but mostly with value. Data can be downloaded from:
> https://s3.amazonaws.com/apache-drill/files/complex.json.gz
> {code}
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select a.id, a.soa[3].str, b.soa[3].str, a.ooa[1].fl from `complex.json` a inner join `complex.json` b on a.soa[3].str=b.soa[3].str order by a.id limit 10;
> +------------+------------+------------+------------+
> | id | EXPR$1 | EXPR$2 | EXPR$3 |
> +------------+------------+------------+------------+
> | 1 | here is a string at row 1 | here is a string at row 1 | {} |
> | 2 | here is a string at row 2 | here is a string at row 2 | {} |
> | 3 | here is a string at row 3 | here is a string at row 3 | {} |
> | 4 | here is a string at row 4 | here is a string at row 4 | {} |
> | 5 | here is a string at row 5 | here is a string at row 5 | {} |
> | 6 | here is a string at row 6 | here is a string at row 6 | {} |
> | 7 | here is a string at row 7 | here is a string at row 7 | {} |
> | 8 | here is a string at row 8 | here is a string at row 8 | {} |
> | 9 | here is a string at row 9 | here is a string at row 9 | {} |
> | 10 | here is a string at row 10 | here is a string at row 10 | {} |
> +------------+------------+------------+------------+
> {code}
> As you can see from the following query, maps is not empty for most of the row IDs.
> {code}
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select a.id, a.ooa[1].fl from `complex.json` a limit 10;
> +------------+------------+
> | id | EXPR$1 |
> +------------+------------+
> | 1 | {"f1":1.6789,"f2":54331.0} |
> | 2 | {} |
> | 3 | {"f1":3.6789,"f2":54351.0} |
> | 4 | {"f1":4.6789,"f2":54361.0} |
> | 5 | {"f1":5.6789,"f2":54371.0} |
> | 6 | {} |
> | 7 | {"f1":7.6789,"f2":54391.0} |
> | 8 | {} |
> | 9 | {"f1":9.6789,"f2":54411.0} |
> | 10 | {"f1":10.6789,"f2":54421.0} |
> +------------+------------+
> {code}
> physical plan:
> {code}
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ> explain plan for select a.id, a.soa[3].str, b.soa[3].str, a.ooa[1].fl from `complex.json` a inner join `complex.json` b on a.soa[3].str=b.soa[3].str order by a.id limit 10;
> +------------+------------+
> | text | json |
> +------------+------------+
> | 00-00 Screen
> 00-01 Project(id=[$0], EXPR$1=[$1], EXPR$2=[$2], EXPR$3=[$3])
> 00-02 SelectionVectorRemover
> 00-03 Limit(fetch=[10])
> 00-04 SingleMergeExchange(sort0=[0 ASC])
> 01-01 SelectionVectorRemover
> 01-02 TopN(limit=[10])
> 01-03 HashToRandomExchange(dist0=[[$0]])
> 02-01 Project(id=[$0], EXPR$1=[$2], EXPR$2=[$5], EXPR$3=[$3])
> 02-02 HashJoin(condition=[=($1, $4)], joinType=[inner])
> 02-04 HashToRandomExchange(dist0=[[$1]])
> 03-01 Project(id=[$2], $f4=[ITEM(ITEM($1, 3), 'str')], ITEM=[ITEM(ITEM($1, 3), 'str')], ITEM3=[ITEM(ITEM($0, 1), 'fl')])
> 03-02 Scan(groupscan=[EasyGroupScan [selectionRoot=/drill/testdata/complex_type/json/complex.json, numFiles=1, columns=[`id`, `soa`[3].`str`, `ooa`[1].`fl`], files=[maprfs:/drill/testdata/complex_type/json/complex.json]]])
> 02-03 Project($f40=[$0], ITEM0=[$1])
> 02-05 HashToRandomExchange(dist0=[[$0]])
> 04-01 Project($f4=[ITEM(ITEM($0, 3), 'str')], ITEM=[ITEM(ITEM($0, 3), 'str')])
> 04-02 Scan(groupscan=[EasyGroupScan [selectionRoot=/drill/testdata/complex_type/json/complex.json, numFiles=1, columns=[`soa`[3].`str`], files=[maprfs:/drill/testdata/complex_type/json/complex.json]]])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)