You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by mg...@apache.org on 2020/05/05 11:50:49 UTC
[hive] branch master updated: HIVE-23372 Project not defined
correctly after reordering a join ADDENDUM - fix sharedwork.q (Krisztian
Kasa, reviewed by Miklos Gergely)
This is an automated email from the ASF dual-hosted git repository.
mgergely pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new dbc04ef HIVE-23372 Project not defined correctly after reordering a join ADDENDUM - fix sharedwork.q (Krisztian Kasa, reviewed by Miklos Gergely)
dbc04ef is described below
commit dbc04ef2861d4bf1d917c7b923be3969af26f718
Author: miklosgergely <mg...@cloudera.com>
AuthorDate: Tue May 5 13:34:34 2020 +0200
HIVE-23372 Project not defined correctly after reordering a join ADDENDUM - fix sharedwork.q (Krisztian Kasa, reviewed by Miklos Gergely)
---
.../results/clientpositive/llap/sharedwork.q.out | 178 ++-------------------
1 file changed, 12 insertions(+), 166 deletions(-)
diff --git a/ql/src/test/results/clientpositive/llap/sharedwork.q.out b/ql/src/test/results/clientpositive/llap/sharedwork.q.out
index 5308daf..175141f 100644
--- a/ql/src/test/results/clientpositive/llap/sharedwork.q.out
+++ b/ql/src/test/results/clientpositive/llap/sharedwork.q.out
@@ -696,72 +696,21 @@ STAGE PLANS:
alias: part
Statistics: Num rows: 26 Data size: 5954 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
-<<<<<<< ours
- Filter Operator
- isSamplingPred: false
- predicate: p_size is not null (type: boolean)
- Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: (p_size + 1) (type: int), p_type (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count(), count(_col1)
- keys: _col0 (type: int)
- minReductionHashAggr: 0.5
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 13 Data size: 260 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- bucketingVersion: 2
- key expressions: _col0 (type: int)
- null sort order: z
- numBuckets: -1
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 260 Basic stats: COMPLETE Column stats: COMPLETE
- tag: -1
- value expressions: _col1 (type: bigint), _col2 (type: bigint)
- auto parallelism: true
- Filter Operator
- isSamplingPred: false
- predicate: (p_size is not null and p_type is not null) (type: boolean)
- Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: p_type (type: string), (p_size + 1) (type: int)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: _col1 (type: int), _col0 (type: string)
- minReductionHashAggr: 0.0
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1404 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- bucketingVersion: 2
- key expressions: _col0 (type: int), _col1 (type: string)
- null sort order: zz
- numBuckets: -1
- sort order: ++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: string)
- Statistics: Num rows: 13 Data size: 1404 Basic stats: COMPLETE Column stats: COMPLETE
- tag: -1
- auto parallelism: true
-=======
Select Operator
expressions: p_name (type: string), p_type (type: string), (p_size + 1) (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 26 Data size: 5954 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ bucketingVersion: 2
key expressions: _col2 (type: int)
null sort order: z
+ numBuckets: -1
sort order: +
Map-reduce partition columns: _col2 (type: int)
Statistics: Num rows: 26 Data size: 5954 Basic stats: COMPLETE Column stats: COMPLETE
tag: 0
value expressions: _col0 (type: string), _col1 (type: string)
auto parallelism: true
->>>>>>> theirs
Execution mode: vectorized, llap
LLAP IO: no inputs
Path -> Alias:
@@ -824,19 +773,6 @@ STAGE PLANS:
filterExpr: (p_size is not null or (p_size is not null and p_type is not null)) (type: boolean)
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
-<<<<<<< ours
- Reduce Output Operator
- bucketingVersion: 2
- key expressions: (p_size + 1) (type: int)
- null sort order: z
- numBuckets: -1
- sort order: +
- Map-reduce partition columns: (p_size + 1) (type: int)
- Statistics: Num rows: 26 Data size: 5954 Basic stats: COMPLETE Column stats: COMPLETE
- tag: 0
- value expressions: p_name (type: string), p_type (type: string), p_size (type: int)
- auto parallelism: true
-=======
Filter Operator
isSamplingPred: false
predicate: p_size is not null (type: boolean)
@@ -853,8 +789,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 13 Data size: 260 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ bucketingVersion: 2
key expressions: _col0 (type: int)
null sort order: z
+ numBuckets: -1
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 13 Data size: 260 Basic stats: COMPLETE Column stats: COMPLETE
@@ -876,14 +814,15 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
Statistics: Num rows: 13 Data size: 1404 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ bucketingVersion: 2
key expressions: _col0 (type: int), _col1 (type: string)
null sort order: zz
+ numBuckets: -1
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: string)
Statistics: Num rows: 13 Data size: 1404 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
auto parallelism: true
->>>>>>> theirs
Execution mode: vectorized, llap
LLAP IO: no inputs
Path -> Alias:
@@ -1045,8 +984,10 @@ STAGE PLANS:
outputColumnNames: _col0
Statistics: Num rows: 13 Data size: 52 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ bucketingVersion: 2
key expressions: _col0 (type: int)
null sort order: z
+ numBuckets: -1
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 13 Data size: 52 Basic stats: COMPLETE Column stats: COMPLETE
@@ -1066,8 +1007,10 @@ STAGE PLANS:
Position of Big Table: 0
Statistics: Num rows: 42 Data size: 9890 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ bucketingVersion: 2
key expressions: _col1 (type: string), _col2 (type: int)
null sort order: zz
+ numBuckets: -1
sort order: ++
Map-reduce partition columns: _col1 (type: string), _col2 (type: int)
Statistics: Num rows: 42 Data size: 9890 Basic stats: COMPLETE Column stats: COMPLETE
@@ -1096,6 +1039,7 @@ STAGE PLANS:
outputColumnNames: _col0
Statistics: Num rows: 55 Data size: 6655 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
+ bucketingVersion: 2
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
@@ -1106,6 +1050,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
+ bucketing_version -1
columns _col0
columns.types string
escape.delim \
@@ -1159,84 +1104,7 @@ STAGE PLANS:
tag: 1
value expressions: _col1 (type: bigint), _col2 (type: bigint)
auto parallelism: true
-<<<<<<< ours
- Reducer 3
- Execution mode: llap
- Needs Tagging: false
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Left Outer Join 0 to 1
- keys:
- 0 (p_size + 1) (type: int)
- 1 _col0 (type: int)
- outputColumnNames: _col1, _col4, _col5, _col13, _col14
- Position of Big Table: 0
- Statistics: Num rows: 32 Data size: 7600 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- bucketingVersion: 2
- key expressions: _col4 (type: string), (_col5 + 1) (type: int)
- null sort order: zz
- numBuckets: -1
- sort order: ++
- Map-reduce partition columns: _col4 (type: string), (_col5 + 1) (type: int)
- Statistics: Num rows: 32 Data size: 7600 Basic stats: COMPLETE Column stats: COMPLETE
- tag: 0
- value expressions: _col1 (type: string), _col13 (type: bigint), _col14 (type: bigint)
- auto parallelism: true
- Reducer 4
- Execution mode: llap
- Needs Tagging: false
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Left Outer Join 0 to 1
- keys:
- 0 _col4 (type: string), (_col5 + 1) (type: int)
- 1 _col0 (type: string), _col1 (type: int)
- outputColumnNames: _col1, _col4, _col13, _col14, _col17
- Position of Big Table: 0
- Statistics: Num rows: 39 Data size: 9231 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col1 (type: string), _col4 (type: string), _col13 (type: bigint), _col14 (type: bigint), _col17 (type: boolean)
- outputColumnNames: _col0, _col1, _col4, _col5, _col8
- Statistics: Num rows: 39 Data size: 9387 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- isSamplingPred: false
- predicate: ((_col8 is null or (_col4 = 0L) or _col4 is null) and ((_col5 < _col4 is not true) or (_col4 = 0L) or _col4 is null or _col8 is not null or _col1 is null) and (_col1 is not null or (_col4 = 0L) or _col4 is null or _col8 is not null)) (type: boolean)
- Statistics: Num rows: 39 Data size: 9387 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 39 Data size: 4719 Basic stats: COMPLETE Column stats: COMPLETE
- File Output Operator
- bucketingVersion: 2
- compressed: false
- GlobalTableId: 0
-#### A masked pattern was here ####
- NumFilesPerFileSink: 1
- Statistics: Num rows: 39 Data size: 4719 Basic stats: COMPLETE Column stats: COMPLETE
-#### A masked pattern was here ####
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- properties:
- bucketing_version -1
- columns _col0
- columns.types string
- escape.delim \
- hive.serialization.extend.additional.nesting.levels true
- serialization.escape.crlf true
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- TotalFiles: 1
- GatherStats: false
- MultiFileSpray: false
- Reducer 5
-=======
Reducer 6
->>>>>>> theirs
Execution mode: vectorized, llap
Needs Tagging: false
Reduce Operator Tree:
@@ -1288,28 +1156,6 @@ STAGE PLANS:
tag: 1
value expressions: _col2 (type: boolean)
auto parallelism: true
-<<<<<<< ours
- Reducer 9
- Execution mode: vectorized, llap
- Needs Tagging: false
- Reduce Operator Tree:
- Group By Operator
- keys: KEY._col0 (type: int)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 52 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- bucketingVersion: 2
- key expressions: _col0 (type: int)
- null sort order: z
- numBuckets: -1
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 52 Basic stats: COMPLETE Column stats: COMPLETE
- tag: 1
- auto parallelism: true
-=======
->>>>>>> theirs
Stage: Stage-0
Fetch Operator