You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by se...@apache.org on 2015/09/30 21:06:32 UTC
[1/3] hive git commit: HIVE-12001 : LLAP: update some out files
(Sergey Shelukhin)
Repository: hive
Updated Branches:
refs/heads/llap 7a3e74579 -> 7273a4c44
http://git-wip-us.apache.org/repos/asf/hive/blob/7273a4c4/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out b/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out
index 9a8379a..b7b9a1b 100644
--- a/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out
+++ b/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out
@@ -321,12 +321,9 @@ POSTHOOK: Input: default@store_sales
18
19
20
-PREHOOK: query: -- The Reduce task has 2 MergePartial GROUP BY operators in a row. Currently,
--- we don't issue startGroup with keys out of the 1st vectorized GROUP BY, so we can't
--- vectorize the 2nd GROUP BY...
-explain
+PREHOOK: query: explain
select
- min(ss_ticket_number)
+ min(ss_ticket_number) m
from
(select
ss_ticket_number
@@ -334,14 +331,11 @@ from
store_sales
group by ss_ticket_number) a
group by ss_ticket_number
-limit 20
+order by m
PREHOOK: type: QUERY
-POSTHOOK: query: -- The Reduce task has 2 MergePartial GROUP BY operators in a row. Currently,
--- we don't issue startGroup with keys out of the 1st vectorized GROUP BY, so we can't
--- vectorize the 2nd GROUP BY...
-explain
+POSTHOOK: query: explain
select
- min(ss_ticket_number)
+ min(ss_ticket_number) m
from
(select
ss_ticket_number
@@ -349,7 +343,7 @@ from
store_sales
group by ss_ticket_number) a
group by ss_ticket_number
-limit 20
+order by m
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-1 is a root stage
@@ -360,6 +354,7 @@ STAGE PLANS:
Tez
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -383,7 +378,7 @@ STAGE PLANS:
Statistics: Num rows: 1000 Data size: 88276 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
Reducer 2
- Execution mode: uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
@@ -400,25 +395,33 @@ STAGE PLANS:
expressions: _col1 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
- Limit
- Number of rows: 20
- Statistics: Num rows: 20 Data size: 1760 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 20 Data size: 1760 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ Reducer 3
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
- limit: 20
+ limit: -1
Processor Tree:
ListSink
PREHOOK: query: select
- min(ss_ticket_number)
+ min(ss_ticket_number) m
from
(select
ss_ticket_number
@@ -426,12 +429,12 @@ from
store_sales
group by ss_ticket_number) a
group by ss_ticket_number
-limit 20
+order by m
PREHOOK: type: QUERY
PREHOOK: Input: default@store_sales
#### A masked pattern was here ####
POSTHOOK: query: select
- min(ss_ticket_number)
+ min(ss_ticket_number) m
from
(select
ss_ticket_number
@@ -439,7 +442,7 @@ from
store_sales
group by ss_ticket_number) a
group by ss_ticket_number
-limit 20
+order by m
POSTHOOK: type: QUERY
POSTHOOK: Input: default@store_sales
#### A masked pattern was here ####
@@ -463,3 +466,1417 @@ POSTHOOK: Input: default@store_sales
18
19
20
+21
+22
+23
+24
+25
+26
+27
+28
+29
+30
+31
+32
+33
+34
+35
+36
+37
+38
+39
+40
+41
+42
+43
+44
+45
+46
+47
+48
+49
+50
+51
+52
+53
+54
+55
+56
+57
+58
+59
+60
+61
+62
+63
+64
+65
+66
+67
+68
+69
+70
+71
+72
+73
+74
+75
+76
+77
+78
+79
+80
+81
+82
+PREHOOK: query: explain
+select
+ ss_ticket_number, sum(ss_item_sk), sum(q)
+from
+ (select
+ ss_ticket_number, ss_item_sk, min(ss_quantity) q
+ from
+ store_sales
+ group by ss_ticket_number, ss_item_sk) a
+group by ss_ticket_number
+order by ss_ticket_number
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ ss_ticket_number, sum(ss_item_sk), sum(q)
+from
+ (select
+ ss_ticket_number, ss_item_sk, min(ss_quantity) q
+ from
+ store_sales
+ group by ss_ticket_number, ss_item_sk) a
+group by ss_ticket_number
+order by ss_ticket_number
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+ Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: store_sales
+ Statistics: Num rows: 1000 Data size: 88276 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: ss_item_sk (type: int), ss_ticket_number (type: int), ss_quantity (type: int)
+ outputColumnNames: ss_item_sk, ss_ticket_number, ss_quantity
+ Statistics: Num rows: 1000 Data size: 88276 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: min(ss_quantity)
+ keys: ss_item_sk (type: int), ss_ticket_number (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1000 Data size: 88276 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int)
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
+ Statistics: Num rows: 1000 Data size: 88276 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col2 (type: int)
+ Execution mode: vectorized, llap
+ Reducer 2
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: min(VALUE._col0)
+ keys: KEY._col0 (type: int), KEY._col1 (type: int)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 500 Data size: 44138 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: sum(_col0), sum(_col2)
+ keys: _col1 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 500 Data size: 44138 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 500 Data size: 44138 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint), _col2 (type: bigint)
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: sum(VALUE._col0), sum(VALUE._col1)
+ keys: KEY._col0 (type: int)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint), _col2 (type: bigint)
+ Reducer 4
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: int), VALUE._col0 (type: bigint), VALUE._col1 (type: bigint)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select
+ ss_ticket_number, sum(ss_item_sk), sum(q)
+from
+ (select
+ ss_ticket_number, ss_item_sk, min(ss_quantity) q
+ from
+ store_sales
+ group by ss_ticket_number, ss_item_sk) a
+group by ss_ticket_number
+order by ss_ticket_number
+PREHOOK: type: QUERY
+PREHOOK: Input: default@store_sales
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ ss_ticket_number, sum(ss_item_sk), sum(q)
+from
+ (select
+ ss_ticket_number, ss_item_sk, min(ss_quantity) q
+ from
+ store_sales
+ group by ss_ticket_number, ss_item_sk) a
+group by ss_ticket_number
+order by ss_ticket_number
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@store_sales
+#### A masked pattern was here ####
+1 85411 816
+2 157365 812
+3 147948 710
+4 69545 411
+5 163232 840
+6 86307 627
+7 114874 563
+8 117953 662
+9 173250 690
+10 60338 602
+11 138545 657
+12 97181 586
+13 109484 555
+14 137333 442
+15 176829 652
+16 115004 654
+17 105008 460
+18 165135 738
+19 128252 831
+20 104789 374
+21 72771 469
+22 128153 449
+23 110253 603
+24 100662 1029
+25 118714 760
+26 81596 502
+27 164068 871
+28 58632 409
+29 133777 417
+30 130451 772
+31 114967 586
+32 142021 592
+33 151818 691
+34 112559 662
+35 137027 780
+36 118285 538
+37 94528 401
+38 81368 521
+39 101064 937
+40 84435 480
+41 112444 688
+42 95731 840
+43 57298 410
+44 159880 839
+45 68919 474
+46 111212 374
+47 78210 416
+48 94459 445
+49 90879 589
+50 37821 407
+51 124927 612
+52 98099 489
+53 138706 609
+54 87478 354
+55 90290 406
+56 78812 372
+57 101175 597
+58 88044 202
+59 104582 753
+60 99218 900
+61 66514 392
+62 126713 527
+63 98778 648
+64 131659 380
+65 86990 494
+66 108808 492
+67 75250 711
+68 91671 548
+69 92821 405
+70 75021 319
+71 124484 748
+72 161470 744
+73 104358 621
+74 88609 688
+75 92940 649
+76 75853 580
+77 124755 873
+78 98285 573
+79 160595 581
+80 151471 704
+81 105109 429
+82 55611 254
+PREHOOK: query: explain
+select
+ ss_ticket_number, ss_item_sk, sum(q)
+from
+ (select
+ ss_ticket_number, ss_item_sk, min(ss_quantity) q
+ from
+ store_sales
+ group by ss_ticket_number, ss_item_sk) a
+group by ss_ticket_number, ss_item_sk
+order by ss_ticket_number, ss_item_sk
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ ss_ticket_number, ss_item_sk, sum(q)
+from
+ (select
+ ss_ticket_number, ss_item_sk, min(ss_quantity) q
+ from
+ store_sales
+ group by ss_ticket_number, ss_item_sk) a
+group by ss_ticket_number, ss_item_sk
+order by ss_ticket_number, ss_item_sk
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: store_sales
+ Statistics: Num rows: 1000 Data size: 88276 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: ss_item_sk (type: int), ss_ticket_number (type: int), ss_quantity (type: int)
+ outputColumnNames: ss_item_sk, ss_ticket_number, ss_quantity
+ Statistics: Num rows: 1000 Data size: 88276 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: min(ss_quantity)
+ keys: ss_item_sk (type: int), ss_ticket_number (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1000 Data size: 88276 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int)
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
+ Statistics: Num rows: 1000 Data size: 88276 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col2 (type: int)
+ Execution mode: vectorized, llap
+ Reducer 2
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: min(VALUE._col0)
+ keys: KEY._col0 (type: int), KEY._col1 (type: int)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 500 Data size: 44138 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: sum(_col2)
+ keys: _col0 (type: int), _col1 (type: int)
+ mode: complete
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col1 (type: int), _col0 (type: int), _col2 (type: bigint)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int)
+ sort order: ++
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col2 (type: bigint)
+ Reducer 3
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: int), KEY.reducesinkkey1 (type: int), VALUE._col0 (type: bigint)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 250 Data size: 22069 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select
+ ss_ticket_number, ss_item_sk, sum(q)
+from
+ (select
+ ss_ticket_number, ss_item_sk, min(ss_quantity) q
+ from
+ store_sales
+ group by ss_ticket_number, ss_item_sk) a
+group by ss_ticket_number, ss_item_sk
+order by ss_ticket_number, ss_item_sk
+PREHOOK: type: QUERY
+PREHOOK: Input: default@store_sales
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ ss_ticket_number, ss_item_sk, sum(q)
+from
+ (select
+ ss_ticket_number, ss_item_sk, min(ss_quantity) q
+ from
+ store_sales
+ group by ss_ticket_number, ss_item_sk) a
+group by ss_ticket_number, ss_item_sk
+order by ss_ticket_number, ss_item_sk
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@store_sales
+#### A masked pattern was here ####
+1 49 5
+1 173 65
+1 1553 50
+1 3248 58
+1 3617 79
+1 4553 100
+1 4583 72
+1 4682 44
+1 5527 88
+1 5981 14
+1 10993 91
+1 13283 37
+1 13538 14
+1 13631 99
+2 1363 4
+2 2930 36
+2 3740 49
+2 6928 65
+2 7654 25
+2 9436 79
+2 10768 30
+2 12068 74
+2 12223 78
+2 13340 71
+2 13927 93
+2 14701 58
+2 15085 88
+2 15782 62
+2 17420 NULL
+3 246 96
+3 1531 NULL
+3 3525 42
+3 4698 98
+3 5355 53
+3 10693 27
+3 12447 82
+3 13021 64
+3 14100 79
+3 14443 4
+3 15786 56
+3 16869 4
+3 17263 17
+3 17971 88
+4 163 17
+4 1576 74
+4 5350 86
+4 5515 23
+4 6988 23
+4 7990 56
+4 8452 27
+4 9685 21
+4 11036 41
+4 12790 43
+5 1808 NULL
+5 1940 60
+5 5842 50
+5 6068 76
+5 6466 36
+5 11324 52
+5 11590 15
+5 12650 66
+5 13562 64
+5 13958 60
+5 14599 83
+5 14686 91
+5 15752 66
+5 16195 50
+5 16792 71
+6 2549 62
+6 2647 100
+6 3049 31
+6 3291 100
+6 6437 72
+6 8621 NULL
+6 10355 94
+6 10895 1
+6 11705 61
+6 13245 64
+6 13513 42
+7 4627 9
+7 4795 73
+7 4833 88
+7 5183 51
+7 5905 69
+7 8955 54
+7 9751 4
+7 10487 52
+7 12571 82
+7 15179 12
+7 15333 NULL
+7 17255 69
+8 665 31
+8 4183 90
+8 5929 83
+8 7115 54
+8 11365 7
+8 11893 95
+8 12041 95
+8 13427 87
+8 16671 20
+8 17119 51
+8 17545 49
+9 69 11
+9 889 6
+9 1185 62
+9 4623 34
+9 7945 83
+9 8334 71
+9 12027 27
+9 12969 59
+9 13483 NULL
+9 13717 53
+9 15133 15
+9 16083 32
+9 16363 54
+9 16461 66
+9 16659 84
+9 17310 33
+10 755 74
+10 1425 92
+10 1511 76
+10 3433 83
+10 3933 52
+10 4357 17
+10 5863 47
+10 9811 28
+10 13803 66
+10 15447 67
+11 157 84
+11 1315 70
+11 7519 68
+11 7608 66
+11 9901 57
+11 10699 33
+11 11490 NULL
+11 11991 38
+11 12438 16
+11 15157 96
+11 15649 33
+11 17226 11
+11 17395 85
+12 373 57
+12 1591 82
+12 4888 56
+12 6148 36
+12 6248 36
+12 9616 66
+12 9788 73
+12 13399 46
+12 14746 26
+12 14944 9
+12 15440 99
+13 868 NULL
+13 1760 12
+13 1898 NULL
+13 2108 9
+13 2191 NULL
+13 4430 73
+13 5971 80
+13 6085 58
+13 6140 15
+13 6682 80
+13 7640 48
+13 7723 27
+13 10096 12
+13 11758 34
+13 16894 87
+13 17240 20
+14 177 41
+14 769 20
+14 4507 4
+14 10175 19
+14 11549 6
+14 11653 60
+14 11817 81
+14 12587 NULL
+14 13069 77
+14 13515 57
+14 13845 17
+14 16741 46
+14 16929 14
+15 4241 21
+15 4505 59
+15 4777 28
+15 7391 98
+15 8336 15
+15 8353 NULL
+15 8690 32
+15 8707 21
+15 10361 39
+15 11659 80
+15 13172 25
+15 16619 81
+15 17267 7
+15 17330 82
+15 17564 26
+15 17857 38
+16 457 60
+16 1888 4
+16 4144 94
+16 6008 59
+16 7504 51
+16 8887 35
+16 9769 42
+16 9790 17
+16 9997 94
+16 11168 86
+16 11920 29
+16 16226 13
+16 17246 70
+17 2092 37
+17 4678 34
+17 6811 70
+17 9214 57
+17 10543 54
+17 11203 21
+17 13177 45
+17 13826 32
+17 15781 76
+17 17683 34
+18 2440 40
+18 5251 41
+18 7378 94
+18 8779 9
+18 8884 18
+18 9886 62
+18 11584 76
+18 11890 7
+18 12602 81
+18 12826 93
+18 12860 18
+18 14011 95
+18 14372 76
+18 14377 15
+18 17995 13
+19 1094 48
+19 3133 96
+19 3376 84
+19 4882 84
+19 6772 97
+19 7087 1
+19 7814 29
+19 8662 97
+19 9094 49
+19 9346 39
+19 10558 82
+19 10651 46
+19 11914 59
+19 16330 NULL
+19 17539 20
+20 1451 89
+20 2618 4
+20 5312 9
+20 5425 15
+20 5483 8
+20 6026 21
+20 7207 90
+20 8714 NULL
+20 9086 4
+20 9800 32
+20 13601 17
+20 14935 NULL
+20 15131 85
+21 230 48
+21 1810 59
+21 2870 50
+21 5170 45
+21 5998 51
+21 6476 49
+21 9187 14
+21 12266 47
+21 14368 18
+21 14396 88
+22 9985 70
+22 10474 31
+22 11599 66
+22 12415 10
+22 15310 15
+22 16396 85
+22 16922 88
+22 17392 14
+22 17660 70
+23 319 86
+23 7242 37
+23 8181 13
+23 8413 1
+23 9093 38
+23 9097 81
+23 11220 91
+23 11257 64
+23 12397 80
+23 15403 96
+23 17631 16
+24 407 53
+24 1389 72
+24 1795 21
+24 2497 85
+24 3103 73
+24 4425 57
+24 4749 28
+24 4873 41
+24 5653 92
+24 6043 1
+24 6751 82
+24 7375 97
+24 10265 93
+24 11551 48
+24 13303 97
+24 16483 89
+25 1333 55
+25 2150 100
+25 2608 76
+25 3454 100
+25 4880 29
+25 5954 34
+25 6955 40
+25 7874 65
+25 9472 48
+25 10159 24
+25 14488 26
+25 14635 68
+25 17000 40
+25 17752 55
+26 1989 26
+26 5053 4
+26 5385 97
+26 5721 81
+26 6647 64
+26 7337 45
+26 9679 18
+26 11895 77
+26 12851 56
+26 15039 34
+27 1305 44
+27 2137 96
+27 2671 92
+27 5831 61
+27 7139 59
+27 8167 28
+27 10757 15
+27 11441 15
+27 11509 65
+27 12237 89
+27 12749 31
+27 13885 66
+27 15025 26
+27 16029 59
+27 16419 65
+27 16767 60
+28 1807 98
+28 2817 8
+28 2967 29
+28 4483 78
+28 5437 15
+28 6411 3
+28 7965 93
+28 8043 58
+28 8407 14
+28 10295 13
+29 20 18
+29 1363 75
+29 2930 23
+29 3740 5
+29 7654 20
+29 9458 33
+29 10795 33
+29 12068 37
+29 12223 59
+29 13340 21
+29 13693 NULL
+29 15085 40
+29 15626 NULL
+29 15782 53
+30 217 91
+30 1951 59
+30 3238 16
+30 3506 15
+30 3928 87
+30 5431 77
+30 6752 69
+30 7870 7
+30 8666 21
+30 12572 33
+30 12670 20
+30 13579 75
+30 14848 62
+30 17348 62
+30 17875 78
+31 913 54
+31 4963 67
+31 6617 11
+31 6917 4
+31 7513 82
+31 11739 95
+31 14575 97
+31 14727 41
+31 15341 31
+31 15411 53
+31 16251 51
+32 1115 61
+32 2095 34
+32 2887 8
+32 4339 6
+32 4537 22
+32 4808 NULL
+32 5798 87
+32 7547 24
+32 9683 26
+32 11005 46
+32 11348 41
+32 12134 21
+32 15001 57
+32 15644 34
+32 16421 74
+32 17659 51
+33 4798 27
+33 7300 3
+33 9649 36
+33 10376 21
+33 11119 92
+33 11756 26
+33 12643 89
+33 12760 54
+33 12964 80
+33 14125 66
+33 14158 82
+33 14692 93
+33 15478 22
+34 1526 91
+34 1717 53
+34 2312 6
+34 4118 88
+34 5197 63
+34 5449 9
+34 6193 61
+34 9325 3
+34 9766 83
+34 12016 42
+34 12290 53
+34 12512 60
+34 13814 20
+34 16324 30
+35 411 51
+35 2377 52
+35 3667 97
+35 4325 56
+35 5179 83
+35 11635 87
+35 11661 81
+35 14239 55
+35 15619 45
+35 15757 9
+35 17341 92
+35 17365 65
+35 17451 7
+36 1115 80
+36 2095 43
+36 2887 31
+36 7547 46
+36 11005 49
+36 11349 80
+36 15001 54
+36 15645 23
+36 16421 25
+36 17561 16
+36 17659 91
+37 2997 94
+37 7283 87
+37 10715 52
+37 10929 88
+37 13171 6
+37 15337 62
+37 16971 12
+37 17125 NULL
+38 757 2
+38 2164 17
+38 3439 84
+38 4154 35
+38 5113 73
+38 6220 98
+38 7018 15
+38 7784 56
+38 8870 15
+38 9710 7
+38 10441 62
+38 15698 57
+39 386 89
+39 1598 64
+39 3476 73
+39 3943 64
+39 4190 86
+39 4957 24
+39 5393 98
+39 7097 78
+39 7118 67
+39 7604 49
+39 7697 24
+39 8078 54
+39 8411 96
+39 15491 54
+39 15625 17
+40 2854 71
+40 3490 65
+40 3985 63
+40 5098 35
+40 5318 87
+40 10094 80
+40 10912 23
+40 12050 NULL
+40 13658 53
+40 16976 3
+41 10 50
+41 64 29
+41 3380 88
+41 5566 11
+41 6310 90
+41 7402 69
+41 7603 94
+41 9322 8
+41 10915 81
+41 14788 15
+41 15242 87
+41 15328 46
+41 16514 20
+42 619 69
+42 976 100
+42 1436 94
+42 2314 74
+42 2392 14
+42 2602 30
+42 3346 74
+42 3613 30
+42 6058 30
+42 6134 92
+42 8462 23
+42 9740 52
+42 10016 57
+42 10471 19
+42 12550 41
+42 15002 41
+43 2923 16
+43 3344 22
+43 3911 26
+43 4364 77
+43 4691 41
+43 5773 85
+43 5852 16
+43 11771 30
+43 14669 97
+44 2351 56
+44 2623 18
+44 7303 14
+44 7527 67
+44 9059 68
+44 11707 83
+44 12341 20
+44 13331 98
+44 13449 45
+44 14149 80
+44 15803 81
+44 16491 56
+44 16837 92
+44 16909 61
+45 811 62
+45 1479 49
+45 3265 98
+45 5309 18
+45 7363 87
+45 10115 68
+45 11095 40
+45 13133 46
+45 16349 6
+46 1960 12
+46 3010 67
+46 7040 33
+46 8065 NULL
+46 11426 72
+46 13042 58
+46 15595 32
+46 16540 30
+46 17150 57
+46 17384 13
+47 254 NULL
+47 481 30
+47 1132 66
+47 1916 71
+47 3085 51
+47 3202 7
+47 3878 NULL
+47 4774 11
+47 5008 82
+47 5305 NULL
+47 5468 7
+47 7214 1
+47 9770 33
+47 13246 47
+47 13477 10
+48 1761 22
+48 2820 4
+48 2829 65
+48 4431 39
+48 5971 29
+48 6085 1
+48 6684 44
+48 9199 88
+48 11259 NULL
+48 12468 62
+48 13153 74
+48 17799 17
+49 749 60
+49 2135 4
+49 5342 69
+49 5852 47
+49 6805 40
+49 7141 94
+49 9049 68
+49 9553 71
+49 12737 48
+49 15155 84
+49 16361 4
+50 1280 69
+50 1312 30
+50 1909 53
+50 1984 40
+50 3097 64
+50 5023 NULL
+50 7135 69
+50 16081 82
+51 422 21
+51 3091 28
+51 4687 6
+51 5029 12
+51 5059 51
+51 6565 33
+51 8384 79
+51 9311 90
+51 10133 54
+51 11234 NULL
+51 12625 53
+51 13199 97
+51 17483 22
+51 17705 66
+52 2420 90
+52 3334 73
+52 6098 NULL
+52 7606 45
+52 11488 76
+52 15649 29
+52 16646 48
+52 17402 91
+52 17456 37
+53 1114 40
+53 2095 62
+53 2786 70
+53 2887 39
+53 7546 58
+53 11348 38
+53 13220 76
+53 13795 38
+53 15991 37
+53 16420 14
+53 16648 79
+53 17296 43
+53 17560 15
+54 702 40
+54 825 50
+54 1165 62
+54 3861 NULL
+54 6517 40
+54 9159 75
+54 14737 38
+54 16059 15
+54 16974 NULL
+54 17479 34
+55 1339 16
+55 3001 7
+55 5137 33
+55 9703 44
+55 12170 92
+55 12205 90
+55 14135 36
+55 14923 71
+55 17677 17
+56 4242 2
+56 4506 57
+56 8353 35
+56 8691 59
+56 8707 68
+56 10362 54
+56 16620 23
+56 17331 74
+57 3253 71
+57 4028 88
+57 4933 22
+57 12596 91
+57 12721 62
+57 12740 52
+57 15182 86
+57 17729 26
+57 17993 99
+58 1829 52
+58 3848 6
+58 5117 2
+58 7649 19
+58 9743 62
+58 10802 14
+58 15635 6
+58 16472 6
+58 16949 35
+59 3133 92
+59 3546 22
+59 5772 70
+59 7087 80
+59 8010 46
+59 8335 36
+59 9348 62
+59 9397 92
+59 10651 100
+59 11916 19
+59 12858 90
+59 14529 44
+60 97 50
+60 555 62
+60 633 71
+60 999 43
+60 1117 78
+60 1573 90
+60 4041 25
+60 4235 28
+60 4513 72
+60 4937 22
+60 7231 95
+60 10277 62
+60 10393 75
+60 13975 14
+60 16887 25
+60 17755 88
+61 1106 4
+61 2264 36
+61 3362 48
+61 4567 26
+61 5528 78
+61 6380 77
+61 7591 78
+61 8924 11
+61 10330 8
+61 16462 26
+62 4093 94
+62 6403 NULL
+62 8457 37
+62 10149 75
+62 12163 29
+62 12199 5
+62 12407 NULL
+62 13559 80
+62 15399 74
+62 15733 40
+62 16151 93
+63 4488 73
+63 5079 79
+63 5217 66
+63 5658 99
+63 9319 80
+63 11370 38
+63 11946 85
+63 13339 19
+63 15793 40
+63 16569 69
+64 1213 NULL
+64 3090 87
+64 3963 NULL
+64 11835 82
+64 13224 NULL
+64 14407 8
+64 15867 59
+64 15936 30
+64 16921 19
+64 17586 78
+64 17617 17
+65 2287 100
+65 4227 42
+65 9625 51
+65 9847 54
+65 13897 40
+65 14905 85
+65 15177 55
+65 17025 67
+66 6507 76
+66 7033 65
+66 7227 66
+66 8197 41
+66 9237 29
+66 10019 10
+66 11419 66
+66 15629 20
+66 16745 91
+66 16795 28
+67 757 77
+67 2133 74
+67 3439 73
+67 4155 87
+67 5113 NULL
+67 7020 79
+67 7507 77
+67 8469 59
+67 8871 71
+67 12087 70
+67 15699 44
+68 1387 74
+68 1603 57
+68 1820 54
+68 2035 22
+68 2296 52
+68 2564 83
+68 5162 23
+68 6763 77
+68 7765 NULL
+68 12526 3
+68 12724 88
+68 17426 2
+68 17600 13
+69 322 45
+69 337 34
+69 4208 9
+69 4267 10
+69 6136 7
+69 7264 67
+69 7822 30
+69 8599 53
+69 11137 68
+69 13489 66
+69 13792 NULL
+69 15448 16
+70 1592 53
+70 2462 NULL
+70 3296 48
+70 3947 NULL
+70 6185 82
+70 6425 NULL
+70 8893 17
+70 9857 20
+70 14549 4
+70 17815 95
+71 457 75
+71 1888 4
+71 2098 51
+71 4144 49
+71 5858 NULL
+71 6008 54
+71 7504 3
+71 8887 10
+71 9274 36
+71 9769 79
+71 9790 96
+71 9997 26
+71 10108 66
+71 10288 30
+71 11168 79
+71 17246 90
+72 1535 9
+72 5917 85
+72 6113 45
+72 6671 13
+72 9860 26
+72 10427 66
+72 10753 16
+72 11741 62
+72 12788 29
+72 12901 57
+72 13085 94
+72 13423 62
+72 13904 37
+72 15587 87
+72 16765 56
+73 247 53
+73 1063 37
+73 3205 82
+73 4946 54
+73 6862 58
+73 10051 49
+73 12502 75
+73 15109 38
+73 16519 97
+73 16585 38
+73 17269 40
+74 326 29
+74 3104 78
+74 3175 23
+74 3278 NULL
+74 3542 96
+74 3754 26
+74 5492 54
+74 7694 17
+74 8653 12
+74 9620 95
+74 10069 99
+74 13208 87
+74 16694 72
+75 607 20
+75 2948 25
+75 4625 73
+75 6938 89
+75 6953 71
+75 8726 6
+75 9905 54
+75 10217 85
+75 11039 70
+75 14186 63
+75 16796 93
+76 257 5
+76 465 2
+76 1107 16
+76 1503 97
+76 2265 98
+76 2869 32
+76 3363 25
+76 4237 48
+76 4567 40
+76 5529 78
+76 6381 50
+76 7591 27
+76 8925 6
+76 10331 3
+76 16463 53
+77 992 62
+77 1399 34
+77 2713 85
+77 3868 89
+77 6289 30
+77 7339 88
+77 7448 95
+77 7486 49
+77 8686 38
+77 9220 90
+77 11918 36
+77 12439 95
+77 13456 48
+77 14815 18
+77 16687 16
+78 901 3
+78 3304 50
+78 3856 27
+78 5965 78
+78 6044 59
+78 6110 43
+78 6500 76
+78 7576 87
+78 8611 79
+78 10507 6
+78 11209 7
+78 12706 19
+78 14996 39
+79 247 NULL
+79 1063 85
+79 3205 48
+79 4947 35
+79 6864 1
+79 10051 10
+79 10524 36
+79 12504 81
+79 14322 41
+79 15109 NULL
+79 15498 3
+79 15888 58
+79 16519 9
+79 16585 93
+79 17269 81
+80 998 93
+80 1519 25
+80 1573 40
+80 4040 66
+80 4513 NULL
+80 4622 1
+80 7231 49
+80 7610 37
+80 10393 5
+80 12968 NULL
+80 13717 91
+80 13975 13
+80 16363 84
+80 16886 77
+80 17308 29
+80 17755 94
+81 4486 31
+81 5078 75
+81 5216 64
+81 5656 24
+81 7166 7
+81 7663 79
+81 8918 37
+81 9319 36
+81 11107 36
+81 11368 26
+81 13339 6
+81 15793 8
+82 2572 53
+82 7862 75
+82 13138 59
+82 14998 49
+82 17041 18
http://git-wip-us.apache.org/repos/asf/hive/blob/7273a4c4/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out b/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out
index 1d2f8dd..afaf2d6 100644
--- a/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out
+++ b/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out
@@ -320,10 +320,10 @@ NULL
-47
-46
PREHOOK: query: explain
-select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint limit 20
+select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint order by ctinyint limit 20
PREHOOK: type: QUERY
POSTHOOK: query: explain
-select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint limit 20
+select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint order by ctinyint limit 20
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-1 is a root stage
@@ -387,11 +387,11 @@ STAGE PLANS:
Processor Tree:
ListSink
-PREHOOK: query: select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint limit 20
+PREHOOK: query: select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint order by ctinyint limit 20
PREHOOK: type: QUERY
PREHOOK: Input: default@alltypesorc
#### A masked pattern was here ####
-POSTHOOK: query: select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint limit 20
+POSTHOOK: query: select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint order by ctinyint limit 20
POSTHOOK: type: QUERY
POSTHOOK: Input: default@alltypesorc
#### A masked pattern was here ####
[2/3] hive git commit: HIVE-12001 : LLAP: update some out files
(Sergey Shelukhin)
Posted by se...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/7273a4c4/ql/src/test/results/clientpositive/llap/vector_auto_smb_mapjoin_14.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_auto_smb_mapjoin_14.q.out b/ql/src/test/results/clientpositive/llap/vector_auto_smb_mapjoin_14.q.out
new file mode 100644
index 0000000..c0103a8
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/vector_auto_smb_mapjoin_14.q.out
@@ -0,0 +1,1918 @@
+PREHOOK: query: -- SORT_QUERY_RESULTS
+
+CREATE TABLE tbl1(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORC
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@tbl1
+POSTHOOK: query: -- SORT_QUERY_RESULTS
+
+CREATE TABLE tbl1(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORC
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@tbl1
+PREHOOK: query: CREATE TABLE tbl2(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORC
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@tbl2
+POSTHOOK: query: CREATE TABLE tbl2(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORC
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@tbl2
+PREHOOK: query: insert overwrite table tbl1
+select * from src where key < 10
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: default@tbl1
+POSTHOOK: query: insert overwrite table tbl1
+select * from src where key < 10
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: default@tbl1
+POSTHOOK: Lineage: tbl1.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: tbl1.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+PREHOOK: query: insert overwrite table tbl2
+select * from src where key < 10
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: default@tbl2
+POSTHOOK: query: insert overwrite table tbl2
+select * from src where key < 10
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: default@tbl2
+POSTHOOK: Lineage: tbl2.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: tbl2.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+PREHOOK: query: -- The join is being performed as part of sub-query. It should be converted to a sort-merge join
+explain
+select count(*) from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+PREHOOK: type: QUERY
+POSTHOOK: query: -- The join is being performed as part of sub-query. It should be converted to a sort-merge join
+explain
+select count(*) from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 key (type: int)
+ 1 key (type: int)
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+22
+PREHOOK: query: -- The join is being performed as part of more than one sub-query. It should be converted to a sort-merge join
+explain
+select count(*) from
+(
+ select key, count(*) from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq1
+ group by key
+) subq2
+PREHOOK: type: QUERY
+POSTHOOK: query: -- The join is being performed as part of more than one sub-query. It should be converted to a sort-merge join
+explain
+select count(*) from
+(
+ select key, count(*) from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq1
+ group by key
+) subq2
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 key (type: int)
+ 1 key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ keys: KEY._col0 (type: int)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Reducer 3
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from
+(
+ select key, count(*) from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq1
+ group by key
+) subq2
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from
+(
+ select key, count(*) from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq1
+ group by key
+) subq2
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+6
+PREHOOK: query: -- A join is being performed across different sub-queries, where a join is being performed in each of them.
+-- Each sub-query should be converted to a sort-merge join.
+explain
+select src1.key, src1.cnt1, src2.cnt1 from
+(
+ select key, count(*) as cnt1 from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq1 group by key
+) src1
+join
+(
+ select key, count(*) as cnt1 from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq2 group by key
+) src2
+on src1.key = src2.key
+PREHOOK: type: QUERY
+POSTHOOK: query: -- A join is being performed across different sub-queries, where a join is being performed in each of them.
+-- Each sub-query should be converted to a sort-merge join.
+explain
+select src1.key, src1.cnt1, src2.cnt1 from
+(
+ select key, count(*) as cnt1 from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq1 group by key
+) src1
+join
+(
+ select key, count(*) as cnt1 from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq2 group by key
+) src2
+on src1.key = src2.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
+ Reducer 6 <- Map 5 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 key (type: int)
+ 1 key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
+ Execution mode: llap
+ Map 5
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 key (type: int)
+ 1 key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ keys: KEY._col0 (type: int)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
+ Reducer 3
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 2 Data size: 224 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: bigint), _col3 (type: bigint)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 2 Data size: 224 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 2 Data size: 224 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 6
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ keys: KEY._col0 (type: int)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select src1.key, src1.cnt1, src2.cnt1 from
+(
+ select key, count(*) as cnt1 from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq1 group by key
+) src1
+join
+(
+ select key, count(*) as cnt1 from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq2 group by key
+) src2
+on src1.key = src2.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select src1.key, src1.cnt1, src2.cnt1 from
+(
+ select key, count(*) as cnt1 from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq1 group by key
+) src1
+join
+(
+ select key, count(*) as cnt1 from
+ (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+ ) subq2 group by key
+) src2
+on src1.key = src2.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+0 9 9
+2 1 1
+4 1 1
+5 9 9
+8 1 1
+9 1 1
+PREHOOK: query: -- The subquery itself is being joined. Since the sub-query only contains selects and filters, it should
+-- be converted to a sort-merge join.
+explain
+select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq2
+ on subq1.key = subq2.key
+PREHOOK: type: QUERY
+POSTHOOK: query: -- The subquery itself is being joined. Since the sub-query only contains selects and filters, it should
+-- be converted to a sort-merge join.
+explain
+select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq2
+ on subq1.key = subq2.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (key < 6) (type: boolean)
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (key < 6) (type: boolean)
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ Statistics: Num rows: 3 Data size: 306 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq2
+ on subq1.key = subq2.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq2
+ on subq1.key = subq2.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+20
+PREHOOK: query: -- The subquery itself is being joined. Since the sub-query only contains selects and filters, it should
+-- be converted to a sort-merge join, although there is more than one level of sub-query
+explain
+select count(*) from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+ join tbl2 b
+ on subq2.key = b.key
+PREHOOK: type: QUERY
+POSTHOOK: query: -- The subquery itself is being joined. Since the sub-query only contains selects and filters, it should
+-- be converted to a sort-merge join, although there is more than one level of sub-query
+explain
+select count(*) from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+ join tbl2 b
+ on subq2.key = b.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (((key < 8) and (key < 6)) and key is not null) (type: boolean)
+ Statistics: Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 key (type: int)
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+ join tbl2 b
+ on subq2.key = b.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+ join tbl2 b
+ on subq2.key = b.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+20
+PREHOOK: query: -- Both the tables are nested sub-queries i.e more then 1 level of sub-query.
+-- The join should be converted to a sort-merge join
+explain
+select count(*) from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+ join
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq3
+ where key < 6
+ ) subq4
+ on subq2.key = subq4.key
+PREHOOK: type: QUERY
+POSTHOOK: query: -- Both the tables are nested sub-queries i.e more then 1 level of sub-query.
+-- The join should be converted to a sort-merge join
+explain
+select count(*) from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+ join
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq3
+ where key < 6
+ ) subq4
+ on subq2.key = subq4.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (((key < 8) and (key < 6)) and key is not null) (type: boolean)
+ Statistics: Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (((key < 8) and (key < 6)) and key is not null) (type: boolean)
+ Statistics: Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ Statistics: Num rows: 1 Data size: 102 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+ join
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq3
+ where key < 6
+ ) subq4
+ on subq2.key = subq4.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+ join
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq3
+ where key < 6
+ ) subq4
+ on subq2.key = subq4.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+#### A masked pattern was here ####
+20
+PREHOOK: query: -- The subquery itself is being joined. Since the sub-query only contains selects and filters and the join key
+-- is not getting modified, it should be converted to a sort-merge join. Note that the sub-query modifies one
+-- item, but that is not part of the join key.
+explain
+select count(*) from
+ (select a.key as key, concat(a.value, a.value) as value from tbl1 a where key < 8) subq1
+ join
+ (select a.key as key, concat(a.value, a.value) as value from tbl2 a where key < 8) subq2
+ on subq1.key = subq2.key
+PREHOOK: type: QUERY
+POSTHOOK: query: -- The subquery itself is being joined. Since the sub-query only contains selects and filters and the join key
+-- is not getting modified, it should be converted to a sort-merge join. Note that the sub-query modifies one
+-- item, but that is not part of the join key.
+explain
+select count(*) from
+ (select a.key as key, concat(a.value, a.value) as value from tbl1 a where key < 8) subq1
+ join
+ (select a.key as key, concat(a.value, a.value) as value from tbl2 a where key < 8) subq2
+ on subq1.key = subq2.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (key < 8) (type: boolean)
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (key < 8) (type: boolean)
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ Statistics: Num rows: 3 Data size: 306 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from
+ (select a.key as key, concat(a.value, a.value) as value from tbl1 a where key < 8) subq1
+ join
+ (select a.key as key, concat(a.value, a.value) as value from tbl2 a where key < 8) subq2
+ on subq1.key = subq2.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from
+ (select a.key as key, concat(a.value, a.value) as value from tbl1 a where key < 8) subq1
+ join
+ (select a.key as key, concat(a.value, a.value) as value from tbl2 a where key < 8) subq2
+ on subq1.key = subq2.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+20
+PREHOOK: query: -- Since the join key is modified by the sub-query, neither sort-merge join not bucketized map-side
+-- join should be performed
+explain
+select count(*) from
+ (select a.key +1 as key, concat(a.value, a.value) as value from tbl1 a) subq1
+ join
+ (select a.key +1 as key, concat(a.value, a.value) as value from tbl2 a) subq2
+ on subq1.key = subq2.key
+PREHOOK: type: QUERY
+POSTHOOK: query: -- Since the join key is modified by the sub-query, neither sort-merge join not bucketized map-side
+-- join should be performed
+explain
+select count(*) from
+ (select a.key +1 as key, concat(a.value, a.value) as value from tbl1 a) subq1
+ join
+ (select a.key +1 as key, concat(a.value, a.value) as value from tbl2 a) subq2
+ on subq1.key = subq2.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: (key + 1) (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: _col0 is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: vectorized, llap
+ Map 4
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: (key + 1) (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: _col0 is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: vectorized, llap
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Reducer 3
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from
+ (select a.key +1 as key, concat(a.value, a.value) as value from tbl1 a) subq1
+ join
+ (select a.key +1 as key, concat(a.value, a.value) as value from tbl2 a) subq2
+ on subq1.key = subq2.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from
+ (select a.key +1 as key, concat(a.value, a.value) as value from tbl1 a) subq1
+ join
+ (select a.key +1 as key, concat(a.value, a.value) as value from tbl2 a) subq2
+ on subq1.key = subq2.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+22
+PREHOOK: query: -- One of the tables is a sub-query and the other is not.
+-- It should be converted to a sort-merge join.
+explain
+select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join tbl2 a on subq1.key = a.key
+PREHOOK: type: QUERY
+POSTHOOK: query: -- One of the tables is a sub-query and the other is not.
+-- It should be converted to a sort-merge join.
+explain
+select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join tbl2 a on subq1.key = a.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (key < 6) (type: boolean)
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 key (type: int)
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join tbl2 a on subq1.key = a.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join tbl2 a on subq1.key = a.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+20
+PREHOOK: query: -- There are more than 2 inputs to the join, all of them being sub-queries.
+-- It should be converted to to a sort-merge join
+explain
+select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq2
+ on (subq1.key = subq2.key)
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq3
+ on (subq1.key = subq3.key)
+PREHOOK: type: QUERY
+POSTHOOK: query: -- There are more than 2 inputs to the join, all of them being sub-queries.
+-- It should be converted to to a sort-merge join
+explain
+select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq2
+ on (subq1.key = subq2.key)
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq3
+ on (subq1.key = subq3.key)
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (key < 6) (type: boolean)
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (key < 6) (type: boolean)
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (key < 6) (type: boolean)
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Inner Join 0 to 2
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ 2 _col0 (type: int)
+ Statistics: Num rows: 6 Data size: 613 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq2
+ on subq1.key = subq2.key
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq3
+ on (subq1.key = subq3.key)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from
+ (select a.key as key, a.value as value from tbl1 a where key < 6) subq1
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq2
+ on subq1.key = subq2.key
+ join
+ (select a.key as key, a.value as value from tbl2 a where key < 6) subq3
+ on (subq1.key = subq3.key)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+56
+PREHOOK: query: -- The join is being performed on a nested sub-query, and an aggregation is performed after that.
+-- The join should be converted to a sort-merge join
+explain
+select count(*) from (
+ select subq2.key as key, subq2.value as value1, b.value as value2 from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+join tbl2 b
+on subq2.key = b.key) a
+PREHOOK: type: QUERY
+POSTHOOK: query: -- The join is being performed on a nested sub-query, and an aggregation is performed after that.
+-- The join should be converted to a sort-merge join
+explain
+select count(*) from (
+ select subq2.key as key, subq2.value as value1, b.value as value2 from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+join tbl2 b
+on subq2.key = b.key) a
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (((key < 8) and (key < 6)) and key is not null) (type: boolean)
+ Statistics: Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 key (type: int)
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from (
+ select subq2.key as key, subq2.value as value1, b.value as value2 from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+join tbl2 b
+on subq2.key = b.key) a
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from (
+ select subq2.key as key, subq2.value as value1, b.value as value2 from
+ (
+ select * from
+ (
+ select a.key as key, a.value as value from tbl1 a where key < 8
+ ) subq1
+ where key < 6
+ ) subq2
+join tbl2 b
+on subq2.key = b.key) a
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+#### A masked pattern was here ####
+20
+PREHOOK: query: CREATE TABLE dest1(key int, value string)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@dest1
+POSTHOOK: query: CREATE TABLE dest1(key int, value string)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@dest1
+PREHOOK: query: CREATE TABLE dest2(key int, val1 string, val2 string)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@dest2
+POSTHOOK: query: CREATE TABLE dest2(key int, val1 string, val2 string)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@dest2
+PREHOOK: query: -- The join is followed by a multi-table insert. It should be converted to
+-- a sort-merge join
+explain
+from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+insert overwrite table dest1 select key, val1
+insert overwrite table dest2 select key, val1, val2
+PREHOOK: type: QUERY
+POSTHOOK: query: -- The join is followed by a multi-table insert. It should be converted to
+-- a sort-merge join
+explain
+from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+insert overwrite table dest1 select key, val1
+insert overwrite table dest2 select key, val1, val2
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-2 is a root stage
+ Stage-3 depends on stages: Stage-2
+ Stage-0 depends on stages: Stage-3
+ Stage-4 depends on stages: Stage-0
+ Stage-1 depends on stages: Stage-3
+ Stage-5 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-2
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 key (type: int)
+ 1 key (type: int)
+ outputColumnNames: _col0, _col1, _col6
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: string), _col6 (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.dest1
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.dest2
+ Execution mode: llap
+
+ Stage: Stage-3
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.dest1
+
+ Stage: Stage-4
+ Stats-Aggr Operator
+
+ Stage: Stage-1
+ Move Operator
+ tables:
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.dest2
+
+ Stage: Stage-5
+ Stats-Aggr Operator
+
+PREHOOK: query: from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+insert overwrite table dest1 select key, val1
+insert overwrite table dest2 select key, val1, val2
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+PREHOOK: Output: default@dest1
+PREHOOK: Output: default@dest2
+POSTHOOK: query: from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+insert overwrite table dest1 select key, val1
+insert overwrite table dest2 select key, val1, val2
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+POSTHOOK: Output: default@dest1
+POSTHOOK: Output: default@dest2
+POSTHOOK: Lineage: dest1.key SIMPLE [(tbl1)a.FieldSchema(name:key, type:int, comment:null), ]
+POSTHOOK: Lineage: dest1.value SIMPLE [(tbl1)a.FieldSchema(name:value, type:string, comment:null), ]
+POSTHOOK: Lineage: dest2.key SIMPLE [(tbl1)a.FieldSchema(name:key, type:int, comment:null), ]
+POSTHOOK: Lineage: dest2.val1 SIMPLE [(tbl1)a.FieldSchema(name:value, type:string, comment:null), ]
+POSTHOOK: Lineage: dest2.val2 SIMPLE [(tbl2)b.FieldSchema(name:value, type:string, comment:null), ]
+PREHOOK: query: select * from dest1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@dest1
+#### A masked pattern was here ####
+POSTHOOK: query: select * from dest1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@dest1
+#### A masked pattern was here ####
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+2 val_2
+4 val_4
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+8 val_8
+9 val_9
+PREHOOK: query: select * from dest2
+PREHOOK: type: QUERY
+PREHOOK: Input: default@dest2
+#### A masked pattern was here ####
+POSTHOOK: query: select * from dest2
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@dest2
+#### A masked pattern was here ####
+0 val_0 val_0
+0 val_0 val_0
+0 val_0 val_0
+0 val_0 val_0
+0 val_0 val_0
+0 val_0 val_0
+0 val_0 val_0
+0 val_0 val_0
+0 val_0 val_0
+2 val_2 val_2
+4 val_4 val_4
+5 val_5 val_5
+5 val_5 val_5
+5 val_5 val_5
+5 val_5 val_5
+5 val_5 val_5
+5 val_5 val_5
+5 val_5 val_5
+5 val_5 val_5
+5 val_5 val_5
+8 val_8 val_8
+9 val_9 val_9
+PREHOOK: query: DROP TABLE dest2
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@dest2
+PREHOOK: Output: default@dest2
+POSTHOOK: query: DROP TABLE dest2
+POSTHOOK: type: DROPTABLE
+POSTHOOK: Input: default@dest2
+POSTHOOK: Output: default@dest2
+PREHOOK: query: CREATE TABLE dest2(key int, cnt int)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@dest2
+POSTHOOK: query: CREATE TABLE dest2(key int, cnt int)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@dest2
+PREHOOK: query: -- The join is followed by a multi-table insert, and one of the inserts involves a reducer.
+-- It should be converted to a sort-merge join
+explain
+from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+insert overwrite table dest1 select key, val1
+insert overwrite table dest2 select key, count(*) group by key
+PREHOOK: type: QUERY
+POSTHOOK: query: -- The join is followed by a multi-table insert, and one of the inserts involves a reducer.
+-- It should be converted to a sort-merge join
+explain
+from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+insert overwrite table dest1 select key, val1
+insert overwrite table dest2 select key, count(*) group by key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-2 is a root stage
+ Stage-3 depends on stages: Stage-2
+ Stage-0 depends on stages: Stage-3
+ Stage-4 depends on stages: Stage-0
+ Stage-1 depends on stages: Stage-3
+ Stage-5 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-2
+ Tez
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 10 Data size: 930 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 5 Data size: 465 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 key (type: int)
+ 1 key (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.dest1
+ Select Operator
+ expressions: _col0 (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 5 Data size: 511 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: vectorized, uber
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ keys: KEY._col0 (type: int)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: int), UDFToInteger(_col1) (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.dest2
+
+ Stage: Stage-3
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.dest1
+
+ Stage: Stage-4
+ Stats-Aggr Operator
+
+ Stage: Stage-1
+ Move Operator
+ tables:
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.dest2
+
+ Stage: Stage-5
+ Stats-Aggr Operator
+
+PREHOOK: query: from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+insert overwrite table dest1 select key, val1
+insert overwrite table dest2 select key, count(*) group by key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl1
+PREHOOK: Input: default@tbl2
+PREHOOK: Output: default@dest1
+PREHOOK: Output: default@dest2
+POSTHOOK: query: from (
+ select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 b on a.key = b.key
+) subq1
+insert overwrite table dest1 select key, val1
+insert overwrite table dest2 select key, count(*) group by key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl1
+POSTHOOK: Input: default@tbl2
+POSTHOOK: Output: default@dest1
+POSTHOOK: Output: default@dest2
+POSTHOOK: Lineage: dest1.key SIMPLE [(tbl1)a.FieldSchema(name:key, type:int, comment:null), ]
+POSTHOOK: Lineage: dest1.value SIMPLE [(tbl1)a.FieldSchema(name:value, type:string, comment:null), ]
+POSTHOOK: Lineage: dest2.cnt EXPRESSION [(tbl1)a.null, (tbl2)b.null, ]
+POSTHOOK: Lineage: dest2.key SIMPLE [(tbl1)a.FieldSchema(name:key, type:int, comment:null), ]
+PREHOOK: query: select * from dest1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@dest1
+#### A masked pattern was here ####
+POSTHOOK: query: select * from dest1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@dest1
+#### A masked pattern was here ####
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+2 val_2
+4 val_4
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+5 val_5
+8 val_8
+9 val_9
+PREHOOK: query: select * from dest2
+PREHOOK: type: QUERY
+PREHOOK: Input: default@dest2
+#### A masked pattern was here ####
+POSTHOOK: query: select * from dest2
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@dest2
+#### A masked pattern was here ####
+0 9
+2 1
+4 1
+5 9
+8 1
+9 1
[3/3] hive git commit: HIVE-12001 : LLAP: update some out files
(Sergey Shelukhin)
Posted by se...@apache.org.
HIVE-12001 : LLAP: update some out files (Sergey Shelukhin)
Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/7273a4c4
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/7273a4c4
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/7273a4c4
Branch: refs/heads/llap
Commit: 7273a4c4405f38ed7de95a4bf915b0309cf75dc0
Parents: 7a3e745
Author: Sergey Shelukhin <se...@apache.org>
Authored: Wed Sep 30 12:07:04 2015 -0700
Committer: Sergey Shelukhin <se...@apache.org>
Committed: Wed Sep 30 12:07:04 2015 -0700
----------------------------------------------------------------------
.../llap/dynpart_sort_opt_vectorization.q.out | 12 +-
.../llap/dynpart_sort_optimization.q.out | 12 +-
.../clientpositive/llap/explainuser_1.q.out | 76 +-
.../llap/vector_auto_smb_mapjoin_14.q.out | 1918 ++++++++++++++++++
.../llap/vector_groupby_reduce.q.out | 1473 +++++++++++++-
.../llap/vectorization_limit.q.out | 8 +-
6 files changed, 3421 insertions(+), 78 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hive/blob/7273a4c4/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out b/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
index 07a87c8..2f70b40 100644
--- a/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
@@ -1821,13 +1821,15 @@ Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
-PREHOOK: query: select * from over1k_part2_orc
+PREHOOK: query: -- SORT_BEFORE_DIFF
+select * from over1k_part2_orc
PREHOOK: type: QUERY
PREHOOK: Input: default@over1k_part2_orc
PREHOOK: Input: default@over1k_part2_orc@ds=foo/t=27
PREHOOK: Input: default@over1k_part2_orc@ds=foo/t=__HIVE_DEFAULT_PARTITION__
#### A masked pattern was here ####
-POSTHOOK: query: select * from over1k_part2_orc
+POSTHOOK: query: -- SORT_BEFORE_DIFF
+select * from over1k_part2_orc
POSTHOOK: type: QUERY
POSTHOOK: Input: default@over1k_part2_orc
POSTHOOK: Input: default@over1k_part2_orc@ds=foo/t=27
@@ -1966,13 +1968,15 @@ Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
-PREHOOK: query: select * from over1k_part2_orc
+PREHOOK: query: -- SORT_BEFORE_DIFF
+select * from over1k_part2_orc
PREHOOK: type: QUERY
PREHOOK: Input: default@over1k_part2_orc
PREHOOK: Input: default@over1k_part2_orc@ds=foo/t=27
PREHOOK: Input: default@over1k_part2_orc@ds=foo/t=__HIVE_DEFAULT_PARTITION__
#### A masked pattern was here ####
-POSTHOOK: query: select * from over1k_part2_orc
+POSTHOOK: query: -- SORT_BEFORE_DIFF
+select * from over1k_part2_orc
POSTHOOK: type: QUERY
POSTHOOK: Input: default@over1k_part2_orc
POSTHOOK: Input: default@over1k_part2_orc@ds=foo/t=27
http://git-wip-us.apache.org/repos/asf/hive/blob/7273a4c4/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out b/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out
index 2534254..fa15157 100644
--- a/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out
@@ -1752,13 +1752,15 @@ Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
-PREHOOK: query: select * from over1k_part2
+PREHOOK: query: -- SORT_BEFORE_DIFF
+select * from over1k_part2
PREHOOK: type: QUERY
PREHOOK: Input: default@over1k_part2
PREHOOK: Input: default@over1k_part2@ds=foo/t=27
PREHOOK: Input: default@over1k_part2@ds=foo/t=__HIVE_DEFAULT_PARTITION__
#### A masked pattern was here ####
-POSTHOOK: query: select * from over1k_part2
+POSTHOOK: query: -- SORT_BEFORE_DIFF
+select * from over1k_part2
POSTHOOK: type: QUERY
POSTHOOK: Input: default@over1k_part2
POSTHOOK: Input: default@over1k_part2@ds=foo/t=27
@@ -1897,13 +1899,15 @@ Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
-PREHOOK: query: select * from over1k_part2
+PREHOOK: query: -- SORT_BEFORE_DIFF
+select * from over1k_part2
PREHOOK: type: QUERY
PREHOOK: Input: default@over1k_part2
PREHOOK: Input: default@over1k_part2@ds=foo/t=27
PREHOOK: Input: default@over1k_part2@ds=foo/t=__HIVE_DEFAULT_PARTITION__
#### A masked pattern was here ####
-POSTHOOK: query: select * from over1k_part2
+POSTHOOK: query: -- SORT_BEFORE_DIFF
+select * from over1k_part2
POSTHOOK: type: QUERY
POSTHOOK: Input: default@over1k_part2
POSTHOOK: Input: default@over1k_part2@ds=foo/t=27
http://git-wip-us.apache.org/repos/asf/hive/blob/7273a4c4/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
index 7d9d99e..d41e02f 100644
--- a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
@@ -1986,7 +1986,7 @@ Stage-0
Statistics:Num rows: 4 Data size: 728 Basic stats: COMPLETE Column stats: COMPLETE
value expressions:_col1 (type: int), _col2 (type: float), _col3 (type: string), _col4 (type: int)
Filter Operator [FIL_27]
- predicate:((((_col1 + _col4) = 2) and _col0 is not null) and ((_col4 + 1) = 2)) (type: boolean)
+ predicate:((((_col1 + _col4) = 2) and ((_col4 + 1) = 2)) and _col0 is not null) (type: boolean)
Statistics:Num rows: 4 Data size: 728 Basic stats: COMPLETE Column stats: COMPLETE
Merge Join Operator [MERGEJOIN_31]
| condition map:[{"":"Outer Join 0 to 1"}]
@@ -2797,7 +2797,7 @@ Stage-0
| outputColumnNames:["_col0","_col1","_col2","_col3"]
| Statistics:Num rows: 1 Data size: 105 Basic stats: COMPLETE Column stats: COMPLETE
| Filter Operator [FIL_61]
- | predicate:(((UDFToDouble(_col0) + UDFToDouble(_col3)) >= 0.0) and ((UDFToDouble(_col0) >= 1.0) or (_col3 >= 1))) (type: boolean)
+ | predicate:(((UDFToDouble(_col0) >= 1.0) or (_col3 >= 1)) and ((UDFToDouble(_col0) + UDFToDouble(_col3)) >= 0.0)) (type: boolean)
| Statistics:Num rows: 1 Data size: 97 Basic stats: COMPLETE Column stats: COMPLETE
| Select Operator [SEL_65]
| outputColumnNames:["_col0","_col1","_col3"]
@@ -4240,7 +4240,7 @@ Stage-0
Select Operator [SEL_13]
Statistics:Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator [FIL_45]
- predicate:((_col0 is null or _col1 is null) and ((_col2 - _col1) > 600.0)) (type: boolean)
+ predicate:(((_col2 - _col1) > 600.0) and (_col0 is null or _col1 is null)) (type: boolean)
Statistics:Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator [GBY_11]
| aggregations:["min(VALUE._col0)","max(VALUE._col1)"]
@@ -6031,36 +6031,36 @@ Stage-0
Reducer 2
File Output Operator [FS_6]
compressed:true
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
Select Operator [SEL_5]
outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
Merge Join Operator [MERGEJOIN_7]
| condition map:[{"":"Inner Join 0 to 1"}]
| keys:{"0":"key (type: int)","1":"key (type: int)"}
| outputColumnNames:["_col0","_col1","_col5","_col6"]
- | Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
|<-Map 1 [SIMPLE_EDGE]
| Reduce Output Operator [RS_2]
| key expressions:key (type: int)
| Map-reduce partition columns:key (type: int)
| sort order:+
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
| value expressions:value (type: int)
| TableScan [TS_0]
| alias:a
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
|<-Map 3 [SIMPLE_EDGE]
Reduce Output Operator [RS_3]
key expressions:key (type: int)
Map-reduce partition columns:key (type: int)
sort order:+
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
value expressions:value (type: int)
TableScan [TS_1]
alias:b
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
PREHOOK: query: explain select /*+ MAPJOIN(a) */ * FROM smb_input1 a JOIN smb_input1 b ON a.key <=> b.key AND a.value <=> b.value
PREHOOK: type: QUERY
@@ -6078,34 +6078,34 @@ Stage-0
Reducer 2
File Output Operator [FS_6]
compressed:true
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
Select Operator [SEL_5]
outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
Merge Join Operator [MERGEJOIN_7]
| condition map:[{"":"Inner Join 0 to 1"}]
| keys:{"0":"key (type: int), value (type: int)","1":"key (type: int), value (type: int)"}
| outputColumnNames:["_col0","_col1","_col5","_col6"]
- | Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
|<-Map 1 [SIMPLE_EDGE]
| Reduce Output Operator [RS_2]
| key expressions:key (type: int), value (type: int)
| Map-reduce partition columns:key (type: int), value (type: int)
| sort order:++
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
| TableScan [TS_0]
| alias:a
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
|<-Map 3 [SIMPLE_EDGE]
Reduce Output Operator [RS_3]
key expressions:key (type: int), value (type: int)
Map-reduce partition columns:key (type: int), value (type: int)
sort order:++
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
TableScan [TS_1]
alias:b
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
PREHOOK: query: explain select /*+ MAPJOIN(a) */ * FROM smb_input1 a RIGHT OUTER JOIN smb_input1 b ON a.key <=> b.key
PREHOOK: type: QUERY
@@ -6123,36 +6123,36 @@ Stage-0
Reducer 2
File Output Operator [FS_6]
compressed:true
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
Select Operator [SEL_5]
outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
Merge Join Operator [MERGEJOIN_7]
| condition map:[{"":"Right Outer Join0 to 1"}]
| keys:{"0":"key (type: int)","1":"key (type: int)"}
| outputColumnNames:["_col0","_col1","_col5","_col6"]
- | Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
|<-Map 1 [SIMPLE_EDGE]
| Reduce Output Operator [RS_2]
| key expressions:key (type: int)
| Map-reduce partition columns:key (type: int)
| sort order:+
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
| value expressions:value (type: int)
| TableScan [TS_0]
| alias:a
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
|<-Map 3 [SIMPLE_EDGE]
Reduce Output Operator [RS_3]
key expressions:key (type: int)
Map-reduce partition columns:key (type: int)
sort order:+
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
value expressions:value (type: int)
TableScan [TS_1]
alias:b
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
PREHOOK: query: explain select /*+ MAPJOIN(b) */ * FROM smb_input1 a JOIN smb_input1 b ON a.key <=> b.key
PREHOOK: type: QUERY
@@ -6170,36 +6170,36 @@ Stage-0
Reducer 2
File Output Operator [FS_6]
compressed:true
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
Select Operator [SEL_5]
outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
Merge Join Operator [MERGEJOIN_7]
| condition map:[{"":"Inner Join 0 to 1"}]
| keys:{"0":"key (type: int)","1":"key (type: int)"}
| outputColumnNames:["_col0","_col1","_col5","_col6"]
- | Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
|<-Map 1 [SIMPLE_EDGE]
| Reduce Output Operator [RS_2]
| key expressions:key (type: int)
| Map-reduce partition columns:key (type: int)
| sort order:+
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
| value expressions:value (type: int)
| TableScan [TS_0]
| alias:a
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
|<-Map 3 [SIMPLE_EDGE]
Reduce Output Operator [RS_3]
key expressions:key (type: int)
Map-reduce partition columns:key (type: int)
sort order:+
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
value expressions:value (type: int)
TableScan [TS_1]
alias:b
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
PREHOOK: query: explain select /*+ MAPJOIN(b) */ * FROM smb_input1 a LEFT OUTER JOIN smb_input1 b ON a.key <=> b.key
PREHOOK: type: QUERY
@@ -6217,36 +6217,36 @@ Stage-0
Reducer 2
File Output Operator [FS_6]
compressed:true
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
Select Operator [SEL_5]
outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
Merge Join Operator [MERGEJOIN_7]
| condition map:[{"":"Left Outer Join0 to 1"}]
| keys:{"0":"key (type: int)","1":"key (type: int)"}
| outputColumnNames:["_col0","_col1","_col5","_col6"]
- | Statistics:Num rows: 28 Data size: 209 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 24 Data size: 174 Basic stats: COMPLETE Column stats: NONE
|<-Map 1 [SIMPLE_EDGE]
| Reduce Output Operator [RS_2]
| key expressions:key (type: int)
| Map-reduce partition columns:key (type: int)
| sort order:+
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
| value expressions:value (type: int)
| TableScan [TS_0]
| alias:a
- | Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ | Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
|<-Map 3 [SIMPLE_EDGE]
Reduce Output Operator [RS_3]
key expressions:key (type: int)
Map-reduce partition columns:key (type: int)
sort order:+
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
value expressions:value (type: int)
TableScan [TS_1]
alias:b
- Statistics:Num rows: 26 Data size: 190 Basic stats: COMPLETE Column stats: NONE
+ Statistics:Num rows: 22 Data size: 159 Basic stats: COMPLETE Column stats: NONE
PREHOOK: query: drop table sales
PREHOOK: type: DROPTABLE