You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by ha...@apache.org on 2017/05/10 05:09:18 UTC
[1/6] hive git commit: HIVE-16330 : Improve plans for scalar subquery
with aggregates (Vineet Garg via Ashutosh Chauhan)
Repository: hive
Updated Branches:
refs/heads/master 4ba48aa5f -> a113ede99
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/results/clientpositive/perf/query9.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/query9.q.out b/ql/src/test/results/clientpositive/perf/query9.q.out
index d714d41..3c913e1 100644
--- a/ql/src/test/results/clientpositive/perf/query9.q.out
+++ b/ql/src/test/results/clientpositive/perf/query9.q.out
@@ -1,33 +1,18 @@
-Warning: Shuffle Join MERGEJOIN[456][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[457][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
-Warning: Shuffle Join MERGEJOIN[458][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
-Warning: Shuffle Join MERGEJOIN[459][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 5' is a cross product
-Warning: Shuffle Join MERGEJOIN[460][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5]] in Stage 'Reducer 6' is a cross product
-Warning: Shuffle Join MERGEJOIN[461][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6]] in Stage 'Reducer 7' is a cross product
-Warning: Shuffle Join MERGEJOIN[462][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7]] in Stage 'Reducer 8' is a cross product
-Warning: Shuffle Join MERGEJOIN[463][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8]] in Stage 'Reducer 9' is a cross product
-Warning: Shuffle Join MERGEJOIN[464][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9]] in Stage 'Reducer 10' is a cross product
-Warning: Shuffle Join MERGEJOIN[465][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10]] in Stage 'Reducer 11' is a cross product
-Warning: Shuffle Join MERGEJOIN[466][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11]] in Stage 'Reducer 12' is a cross product
-Warning: Shuffle Join MERGEJOIN[467][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12]] in Stage 'Reducer 13' is a cross product
-Warning: Shuffle Join MERGEJOIN[468][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13]] in Stage 'Reducer 14' is a cross product
-Warning: Shuffle Join MERGEJOIN[469][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14]] in Stage 'Reducer 15' is a cross product
-Warning: Shuffle Join MERGEJOIN[470][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15]] in Stage 'Reducer 16' is a cross product
-Warning: Shuffle Join MERGEJOIN[471][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16]] in Stage 'Reducer 17' is a cross product
-Warning: Shuffle Join MERGEJOIN[472][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17]] in Stage 'Reducer 18' is a cross product
-Warning: Shuffle Join MERGEJOIN[473][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18]] in Stage 'Reducer 19' is a cross product
-Warning: Shuffle Join MERGEJOIN[474][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19]] in Stage 'Reducer 20' is a cross product
-Warning: Shuffle Join MERGEJOIN[475][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20]] in Stage 'Reducer 21' is a cross product
-Warning: Shuffle Join MERGEJOIN[476][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21]] in Stage 'Reducer 22' is a cross product
-Warning: Shuffle Join MERGEJOIN[477][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21, $hdt$_22]] in Stage 'Reducer 23' is a cross product
-Warning: Shuffle Join MERGEJOIN[478][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21, $hdt$_22, $hdt$_23]] in Stage 'Reducer 24' is a cross product
-Warning: Shuffle Join MERGEJOIN[479][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21, $hdt$_22, $hdt$_23, $hdt$_24]] in Stage 'Reducer 25' is a cross product
-Warning: Shuffle Join MERGEJOIN[480][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21, $hdt$_22, $hdt$_23, $hdt$_24, $hdt$_25]] in Stage 'Reducer 26' is a cross product
-Warning: Shuffle Join MERGEJOIN[481][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21, $hdt$_22, $hdt$_23, $hdt$_24, $hdt$_25, $hdt$_26]] in Stage 'Reducer 27' is a cross product
-Warning: Shuffle Join MERGEJOIN[482][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21, $hdt$_22, $hdt$_23, $hdt$_24, $hdt$_25, $hdt$_26, $hdt$_27]] in Stage 'Reducer 28' is a cross product
-Warning: Shuffle Join MERGEJOIN[483][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21, $hdt$_22, $hdt$_23, $hdt$_24, $hdt$_25, $hdt$_26, $hdt$_27, $hdt$_28]] in Stage 'Reducer 29' is a cross product
-Warning: Shuffle Join MERGEJOIN[484][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21, $hdt$_22, $hdt$_23, $hdt$_24, $hdt$_25, $hdt$_26, $hdt$_27, $hdt$_28, $hdt$_29]] in Stage 'Reducer 30' is a cross product
-Warning: Shuffle Join MERGEJOIN[485][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15, $hdt$_16, $hdt$_17, $hdt$_18, $hdt$_19, $hdt$_20, $hdt$_21, $hdt$_22, $hdt$_23, $hdt$_24, $hdt$_25, $hdt$_26, $hdt$_27, $hdt$_28, $hdt$_29, $hdt$_30]] in Stage 'Reducer 31' is a cross product
+Warning: Shuffle Join MERGEJOIN[171][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[172][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[173][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[174][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 5' is a cross product
+Warning: Shuffle Join MERGEJOIN[175][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5]] in Stage 'Reducer 6' is a cross product
+Warning: Shuffle Join MERGEJOIN[176][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6]] in Stage 'Reducer 7' is a cross product
+Warning: Shuffle Join MERGEJOIN[177][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7]] in Stage 'Reducer 8' is a cross product
+Warning: Shuffle Join MERGEJOIN[178][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8]] in Stage 'Reducer 9' is a cross product
+Warning: Shuffle Join MERGEJOIN[179][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9]] in Stage 'Reducer 10' is a cross product
+Warning: Shuffle Join MERGEJOIN[180][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10]] in Stage 'Reducer 11' is a cross product
+Warning: Shuffle Join MERGEJOIN[181][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11]] in Stage 'Reducer 12' is a cross product
+Warning: Shuffle Join MERGEJOIN[182][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12]] in Stage 'Reducer 13' is a cross product
+Warning: Shuffle Join MERGEJOIN[183][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13]] in Stage 'Reducer 14' is a cross product
+Warning: Shuffle Join MERGEJOIN[184][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14]] in Stage 'Reducer 15' is a cross product
+Warning: Shuffle Join MERGEJOIN[185][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6, $hdt$_7, $hdt$_8, $hdt$_9, $hdt$_10, $hdt$_11, $hdt$_12, $hdt$_13, $hdt$_14, $hdt$_15]] in Stage 'Reducer 16' is a cross product
PREHOOK: query: explain select case when (select count(*)
from store_sales
where ss_quantity between 1 and 20) > 409437
@@ -127,703 +112,313 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Vertex dependency in root stage
-Reducer 10 <- Reducer 49 (CUSTOM_SIMPLE_EDGE), Reducer 9 (CUSTOM_SIMPLE_EDGE)
-Reducer 11 <- Reducer 10 (CUSTOM_SIMPLE_EDGE), Reducer 51 (CUSTOM_SIMPLE_EDGE)
-Reducer 12 <- Reducer 11 (CUSTOM_SIMPLE_EDGE), Reducer 53 (CUSTOM_SIMPLE_EDGE)
-Reducer 13 <- Reducer 12 (CUSTOM_SIMPLE_EDGE), Reducer 55 (CUSTOM_SIMPLE_EDGE)
-Reducer 14 <- Reducer 13 (CUSTOM_SIMPLE_EDGE), Reducer 57 (CUSTOM_SIMPLE_EDGE)
-Reducer 15 <- Reducer 14 (CUSTOM_SIMPLE_EDGE), Reducer 59 (CUSTOM_SIMPLE_EDGE)
-Reducer 16 <- Reducer 15 (CUSTOM_SIMPLE_EDGE), Reducer 61 (CUSTOM_SIMPLE_EDGE)
-Reducer 17 <- Reducer 16 (CUSTOM_SIMPLE_EDGE), Reducer 63 (CUSTOM_SIMPLE_EDGE)
-Reducer 18 <- Reducer 17 (CUSTOM_SIMPLE_EDGE), Reducer 65 (CUSTOM_SIMPLE_EDGE)
-Reducer 19 <- Reducer 18 (CUSTOM_SIMPLE_EDGE), Reducer 67 (CUSTOM_SIMPLE_EDGE)
-Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 33 (CUSTOM_SIMPLE_EDGE)
-Reducer 20 <- Reducer 19 (CUSTOM_SIMPLE_EDGE), Reducer 69 (CUSTOM_SIMPLE_EDGE)
-Reducer 21 <- Reducer 20 (CUSTOM_SIMPLE_EDGE), Reducer 71 (CUSTOM_SIMPLE_EDGE)
-Reducer 22 <- Reducer 21 (CUSTOM_SIMPLE_EDGE), Reducer 73 (CUSTOM_SIMPLE_EDGE)
-Reducer 23 <- Reducer 22 (CUSTOM_SIMPLE_EDGE), Reducer 75 (CUSTOM_SIMPLE_EDGE)
-Reducer 24 <- Reducer 23 (CUSTOM_SIMPLE_EDGE), Reducer 77 (CUSTOM_SIMPLE_EDGE)
-Reducer 25 <- Reducer 24 (CUSTOM_SIMPLE_EDGE), Reducer 79 (CUSTOM_SIMPLE_EDGE)
-Reducer 26 <- Reducer 25 (CUSTOM_SIMPLE_EDGE), Reducer 81 (CUSTOM_SIMPLE_EDGE)
-Reducer 27 <- Reducer 26 (CUSTOM_SIMPLE_EDGE), Reducer 83 (CUSTOM_SIMPLE_EDGE)
-Reducer 28 <- Reducer 27 (CUSTOM_SIMPLE_EDGE), Reducer 85 (CUSTOM_SIMPLE_EDGE)
-Reducer 29 <- Reducer 28 (CUSTOM_SIMPLE_EDGE), Reducer 87 (CUSTOM_SIMPLE_EDGE)
-Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 35 (CUSTOM_SIMPLE_EDGE)
-Reducer 30 <- Reducer 29 (CUSTOM_SIMPLE_EDGE), Reducer 89 (CUSTOM_SIMPLE_EDGE)
-Reducer 31 <- Reducer 30 (CUSTOM_SIMPLE_EDGE), Reducer 91 (CUSTOM_SIMPLE_EDGE)
-Reducer 33 <- Map 32 (CUSTOM_SIMPLE_EDGE)
-Reducer 35 <- Map 34 (CUSTOM_SIMPLE_EDGE)
-Reducer 37 <- Map 36 (CUSTOM_SIMPLE_EDGE)
-Reducer 39 <- Map 38 (CUSTOM_SIMPLE_EDGE)
-Reducer 4 <- Reducer 3 (CUSTOM_SIMPLE_EDGE), Reducer 37 (CUSTOM_SIMPLE_EDGE)
-Reducer 41 <- Map 40 (CUSTOM_SIMPLE_EDGE)
-Reducer 43 <- Map 42 (CUSTOM_SIMPLE_EDGE)
-Reducer 45 <- Map 44 (CUSTOM_SIMPLE_EDGE)
-Reducer 47 <- Map 46 (CUSTOM_SIMPLE_EDGE)
-Reducer 49 <- Map 48 (CUSTOM_SIMPLE_EDGE)
-Reducer 5 <- Reducer 39 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
-Reducer 51 <- Map 50 (CUSTOM_SIMPLE_EDGE)
-Reducer 53 <- Map 52 (CUSTOM_SIMPLE_EDGE)
-Reducer 55 <- Map 54 (CUSTOM_SIMPLE_EDGE)
-Reducer 57 <- Map 56 (CUSTOM_SIMPLE_EDGE)
-Reducer 59 <- Map 58 (CUSTOM_SIMPLE_EDGE)
-Reducer 6 <- Reducer 41 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
-Reducer 61 <- Map 60 (CUSTOM_SIMPLE_EDGE)
-Reducer 63 <- Map 62 (CUSTOM_SIMPLE_EDGE)
-Reducer 65 <- Map 64 (CUSTOM_SIMPLE_EDGE)
-Reducer 67 <- Map 66 (CUSTOM_SIMPLE_EDGE)
-Reducer 69 <- Map 68 (CUSTOM_SIMPLE_EDGE)
-Reducer 7 <- Reducer 43 (CUSTOM_SIMPLE_EDGE), Reducer 6 (CUSTOM_SIMPLE_EDGE)
-Reducer 71 <- Map 70 (CUSTOM_SIMPLE_EDGE)
-Reducer 73 <- Map 72 (CUSTOM_SIMPLE_EDGE)
-Reducer 75 <- Map 74 (CUSTOM_SIMPLE_EDGE)
-Reducer 77 <- Map 76 (CUSTOM_SIMPLE_EDGE)
-Reducer 79 <- Map 78 (CUSTOM_SIMPLE_EDGE)
-Reducer 8 <- Reducer 45 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
-Reducer 81 <- Map 80 (CUSTOM_SIMPLE_EDGE)
-Reducer 83 <- Map 82 (CUSTOM_SIMPLE_EDGE)
-Reducer 85 <- Map 84 (CUSTOM_SIMPLE_EDGE)
-Reducer 87 <- Map 86 (CUSTOM_SIMPLE_EDGE)
-Reducer 89 <- Map 88 (CUSTOM_SIMPLE_EDGE)
-Reducer 9 <- Reducer 47 (CUSTOM_SIMPLE_EDGE), Reducer 8 (CUSTOM_SIMPLE_EDGE)
-Reducer 91 <- Map 90 (CUSTOM_SIMPLE_EDGE)
+Reducer 10 <- Reducer 34 (CUSTOM_SIMPLE_EDGE), Reducer 9 (CUSTOM_SIMPLE_EDGE)
+Reducer 11 <- Reducer 10 (CUSTOM_SIMPLE_EDGE), Reducer 36 (CUSTOM_SIMPLE_EDGE)
+Reducer 12 <- Reducer 11 (CUSTOM_SIMPLE_EDGE), Reducer 38 (CUSTOM_SIMPLE_EDGE)
+Reducer 13 <- Reducer 12 (CUSTOM_SIMPLE_EDGE), Reducer 40 (CUSTOM_SIMPLE_EDGE)
+Reducer 14 <- Reducer 13 (CUSTOM_SIMPLE_EDGE), Reducer 42 (CUSTOM_SIMPLE_EDGE)
+Reducer 15 <- Reducer 14 (CUSTOM_SIMPLE_EDGE), Reducer 44 (CUSTOM_SIMPLE_EDGE)
+Reducer 16 <- Reducer 15 (CUSTOM_SIMPLE_EDGE), Reducer 46 (CUSTOM_SIMPLE_EDGE)
+Reducer 18 <- Map 17 (CUSTOM_SIMPLE_EDGE)
+Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 18 (CUSTOM_SIMPLE_EDGE)
+Reducer 20 <- Map 19 (CUSTOM_SIMPLE_EDGE)
+Reducer 22 <- Map 21 (CUSTOM_SIMPLE_EDGE)
+Reducer 24 <- Map 23 (CUSTOM_SIMPLE_EDGE)
+Reducer 26 <- Map 25 (CUSTOM_SIMPLE_EDGE)
+Reducer 28 <- Map 27 (CUSTOM_SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 20 (CUSTOM_SIMPLE_EDGE)
+Reducer 30 <- Map 29 (CUSTOM_SIMPLE_EDGE)
+Reducer 32 <- Map 31 (CUSTOM_SIMPLE_EDGE)
+Reducer 34 <- Map 33 (CUSTOM_SIMPLE_EDGE)
+Reducer 36 <- Map 35 (CUSTOM_SIMPLE_EDGE)
+Reducer 38 <- Map 37 (CUSTOM_SIMPLE_EDGE)
+Reducer 4 <- Reducer 22 (CUSTOM_SIMPLE_EDGE), Reducer 3 (CUSTOM_SIMPLE_EDGE)
+Reducer 40 <- Map 39 (CUSTOM_SIMPLE_EDGE)
+Reducer 42 <- Map 41 (CUSTOM_SIMPLE_EDGE)
+Reducer 44 <- Map 43 (CUSTOM_SIMPLE_EDGE)
+Reducer 46 <- Map 45 (CUSTOM_SIMPLE_EDGE)
+Reducer 5 <- Reducer 24 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
+Reducer 6 <- Reducer 26 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
+Reducer 7 <- Reducer 28 (CUSTOM_SIMPLE_EDGE), Reducer 6 (CUSTOM_SIMPLE_EDGE)
+Reducer 8 <- Reducer 30 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
+Reducer 9 <- Reducer 32 (CUSTOM_SIMPLE_EDGE), Reducer 8 (CUSTOM_SIMPLE_EDGE)
Stage-0
Fetch Operator
limit:-1
Stage-1
- Reducer 31
- File Output Operator [FS_424]
- Select Operator [SEL_423] (rows=36 width=3270)
+ Reducer 16
+ File Output Operator [FS_154]
+ Select Operator [SEL_153] (rows=36 width=3135)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_485] (rows=36 width=3270)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20","_col22","_col24","_col26","_col28","_col30"]
- <-Reducer 30 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_420]
- Merge Join Operator [MERGEJOIN_484] (rows=36 width=2981)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20","_col22","_col24","_col26","_col28"]
- <-Reducer 29 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_417]
- Merge Join Operator [MERGEJOIN_483] (rows=36 width=2972)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20","_col22","_col24","_col26","_col28"]
- <-Reducer 28 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_414]
- Merge Join Operator [MERGEJOIN_482] (rows=36 width=2683)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20","_col22","_col24","_col26"]
- <-Reducer 27 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_411]
- Merge Join Operator [MERGEJOIN_481] (rows=36 width=2674)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20","_col22","_col24","_col26"]
- <-Reducer 26 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_408]
- Merge Join Operator [MERGEJOIN_480] (rows=36 width=2665)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20","_col22","_col24"]
- <-Reducer 25 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_405]
- Merge Join Operator [MERGEJOIN_479] (rows=36 width=2656)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20","_col22","_col24"]
- <-Reducer 24 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_402]
- Merge Join Operator [MERGEJOIN_478] (rows=36 width=2367)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20","_col22"]
- <-Reducer 23 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_399]
- Merge Join Operator [MERGEJOIN_477] (rows=36 width=2358)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20","_col22"]
- <-Reducer 22 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_396]
- Merge Join Operator [MERGEJOIN_476] (rows=36 width=2069)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20"]
- <-Reducer 21 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_393]
- Merge Join Operator [MERGEJOIN_475] (rows=36 width=2060)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18","_col20"]
- <-Reducer 20 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_390]
- Merge Join Operator [MERGEJOIN_474] (rows=36 width=2051)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18"]
- <-Reducer 19 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_387]
- Merge Join Operator [MERGEJOIN_473] (rows=36 width=2042)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16","_col18"]
- <-Reducer 18 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_384]
- Merge Join Operator [MERGEJOIN_472] (rows=36 width=1753)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16"]
- <-Reducer 17 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_381]
- Merge Join Operator [MERGEJOIN_471] (rows=36 width=1744)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14","_col16"]
- <-Reducer 16 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_378]
- Merge Join Operator [MERGEJOIN_470] (rows=36 width=1455)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14"]
- <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_375]
- Merge Join Operator [MERGEJOIN_469] (rows=36 width=1446)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12","_col14"]
- <-Reducer 14 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_372]
- Merge Join Operator [MERGEJOIN_468] (rows=36 width=1437)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10","_col12"]
- <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_369]
- Merge Join Operator [MERGEJOIN_467] (rows=36 width=1428)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10","_col12"]
- <-Reducer 12 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_366]
- Merge Join Operator [MERGEJOIN_466] (rows=36 width=1139)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8","_col10"]
- <-Reducer 11 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_363]
- Merge Join Operator [MERGEJOIN_465] (rows=36 width=1130)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8","_col10"]
- <-Reducer 10 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_360]
- Merge Join Operator [MERGEJOIN_464] (rows=36 width=841)
- Conds:(Inner),Output:["_col2","_col4","_col6","_col8"]
- <-Reducer 49 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_358]
- Select Operator [SEL_105] (rows=1 width=8)
- Filter Operator [FIL_104] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_102] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_97] (rows=1 width=8)
- Group By Operator [GBY_96] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 48 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_95]
- Group By Operator [GBY_94] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_93] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_434] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 21 AND 40
- TableScan [TS_91] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 9 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_357]
- Merge Join Operator [MERGEJOIN_463] (rows=36 width=832)
- Conds:(Left Outer),Output:["_col2","_col4","_col6","_col8"]
- <-Reducer 47 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_355]
- Group By Operator [GBY_89] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 46 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_88]
- Group By Operator [GBY_87] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_86] (rows=63999515 width=88)
- Filter Operator [FIL_433] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 21 AND 40
- TableScan [TS_84] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity"]
- <-Reducer 8 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_354]
- Merge Join Operator [MERGEJOIN_462] (rows=36 width=823)
- Conds:(Inner),Output:["_col2","_col4","_col6"]
- <-Reducer 45 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_352]
- Select Operator [SEL_83] (rows=1 width=8)
- Filter Operator [FIL_82] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_80] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_75] (rows=1 width=8)
- Group By Operator [GBY_74] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 44 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_73]
- Group By Operator [GBY_72] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_71] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_432] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 21 AND 40
- TableScan [TS_69] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 7 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_351]
- Merge Join Operator [MERGEJOIN_461] (rows=36 width=814)
- Conds:(Left Outer),Output:["_col2","_col4","_col6"]
- <-Reducer 43 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_349]
- Group By Operator [GBY_67] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 42 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_66]
- Group By Operator [GBY_65] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_net_paid_inc_tax)"]
- Select Operator [SEL_64] (rows=63999515 width=88)
- Output:["ss_net_paid_inc_tax"]
- Filter Operator [FIL_431] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 1 AND 20
- TableScan [TS_62] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_net_paid_inc_tax"]
- <-Reducer 6 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_348]
- Merge Join Operator [MERGEJOIN_460] (rows=36 width=525)
- Conds:(Inner),Output:["_col2","_col4"]
- <-Reducer 41 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_346]
- Select Operator [SEL_61] (rows=1 width=8)
- Filter Operator [FIL_60] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_58] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_53] (rows=1 width=8)
- Group By Operator [GBY_52] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 40 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_51]
- Group By Operator [GBY_50] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_49] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_430] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 1 AND 20
- TableScan [TS_47] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 5 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_345]
- Merge Join Operator [MERGEJOIN_459] (rows=36 width=516)
- Conds:(Left Outer),Output:["_col2","_col4"]
- <-Reducer 39 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_343]
- Group By Operator [GBY_45] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 38 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_44]
- Group By Operator [GBY_43] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_ext_list_price)"]
- Select Operator [SEL_42] (rows=63999515 width=88)
- Output:["ss_ext_list_price"]
- Filter Operator [FIL_429] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 1 AND 20
- TableScan [TS_40] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_ext_list_price"]
- <-Reducer 4 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_342]
- Merge Join Operator [MERGEJOIN_458] (rows=36 width=227)
- Conds:(Inner),Output:["_col2"]
- <-Reducer 3 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_339]
- Merge Join Operator [MERGEJOIN_457] (rows=36 width=218)
- Conds:(Left Outer),Output:["_col2"]
- <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_336]
- Merge Join Operator [MERGEJOIN_456] (rows=36 width=209)
- Conds:(Inner)
- <-Map 1 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_333]
- Select Operator [SEL_2] (rows=36 width=200)
- Filter Operator [FIL_425] (rows=36 width=200)
- predicate:(r_reason_sk = 1)
- TableScan [TS_0] (rows=72 width=200)
- default@reason,reason,Tbl:COMPLETE,Col:NONE,Output:["r_reason_sk"]
- <-Reducer 33 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_334]
- Select Operator [SEL_17] (rows=1 width=8)
- Filter Operator [FIL_16] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_14] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_9] (rows=1 width=8)
- Group By Operator [GBY_8] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 32 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_7]
- Group By Operator [GBY_6] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_5] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_426] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 1 AND 20
- TableScan [TS_3] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 35 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_337]
- Group By Operator [GBY_23] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 34 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_22]
- Group By Operator [GBY_21] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_20] (rows=63999515 width=88)
- Filter Operator [FIL_427] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 1 AND 20
- TableScan [TS_18] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity"]
- <-Reducer 37 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_340]
- Select Operator [SEL_39] (rows=1 width=8)
- Filter Operator [FIL_38] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_36] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_31] (rows=1 width=8)
- Group By Operator [GBY_30] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 36 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_29]
- Group By Operator [GBY_28] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_27] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_428] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 1 AND 20
- TableScan [TS_25] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 51 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_361]
- Group By Operator [GBY_111] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 50 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_110]
- Group By Operator [GBY_109] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_ext_list_price)"]
- Select Operator [SEL_108] (rows=63999515 width=88)
- Output:["ss_ext_list_price"]
- Filter Operator [FIL_435] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 21 AND 40
- TableScan [TS_106] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_ext_list_price"]
- <-Reducer 53 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_364]
- Select Operator [SEL_127] (rows=1 width=8)
- Filter Operator [FIL_126] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_124] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_119] (rows=1 width=8)
- Group By Operator [GBY_118] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 52 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_117]
- Group By Operator [GBY_116] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_115] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_436] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 21 AND 40
- TableScan [TS_113] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 55 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_367]
- Group By Operator [GBY_133] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 54 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_132]
- Group By Operator [GBY_131] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_net_paid_inc_tax)"]
- Select Operator [SEL_130] (rows=63999515 width=88)
- Output:["ss_net_paid_inc_tax"]
- Filter Operator [FIL_437] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 21 AND 40
- TableScan [TS_128] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_net_paid_inc_tax"]
- <-Reducer 57 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_370]
- Select Operator [SEL_149] (rows=1 width=8)
- Filter Operator [FIL_148] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_146] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_141] (rows=1 width=8)
- Group By Operator [GBY_140] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 56 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_139]
- Group By Operator [GBY_138] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_137] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_438] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 41 AND 60
- TableScan [TS_135] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 59 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_373]
- Group By Operator [GBY_155] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 58 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_154]
- Group By Operator [GBY_153] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_152] (rows=63999515 width=88)
- Filter Operator [FIL_439] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 41 AND 60
- TableScan [TS_150] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity"]
- <-Reducer 61 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_376]
- Select Operator [SEL_171] (rows=1 width=8)
- Filter Operator [FIL_170] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_168] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_163] (rows=1 width=8)
- Group By Operator [GBY_162] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 60 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_161]
- Group By Operator [GBY_160] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_159] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_440] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 41 AND 60
- TableScan [TS_157] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 63 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_379]
- Group By Operator [GBY_177] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 62 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_176]
- Group By Operator [GBY_175] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_ext_list_price)"]
- Select Operator [SEL_174] (rows=63999515 width=88)
- Output:["ss_ext_list_price"]
- Filter Operator [FIL_441] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 41 AND 60
- TableScan [TS_172] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_ext_list_price"]
- <-Reducer 65 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_382]
- Select Operator [SEL_193] (rows=1 width=8)
- Filter Operator [FIL_192] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_190] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_185] (rows=1 width=8)
- Group By Operator [GBY_184] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 64 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_183]
- Group By Operator [GBY_182] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_181] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_442] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 41 AND 60
- TableScan [TS_179] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 67 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_385]
- Group By Operator [GBY_199] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 66 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_198]
- Group By Operator [GBY_197] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_net_paid_inc_tax)"]
- Select Operator [SEL_196] (rows=63999515 width=88)
- Output:["ss_net_paid_inc_tax"]
- Filter Operator [FIL_443] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 41 AND 60
- TableScan [TS_194] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_net_paid_inc_tax"]
- <-Reducer 69 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_388]
- Select Operator [SEL_215] (rows=1 width=8)
- Filter Operator [FIL_214] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_212] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_207] (rows=1 width=8)
- Group By Operator [GBY_206] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 68 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_205]
- Group By Operator [GBY_204] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_203] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_444] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 61 AND 80
- TableScan [TS_201] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 71 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_391]
- Group By Operator [GBY_221] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 70 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_220]
- Group By Operator [GBY_219] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_218] (rows=63999515 width=88)
- Filter Operator [FIL_445] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 61 AND 80
- TableScan [TS_216] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity"]
- <-Reducer 73 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_394]
- Select Operator [SEL_237] (rows=1 width=8)
- Filter Operator [FIL_236] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_234] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_229] (rows=1 width=8)
- Group By Operator [GBY_228] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 72 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_227]
- Group By Operator [GBY_226] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_225] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_446] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 61 AND 80
- TableScan [TS_223] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 75 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_397]
- Group By Operator [GBY_243] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 74 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_242]
- Group By Operator [GBY_241] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_ext_list_price)"]
- Select Operator [SEL_240] (rows=63999515 width=88)
- Output:["ss_ext_list_price"]
- Filter Operator [FIL_447] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 61 AND 80
- TableScan [TS_238] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_ext_list_price"]
- <-Reducer 77 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_400]
- Select Operator [SEL_259] (rows=1 width=8)
- Filter Operator [FIL_258] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_256] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_251] (rows=1 width=8)
- Group By Operator [GBY_250] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Map 76 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_249]
- Group By Operator [GBY_248] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(ss_sold_date_sk)"]
- Select Operator [SEL_247] (rows=63999515 width=88)
- Output:["ss_sold_date_sk"]
- Filter Operator [FIL_448] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 61 AND 80
- TableScan [TS_245] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 79 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_403]
- Group By Operator [GBY_265] (rows=1 width=288)
+ Merge Join Operator [MERGEJOIN_185] (rows=36 width=3135)
+ Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
+ <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_150]
+ Merge Join Operator [MERGEJOIN_184] (rows=36 width=2846)
+ Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
+ <-Reducer 14 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_147]
+ Merge Join Operator [MERGEJOIN_183] (rows=36 width=2557)
+ Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
+ <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_144]
+ Merge Join Operator [MERGEJOIN_182] (rows=36 width=2548)
+ Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12"]
+ <-Reducer 12 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_141]
+ Merge Join Operator [MERGEJOIN_181] (rows=36 width=2259)
+ Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ <-Reducer 11 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_138]
+ Merge Join Operator [MERGEJOIN_180] (rows=36 width=1970)
+ Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"]
+ <-Reducer 10 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_135]
+ Merge Join Operator [MERGEJOIN_179] (rows=36 width=1961)
+ Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
+ <-Reducer 34 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_133]
+ Group By Operator [GBY_64] (rows=1 width=288)
Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 78 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_264]
- Group By Operator [GBY_263] (rows=1 width=288)
+ <-Map 33 [CUSTOM_SIMPLE_EDGE]
+
<TRUNCATED>
[2/6] hive git commit: HIVE-16330 : Improve plans for scalar subquery
with aggregates (Vineet Garg via Ashutosh Chauhan)
Posted by ha...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/results/clientpositive/perf/query81.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/query81.q.out b/ql/src/test/results/clientpositive/perf/query81.q.out
index a09d5c9..769a3d1 100644
--- a/ql/src/test/results/clientpositive/perf/query81.q.out
+++ b/ql/src/test/results/clientpositive/perf/query81.q.out
@@ -59,222 +59,159 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Vertex dependency in root stage
-Reducer 10 <- Reducer 9 (SIMPLE_EDGE)
-Reducer 11 <- Reducer 10 (SIMPLE_EDGE), Reducer 17 (SIMPLE_EDGE)
-Reducer 15 <- Map 14 (SIMPLE_EDGE), Map 18 (SIMPLE_EDGE)
-Reducer 16 <- Map 19 (SIMPLE_EDGE), Reducer 15 (SIMPLE_EDGE)
-Reducer 17 <- Reducer 16 (SIMPLE_EDGE)
-Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE)
-Reducer 21 <- Map 20 (SIMPLE_EDGE), Map 24 (SIMPLE_EDGE)
-Reducer 22 <- Map 25 (SIMPLE_EDGE), Reducer 21 (SIMPLE_EDGE)
-Reducer 23 <- Reducer 22 (SIMPLE_EDGE)
-Reducer 3 <- Reducer 11 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
-Reducer 4 <- Reducer 23 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
-Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
-Reducer 8 <- Map 12 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
-Reducer 9 <- Map 13 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
+Reducer 10 <- Reducer 16 (SIMPLE_EDGE), Reducer 9 (SIMPLE_EDGE)
+Reducer 14 <- Map 13 (SIMPLE_EDGE), Map 17 (SIMPLE_EDGE)
+Reducer 15 <- Map 18 (SIMPLE_EDGE), Reducer 14 (SIMPLE_EDGE)
+Reducer 16 <- Reducer 15 (SIMPLE_EDGE)
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 10 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
+Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
+Reducer 7 <- Map 11 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE)
+Reducer 8 <- Map 12 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
+Reducer 9 <- Reducer 8 (SIMPLE_EDGE)
Stage-0
Fetch Operator
limit:100
Stage-1
- Reducer 5
- File Output Operator [FS_101]
- Limit [LIM_100] (rows=100 width=860)
+ Reducer 4
+ File Output Operator [FS_67]
+ Limit [LIM_66] (rows=100 width=860)
Number of rows:100
- Select Operator [SEL_99] (rows=35493335 width=860)
+ Select Operator [SEL_65] (rows=32266667 width=860)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_98]
- Select Operator [SEL_97] (rows=35493335 width=860)
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_64]
+ Select Operator [SEL_63] (rows=32266667 width=860)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col11","_col12","_col13","_col14","_col15"]
- Filter Operator [FIL_96] (rows=35493335 width=860)
- predicate:(_col2 > CASE WHEN (_col24 is null) THEN (null) ELSE (_col23) END)
- Merge Join Operator [MERGEJOIN_153] (rows=106480005 width=860)
- Conds:RS_93._col1=RS_94._col2(Left Outer),Output:["_col2","_col4","_col5","_col6","_col7","_col8","_col9","_col11","_col12","_col13","_col14","_col16","_col18","_col19","_col20","_col23","_col24"]
- <-Reducer 23 [SIMPLE_EDGE]
- SHUFFLE [RS_94]
- PartitionCols:_col2
- Select Operator [SEL_92] (rows=11000000 width=1014)
- Output:["_col0","_col1","_col2"]
- Group By Operator [GBY_91] (rows=11000000 width=1014)
- Output:["_col0","_col1"],aggregations:["avg(_col2)"],keys:_col0
- Select Operator [SEL_87] (rows=22000000 width=1014)
- Output:["_col0","_col2"]
- Group By Operator [GBY_86] (rows=22000000 width=1014)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
- <-Reducer 22 [SIMPLE_EDGE]
- SHUFFLE [RS_85]
- PartitionCols:_col0
- Group By Operator [GBY_84] (rows=44000000 width=1014)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col7, _col1
- Merge Join Operator [MERGEJOIN_150] (rows=44000000 width=1014)
- Conds:RS_80._col2=RS_81._col0(Inner),Output:["_col1","_col3","_col7"]
- <-Map 25 [SIMPLE_EDGE]
- SHUFFLE [RS_81]
- PartitionCols:_col0
- Select Operator [SEL_76] (rows=40000000 width=1014)
- Output:["_col0","_col1"]
- Filter Operator [FIL_143] (rows=40000000 width=1014)
- predicate:(ca_address_sk is not null and ca_state is not null)
- TableScan [TS_74] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
- <-Reducer 21 [SIMPLE_EDGE]
- SHUFFLE [RS_80]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_149] (rows=31678769 width=106)
- Conds:RS_77._col0=RS_78._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 20 [SIMPLE_EDGE]
- SHUFFLE [RS_77]
- PartitionCols:_col0
- Select Operator [SEL_70] (rows=28798881 width=106)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_141] (rows=28798881 width=106)
- predicate:(cr_returned_date_sk is not null and cr_returning_addr_sk is not null)
- TableScan [TS_68] (rows=28798881 width=106)
- default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_returned_date_sk","cr_returning_customer_sk","cr_returning_addr_sk","cr_return_amt_inc_tax"]
- <-Map 24 [SIMPLE_EDGE]
- SHUFFLE [RS_78]
- PartitionCols:_col0
- Select Operator [SEL_73] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_142] (rows=36524 width=1119)
- predicate:((d_year = 1998) and d_date_sk is not null)
- TableScan [TS_71] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_93]
- PartitionCols:_col1
- Select Operator [SEL_67] (rows=96800003 width=860)
- Output:["_col1","_col2","_col4","_col5","_col6","_col7","_col8","_col9","_col11","_col12","_col13","_col14","_col16","_col18","_col19","_col20"]
- Merge Join Operator [MERGEJOIN_152] (rows=96800003 width=860)
- Conds:RS_64._col0=RS_65._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col7","_col8","_col9","_col10","_col11","_col12","_col14","_col15","_col16","_col17","_col19","_col20"]
- <-Reducer 11 [SIMPLE_EDGE]
- SHUFFLE [RS_65]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_151] (rows=24200000 width=1014)
- Conds:RS_57._col1=RS_58._col0(Left Outer),Output:["_col0","_col1","_col2"]
- <-Reducer 10 [SIMPLE_EDGE]
- SHUFFLE [RS_57]
- PartitionCols:_col1
- Select Operator [SEL_25] (rows=22000000 width=1014)
- Output:["_col0","_col1","_col2"]
- Group By Operator [GBY_24] (rows=22000000 width=1014)
+ Filter Operator [FIL_62] (rows=32266667 width=860)
+ predicate:(_col2 > CASE WHEN (_col22 is null) THEN (null) ELSE (_col21) END)
+ Select Operator [SEL_61] (rows=96800003 width=860)
+ Output:["_col2","_col4","_col5","_col6","_col7","_col8","_col9","_col11","_col12","_col13","_col14","_col16","_col18","_col19","_col20","_col21","_col22"]
+ Merge Join Operator [MERGEJOIN_105] (rows=96800003 width=860)
+ Conds:RS_58._col0=RS_59._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col7","_col8","_col9","_col10","_col11","_col12","_col14","_col15","_col16","_col17","_col20","_col21","_col22"]
+ <-Reducer 10 [SIMPLE_EDGE]
+ SHUFFLE [RS_59]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_104] (rows=24200000 width=1014)
+ Conds:RS_51._col1=RS_52._col2(Left Outer),Output:["_col0","_col2","_col3","_col4"]
+ <-Reducer 16 [SIMPLE_EDGE]
+ SHUFFLE [RS_52]
+ PartitionCols:_col2
+ Select Operator [SEL_50] (rows=11000000 width=1014)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_49] (rows=11000000 width=1014)
+ Output:["_col0","_col1"],aggregations:["avg(_col2)"],keys:_col0
+ Select Operator [SEL_45] (rows=22000000 width=1014)
+ Output:["_col0","_col2"]
+ Group By Operator [GBY_44] (rows=22000000 width=1014)
Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
- <-Reducer 9 [SIMPLE_EDGE]
- SHUFFLE [RS_23]
- PartitionCols:_col0, _col1
- Group By Operator [GBY_22] (rows=44000000 width=1014)
+ <-Reducer 15 [SIMPLE_EDGE]
+ SHUFFLE [RS_43]
+ PartitionCols:_col0
+ Group By Operator [GBY_42] (rows=44000000 width=1014)
Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col7, _col1
- Merge Join Operator [MERGEJOIN_146] (rows=44000000 width=1014)
- Conds:RS_18._col2=RS_19._col0(Inner),Output:["_col1","_col3","_col7"]
- <-Map 13 [SIMPLE_EDGE]
- SHUFFLE [RS_19]
+ Merge Join Operator [MERGEJOIN_103] (rows=44000000 width=1014)
+ Conds:RS_38._col2=RS_39._col0(Inner),Output:["_col1","_col3","_col7"]
+ <-Map 18 [SIMPLE_EDGE]
+ SHUFFLE [RS_39]
PartitionCols:_col0
- Select Operator [SEL_14] (rows=40000000 width=1014)
+ Select Operator [SEL_34] (rows=40000000 width=1014)
Output:["_col0","_col1"]
- Filter Operator [FIL_137] (rows=40000000 width=1014)
- predicate:ca_address_sk is not null
- TableScan [TS_12] (rows=40000000 width=1014)
+ Filter Operator [FIL_98] (rows=40000000 width=1014)
+ predicate:(ca_address_sk is not null and ca_state is not null)
+ TableScan [TS_32] (rows=40000000 width=1014)
default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
- <-Reducer 8 [SIMPLE_EDGE]
- SHUFFLE [RS_18]
+ <-Reducer 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_38]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_145] (rows=31678769 width=106)
- Conds:RS_15._col0=RS_16._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 12 [SIMPLE_EDGE]
- SHUFFLE [RS_16]
+ Merge Join Operator [MERGEJOIN_102] (rows=31678769 width=106)
+ Conds:RS_35._col0=RS_36._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 13 [SIMPLE_EDGE]
+ SHUFFLE [RS_35]
+ PartitionCols:_col0
+ Select Operator [SEL_28] (rows=28798881 width=106)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_96] (rows=28798881 width=106)
+ predicate:(cr_returned_date_sk is not null and cr_returning_addr_sk is not null)
+ TableScan [TS_26] (rows=28798881 width=106)
+ default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_returned_date_sk","cr_returning_customer_sk","cr_returning_addr_sk","cr_return_amt_inc_tax"]
+ <-Map 17 [SIMPLE_EDGE]
+ SHUFFLE [RS_36]
PartitionCols:_col0
- Select Operator [SEL_11] (rows=36524 width=1119)
+ Select Operator [SEL_31] (rows=36524 width=1119)
Output:["_col0"]
- Filter Operator [FIL_136] (rows=36524 width=1119)
+ Filter Operator [FIL_97] (rows=36524 width=1119)
predicate:((d_year = 1998) and d_date_sk is not null)
- TableScan [TS_9] (rows=73049 width=1119)
+ TableScan [TS_29] (rows=73049 width=1119)
default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Map 7 [SIMPLE_EDGE]
- SHUFFLE [RS_15]
- PartitionCols:_col0
- Select Operator [SEL_8] (rows=28798881 width=106)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_135] (rows=28798881 width=106)
- predicate:(cr_returned_date_sk is not null and cr_returning_addr_sk is not null and cr_returning_customer_sk is not null)
- TableScan [TS_6] (rows=28798881 width=106)
- default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_returned_date_sk","cr_returning_customer_sk","cr_returning_addr_sk","cr_return_amt_inc_tax"]
- <-Reducer 17 [SIMPLE_EDGE]
- SHUFFLE [RS_58]
- PartitionCols:_col0
- Select Operator [SEL_56] (rows=1833333 width=1014)
- Output:["_col0"]
- Filter Operator [FIL_55] (rows=1833333 width=1014)
- predicate:(sq_count_check(_col1) <= 1)
- Group By Operator [GBY_54] (rows=5500000 width=1014)
- Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
- Group By Operator [GBY_49] (rows=11000000 width=1014)
- Output:["_col0"],keys:_col0
- Select Operator [SEL_45] (rows=22000000 width=1014)
- Output:["_col0"]
- Group By Operator [GBY_44] (rows=22000000 width=1014)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
- <-Reducer 16 [SIMPLE_EDGE]
- SHUFFLE [RS_43]
- PartitionCols:_col0
- Group By Operator [GBY_42] (rows=44000000 width=1014)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col7, _col1
- Merge Join Operator [MERGEJOIN_148] (rows=44000000 width=1014)
- Conds:RS_38._col2=RS_39._col0(Inner),Output:["_col1","_col3","_col7"]
- <-Map 19 [SIMPLE_EDGE]
- SHUFFLE [RS_39]
- PartitionCols:_col0
- Select Operator [SEL_34] (rows=40000000 width=1014)
- Output:["_col0","_col1"]
- Filter Operator [FIL_140] (rows=40000000 width=1014)
- predicate:(ca_address_sk is not null and ca_state is not null)
- TableScan [TS_32] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
- <-Reducer 15 [SIMPLE_EDGE]
- SHUFFLE [RS_38]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_147] (rows=31678769 width=106)
- Conds:RS_35._col0=RS_36._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 14 [SIMPLE_EDGE]
- SHUFFLE [RS_35]
- PartitionCols:_col0
- Select Operator [SEL_28] (rows=28798881 width=106)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_138] (rows=28798881 width=106)
- predicate:(cr_returned_date_sk is not null and cr_returning_addr_sk is not null)
- TableScan [TS_26] (rows=28798881 width=106)
- default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_returned_date_sk","cr_returning_customer_sk","cr_returning_addr_sk","cr_return_amt_inc_tax"]
- <-Map 18 [SIMPLE_EDGE]
- SHUFFLE [RS_36]
- PartitionCols:_col0
- Select Operator [SEL_31] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_139] (rows=36524 width=1119)
- predicate:((d_year = 1998) and d_date_sk is not null)
- TableScan [TS_29] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_64]
+ <-Reducer 9 [SIMPLE_EDGE]
+ SHUFFLE [RS_51]
+ PartitionCols:_col1
+ Select Operator [SEL_25] (rows=22000000 width=1014)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_24] (rows=22000000 width=1014)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
+ <-Reducer 8 [SIMPLE_EDGE]
+ SHUFFLE [RS_23]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_22] (rows=44000000 width=1014)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col7, _col1
+ Merge Join Operator [MERGEJOIN_101] (rows=44000000 width=1014)
+ Conds:RS_18._col2=RS_19._col0(Inner),Output:["_col1","_col3","_col7"]
+ <-Map 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_19]
+ PartitionCols:_col0
+ Select Operator [SEL_14] (rows=40000000 width=1014)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_95] (rows=40000000 width=1014)
+ predicate:ca_address_sk is not null
+ TableScan [TS_12] (rows=40000000 width=1014)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
+ <-Reducer 7 [SIMPLE_EDGE]
+ SHUFFLE [RS_18]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_100] (rows=31678769 width=106)
+ Conds:RS_15._col0=RS_16._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_16]
+ PartitionCols:_col0
+ Select Operator [SEL_11] (rows=36524 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_94] (rows=36524 width=1119)
+ predicate:((d_year = 1998) and d_date_sk is not null)
+ TableScan [TS_9] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ <-Map 6 [SIMPLE_EDGE]
+ SHUFFLE [RS_15]
+ PartitionCols:_col0
+ Select Operator [SEL_8] (rows=28798881 width=106)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_93] (rows=28798881 width=106)
+ predicate:(cr_returned_date_sk is not null and cr_returning_addr_sk is not null and cr_returning_customer_sk is not null)
+ TableScan [TS_6] (rows=28798881 width=106)
+ default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_returned_date_sk","cr_returning_customer_sk","cr_returning_addr_sk","cr_return_amt_inc_tax"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_58]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_99] (rows=88000001 width=860)
+ Conds:RS_55._col2=RS_56._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col7","_col8","_col9","_col10","_col11","_col12","_col14","_col15","_col16","_col17"]
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_55]
+ PartitionCols:_col2
+ Select Operator [SEL_2] (rows=80000000 width=860)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_91] (rows=80000000 width=860)
+ predicate:(c_customer_sk is not null and c_current_addr_sk is not null)
+ TableScan [TS_0] (rows=80000000 width=860)
+ default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_customer_id","c_current_addr_sk","c_salutation","c_first_name","c_last_name"]
+ <-Map 5 [SIMPLE_EDGE]
+ SHUFFLE [RS_56]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_144] (rows=88000001 width=860)
- Conds:RS_61._col2=RS_62._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col7","_col8","_col9","_col10","_col11","_col12","_col14","_col15","_col16","_col17"]
- <-Map 1 [SIMPLE_EDGE]
- SHUFFLE [RS_61]
- PartitionCols:_col2
- Select Operator [SEL_2] (rows=80000000 width=860)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_133] (rows=80000000 width=860)
- predicate:(c_customer_sk is not null and c_current_addr_sk is not null)
- TableScan [TS_0] (rows=80000000 width=860)
- default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_customer_id","c_current_addr_sk","c_salutation","c_first_name","c_last_name"]
- <-Map 6 [SIMPLE_EDGE]
- SHUFFLE [RS_62]
- PartitionCols:_col0
- Select Operator [SEL_5] (rows=20000000 width=1014)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col8","_col9","_col10","_col11"]
- Filter Operator [FIL_134] (rows=20000000 width=1014)
- predicate:((ca_state = 'IL') and ca_address_sk is not null)
- TableScan [TS_3] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_street_number","ca_street_name","ca_street_type","ca_suite_number","ca_city","ca_county","ca_state","ca_zip","ca_country","ca_gmt_offset","ca_location_type"]
+ Select Operator [SEL_5] (rows=20000000 width=1014)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col8","_col9","_col10","_col11"]
+ Filter Operator [FIL_92] (rows=20000000 width=1014)
+ predicate:((ca_state = 'IL') and ca_address_sk is not null)
+ TableScan [TS_3] (rows=40000000 width=1014)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_street_number","ca_street_name","ca_street_type","ca_suite_number","ca_city","ca_county","ca_state","ca_zip","ca_country","ca_gmt_offset","ca_location_type"]
[6/6] hive git commit: HIVE-16330 : Improve plans for scalar subquery
with aggregates (Vineet Garg via Ashutosh Chauhan)
Posted by ha...@apache.org.
HIVE-16330 : Improve plans for scalar subquery with aggregates (Vineet Garg via Ashutosh Chauhan)
Signed-off-by: Ashutosh Chauhan <ha...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/a113ede9
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/a113ede9
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/a113ede9
Branch: refs/heads/master
Commit: a113ede99e82a11384a144616938334696032dd8
Parents: 4ba48aa
Author: Vineet Garg <vg...@hortonworks.com>
Authored: Tue May 9 21:07:42 2017 -0700
Committer: Ashutosh Chauhan <ha...@apache.org>
Committed: Tue May 9 22:06:52 2017 -0700
----------------------------------------------------------------------
.../optimizer/calcite/HivePlannerContext.java | 12 +-
.../calcite/rules/HiveSubQueryRemoveRule.java | 59 +-
.../hadoop/hive/ql/parse/CalcitePlanner.java | 25 +-
.../apache/hadoop/hive/ql/parse/QBSubQuery.java | 62 +-
.../calcite/TestCBORuleFiredOnlyOnce.java | 3 +-
.../subquery_scalar_corr_multi_rows.q | 3 +-
.../subquery_scalar_corr_multi_rows.q.out | 3 +-
.../clientpositive/llap/subquery_scalar.q.out | 2423 +++------------
.../clientpositive/llap/subquery_select.q.out | 2784 +++---------------
.../results/clientpositive/perf/query1.q.out | 240 +-
.../results/clientpositive/perf/query30.q.out | 335 +--
.../results/clientpositive/perf/query6.q.out | 349 +--
.../results/clientpositive/perf/query81.q.out | 335 +--
.../results/clientpositive/perf/query9.q.out | 1019 ++-----
14 files changed, 1859 insertions(+), 5793 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HivePlannerContext.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HivePlannerContext.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HivePlannerContext.java
index d0b1757..bdf9955 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HivePlannerContext.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HivePlannerContext.java
@@ -22,6 +22,7 @@ import org.apache.calcite.plan.Context;
import org.apache.calcite.rel.RelNode;
import org.apache.hadoop.hive.ql.optimizer.calcite.cost.HiveAlgorithmsConf;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRulesRegistry;
+
import java.util.Set;
@@ -29,17 +30,18 @@ public class HivePlannerContext implements Context {
private HiveAlgorithmsConf algoConfig;
private HiveRulesRegistry registry;
private CalciteConnectionConfig calciteConfig;
- private Set<RelNode> corrScalarRexSQWithAgg;
+ private SubqueryConf subqueryConfig;
public HivePlannerContext(HiveAlgorithmsConf algoConfig, HiveRulesRegistry registry,
- CalciteConnectionConfig calciteConfig, Set<RelNode> corrScalarRexSQWithAgg) {
+ CalciteConnectionConfig calciteConfig, Set<RelNode> corrScalarRexSQWithAgg,
+ Set<RelNode> scalarAggNoGbyWindowing) {
this.algoConfig = algoConfig;
this.registry = registry;
this.calciteConfig = calciteConfig;
// this is to keep track if a subquery is correlated and contains aggregate
// this is computed in CalcitePlanner while planning and is later required by subuery remove rule
// hence this is passed using HivePlannerContext
- this.corrScalarRexSQWithAgg = corrScalarRexSQWithAgg;
+ this.subqueryConfig = new SubqueryConf(corrScalarRexSQWithAgg, scalarAggNoGbyWindowing);
}
public <T> T unwrap(Class<T> clazz) {
@@ -52,8 +54,8 @@ public class HivePlannerContext implements Context {
if (clazz.isInstance(calciteConfig)) {
return clazz.cast(calciteConfig);
}
- if(clazz.isInstance(corrScalarRexSQWithAgg)) {
- return clazz.cast(corrScalarRexSQWithAgg);
+ if(clazz.isInstance(subqueryConfig)) {
+ return clazz.cast(subqueryConfig);
}
return null;
}
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
index c692cc0..83d3f74 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
@@ -56,6 +56,7 @@ import java.util.Set;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveSubQRemoveRelBuilder;
+import org.apache.hadoop.hive.ql.optimizer.calcite.SubqueryConf;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter;
/**
@@ -99,11 +100,12 @@ public abstract class HiveSubQueryRemoveRule extends RelOptRule{
final int fieldCount = builder.peek().getRowType().getFieldCount();
assert(filter instanceof HiveFilter);
- Set<RelNode> corrScalarQueries = filter.getCluster().getPlanner().getContext().unwrap(Set.class);
- boolean isCorrScalarQuery = corrScalarQueries.contains(e.rel);
+ SubqueryConf subqueryConfig = filter.getCluster().getPlanner().getContext().unwrap(SubqueryConf.class);
+ boolean isCorrScalarQuery = subqueryConfig.getCorrScalarRexSQWithAgg().contains(e.rel);
+ boolean hasNoWindowingAndNoGby = subqueryConfig.getScalarAggWithoutGbyWindowing().contains(e.rel);
final RexNode target = apply(e, HiveFilter.getVariablesSet(e), logic,
- builder, 1, fieldCount, isCorrScalarQuery);
+ builder, 1, fieldCount, isCorrScalarQuery, hasNoWindowingAndNoGby);
final RexShuttle shuttle = new ReplaceSubQueryShuttle(e, target);
builder.filter(shuttle.apply(filter.getCondition()));
builder.project(fields(builder, filter.getRowType().getFieldCount()));
@@ -122,11 +124,12 @@ public abstract class HiveSubQueryRemoveRule extends RelOptRule{
builder.push(project.getInput());
final int fieldCount = builder.peek().getRowType().getFieldCount();
- Set<RelNode> corrScalarQueries = project.getCluster().getPlanner().getContext().unwrap(Set.class);
- boolean isCorrScalarQuery = corrScalarQueries.contains(e.rel);
+ SubqueryConf subqueryConfig = project.getCluster().getPlanner().getContext().unwrap(SubqueryConf.class);
+ boolean isCorrScalarQuery = subqueryConfig.getCorrScalarRexSQWithAgg().contains(e.rel);
+ boolean hasNoWindowingAndNoGby = subqueryConfig.getScalarAggWithoutGbyWindowing().contains(e.rel);
final RexNode target = apply(e, HiveFilter.getVariablesSet(e),
- logic, builder, 1, fieldCount, isCorrScalarQuery);
+ logic, builder, 1, fieldCount, isCorrScalarQuery, hasNoWindowingAndNoGby);
final RexShuttle shuttle = new ReplaceSubQueryShuttle(e, target);
builder.project(shuttle.apply(project.getProjects()),
project.getRowType().getFieldNames());
@@ -165,28 +168,32 @@ public abstract class HiveSubQueryRemoveRule extends RelOptRule{
protected RexNode apply(RexSubQuery e, Set<CorrelationId> variablesSet,
RelOptUtil.Logic logic,
HiveSubQRemoveRelBuilder builder, int inputCount, int offset,
- boolean isCorrScalarAgg) {
+ boolean isCorrScalarAgg,
+ boolean hasNoWindowingAndNoGby ) {
switch (e.getKind()) {
case SCALAR_QUERY:
- builder.push(e.rel);
- // returns single row/column
- builder.aggregate(builder.groupKey(),
- builder.count(false, "cnt"));
-
- SqlFunction countCheck = new SqlFunction("sq_count_check", SqlKind.OTHER_FUNCTION, ReturnTypes.BIGINT,
- InferTypes.RETURN_TYPE, OperandTypes.NUMERIC, SqlFunctionCategory.USER_DEFINED_FUNCTION);
-
- // we create FILTER (sq_count_check(count()) <= 1) instead of PROJECT because RelFieldTrimmer
- // ends up getting rid of Project since it is not used further up the tree
- builder.filter(builder.call(SqlStdOperatorTable.LESS_THAN_OR_EQUAL,
- builder.call(countCheck, builder.field("cnt")),
- builder.literal(1)));
- if( !variablesSet.isEmpty())
- {
- builder.join(JoinRelType.LEFT, builder.literal(true), variablesSet);
+ // if scalar query has aggregate and no windowing and no gby avoid adding sq_count_check
+ // since it is guaranteed to produce at most one row
+ if(!hasNoWindowingAndNoGby) {
+ builder.push(e.rel);
+ // returns single row/column
+ builder.aggregate(builder.groupKey(), builder.count(false, "cnt"));
+
+ SqlFunction countCheck =
+ new SqlFunction("sq_count_check", SqlKind.OTHER_FUNCTION, ReturnTypes.BIGINT,
+ InferTypes.RETURN_TYPE, OperandTypes.NUMERIC, SqlFunctionCategory.USER_DEFINED_FUNCTION);
+
+ // we create FILTER (sq_count_check(count()) <= 1) instead of PROJECT because RelFieldTrimmer
+ // ends up getting rid of Project since it is not used further up the tree
+ builder.filter(builder.call(SqlStdOperatorTable.LESS_THAN_OR_EQUAL,
+ builder.call(countCheck, builder.field("cnt")), builder.literal(1)));
+ if (!variablesSet.isEmpty()) {
+ builder.join(JoinRelType.LEFT, builder.literal(true), variablesSet);
+ } else
+ builder.join(JoinRelType.INNER, builder.literal(true), variablesSet);
+
+ offset++;
}
- else
- builder.join(JoinRelType.INNER, builder.literal(true), variablesSet);
if(isCorrScalarAgg) {
// Transformation :
// Outer Query Left Join (inner query) on correlated predicate and preserve rows only from left side.
@@ -218,10 +225,8 @@ public abstract class HiveSubQueryRemoveRule extends RelOptRule{
//Transformation is to left join for correlated predicates and inner join otherwise,
// but do a count on inner side before that to make sure it generates atmost 1 row.
-
builder.push(e.rel);
builder.join(JoinRelType.LEFT, builder.literal(true), variablesSet);
- offset++;
return field(builder, inputCount, offset);
case IN:
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java b/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
index 5d640be..fa96e94 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
@@ -1299,6 +1299,7 @@ public class CalcitePlanner extends SemanticAnalyzer {
// this is to keep track if a subquery is correlated and contains aggregate
// since this is special cased when it is rewritten in SubqueryRemoveRule
Set<RelNode> corrScalarRexSQWithAgg = new HashSet<RelNode>();
+ Set<RelNode> scalarAggNoGbyNoWin = new HashSet<RelNode>();
// TODO: Do we need to keep track of RR, ColNameToPosMap for every op or
// just last one.
@@ -1332,7 +1333,7 @@ public class CalcitePlanner extends SemanticAnalyzer {
Boolean.FALSE.toString());
CalciteConnectionConfig calciteConfig = new CalciteConnectionConfigImpl(calciteConfigProperties);
HivePlannerContext confContext = new HivePlannerContext(algorithmsConf, registry, calciteConfig,
- corrScalarRexSQWithAgg);
+ corrScalarRexSQWithAgg, scalarAggNoGbyNoWin);
RelOptPlanner planner = HiveVolcanoPlanner.createPlanner(confContext);
final RexBuilder rexBuilder = cluster.getRexBuilder();
final RelOptCluster optCluster = RelOptCluster.create(planner, rexBuilder);
@@ -2425,8 +2426,8 @@ public class CalcitePlanner extends SemanticAnalyzer {
}
private void subqueryRestrictionCheck(QB qb, ASTNode searchCond, RelNode srcRel,
- boolean forHavingClause,
- Set<ASTNode> corrScalarQueries) throws SemanticException {
+ boolean forHavingClause, Set<ASTNode> corrScalarQueries,
+ Set<ASTNode> scalarQueriesWithAggNoWinNoGby) throws SemanticException {
List<ASTNode> subQueriesInOriginalTree = SubQueryUtils.findSubQueries(searchCond);
ASTNode clonedSearchCond = (ASTNode) SubQueryUtils.adaptor.dupTree(searchCond);
@@ -2461,18 +2462,25 @@ public class CalcitePlanner extends SemanticAnalyzer {
String havingInputAlias = null;
- boolean isCorrScalarWithAgg = subQuery.subqueryRestrictionsCheck(inputRR, forHavingClause, havingInputAlias);
- if(isCorrScalarWithAgg) {
+ boolean [] subqueryConfig = {false, false};
+ subQuery.subqueryRestrictionsCheck(inputRR, forHavingClause,
+ havingInputAlias, subqueryConfig);
+ if(subqueryConfig[0]) {
corrScalarQueries.add(originalSubQueryAST);
}
+ if(subqueryConfig[1]) {
+ scalarQueriesWithAggNoWinNoGby.add(originalSubQueryAST);
+ }
}
}
private boolean genSubQueryRelNode(QB qb, ASTNode node, RelNode srcRel, boolean forHavingClause,
Map<ASTNode, RelNode> subQueryToRelNode) throws SemanticException {
Set<ASTNode> corrScalarQueriesWithAgg = new HashSet<ASTNode>();
+ Set<ASTNode> scalarQueriesWithAggNoWinNoGby= new HashSet<ASTNode>();
//disallow subqueries which HIVE doesn't currently support
- subqueryRestrictionCheck(qb, node, srcRel, forHavingClause, corrScalarQueriesWithAgg);
+ subqueryRestrictionCheck(qb, node, srcRel, forHavingClause, corrScalarQueriesWithAgg,
+ scalarQueriesWithAggNoWinNoGby);
Deque<ASTNode> stack = new ArrayDeque<ASTNode>();
stack.push(node);
@@ -2502,9 +2510,14 @@ public class CalcitePlanner extends SemanticAnalyzer {
subQueryToRelNode.put(next, subQueryRelNode);
//keep track of subqueries which are scalar, correlated and contains aggregate
// subquery expression. This will later be special cased in Subquery remove rule
+ // for correlated scalar queries with aggregate we have take care of the case where
+ // inner aggregate happens on empty result
if(corrScalarQueriesWithAgg.contains(next)) {
corrScalarRexSQWithAgg.add(subQueryRelNode);
}
+ if(scalarQueriesWithAggNoWinNoGby.contains(next)) {
+ scalarAggNoGbyNoWin.add(subQueryRelNode);
+ }
isSubQuery = true;
break;
default:
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java b/ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java
index ec52741..0097a04 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java
@@ -526,9 +526,9 @@ public class QBSubQuery implements ISubQueryJoinInfo {
* @return true if it is correlated scalar subquery with an aggregate
* @throws SemanticException
*/
- boolean subqueryRestrictionsCheck(RowResolver parentQueryRR,
+ void subqueryRestrictionsCheck(RowResolver parentQueryRR,
boolean forHavingClause,
- String outerQueryAlias)
+ String outerQueryAlias, boolean [] subqueryConfig)
throws SemanticException {
ASTNode insertClause = getChildFromSubqueryAST("Insert", HiveParser.TOK_INSERT);
@@ -568,37 +568,35 @@ public class QBSubQuery implements ISubQueryJoinInfo {
hasCount = hasCount | ( r == 2 );
}
-
-
- ASTNode whereClause = SubQueryUtils.subQueryWhere(insertClause);
-
- if ( whereClause == null ) {
- return false;
- }
- ASTNode searchCond = (ASTNode) whereClause.getChild(0);
- List<ASTNode> conjuncts = new ArrayList<ASTNode>();
- SubQueryUtils.extractConjuncts(searchCond, conjuncts);
-
- ConjunctAnalyzer conjunctAnalyzer = new ConjunctAnalyzer(parentQueryRR,
- forHavingClause, outerQueryAlias);
-
+ // figure out correlation and presence of non-equi join predicate
boolean hasCorrelation = false;
boolean hasNonEquiJoinPred = false;
- for(ASTNode conjunctAST : conjuncts) {
- Conjunct conjunct = conjunctAnalyzer.analyzeConjunct(conjunctAST);
- if(conjunct.isCorrelated()){
- hasCorrelation = true;
- }
- if ( conjunct.eitherSideRefersBoth() && conjunctAST.getType() != HiveParser.EQUAL) {
- hasNonEquiJoinPred = true;
+
+ ASTNode whereClause = SubQueryUtils.subQueryWhere(insertClause);
+ if ( whereClause != null ) {
+ ASTNode searchCond = (ASTNode) whereClause.getChild(0);
+ List<ASTNode> conjuncts = new ArrayList<ASTNode>();
+ SubQueryUtils.extractConjuncts(searchCond, conjuncts);
+
+ ConjunctAnalyzer conjunctAnalyzer =
+ new ConjunctAnalyzer(parentQueryRR, forHavingClause, outerQueryAlias);
+
+ for (ASTNode conjunctAST : conjuncts) {
+ Conjunct conjunct = conjunctAnalyzer.analyzeConjunct(conjunctAST);
+ if (conjunct.isCorrelated()) {
+ hasCorrelation = true;
+ }
+ if (conjunct.eitherSideRefersBoth() && conjunctAST.getType() != HiveParser.EQUAL) {
+ hasNonEquiJoinPred = true;
+ }
}
}
+
+ // figure out if there is group by
boolean noImplicityGby = true;
- if ( insertClause.getChild(1).getChildCount() > 3 &&
- insertClause.getChild(1).getChild(3).getType() == HiveParser.TOK_GROUPBY ) {
- if((ASTNode) insertClause.getChild(1).getChild(3) != null){
+ if ( insertClause.getChildCount() > 3 &&
+ insertClause.getChild(3).getType() == HiveParser.TOK_GROUPBY ) {
noImplicityGby = false;
- }
}
/*
@@ -643,22 +641,24 @@ public class QBSubQuery implements ISubQueryJoinInfo {
subQueryAST,
"Scalar subqueries with aggregate cannot have non-equi join predicate"));
}
+ if(!hasWindowing) {
+ subqueryConfig[1] = true;
+ }
if(hasCorrelation) {
- return true;
+ subqueryConfig[0] = true;
}
}
else if(operator.getType() == SubQueryType.IN) {
if(hasCount && hasCorrelation) {
- return true;
+ subqueryConfig[0] = true;
}
}
else if (operator.getType() == SubQueryType.NOT_IN) {
if(hasCorrelation) {
- return true;
+ subqueryConfig[0] = true;
}
}
}
- return false;
}
void validateAndRewriteAST(RowResolver outerQueryRR,
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/TestCBORuleFiredOnlyOnce.java
----------------------------------------------------------------------
diff --git a/ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/TestCBORuleFiredOnlyOnce.java b/ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/TestCBORuleFiredOnlyOnce.java
index 4823950..884e034 100644
--- a/ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/TestCBORuleFiredOnlyOnce.java
+++ b/ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/TestCBORuleFiredOnlyOnce.java
@@ -61,7 +61,8 @@ public class TestCBORuleFiredOnlyOnce {
// Create rules registry to not trigger a rule more than once
HiveRulesRegistry registry = new HiveRulesRegistry();
- HivePlannerContext context = new HivePlannerContext(null, registry, null, null);
+ HivePlannerContext context = new HivePlannerContext(null, registry, null,
+ null, null);
HepPlanner planner = new HepPlanner(programBuilder.build(), context);
// Cluster
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/queries/clientnegative/subquery_scalar_corr_multi_rows.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientnegative/subquery_scalar_corr_multi_rows.q b/ql/src/test/queries/clientnegative/subquery_scalar_corr_multi_rows.q
index e9ea703..e71a60d 100644
--- a/ql/src/test/queries/clientnegative/subquery_scalar_corr_multi_rows.q
+++ b/ql/src/test/queries/clientnegative/subquery_scalar_corr_multi_rows.q
@@ -1,2 +1,3 @@
-- inner query produces more than one row
-select * from part where p_size > (select count(*) from part p where p.p_mfgr = part.p_mfgr group by p_type);
\ No newline at end of file
+select * from part where p_size >
+ (select count(*) from part p where p.p_mfgr = part.p_mfgr group by p_type);
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/results/clientnegative/subquery_scalar_corr_multi_rows.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientnegative/subquery_scalar_corr_multi_rows.q.out b/ql/src/test/results/clientnegative/subquery_scalar_corr_multi_rows.q.out
index 3235048..8377085 100644
--- a/ql/src/test/results/clientnegative/subquery_scalar_corr_multi_rows.q.out
+++ b/ql/src/test/results/clientnegative/subquery_scalar_corr_multi_rows.q.out
@@ -1,4 +1,5 @@
-PREHOOK: query: select * from part where p_size > (select count(*) from part p where p.p_mfgr = part.p_mfgr group by p_type)
+PREHOOK: query: select * from part where p_size >
+ (select count(*) from part p where p.p_mfgr = part.p_mfgr group by p_type)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
[4/6] hive git commit: HIVE-16330 : Improve plans for scalar subquery
with aggregates (Vineet Garg via Ashutosh Chauhan)
Posted by ha...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/results/clientpositive/llap/subquery_select.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/subquery_select.q.out b/ql/src/test/results/clientpositive/llap/subquery_select.q.out
index 8eaec9e..e96067f 100644
--- a/ql/src/test/results/clientpositive/llap/subquery_select.q.out
+++ b/ql/src/test/results/clientpositive/llap/subquery_select.q.out
@@ -1447,9 +1447,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
- Reducer 6 <- Map 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1473,26 +1472,6 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p
- Statistics: Num rows: 26 Data size: 2704 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: p_type is not null (type: boolean)
- Statistics: Num rows: 26 Data size: 2704 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: p_type (type: string)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: p
Statistics: Num rows: 26 Data size: 5850 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_type is not null (type: boolean)
@@ -1517,20 +1496,18 @@ STAGE PLANS:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col0 (type: string)
- 1 _col0 (type: string)
- 2 _col2 (type: string)
- outputColumnNames: _col1, _col4, _col5
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
+ 1 _col2 (type: string)
+ outputColumnNames: _col1, _col2, _col3
+ Statistics: Num rows: 26 Data size: 416 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col1 (type: int), CASE WHEN (_col5 is null) THEN (0) ELSE (_col4) END (type: bigint)
+ expressions: _col1 (type: int), CASE WHEN (_col3 is null) THEN (0) ELSE (_col2) END (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 26 Data size: 312 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 26 Data size: 312 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1539,32 +1516,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- keys: KEY._col0 (type: string)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- keys: _col0 (type: string)
- mode: complete
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col1) <= 1) (type: boolean)
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
@@ -1634,9 +1585,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
- Reducer 6 <- Map 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1660,26 +1610,6 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p
- Statistics: Num rows: 26 Data size: 2704 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: p_type is not null (type: boolean)
- Statistics: Num rows: 26 Data size: 2704 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: p_type (type: string)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: p
Statistics: Num rows: 26 Data size: 5850 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_type is not null (type: boolean)
@@ -1704,20 +1634,18 @@ STAGE PLANS:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col0 (type: string)
- 1 _col0 (type: string)
- 2 _col2 (type: string)
- outputColumnNames: _col1, _col4, _col5
- Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
+ 1 _col2 (type: string)
+ outputColumnNames: _col1, _col2, _col3
+ Statistics: Num rows: 26 Data size: 4992 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col1 (type: int), CASE WHEN (_col5 is null) THEN (null) ELSE (_col4) END (type: string)
+ expressions: _col1 (type: int), CASE WHEN (_col3 is null) THEN (null) ELSE (_col2) END (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 26 Data size: 4888 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 26 Data size: 4888 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1726,32 +1654,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- keys: KEY._col0 (type: string)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- keys: _col0 (type: string)
- mode: complete
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col1) <= 1) (type: boolean)
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: max(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
@@ -1808,8 +1710,7 @@ POSTHOOK: Input: default@part
7 NULL
12 NULL
23 NULL
-Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[31][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain SELECT p_size, (SELECT max(p_size) FROM part)
FROM part
PREHOOK: type: QUERY
@@ -1825,10 +1726,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
- Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE)
- Reducer 7 <- Map 6 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
+ Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1846,27 +1745,7 @@ STAGE PLANS:
value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: part
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 6
+ Map 3
Map Operator Tree:
TableScan
alias: part
@@ -1891,62 +1770,20 @@ STAGE PLANS:
Reduce Operator Tree:
Merge Join Operator
condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: int)
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
Left Outer Join0 to 1
keys:
0
1
- outputColumnNames: _col0, _col2
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: int), _col2 (type: int)
- outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 5
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 7
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 4
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -1965,8 +1802,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[31][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: SELECT p_size, (SELECT max(p_size) FROM part)
FROM part
PREHOOK: type: QUERY
@@ -2003,8 +1839,7 @@ POSTHOOK: Input: default@part
2 46
46 46
23 46
-Warning: Shuffle Join MERGEJOIN[48][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 4' is a cross product
-Warning: Shuffle Join MERGEJOIN[49][tables = [$hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 5' is a cross product
+Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 4' is a cross product
PREHOOK: query: explain
select *
from src b
@@ -2032,12 +1867,10 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 10 <- Map 9 (CUSTOM_SIMPLE_EDGE)
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
- Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE), Reducer 8 (CUSTOM_SIMPLE_EDGE)
- Reducer 5 <- Reducer 10 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
- Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
- Reducer 8 <- Map 7 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
+ Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
+ Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
+ Reducer 7 <- Map 6 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -2074,27 +1907,7 @@ STAGE PLANS:
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
- Map 7
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: key
- Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count(key)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 9
+ Map 6
Map Operator Tree:
TableScan
alias: src
@@ -2114,18 +1927,6 @@ STAGE PLANS:
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
- Reducer 10
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: max(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: string)
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -2149,29 +1950,14 @@ STAGE PLANS:
Reduce Operator Tree:
Merge Join Operator
condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col1
- Statistics: Num rows: 83 Data size: 7553 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 83 Data size: 7553 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col1 (type: string)
- Reducer 5
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
Left Outer Join0 to 1
keys:
0
1
- outputColumnNames: _col1, _col3
+ outputColumnNames: _col1, _col2
Statistics: Num rows: 83 Data size: 22825 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
- keys: _col3 (type: string), _col1 (type: string)
+ keys: _col2 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 36 Data size: 9900 Basic stats: COMPLETE Column stats: COMPLETE
@@ -2180,7 +1966,7 @@ STAGE PLANS:
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
Statistics: Num rows: 36 Data size: 9900 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 6
+ Reducer 5
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -2193,29 +1979,18 @@ STAGE PLANS:
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
Statistics: Num rows: 36 Data size: 9900 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 8
+ Reducer 7
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
+ aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: string)
Stage: Stage-0
Fetch Operator
@@ -2223,8 +1998,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[48][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 4' is a cross product
-Warning: Shuffle Join MERGEJOIN[49][tables = [$hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 5' is a cross product
+Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 4' is a cross product
PREHOOK: query: select *
from src b
where b.key in
@@ -2275,9 +2049,8 @@ STAGE PLANS:
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
- Reducer 4 <- Map 3 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
+ Reducer 4 <- Map 3 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 6 <- Map 5 (SIMPLE_EDGE)
- Reducer 8 <- Map 7 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -2319,36 +2092,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: sc
- Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
+ aggregations: max(key)
keys: value (type: string)
mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 214 Data size: 19474 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 214 Data size: 19474 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: no inputs
- Map 7
- Map Operator Tree:
- TableScan
- alias: sc
- Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: max(key)
- keys: value (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 214 Data size: 58850 Basic stats: COMPLETE Column stats: COMPLETE
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 214 Data size: 58850 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
@@ -2381,15 +2134,13 @@ STAGE PLANS:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col1 (type: string)
- 1 _col0 (type: string)
- 2 _col2 (type: string)
- outputColumnNames: _col1, _col4, _col5
+ 1 _col2 (type: string)
+ outputColumnNames: _col1, _col2, _col3
Statistics: Num rows: 83 Data size: 23157 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: CASE WHEN (_col5 is null) THEN (null) ELSE (_col4) END (type: string), _col1 (type: string)
+ expressions: CASE WHEN (_col3 is null) THEN (null) ELSE (_col2) END (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 83 Data size: 22825 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
@@ -2406,32 +2157,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- keys: KEY._col0 (type: string)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 214 Data size: 19474 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- keys: _col0 (type: string)
- mode: complete
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col1) <= 1) (type: boolean)
- Statistics: Num rows: 71 Data size: 7029 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 71 Data size: 7029 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 71 Data size: 7029 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 8
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: max(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
@@ -2691,8 +2416,7 @@ POSTHOOK: query: CREATE TABLE tempty(i int)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@tempty
-Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[31][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select p_size, (select count(*) from tempty) from part
PREHOOK: type: QUERY
POSTHOOK: query: explain select p_size, (select count(*) from tempty) from part
@@ -2706,10 +2430,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
- Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE)
- Reducer 7 <- Map 6 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
+ Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -2727,27 +2449,7 @@ STAGE PLANS:
value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: tempty
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: i (type: int)
- outputColumnNames: i
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Group By Operator
- aggregations: count(i)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 6
+ Map 3
Map Operator Tree:
TableScan
alias: tempty
@@ -2770,62 +2472,20 @@ STAGE PLANS:
Reduce Operator Tree:
Merge Join Operator
condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0
- Statistics: Num rows: 26 Data size: 338 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 338 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: int)
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
Left Outer Join0 to 1
keys:
0
1
- outputColumnNames: _col0, _col2
- Statistics: Num rows: 26 Data size: 572 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int), _col2 (type: bigint)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 26 Data size: 572 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 26 Data size: 572 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 5
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reducer 7
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 26 Data size: 312 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 26 Data size: 312 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 4
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -2844,8 +2504,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[31][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select p_size, (select count(*) from tempty) from part
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -2882,8 +2541,7 @@ POSTHOOK: Input: default@tempty
2 0
46 0
23 0
-Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[31][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select p_size, (select max(i) from tempty) from part
PREHOOK: type: QUERY
POSTHOOK: query: explain select p_size, (select max(i) from tempty) from part
@@ -2897,10 +2555,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
- Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE)
- Reducer 7 <- Map 6 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
+ Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -2918,27 +2574,7 @@ STAGE PLANS:
value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: tempty
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: i (type: int)
- outputColumnNames: i
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Group By Operator
- aggregations: count(i)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 6
+ Map 3
Map Operator Tree:
TableScan
alias: tempty
@@ -2963,62 +2599,20 @@ STAGE PLANS:
Reduce Operator Tree:
Merge Join Operator
condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0
- Statistics: Num rows: 26 Data size: 338 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 338 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: int)
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
Left Outer Join0 to 1
keys:
0
1
- outputColumnNames: _col0, _col2
- Statistics: Num rows: 26 Data size: 468 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int), _col2 (type: int)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 26 Data size: 468 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 26 Data size: 468 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 5
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reducer 7
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 26 Data size: 234 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 26 Data size: 234 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 4
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -3037,8 +2631,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[31][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select p_size, (select max(i) from tempty) from part
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -3106,9 +2699,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
- Reducer 6 <- Map 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -3132,26 +2724,6 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p
- Statistics: Num rows: 26 Data size: 2704 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: p_type is not null (type: boolean)
- Statistics: Num rows: 26 Data size: 2704 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: p_type (type: string)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: p
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_type is not null (type: boolean)
@@ -3176,20 +2748,18 @@ STAGE PLANS:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col0 (type: string)
- 1 _col0 (type: string)
- 2 _col2 (type: string)
- outputColumnNames: _col1, _col4, _col5
- Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ 1 _col2 (type: string)
+ outputColumnNames: _col1, _col2, _col3
+ Statistics: Num rows: 26 Data size: 312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col1 (type: int), (1 + CASE WHEN (_col5 is null) THEN (null) ELSE (_col4) END) (type: int)
+ expressions: _col1 (type: int), (1 + CASE WHEN (_col3 is null) THEN (null) ELSE (_col2) END) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -3198,32 +2768,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- keys: KEY._col0 (type: string)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- keys: _col0 (type: string)
- mode: complete
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col1) <= 1) (type: boolean)
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: max(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
@@ -3297,9 +2841,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
- Reducer 6 <- Map 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -3323,26 +2866,6 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p
- Statistics: Num rows: 26 Data size: 2704 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: p_type is not null (type: boolean)
- Statistics: Num rows: 26 Data size: 2704 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: p_type (type: string)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: p
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_type is not null (type: boolean)
@@ -3367,20 +2890,18 @@ STAGE PLANS:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col0 (type: string)
- 1 _col0 (type: string)
- 2 _col2 (type: string)
- outputColumnNames: _col1, _col4, _col5
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
+ 1 _col2 (type: string)
+ outputColumnNames: _col1, _col2, _col3
+ Statistics: Num rows: 26 Data size: 416 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col1 (type: int), CASE WHEN (_col5 is null) THEN (false) ELSE (_col4 is null) END (type: boolean)
+ expressions: _col1 (type: int), CASE WHEN (_col3 is null) THEN (false) ELSE (_col2 is null) END (type: boolean)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -3389,32 +2910,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- keys: KEY._col0 (type: string)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- keys: _col0 (type: string)
- mode: complete
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col1) <= 1) (type: boolean)
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 4 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
@@ -4139,11 +3634,9 @@ POSTHOOK: Input: default@part
6 false
18 false
45 false
-Warning: Shuffle Join MERGEJOIN[85][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 5' is a cross product
-Warning: Shuffle Join MERGEJOIN[86][tables = [$hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 6' is a cross product
-Warning: Shuffle Join MERGEJOIN[87][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[88][tables = [$hdt$_2, $hdt$_3]] in Stage 'Reducer 14' is a cross product
-Warning: Shuffle Join MERGEJOIN[89][tables = [$hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 15' is a cross product
+Warning: Shuffle Join MERGEJOIN[50][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 5' is a cross product
+Warning: Shuffle Join MERGEJOIN[51][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[52][tables = [$hdt$_2, $hdt$_3]] in Stage 'Reducer 11' is a cross product
PREHOOK: query: explain select p_size, (p_size IN
(select (select max(p_size) from part) as sb from part order by sb limit 1)) = true
from part
@@ -4161,19 +3654,15 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 10 <- Map 9 (CUSTOM_SIMPLE_EDGE)
- Reducer 12 <- Map 11 (CUSTOM_SIMPLE_EDGE)
- Reducer 14 <- Map 13 (CUSTOM_SIMPLE_EDGE), Reducer 18 (CUSTOM_SIMPLE_EDGE)
- Reducer 15 <- Reducer 14 (CUSTOM_SIMPLE_EDGE), Reducer 20 (CUSTOM_SIMPLE_EDGE)
- Reducer 16 <- Reducer 15 (SIMPLE_EDGE)
- Reducer 18 <- Map 17 (CUSTOM_SIMPLE_EDGE)
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 8 (CUSTOM_SIMPLE_EDGE)
- Reducer 20 <- Map 19 (CUSTOM_SIMPLE_EDGE)
- Reducer 3 <- Reducer 16 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
- Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE), Reducer 10 (CUSTOM_SIMPLE_EDGE)
- Reducer 6 <- Reducer 12 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
- Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
- Reducer 8 <- Reducer 7 (CUSTOM_SIMPLE_EDGE)
+ Reducer 11 <- Map 10 (CUSTOM_SIMPLE_EDGE), Reducer 14 (CUSTOM_SIMPLE_EDGE)
+ Reducer 12 <- Reducer 11 (SIMPLE_EDGE)
+ Reducer 14 <- Map 13 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
+ Reducer 3 <- Reducer 12 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
+ Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE), Reducer 9 (CUSTOM_SIMPLE_EDGE)
+ Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
+ Reducer 7 <- Reducer 6 (CUSTOM_SIMPLE_EDGE)
+ Reducer 9 <- Map 8 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -4191,27 +3680,7 @@ STAGE PLANS:
value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Map 11
- Map Operator Tree:
- TableScan
- alias: part
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: p_size (type: int)
- outputColumnNames: p_size
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: max(p_size)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: int)
- Execution mode: llap
- LLAP IO: no inputs
- Map 13
+ Map 10
Map Operator Tree:
TableScan
alias: part
@@ -4223,27 +3692,7 @@ STAGE PLANS:
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
- Map 17
- Map Operator Tree:
- TableScan
- alias: part
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 19
+ Map 13
Map Operator Tree:
TableScan
alias: part
@@ -4275,75 +3724,27 @@ STAGE PLANS:
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
- Map 9
+ Map 8
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
+ expressions: p_size (type: int)
+ outputColumnNames: p_size
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
- aggregations: count(p_partkey)
+ aggregations: max(p_size)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Reducer 10
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 12
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: max(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: int)
- Reducer 14
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 15
+ Reducer 11
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
@@ -4352,10 +3753,10 @@ STAGE PLANS:
keys:
0
1
- outputColumnNames: _col2
+ outputColumnNames: _col1
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col2 (type: int)
+ expressions: _col1 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
@@ -4363,7 +3764,7 @@ STAGE PLANS:
sort order: +
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.1
- Reducer 16
+ Reducer 12
Execution mode: llap
Reduce Operator Tree:
Select Operator
@@ -4383,29 +3784,18 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
- Reducer 18
+ Reducer 14
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
+ aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int)
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -4423,18 +3813,6 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 26 Data size: 520 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
- Reducer 20
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: max(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: int)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -4462,34 +3840,21 @@ STAGE PLANS:
Reduce Operator Tree:
Merge Join Operator
condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
Left Outer Join0 to 1
keys:
0
1
- outputColumnNames: _col2
+ outputColumnNames: _col1
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col2 (type: int)
+ expressions: _col1 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 7
+ Reducer 6
Execution mode: llap
Reduce Operator Tree:
Select Operator
@@ -4508,7 +3873,7 @@ STAGE PLANS:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
- Reducer 8
+ Reducer 7
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -4520,6 +3885,18 @@ STAGE PLANS:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
+ Reducer 9
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: max(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int)
Stage: Stage-0
Fetch Operator
@@ -4527,11 +3904,9 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[85][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 5' is a cross product
-Warning: Shuffle Join MERGEJOIN[86][tables = [$hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 6' is a cross product
-Warning: Shuffle Join MERGEJOIN[87][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[88][tables = [$hdt$_2, $hdt$_3]] in Stage 'Reducer 14' is a cross product
-Warning: Shuffle Join MERGEJOIN[89][tables = [$hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 15' is a cross product
+Warning: Shuffle Join MERGEJOIN[50][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 5' is a cross product
+Warning: Shuffle Join MERGEJOIN[51][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[52][tables = [$hdt$_2, $hdt$_3]] in Stage 'Reducer 11' is a cross product
PREHOOK: query: select p_size, (p_size IN
(select (select max(p_size) from part) as sb from part order by sb limit 1)) = true
from part
@@ -4570,12 +3945,9 @@ POSTHOOK: Input: default@part
42 false
45 false
46 true
-Warning: Shuffle Join MERGEJOIN[94][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[95][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
-Warning: Shuffle Join MERGEJOIN[96][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
-Warning: Shuffle Join MERGEJOIN[97][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 5' is a cross product
-Warning: Shuffle Join MERGEJOIN[98][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5]] in Stage 'Reducer 6' is a cross product
-Warning: Shuffle Join MERGEJOIN[99][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6]] in Stage 'Reducer 7' is a cross product
+Warning: Shuffle Join MERGEJOIN[37][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[38][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[39][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
PREHOOK: query: explain select case when (select count(*)
from part
where p_size between 1 and 20) > 409437
@@ -4607,18 +3979,12 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 11 <- Map 10 (CUSTOM_SIMPLE_EDGE)
- Reducer 13 <- Map 12 (CUSTOM_SIMPLE_EDGE)
- Reducer 15 <- Map 14 (CUSTOM_SIMPLE_EDGE)
- Reducer 17 <- Map 16 (CUSTOM_SIMPLE_EDGE)
- Reducer 19 <- Map 18 (CUSTOM_SIMPLE_EDGE)
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 9 (CUSTOM_SIMPLE_EDGE)
- Reducer 3 <- Reducer 11 (CUSTOM_SIMPLE_EDGE), Reducer 2 (CUSTOM_SIMPLE_EDGE)
- Reducer 4 <- Reducer 13 (CUSTOM_SIMPLE_EDGE), Reducer 3 (CUSTOM_SIMPLE_EDGE)
- Reducer 5 <- Reducer 15 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
- Reducer 6 <- Reducer 17 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
- Reducer 7 <- Reducer 19 (CUSTOM_SIMPLE_EDGE), Reducer 6 (CUSTOM_SIMPLE_EDGE)
- Reducer 9 <- Map 8 (CUSTOM_SIMPLE_EDGE)
+ Reducer 10 <- Map 9 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 6 (CUSTOM_SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 8 (CUSTOM_SIMPLE_EDGE)
+ Reducer 4 <- Reducer 10 (CUSTOM_SIMPLE_EDGE), Reducer 3 (CUSTOM_SIMPLE_EDGE)
+ Reducer 6 <- Map 5 (CUSTOM_SIMPLE_EDGE)
+ Reducer 8 <- Map 7 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -4633,7 +3999,7 @@ STAGE PLANS:
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
- Map 10
+ Map 5
Map Operator Tree:
TableScan
alias: part
@@ -4654,26 +4020,7 @@ STAGE PLANS:
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
- Map 12
- Map Operator Tree:
- TableScan
- alias: part
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: p_partkey BETWEEN 1 AND 20 (type: boolean)
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 14
+ Map 7
Map Operator Tree:
TableScan
alias: part
@@ -4692,26 +4039,7 @@ STAGE PLANS:
value expressions: _col0 (type: struct<count:bigint,sum:double,input:int>)
Execution mode: llap
LLAP IO: no inputs
- Map 16
- Map Operator Tree:
- TableScan
- alias: part
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: p_partkey BETWEEN 10000 AND 20000 (type: boolean)
- Statistics: Num rows: 8 Data size: 32 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 18
+ Map 9
Map Operator Tree:
TableScan
alias: part
@@ -4734,100 +4062,7 @@ STAGE PLANS:
value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Map 8
- Map Operator Tree:
- TableScan
- alias: part
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: p_size BETWEEN 1 AND 20 (type: boolean)
- Statistics: Num rows: 8 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
- Statistics: Num rows: 8 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 11
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Reducer 13
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 15
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: avg(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: double)
- Reducer 17
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 19
+ Reducer 10
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -4844,45 +4079,17 @@ STAGE PLANS:
Reduce Operator Tree:
Merge Join Operator
condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
Left Outer Join0 to 1
keys:
0
1
- outputColumnNames: _col2
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col2 (type: bigint)
- Reducer 4
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col2
+ outputColumnNames: _col1
Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col2 (type: bigint)
- Reducer 5
+ value expressions: _col1 (type: bigint)
+ Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
@@ -4891,28 +4098,13 @@ STAGE PLANS:
keys:
0
1
- outputColumnNames: _col2, _col4
- Statistics: Num rows: 26 Data size: 416 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 416 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col2 (type: bigint), _col4 (type: double)
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col2, _col4
+ outputColumnNames: _col1, _col2
Statistics: Num rows: 26 Data size: 416 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
Statistics: Num rows: 26 Data size: 416 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col2 (type: bigint), _col4 (type: double)
- Reducer 7
+ value expressions: _col1 (type: bigint), _col2 (type: double)
+ Reducer 4
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
@@ -4921,10 +4113,10 @@ STAGE PLANS:
keys:
0
1
- outputColumnNames: _col2, _col4, _col6
+ outputColumnNames: _col1, _col2, _col3
Statistics: Num rows: 26 Data size: 520 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: CASE WHEN ((_col2 > 409437)) THEN (_col4) ELSE (_col6) END (type: double)
+ expressions: CASE WHEN ((_col1 > 409437)) THEN (_col2) ELSE (_col3) END (type: double)
outputColumnNames: _col0
Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
@@ -4934,7 +4126,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 9
+ Reducer 6
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -4942,21 +4134,22 @@ STAGE PLANS:
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
+ Reduce Output Operator
+ sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
-
<TRUNCATED>
[3/6] hive git commit: HIVE-16330 : Improve plans for scalar subquery
with aggregates (Vineet Garg via Ashutosh Chauhan)
Posted by ha...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/results/clientpositive/perf/query1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/query1.q.out b/ql/src/test/results/clientpositive/perf/query1.q.out
index 07828da..d84c76d 100644
--- a/ql/src/test/results/clientpositive/perf/query1.q.out
+++ b/ql/src/test/results/clientpositive/perf/query1.q.out
@@ -47,175 +47,127 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Vertex dependency in root stage
-Reducer 13 <- Map 12 (SIMPLE_EDGE), Map 15 (SIMPLE_EDGE)
-Reducer 14 <- Reducer 13 (SIMPLE_EDGE)
-Reducer 17 <- Map 16 (SIMPLE_EDGE), Map 19 (SIMPLE_EDGE)
-Reducer 18 <- Reducer 17 (SIMPLE_EDGE)
-Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 9 (SIMPLE_EDGE)
+Reducer 12 <- Map 11 (SIMPLE_EDGE), Map 14 (SIMPLE_EDGE)
+Reducer 13 <- Reducer 12 (SIMPLE_EDGE)
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 8 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
-Reducer 4 <- Map 10 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
-Reducer 5 <- Map 11 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
-Reducer 6 <- Reducer 14 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
-Reducer 7 <- Reducer 18 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
-Reducer 8 <- Reducer 7 (SIMPLE_EDGE)
+Reducer 4 <- Map 9 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
+Reducer 5 <- Map 10 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
+Reducer 6 <- Reducer 13 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
+Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
Stage-0
Fetch Operator
limit:100
Stage-1
- Reducer 8
- File Output Operator [FS_82]
- Limit [LIM_81] (rows=100 width=860)
+ Reducer 7
+ File Output Operator [FS_54]
+ Limit [LIM_53] (rows=100 width=860)
Number of rows:100
- Select Operator [SEL_80] (rows=35493335 width=860)
+ Select Operator [SEL_52] (rows=32266667 width=860)
Output:["_col0"]
- <-Reducer 7 [SIMPLE_EDGE]
- SHUFFLE [RS_79]
- Select Operator [SEL_78] (rows=35493335 width=860)
+ <-Reducer 6 [SIMPLE_EDGE]
+ SHUFFLE [RS_51]
+ Select Operator [SEL_50] (rows=32266667 width=860)
Output:["_col0"]
- Filter Operator [FIL_77] (rows=35493335 width=860)
- predicate:(_col2 > CASE WHEN (_col10 is null) THEN (null) ELSE (_col9) END)
- Merge Join Operator [MERGEJOIN_114] (rows=106480005 width=860)
- Conds:RS_74._col1=RS_75._col2(Left Outer),Output:["_col2","_col6","_col9","_col10"]
- <-Reducer 18 [SIMPLE_EDGE]
- SHUFFLE [RS_75]
+ Filter Operator [FIL_49] (rows=32266667 width=860)
+ predicate:(_col2 > CASE WHEN (_col8 is null) THEN (null) ELSE (_col7) END)
+ Merge Join Operator [MERGEJOIN_78] (rows=96800003 width=860)
+ Conds:RS_45._col1=RS_46._col2(Left Outer),Output:["_col2","_col6","_col7","_col8"]
+ <-Reducer 13 [SIMPLE_EDGE]
+ SHUFFLE [RS_46]
PartitionCols:_col2
- Select Operator [SEL_73] (rows=15837566 width=77)
+ Select Operator [SEL_38] (rows=15837566 width=77)
Output:["_col0","_col1","_col2"]
- Group By Operator [GBY_72] (rows=15837566 width=77)
+ Group By Operator [GBY_37] (rows=15837566 width=77)
Output:["_col0","_col1"],aggregations:["avg(_col2)"],keys:_col1
- Select Operator [SEL_68] (rows=31675133 width=77)
+ Select Operator [SEL_33] (rows=31675133 width=77)
Output:["_col1","_col2"]
- Group By Operator [GBY_67] (rows=31675133 width=77)
+ Group By Operator [GBY_32] (rows=31675133 width=77)
Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
- <-Reducer 17 [SIMPLE_EDGE]
- SHUFFLE [RS_66]
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_31]
PartitionCols:_col0
- Group By Operator [GBY_65] (rows=63350266 width=77)
+ Group By Operator [GBY_30] (rows=63350266 width=77)
Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col2, _col1
- Merge Join Operator [MERGEJOIN_112] (rows=63350266 width=77)
- Conds:RS_61._col0=RS_62._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 16 [SIMPLE_EDGE]
- SHUFFLE [RS_61]
+ Merge Join Operator [MERGEJOIN_77] (rows=63350266 width=77)
+ Conds:RS_26._col0=RS_27._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_26]
PartitionCols:_col0
- Select Operator [SEL_57] (rows=57591150 width=77)
+ Select Operator [SEL_22] (rows=57591150 width=77)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_106] (rows=57591150 width=77)
+ Filter Operator [FIL_72] (rows=57591150 width=77)
predicate:(sr_returned_date_sk is not null and sr_store_sk is not null)
- TableScan [TS_55] (rows=57591150 width=77)
+ TableScan [TS_20] (rows=57591150 width=77)
default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_customer_sk","sr_store_sk","sr_fee"]
- <-Map 19 [SIMPLE_EDGE]
- SHUFFLE [RS_62]
+ <-Map 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_27]
PartitionCols:_col0
- Select Operator [SEL_60] (rows=36524 width=1119)
+ Select Operator [SEL_25] (rows=36524 width=1119)
Output:["_col0"]
- Filter Operator [FIL_107] (rows=36524 width=1119)
+ Filter Operator [FIL_73] (rows=36524 width=1119)
predicate:((d_year = 2000) and d_date_sk is not null)
- TableScan [TS_58] (rows=73049 width=1119)
+ TableScan [TS_23] (rows=73049 width=1119)
default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Reducer 6 [SIMPLE_EDGE]
- SHUFFLE [RS_74]
+ <-Reducer 5 [SIMPLE_EDGE]
+ SHUFFLE [RS_45]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_113] (rows=96800003 width=860)
- Conds:RS_51._col1=RS_52._col0(Left Outer),Output:["_col1","_col2","_col6"]
- <-Reducer 14 [SIMPLE_EDGE]
- SHUFFLE [RS_52]
+ Merge Join Operator [MERGEJOIN_76] (rows=88000001 width=860)
+ Conds:RS_42._col0=RS_43._col0(Inner),Output:["_col1","_col2","_col6"]
+ <-Map 10 [SIMPLE_EDGE]
+ SHUFFLE [RS_43]
PartitionCols:_col0
- Select Operator [SEL_44] (rows=2639594 width=77)
- Output:["_col0"]
- Filter Operator [FIL_43] (rows=2639594 width=77)
- predicate:(sq_count_check(_col1) <= 1)
- Group By Operator [GBY_42] (rows=7918783 width=77)
- Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
- Group By Operator [GBY_37] (rows=15837566 width=77)
- Output:["_col0"],keys:_col1
- Select Operator [SEL_33] (rows=31675133 width=77)
- Output:["_col1"]
- Group By Operator [GBY_32] (rows=31675133 width=77)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
- <-Reducer 13 [SIMPLE_EDGE]
- SHUFFLE [RS_31]
- PartitionCols:_col0
- Group By Operator [GBY_30] (rows=63350266 width=77)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col2, _col1
- Merge Join Operator [MERGEJOIN_111] (rows=63350266 width=77)
- Conds:RS_26._col0=RS_27._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 12 [SIMPLE_EDGE]
- SHUFFLE [RS_26]
- PartitionCols:_col0
- Select Operator [SEL_22] (rows=57591150 width=77)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_104] (rows=57591150 width=77)
- predicate:(sr_returned_date_sk is not null and sr_store_sk is not null)
- TableScan [TS_20] (rows=57591150 width=77)
- default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_customer_sk","sr_store_sk","sr_fee"]
- <-Map 15 [SIMPLE_EDGE]
- SHUFFLE [RS_27]
- PartitionCols:_col0
- Select Operator [SEL_25] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_105] (rows=36524 width=1119)
- predicate:((d_year = 2000) and d_date_sk is not null)
- TableScan [TS_23] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Reducer 5 [SIMPLE_EDGE]
- SHUFFLE [RS_51]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_110] (rows=88000001 width=860)
- Conds:RS_48._col0=RS_49._col0(Inner),Output:["_col1","_col2","_col6"]
- <-Map 11 [SIMPLE_EDGE]
- SHUFFLE [RS_49]
- PartitionCols:_col0
- Select Operator [SEL_19] (rows=80000000 width=860)
- Output:["_col0","_col1"]
- Filter Operator [FIL_103] (rows=80000000 width=860)
- predicate:c_customer_sk is not null
- TableScan [TS_17] (rows=80000000 width=860)
- default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_customer_id"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_48]
+ Select Operator [SEL_19] (rows=80000000 width=860)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_71] (rows=80000000 width=860)
+ predicate:c_customer_sk is not null
+ TableScan [TS_17] (rows=80000000 width=860)
+ default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_customer_id"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_42]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_75] (rows=34842647 width=77)
+ Conds:RS_39._col1=RS_40._col0(Inner),Output:["_col0","_col1","_col2"]
+ <-Map 9 [SIMPLE_EDGE]
+ SHUFFLE [RS_40]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_109] (rows=34842647 width=77)
- Conds:RS_45._col1=RS_46._col0(Inner),Output:["_col0","_col1","_col2"]
- <-Map 10 [SIMPLE_EDGE]
- SHUFFLE [RS_46]
- PartitionCols:_col0
- Select Operator [SEL_16] (rows=852 width=1910)
- Output:["_col0"]
- Filter Operator [FIL_102] (rows=852 width=1910)
- predicate:((s_state = 'NM') and s_store_sk is not null)
- TableScan [TS_14] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_state"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_45]
- PartitionCols:_col1
- Select Operator [SEL_13] (rows=31675133 width=77)
- Output:["_col0","_col1","_col2"]
- Group By Operator [GBY_12] (rows=31675133 width=77)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_11]
- PartitionCols:_col0, _col1
- Group By Operator [GBY_10] (rows=63350266 width=77)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col2, _col1
- Merge Join Operator [MERGEJOIN_108] (rows=63350266 width=77)
- Conds:RS_6._col0=RS_7._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 1 [SIMPLE_EDGE]
- SHUFFLE [RS_6]
- PartitionCols:_col0
- Select Operator [SEL_2] (rows=57591150 width=77)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_100] (rows=57591150 width=77)
- predicate:(sr_returned_date_sk is not null and sr_store_sk is not null and sr_customer_sk is not null)
- TableScan [TS_0] (rows=57591150 width=77)
- default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_customer_sk","sr_store_sk","sr_fee"]
- <-Map 9 [SIMPLE_EDGE]
- SHUFFLE [RS_7]
- PartitionCols:_col0
- Select Operator [SEL_5] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_101] (rows=36524 width=1119)
- predicate:((d_year = 2000) and d_date_sk is not null)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ Select Operator [SEL_16] (rows=852 width=1910)
+ Output:["_col0"]
+ Filter Operator [FIL_70] (rows=852 width=1910)
+ predicate:((s_state = 'NM') and s_store_sk is not null)
+ TableScan [TS_14] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_state"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_39]
+ PartitionCols:_col1
+ Select Operator [SEL_13] (rows=31675133 width=77)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_12] (rows=31675133 width=77)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_11]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_10] (rows=63350266 width=77)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col2, _col1
+ Merge Join Operator [MERGEJOIN_74] (rows=63350266 width=77)
+ Conds:RS_6._col0=RS_7._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_6]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=57591150 width=77)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_68] (rows=57591150 width=77)
+ predicate:(sr_returned_date_sk is not null and sr_store_sk is not null and sr_customer_sk is not null)
+ TableScan [TS_0] (rows=57591150 width=77)
+ default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_customer_sk","sr_store_sk","sr_fee"]
+ <-Map 8 [SIMPLE_EDGE]
+ SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=36524 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_69] (rows=36524 width=1119)
+ predicate:((d_year = 2000) and d_date_sk is not null)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/results/clientpositive/perf/query30.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/query30.q.out b/ql/src/test/results/clientpositive/perf/query30.q.out
index 72871f4..4b7d949 100644
--- a/ql/src/test/results/clientpositive/perf/query30.q.out
+++ b/ql/src/test/results/clientpositive/perf/query30.q.out
@@ -59,222 +59,159 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Vertex dependency in root stage
-Reducer 10 <- Reducer 9 (SIMPLE_EDGE)
-Reducer 11 <- Reducer 10 (SIMPLE_EDGE), Reducer 17 (SIMPLE_EDGE)
-Reducer 15 <- Map 14 (SIMPLE_EDGE), Map 18 (SIMPLE_EDGE)
-Reducer 16 <- Map 19 (SIMPLE_EDGE), Reducer 15 (SIMPLE_EDGE)
-Reducer 17 <- Reducer 16 (SIMPLE_EDGE)
-Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE)
-Reducer 21 <- Map 20 (SIMPLE_EDGE), Map 24 (SIMPLE_EDGE)
-Reducer 22 <- Map 25 (SIMPLE_EDGE), Reducer 21 (SIMPLE_EDGE)
-Reducer 23 <- Reducer 22 (SIMPLE_EDGE)
-Reducer 3 <- Reducer 11 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
-Reducer 4 <- Reducer 23 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
-Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
-Reducer 8 <- Map 12 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
-Reducer 9 <- Map 13 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
+Reducer 10 <- Reducer 16 (SIMPLE_EDGE), Reducer 9 (SIMPLE_EDGE)
+Reducer 14 <- Map 13 (SIMPLE_EDGE), Map 17 (SIMPLE_EDGE)
+Reducer 15 <- Map 18 (SIMPLE_EDGE), Reducer 14 (SIMPLE_EDGE)
+Reducer 16 <- Reducer 15 (SIMPLE_EDGE)
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 10 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
+Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
+Reducer 7 <- Map 11 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE)
+Reducer 8 <- Map 12 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
+Reducer 9 <- Reducer 8 (SIMPLE_EDGE)
Stage-0
Fetch Operator
limit:100
Stage-1
- Reducer 5
- File Output Operator [FS_101]
- Limit [LIM_100] (rows=100 width=860)
+ Reducer 4
+ File Output Operator [FS_67]
+ Limit [LIM_66] (rows=100 width=860)
Number of rows:100
- Select Operator [SEL_99] (rows=35493335 width=860)
+ Select Operator [SEL_65] (rows=32266667 width=860)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_98]
- Select Operator [SEL_97] (rows=35493335 width=860)
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_64]
+ Select Operator [SEL_63] (rows=32266667 width=860)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12"]
- Filter Operator [FIL_96] (rows=35493335 width=860)
- predicate:(_col2 > CASE WHEN (_col22 is null) THEN (null) ELSE (_col21) END)
- Merge Join Operator [MERGEJOIN_153] (rows=106480005 width=860)
- Conds:RS_93._col1=RS_94._col2(Left Outer),Output:["_col2","_col6","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col21","_col22"]
- <-Reducer 23 [SIMPLE_EDGE]
- SHUFFLE [RS_94]
- PartitionCols:_col2
- Select Operator [SEL_92] (rows=11000000 width=1014)
- Output:["_col0","_col1","_col2"]
- Group By Operator [GBY_91] (rows=11000000 width=1014)
- Output:["_col0","_col1"],aggregations:["avg(_col2)"],keys:_col0
- Select Operator [SEL_87] (rows=22000000 width=1014)
- Output:["_col0","_col2"]
- Group By Operator [GBY_86] (rows=22000000 width=1014)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
- <-Reducer 22 [SIMPLE_EDGE]
- SHUFFLE [RS_85]
- PartitionCols:_col0
- Group By Operator [GBY_84] (rows=44000000 width=1014)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col7, _col1
- Merge Join Operator [MERGEJOIN_150] (rows=44000000 width=1014)
- Conds:RS_80._col2=RS_81._col0(Inner),Output:["_col1","_col3","_col7"]
- <-Map 25 [SIMPLE_EDGE]
- SHUFFLE [RS_81]
- PartitionCols:_col0
- Select Operator [SEL_76] (rows=40000000 width=1014)
- Output:["_col0","_col1"]
- Filter Operator [FIL_143] (rows=40000000 width=1014)
- predicate:(ca_address_sk is not null and ca_state is not null)
- TableScan [TS_74] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
- <-Reducer 21 [SIMPLE_EDGE]
- SHUFFLE [RS_80]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_149] (rows=15838314 width=92)
- Conds:RS_77._col0=RS_78._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 20 [SIMPLE_EDGE]
- SHUFFLE [RS_77]
- PartitionCols:_col0
- Select Operator [SEL_70] (rows=14398467 width=92)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_141] (rows=14398467 width=92)
- predicate:(wr_returned_date_sk is not null and wr_returning_addr_sk is not null)
- TableScan [TS_68] (rows=14398467 width=92)
- default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_returned_date_sk","wr_returning_customer_sk","wr_returning_addr_sk","wr_return_amt"]
- <-Map 24 [SIMPLE_EDGE]
- SHUFFLE [RS_78]
- PartitionCols:_col0
- Select Operator [SEL_73] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_142] (rows=36524 width=1119)
- predicate:((d_year = 2002) and d_date_sk is not null)
- TableScan [TS_71] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_93]
- PartitionCols:_col1
- Select Operator [SEL_67] (rows=96800003 width=860)
- Output:["_col1","_col2","_col6","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18"]
- Merge Join Operator [MERGEJOIN_152] (rows=96800003 width=860)
- Conds:RS_64._col0=RS_65._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col17","_col18"]
- <-Reducer 11 [SIMPLE_EDGE]
- SHUFFLE [RS_65]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_151] (rows=24200000 width=1014)
- Conds:RS_57._col1=RS_58._col0(Left Outer),Output:["_col0","_col1","_col2"]
- <-Reducer 10 [SIMPLE_EDGE]
- SHUFFLE [RS_57]
- PartitionCols:_col1
- Select Operator [SEL_25] (rows=22000000 width=1014)
- Output:["_col0","_col1","_col2"]
- Group By Operator [GBY_24] (rows=22000000 width=1014)
+ Filter Operator [FIL_62] (rows=32266667 width=860)
+ predicate:(_col2 > CASE WHEN (_col20 is null) THEN (null) ELSE (_col19) END)
+ Select Operator [SEL_61] (rows=96800003 width=860)
+ Output:["_col2","_col6","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20"]
+ Merge Join Operator [MERGEJOIN_105] (rows=96800003 width=860)
+ Conds:RS_58._col0=RS_59._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col18","_col19","_col20"]
+ <-Reducer 10 [SIMPLE_EDGE]
+ SHUFFLE [RS_59]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_104] (rows=24200000 width=1014)
+ Conds:RS_51._col1=RS_52._col2(Left Outer),Output:["_col0","_col2","_col3","_col4"]
+ <-Reducer 16 [SIMPLE_EDGE]
+ SHUFFLE [RS_52]
+ PartitionCols:_col2
+ Select Operator [SEL_50] (rows=11000000 width=1014)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_49] (rows=11000000 width=1014)
+ Output:["_col0","_col1"],aggregations:["avg(_col2)"],keys:_col0
+ Select Operator [SEL_45] (rows=22000000 width=1014)
+ Output:["_col0","_col2"]
+ Group By Operator [GBY_44] (rows=22000000 width=1014)
Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
- <-Reducer 9 [SIMPLE_EDGE]
- SHUFFLE [RS_23]
- PartitionCols:_col0, _col1
- Group By Operator [GBY_22] (rows=44000000 width=1014)
+ <-Reducer 15 [SIMPLE_EDGE]
+ SHUFFLE [RS_43]
+ PartitionCols:_col0
+ Group By Operator [GBY_42] (rows=44000000 width=1014)
Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col7, _col1
- Merge Join Operator [MERGEJOIN_146] (rows=44000000 width=1014)
- Conds:RS_18._col2=RS_19._col0(Inner),Output:["_col1","_col3","_col7"]
- <-Map 13 [SIMPLE_EDGE]
- SHUFFLE [RS_19]
+ Merge Join Operator [MERGEJOIN_103] (rows=44000000 width=1014)
+ Conds:RS_38._col2=RS_39._col0(Inner),Output:["_col1","_col3","_col7"]
+ <-Map 18 [SIMPLE_EDGE]
+ SHUFFLE [RS_39]
PartitionCols:_col0
- Select Operator [SEL_14] (rows=40000000 width=1014)
+ Select Operator [SEL_34] (rows=40000000 width=1014)
Output:["_col0","_col1"]
- Filter Operator [FIL_137] (rows=40000000 width=1014)
- predicate:ca_address_sk is not null
- TableScan [TS_12] (rows=40000000 width=1014)
+ Filter Operator [FIL_98] (rows=40000000 width=1014)
+ predicate:(ca_address_sk is not null and ca_state is not null)
+ TableScan [TS_32] (rows=40000000 width=1014)
default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
- <-Reducer 8 [SIMPLE_EDGE]
- SHUFFLE [RS_18]
+ <-Reducer 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_38]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_145] (rows=15838314 width=92)
- Conds:RS_15._col0=RS_16._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 12 [SIMPLE_EDGE]
- SHUFFLE [RS_16]
+ Merge Join Operator [MERGEJOIN_102] (rows=15838314 width=92)
+ Conds:RS_35._col0=RS_36._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 13 [SIMPLE_EDGE]
+ SHUFFLE [RS_35]
+ PartitionCols:_col0
+ Select Operator [SEL_28] (rows=14398467 width=92)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_96] (rows=14398467 width=92)
+ predicate:(wr_returned_date_sk is not null and wr_returning_addr_sk is not null)
+ TableScan [TS_26] (rows=14398467 width=92)
+ default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_returned_date_sk","wr_returning_customer_sk","wr_returning_addr_sk","wr_return_amt"]
+ <-Map 17 [SIMPLE_EDGE]
+ SHUFFLE [RS_36]
PartitionCols:_col0
- Select Operator [SEL_11] (rows=36524 width=1119)
+ Select Operator [SEL_31] (rows=36524 width=1119)
Output:["_col0"]
- Filter Operator [FIL_136] (rows=36524 width=1119)
+ Filter Operator [FIL_97] (rows=36524 width=1119)
predicate:((d_year = 2002) and d_date_sk is not null)
- TableScan [TS_9] (rows=73049 width=1119)
+ TableScan [TS_29] (rows=73049 width=1119)
default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Map 7 [SIMPLE_EDGE]
- SHUFFLE [RS_15]
- PartitionCols:_col0
- Select Operator [SEL_8] (rows=14398467 width=92)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_135] (rows=14398467 width=92)
- predicate:(wr_returned_date_sk is not null and wr_returning_addr_sk is not null and wr_returning_customer_sk is not null)
- TableScan [TS_6] (rows=14398467 width=92)
- default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_returned_date_sk","wr_returning_customer_sk","wr_returning_addr_sk","wr_return_amt"]
- <-Reducer 17 [SIMPLE_EDGE]
- SHUFFLE [RS_58]
- PartitionCols:_col0
- Select Operator [SEL_56] (rows=1833333 width=1014)
- Output:["_col0"]
- Filter Operator [FIL_55] (rows=1833333 width=1014)
- predicate:(sq_count_check(_col1) <= 1)
- Group By Operator [GBY_54] (rows=5500000 width=1014)
- Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
- Group By Operator [GBY_49] (rows=11000000 width=1014)
- Output:["_col0"],keys:_col0
- Select Operator [SEL_45] (rows=22000000 width=1014)
- Output:["_col0"]
- Group By Operator [GBY_44] (rows=22000000 width=1014)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
- <-Reducer 16 [SIMPLE_EDGE]
- SHUFFLE [RS_43]
- PartitionCols:_col0
- Group By Operator [GBY_42] (rows=44000000 width=1014)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col7, _col1
- Merge Join Operator [MERGEJOIN_148] (rows=44000000 width=1014)
- Conds:RS_38._col2=RS_39._col0(Inner),Output:["_col1","_col3","_col7"]
- <-Map 19 [SIMPLE_EDGE]
- SHUFFLE [RS_39]
- PartitionCols:_col0
- Select Operator [SEL_34] (rows=40000000 width=1014)
- Output:["_col0","_col1"]
- Filter Operator [FIL_140] (rows=40000000 width=1014)
- predicate:(ca_address_sk is not null and ca_state is not null)
- TableScan [TS_32] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
- <-Reducer 15 [SIMPLE_EDGE]
- SHUFFLE [RS_38]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_147] (rows=15838314 width=92)
- Conds:RS_35._col0=RS_36._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 14 [SIMPLE_EDGE]
- SHUFFLE [RS_35]
- PartitionCols:_col0
- Select Operator [SEL_28] (rows=14398467 width=92)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_138] (rows=14398467 width=92)
- predicate:(wr_returned_date_sk is not null and wr_returning_addr_sk is not null)
- TableScan [TS_26] (rows=14398467 width=92)
- default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_returned_date_sk","wr_returning_customer_sk","wr_returning_addr_sk","wr_return_amt"]
- <-Map 18 [SIMPLE_EDGE]
- SHUFFLE [RS_36]
- PartitionCols:_col0
- Select Operator [SEL_31] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_139] (rows=36524 width=1119)
- predicate:((d_year = 2002) and d_date_sk is not null)
- TableScan [TS_29] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_64]
+ <-Reducer 9 [SIMPLE_EDGE]
+ SHUFFLE [RS_51]
+ PartitionCols:_col1
+ Select Operator [SEL_25] (rows=22000000 width=1014)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_24] (rows=22000000 width=1014)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
+ <-Reducer 8 [SIMPLE_EDGE]
+ SHUFFLE [RS_23]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_22] (rows=44000000 width=1014)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col7, _col1
+ Merge Join Operator [MERGEJOIN_101] (rows=44000000 width=1014)
+ Conds:RS_18._col2=RS_19._col0(Inner),Output:["_col1","_col3","_col7"]
+ <-Map 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_19]
+ PartitionCols:_col0
+ Select Operator [SEL_14] (rows=40000000 width=1014)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_95] (rows=40000000 width=1014)
+ predicate:ca_address_sk is not null
+ TableScan [TS_12] (rows=40000000 width=1014)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
+ <-Reducer 7 [SIMPLE_EDGE]
+ SHUFFLE [RS_18]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_100] (rows=15838314 width=92)
+ Conds:RS_15._col0=RS_16._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_16]
+ PartitionCols:_col0
+ Select Operator [SEL_11] (rows=36524 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_94] (rows=36524 width=1119)
+ predicate:((d_year = 2002) and d_date_sk is not null)
+ TableScan [TS_9] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ <-Map 6 [SIMPLE_EDGE]
+ SHUFFLE [RS_15]
+ PartitionCols:_col0
+ Select Operator [SEL_8] (rows=14398467 width=92)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_93] (rows=14398467 width=92)
+ predicate:(wr_returned_date_sk is not null and wr_returning_addr_sk is not null and wr_returning_customer_sk is not null)
+ TableScan [TS_6] (rows=14398467 width=92)
+ default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_returned_date_sk","wr_returning_customer_sk","wr_returning_addr_sk","wr_return_amt"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_58]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_99] (rows=88000001 width=860)
+ Conds:RS_55._col2=RS_56._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_55]
+ PartitionCols:_col2
+ Select Operator [SEL_2] (rows=80000000 width=860)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
+ Filter Operator [FIL_91] (rows=80000000 width=860)
+ predicate:(c_customer_sk is not null and c_current_addr_sk is not null)
+ TableScan [TS_0] (rows=80000000 width=860)
+ default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_customer_id","c_current_addr_sk","c_salutation","c_first_name","c_last_name","c_preferred_cust_flag","c_birth_day","c_birth_month","c_birth_year","c_birth_country","c_login","c_email_address","c_last_review_date"]
+ <-Map 5 [SIMPLE_EDGE]
+ SHUFFLE [RS_56]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_144] (rows=88000001 width=860)
- Conds:RS_61._col2=RS_62._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- <-Map 1 [SIMPLE_EDGE]
- SHUFFLE [RS_61]
- PartitionCols:_col2
- Select Operator [SEL_2] (rows=80000000 width=860)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- Filter Operator [FIL_133] (rows=80000000 width=860)
- predicate:(c_customer_sk is not null and c_current_addr_sk is not null)
- TableScan [TS_0] (rows=80000000 width=860)
- default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_customer_id","c_current_addr_sk","c_salutation","c_first_name","c_last_name","c_preferred_cust_flag","c_birth_day","c_birth_month","c_birth_year","c_birth_country","c_login","c_email_address","c_last_review_date"]
- <-Map 6 [SIMPLE_EDGE]
- SHUFFLE [RS_62]
- PartitionCols:_col0
- Select Operator [SEL_5] (rows=20000000 width=1014)
- Output:["_col0"]
- Filter Operator [FIL_134] (rows=20000000 width=1014)
- predicate:((ca_state = 'IL') and ca_address_sk is not null)
- TableScan [TS_3] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
+ Select Operator [SEL_5] (rows=20000000 width=1014)
+ Output:["_col0"]
+ Filter Operator [FIL_92] (rows=20000000 width=1014)
+ predicate:((ca_state = 'IL') and ca_address_sk is not null)
+ TableScan [TS_3] (rows=40000000 width=1014)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/results/clientpositive/perf/query6.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/query6.q.out b/ql/src/test/results/clientpositive/perf/query6.q.out
index aede0d7..183c2ec 100644
--- a/ql/src/test/results/clientpositive/perf/query6.q.out
+++ b/ql/src/test/results/clientpositive/perf/query6.q.out
@@ -1,4 +1,4 @@
-Warning: Shuffle Join MERGEJOIN[129][tables = [$hdt$_5, $hdt$_6]] in Stage 'Reducer 17' is a cross product
+Warning: Shuffle Join MERGEJOIN[111][tables = [$hdt$_5, $hdt$_6]] in Stage 'Reducer 16' is a cross product
PREHOOK: query: explain select a.ca_state state, count(*) cnt
from customer_address a
,customer c
@@ -50,207 +50,180 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Vertex dependency in root stage
-Reducer 10 <- Map 9 (SIMPLE_EDGE)
-Reducer 13 <- Map 12 (SIMPLE_EDGE), Map 14 (SIMPLE_EDGE)
-Reducer 16 <- Map 15 (SIMPLE_EDGE)
-Reducer 17 <- Reducer 16 (CUSTOM_SIMPLE_EDGE), Reducer 21 (CUSTOM_SIMPLE_EDGE)
-Reducer 18 <- Map 22 (SIMPLE_EDGE), Reducer 17 (SIMPLE_EDGE)
-Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 10 (SIMPLE_EDGE)
-Reducer 20 <- Map 19 (SIMPLE_EDGE)
-Reducer 21 <- Reducer 20 (CUSTOM_SIMPLE_EDGE)
-Reducer 24 <- Map 23 (SIMPLE_EDGE)
-Reducer 3 <- Map 11 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
-Reducer 4 <- Reducer 13 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
-Reducer 5 <- Reducer 18 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
-Reducer 6 <- Reducer 24 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
+Reducer 12 <- Map 11 (SIMPLE_EDGE), Map 13 (SIMPLE_EDGE)
+Reducer 15 <- Map 14 (SIMPLE_EDGE)
+Reducer 16 <- Reducer 15 (CUSTOM_SIMPLE_EDGE), Reducer 20 (CUSTOM_SIMPLE_EDGE)
+Reducer 17 <- Map 21 (SIMPLE_EDGE), Reducer 16 (SIMPLE_EDGE)
+Reducer 19 <- Map 18 (SIMPLE_EDGE)
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 9 (SIMPLE_EDGE)
+Reducer 20 <- Reducer 19 (CUSTOM_SIMPLE_EDGE)
+Reducer 3 <- Map 10 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
+Reducer 4 <- Reducer 12 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
+Reducer 5 <- Reducer 17 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
+Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
-Reducer 8 <- Reducer 7 (SIMPLE_EDGE)
+Reducer 9 <- Map 8 (SIMPLE_EDGE)
Stage-0
Fetch Operator
limit:100
Stage-1
- Reducer 8
- File Output Operator [FS_93]
- Limit [LIM_92] (rows=100 width=88)
+ Reducer 7
+ File Output Operator [FS_77]
+ Limit [LIM_76] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_91] (rows=46850848 width=88)
+ Select Operator [SEL_75] (rows=42591679 width=88)
Output:["_col0","_col1"]
- <-Reducer 7 [SIMPLE_EDGE]
- SHUFFLE [RS_90]
- Filter Operator [FIL_88] (rows=46850848 width=88)
+ <-Reducer 6 [SIMPLE_EDGE]
+ SHUFFLE [RS_74]
+ Filter Operator [FIL_72] (rows=42591679 width=88)
predicate:(_col1 >= 10)
- Group By Operator [GBY_87] (rows=140552546 width=88)
+ Group By Operator [GBY_71] (rows=127775039 width=88)
Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
- <-Reducer 6 [SIMPLE_EDGE]
- SHUFFLE [RS_86]
+ <-Reducer 5 [SIMPLE_EDGE]
+ SHUFFLE [RS_70]
PartitionCols:_col0
- Group By Operator [GBY_85] (rows=281105093 width=88)
+ Group By Operator [GBY_69] (rows=255550079 width=88)
Output:["_col0","_col1"],aggregations:["count()"],keys:_col1
- Select Operator [SEL_84] (rows=281105093 width=88)
+ Select Operator [SEL_68] (rows=255550079 width=88)
Output:["_col1"]
- Filter Operator [FIL_83] (rows=281105093 width=88)
- predicate:(_col10 > (1.2 * CASE WHEN (_col17 is null) THEN (null) ELSE (_col16) END))
- Merge Join Operator [MERGEJOIN_133] (rows=843315281 width=88)
- Conds:RS_80._col11=RS_81._col2(Left Outer),Output:["_col1","_col10","_col16","_col17"]
- <-Reducer 24 [SIMPLE_EDGE]
- SHUFFLE [RS_81]
- PartitionCols:_col2
- Select Operator [SEL_79] (rows=231000 width=1436)
- Output:["_col0","_col1","_col2"]
- Group By Operator [GBY_78] (rows=231000 width=1436)
- Output:["_col0","_col1"],aggregations:["avg(VALUE._col0)"],keys:KEY._col0
- <-Map 23 [SIMPLE_EDGE]
- SHUFFLE [RS_77]
- PartitionCols:_col0
- Group By Operator [GBY_76] (rows=462000 width=1436)
- Output:["_col0","_col1"],aggregations:["avg(i_current_price)"],keys:i_category
- Filter Operator [FIL_125] (rows=462000 width=1436)
- predicate:i_category is not null
- TableScan [TS_73] (rows=462000 width=1436)
- default@item,j,Tbl:COMPLETE,Col:NONE,Output:["i_current_price","i_category"]
- <-Reducer 5 [SIMPLE_EDGE]
- SHUFFLE [RS_80]
- PartitionCols:_col11
- Select Operator [SEL_72] (rows=766650239 width=88)
- Output:["_col1","_col10","_col11"]
- Merge Join Operator [MERGEJOIN_132] (rows=766650239 width=88)
- Conds:RS_69._col5=RS_70._col0(Inner),Output:["_col1","_col2","_col11"]
- <-Reducer 18 [SIMPLE_EDGE]
- SHUFFLE [RS_70]
+ Filter Operator [FIL_67] (rows=255550079 width=88)
+ predicate:(_col10 > (1.2 * CASE WHEN (_col15 is null) THEN (null) ELSE (_col14) END))
+ Select Operator [SEL_66] (rows=766650239 width=88)
+ Output:["_col1","_col10","_col14","_col15"]
+ Merge Join Operator [MERGEJOIN_114] (rows=766650239 width=88)
+ Conds:RS_63._col6=RS_64._col0(Inner),Output:["_col1","_col3","_col4","_col12"]
+ <-Reducer 17 [SIMPLE_EDGE]
+ SHUFFLE [RS_64]
+ PartitionCols:_col0
+ Select Operator [SEL_53] (rows=80353 width=1119)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_112] (rows=80353 width=1119)
+ Conds:RS_50._col0=RS_51._col1(Inner),Output:["_col2"]
+ <-Map 21 [SIMPLE_EDGE]
+ SHUFFLE [RS_51]
+ PartitionCols:_col1
+ Select Operator [SEL_46] (rows=73049 width=1119)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_107] (rows=73049 width=1119)
+ predicate:(d_date_sk is not null and d_month_seq is not null)
+ TableScan [TS_44] (rows=73049 width=1119)
+ default@date_dim,d,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_month_seq"]
+ <-Reducer 16 [SIMPLE_EDGE]
+ SHUFFLE [RS_50]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_111] (rows=9131 width=1128)
+ Conds:(Inner),Output:["_col0"]
+ <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_47]
+ Group By Operator [GBY_28] (rows=9131 width=1119)
+ Output:["_col0"],keys:KEY._col0
+ <-Map 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_27]
+ PartitionCols:_col0
+ Group By Operator [GBY_26] (rows=18262 width=1119)
+ Output:["_col0"],keys:d_month_seq
+ Select Operator [SEL_25] (rows=18262 width=1119)
+ Output:["d_month_seq"]
+ Filter Operator [FIL_105] (rows=18262 width=1119)
+ predicate:((d_year = 2000) and (d_moy = 2) and d_month_seq is not null)
+ TableScan [TS_23] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_month_seq","d_year","d_moy"]
+ <-Reducer 20 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_48]
+ Select Operator [SEL_43] (rows=1 width=8)
+ Filter Operator [FIL_42] (rows=1 width=8)
+ predicate:(sq_count_check(_col0) <= 1)
+ Group By Operator [GBY_40] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(VALUE._col0)"]
+ <-Reducer 19 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_39]
+ Group By Operator [GBY_38] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count()"]
+ Select Operator [SEL_36] (rows=9131 width=1119)
+ Group By Operator [GBY_35] (rows=9131 width=1119)
+ Output:["_col0"],keys:KEY._col0
+ <-Map 18 [SIMPLE_EDGE]
+ SHUFFLE [RS_34]
+ PartitionCols:_col0
+ Group By Operator [GBY_33] (rows=18262 width=1119)
+ Output:["_col0"],keys:d_month_seq
+ Select Operator [SEL_32] (rows=18262 width=1119)
+ Output:["d_month_seq"]
+ Filter Operator [FIL_106] (rows=18262 width=1119)
+ predicate:((d_year = 2000) and (d_moy = 2))
+ TableScan [TS_30] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_month_seq","d_year","d_moy"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_63]
+ PartitionCols:_col6
+ Merge Join Operator [MERGEJOIN_113] (rows=696954748 width=88)
+ Conds:RS_60._col8=RS_61._col0(Inner),Output:["_col1","_col3","_col4","_col6","_col12"]
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_61]
PartitionCols:_col0
- Select Operator [SEL_59] (rows=80353 width=1119)
- Output:["_col0"]
- Merge Join Operator [MERGEJOIN_130] (rows=80353 width=1119)
- Conds:RS_56._col0=RS_57._col1(Inner),Output:["_col2"]
- <-Map 22 [SIMPLE_EDGE]
- SHUFFLE [RS_57]
- PartitionCols:_col1
- Select Operator [SEL_52] (rows=73049 width=1119)
- Output:["_col0","_col1"]
- Filter Operator [FIL_124] (rows=73049 width=1119)
- predicate:(d_date_sk is not null and d_month_seq is not null)
- TableScan [TS_50] (rows=73049 width=1119)
- default@date_dim,d,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_month_seq"]
- <-Reducer 17 [SIMPLE_EDGE]
- SHUFFLE [RS_56]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_129] (rows=9131 width=1128)
- Conds:(Inner),Output:["_col0"]
- <-Reducer 16 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_53]
- Group By Operator [GBY_34] (rows=9131 width=1119)
- Output:["_col0"],keys:KEY._col0
- <-Map 15 [SIMPLE_EDGE]
- SHUFFLE [RS_33]
- PartitionCols:_col0
- Group By Operator [GBY_32] (rows=18262 width=1119)
- Output:["_col0"],keys:d_month_seq
- Select Operator [SEL_31] (rows=18262 width=1119)
- Output:["d_month_seq"]
- Filter Operator [FIL_122] (rows=18262 width=1119)
- predicate:((d_year = 2000) and (d_moy = 2) and d_month_seq is not null)
- TableScan [TS_29] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_month_seq","d_year","d_moy"]
- <-Reducer 21 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_54]
- Select Operator [SEL_49] (rows=1 width=8)
- Filter Operator [FIL_48] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_46] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Reducer 20 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_45]
- Group By Operator [GBY_44] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_42] (rows=9131 width=1119)
- Group By Operator [GBY_41] (rows=9131 width=1119)
- Output:["_col0"],keys:KEY._col0
- <-Map 19 [SIMPLE_EDGE]
- SHUFFLE [RS_40]
- PartitionCols:_col0
- Group By Operator [GBY_39] (rows=18262 width=1119)
- Output:["_col0"],keys:d_month_seq
- Select Operator [SEL_38] (rows=18262 width=1119)
- Output:["d_month_seq"]
- Filter Operator [FIL_123] (rows=18262 width=1119)
- predicate:((d_year = 2000) and (d_moy = 2))
- TableScan [TS_36] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_month_seq","d_year","d_moy"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_69]
- PartitionCols:_col5
- Merge Join Operator [MERGEJOIN_131] (rows=696954748 width=88)
- Conds:RS_66._col7=RS_67._col0(Inner),Output:["_col1","_col2","_col5","_col11"]
- <-Reducer 13 [SIMPLE_EDGE]
- SHUFFLE [RS_67]
+ Merge Join Operator [MERGEJOIN_110] (rows=88000001 width=860)
+ Conds:RS_19._col1=RS_20._col0(Inner),Output:["_col0","_col3"]
+ <-Map 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_19]
+ PartitionCols:_col1
+ Select Operator [SEL_15] (rows=80000000 width=860)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_103] (rows=80000000 width=860)
+ predicate:(c_current_addr_sk is not null and c_customer_sk is not null)
+ TableScan [TS_13] (rows=80000000 width=860)
+ default@customer,c,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_addr_sk"]
+ <-Map 13 [SIMPLE_EDGE]
+ SHUFFLE [RS_20]
+ PartitionCols:_col0
+ Select Operator [SEL_18] (rows=40000000 width=1014)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_104] (rows=40000000 width=1014)
+ predicate:ca_address_sk is not null
+ TableScan [TS_16] (rows=40000000 width=1014)
+ default@customer_address,a,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_60]
+ PartitionCols:_col8
+ Merge Join Operator [MERGEJOIN_109] (rows=633595212 width=88)
+ Conds:RS_57._col0=RS_58._col1(Inner),Output:["_col1","_col3","_col4","_col6","_col8"]
+ <-Map 10 [SIMPLE_EDGE]
+ SHUFFLE [RS_58]
+ PartitionCols:_col1
+ Select Operator [SEL_12] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_102] (rows=575995635 width=88)
+ predicate:(ss_customer_sk is not null and ss_sold_date_sk is not null and ss_item_sk is not null)
+ TableScan [TS_10] (rows=575995635 width=88)
+ default@store_sales,s,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_57]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_128] (rows=88000001 width=860)
- Conds:RS_25._col1=RS_26._col0(Inner),Output:["_col0","_col3"]
- <-Map 12 [SIMPLE_EDGE]
- SHUFFLE [RS_25]
- PartitionCols:_col1
- Select Operator [SEL_21] (rows=80000000 width=860)
- Output:["_col0","_col1"]
- Filter Operator [FIL_120] (rows=80000000 width=860)
- predicate:(c_current_addr_sk is not null and c_customer_sk is not null)
- TableScan [TS_19] (rows=80000000 width=860)
- default@customer,c,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_addr_sk"]
- <-Map 14 [SIMPLE_EDGE]
- SHUFFLE [RS_26]
- PartitionCols:_col0
- Select Operator [SEL_24] (rows=40000000 width=1014)
- Output:["_col0","_col1"]
- Filter Operator [FIL_121] (rows=40000000 width=1014)
- predicate:ca_address_sk is not null
- TableScan [TS_22] (rows=40000000 width=1014)
- default@customer_address,a,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_66]
- PartitionCols:_col7
- Merge Join Operator [MERGEJOIN_127] (rows=633595212 width=88)
- Conds:RS_63._col0=RS_64._col1(Inner),Output:["_col1","_col2","_col5","_col7"]
- <-Map 11 [SIMPLE_EDGE]
- SHUFFLE [RS_64]
- PartitionCols:_col1
- Select Operator [SEL_18] (rows=575995635 width=88)
+ Merge Join Operator [MERGEJOIN_108] (rows=508200 width=1436)
+ Conds:RS_54._col2=RS_55._col2(Left Outer),Output:["_col0","_col1","_col3","_col4"]
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_54]
+ PartitionCols:_col2
+ Select Operator [SEL_2] (rows=462000 width=1436)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_119] (rows=575995635 width=88)
- predicate:(ss_customer_sk is not null and ss_sold_date_sk is not null and ss_item_sk is not null)
- TableScan [TS_16] (rows=575995635 width=88)
- default@store_sales,s,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_63]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_126] (rows=508200 width=1436)
- Conds:RS_60._col2=RS_61._col0(Left Outer),Output:["_col0","_col1","_col2"]
- <-Map 1 [SIMPLE_EDGE]
- SHUFFLE [RS_60]
- PartitionCols:_col2
- Select Operator [SEL_2] (rows=462000 width=1436)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_117] (rows=462000 width=1436)
- predicate:i_item_sk is not null
- TableScan [TS_0] (rows=462000 width=1436)
- default@item,i,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_current_price","i_category"]
- <-Reducer 10 [SIMPLE_EDGE]
- SHUFFLE [RS_61]
- PartitionCols:_col0
- Select Operator [SEL_15] (rows=38500 width=1436)
- Output:["_col0"]
- Filter Operator [FIL_14] (rows=38500 width=1436)
- predicate:(sq_count_check(_col1) <= 1)
- Group By Operator [GBY_13] (rows=115500 width=1436)
- Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
- Group By Operator [GBY_8] (rows=231000 width=1436)
- Output:["_col0"],keys:KEY._col0
- <-Map 9 [SIMPLE_EDGE]
- SHUFFLE [RS_7]
- PartitionCols:_col0
- Group By Operator [GBY_6] (rows=462000 width=1436)
- Output:["_col0"],keys:i_category
- Filter Operator [FIL_118] (rows=462000 width=1436)
- predicate:i_category is not null
- TableScan [TS_3] (rows=462000 width=1436)
- default@item,j,Tbl:COMPLETE,Col:NONE,Output:["i_category"]
+ Filter Operator [FIL_100] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ TableScan [TS_0] (rows=462000 width=1436)
+ default@item,i,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_current_price","i_category"]
+ <-Reducer 9 [SIMPLE_EDGE]
+ SHUFFLE [RS_55]
+ PartitionCols:_col2
+ Select Operator [SEL_9] (rows=231000 width=1436)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_8] (rows=231000 width=1436)
+ Output:["_col0","_col1"],aggregations:["avg(VALUE._col0)"],keys:KEY._col0
+ <-Map 8 [SIMPLE_EDGE]
+ SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Group By Operator [GBY_6] (rows=462000 width=1436)
+ Output:["_col0","_col1"],aggregations:["avg(i_current_price)"],keys:i_category
+ Filter Operator [FIL_101] (rows=462000 width=1436)
+ predicate:i_category is not null
+ TableScan [TS_3] (rows=462000 width=1436)
+ default@item,j,Tbl:COMPLETE,Col:NONE,Output:["i_current_price","i_category"]
[5/6] hive git commit: HIVE-16330 : Improve plans for scalar subquery
with aggregates (Vineet Garg via Ashutosh Chauhan)
Posted by ha...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/a113ede9/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out b/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out
index b2b5458..c8524df 100644
--- a/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out
+++ b/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out
@@ -75,7 +75,7 @@ POSTHOOK: Lineage: part_null.p_partkey EXPRESSION [(values__tmp__table__2)values
POSTHOOK: Lineage: part_null.p_retailprice EXPRESSION [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col8, type:string, comment:), ]
POSTHOOK: Lineage: part_null.p_size EXPRESSION [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col6, type:string, comment:), ]
POSTHOOK: Lineage: part_null.p_type SIMPLE [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col5, type:string, comment:), ]
-Warning: Shuffle Join MERGEJOIN[29][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[14][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select * from part where p_size > (select avg(p_size) from part_null)
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from part where p_size > (select avg(p_size) from part_null)
@@ -89,9 +89,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE), Reducer 6 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE)
- Reducer 6 <- Map 5 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -115,26 +114,6 @@ STAGE PLANS:
alias: part_null
Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
- Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: part_null
- Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Select Operator
expressions: p_size (type: int)
outputColumnNames: p_size
Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
@@ -155,23 +134,21 @@ STAGE PLANS:
Merge Join Operator
condition map:
Inner Join 0 to 1
- Inner Join 0 to 2
keys:
0
1
- 2
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
- Statistics: Num rows: 26 Data size: 18304 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
+ Statistics: Num rows: 26 Data size: 18096 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (UDFToDouble(_col5) > _col10) (type: boolean)
- Statistics: Num rows: 8 Data size: 5632 Basic stats: COMPLETE Column stats: NONE
+ predicate: (UDFToDouble(_col5) > _col9) (type: boolean)
+ Statistics: Num rows: 8 Data size: 5568 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 8 Data size: 5632 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 8 Data size: 5568 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 8 Data size: 5632 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 8 Data size: 5568 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -180,29 +157,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: avg(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
@@ -218,7 +172,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[29][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[14][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select * from part where p_size > (select avg(p_size) from part_null)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -698,7 +652,7 @@ POSTHOOK: Input: default@tnull
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
86428 almond aquamarine burnished black steel Manufacturer#1 Brand#12 STANDARD ANODIZED STEEL 28 WRAP BAG 1414.42 arefully
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
-Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[15][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select * from part where (select max(p_name) from part_null) is not null
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from part where (select max(p_name) from part_null) is not null
@@ -712,9 +666,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE), Reducer 6 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE)
- Reducer 6 <- Map 5 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -736,26 +689,6 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part_null
- Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
- Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: part_null
Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: p_name (type: string)
@@ -778,16 +711,14 @@ STAGE PLANS:
Merge Join Operator
condition map:
Inner Join 0 to 1
- Inner Join 0 to 2
keys:
0
1
- 2
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 26 Data size: 21112 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 20904 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 21112 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 20904 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -796,29 +727,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
@@ -838,7 +746,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[15][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select * from part where (select max(p_name) from part_null) is not null
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -875,10 +783,8 @@ POSTHOOK: Input: default@part_null
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
86428 almond aquamarine burnished black steel Manufacturer#1 Brand#12 STANDARD ANODIZED STEEL 28 WRAP BAG 1414.42 arefully
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
-Warning: Shuffle Join MERGEJOIN[57][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[58][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
-Warning: Shuffle Join MERGEJOIN[59][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
-Warning: Shuffle Join MERGEJOIN[60][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 5' is a cross product
+Warning: Shuffle Join MERGEJOIN[23][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[24][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where p_size between (select min(p_size) from part) and (select avg(p_size) from part)
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from part where p_size between (select min(p_size) from part) and (select avg(p_size) from part)
@@ -892,14 +798,10 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 11 <- Map 10 (CUSTOM_SIMPLE_EDGE)
- Reducer 13 <- Map 12 (CUSTOM_SIMPLE_EDGE)
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 9 (CUSTOM_SIMPLE_EDGE)
- Reducer 4 <- Reducer 11 (CUSTOM_SIMPLE_EDGE), Reducer 3 (CUSTOM_SIMPLE_EDGE)
- Reducer 5 <- Reducer 13 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
+ Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE)
Reducer 7 <- Map 6 (CUSTOM_SIMPLE_EDGE)
- Reducer 9 <- Map 8 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -917,27 +819,27 @@ STAGE PLANS:
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Execution mode: llap
LLAP IO: no inputs
- Map 10
+ Map 4
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
+ expressions: p_size (type: int)
+ outputColumnNames: p_size
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
- aggregations: count(p_partkey)
+ aggregations: min(p_size)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Map 12
+ Map 6
Map Operator Tree:
TableScan
alias: part
@@ -957,127 +859,22 @@ STAGE PLANS:
value expressions: _col0 (type: struct<count:bigint,sum:double,input:int>)
Execution mode: llap
LLAP IO: no inputs
- Map 6
- Map Operator Tree:
- TableScan
- alias: part
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 8
- Map Operator Tree:
- TableScan
- alias: part
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: p_size (type: int)
- outputColumnNames: p_size
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: min(p_size)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: int)
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 11
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 13
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: avg(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: double)
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
Left Outer Join0 to 1
keys:
0
1
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
- Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col10 (type: int)
- Reducer 4
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col10 (type: int)
- Reducer 5
+ value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col9 (type: int)
+ Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
@@ -1086,10 +883,10 @@ STAGE PLANS:
keys:
0
1
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col12
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10
Statistics: Num rows: 26 Data size: 16406 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: UDFToDouble(_col5) BETWEEN UDFToDouble(_col10) AND _col12 (type: boolean)
+ predicate: UDFToDouble(_col5) BETWEEN UDFToDouble(_col9) AND _col10 (type: boolean)
Statistics: Num rows: 2 Data size: 1262 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
@@ -1102,41 +899,30 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 7
+ Reducer 5
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
+ aggregations: min(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 9
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int)
+ Reducer 7
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: min(VALUE._col0)
+ aggregations: avg(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: int)
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: double)
Stage: Stage-0
Fetch Operator
@@ -1144,10 +930,8 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[57][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[58][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
-Warning: Shuffle Join MERGEJOIN[59][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
-Warning: Shuffle Join MERGEJOIN[60][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 5' is a cross product
+Warning: Shuffle Join MERGEJOIN[23][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[24][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where p_size between (select min(p_size) from part) and (select avg(p_size) from part)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -1426,7 +1210,7 @@ Manufacturer#5 almond antique medium spring khaki 6
Manufacturer#5 almond antique sky peru orange 2
Manufacturer#5 almond aquamarine dodger light gainsboro 46
Manufacturer#5 almond azure blanched chiffon midnight 23
-Warning: Shuffle Join MERGEJOIN[29][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[14][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select * from part where (p_partkey*p_size) <> (select min(p_partkey) from part)
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from part where (p_partkey*p_size) <> (select min(p_partkey) from part)
@@ -1440,9 +1224,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE), Reducer 6 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE)
- Reducer 6 <- Map 5 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1470,26 +1253,6 @@ STAGE PLANS:
outputColumnNames: p_partkey
Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: part
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
aggregations: min(p_partkey)
mode: hash
outputColumnNames: _col0
@@ -1506,15 +1269,13 @@ STAGE PLANS:
Merge Join Operator
condition map:
Inner Join 0 to 1
- Inner Join 0 to 2
keys:
0
1
- 2
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: ((_col0 * _col5) <> _col10) (type: boolean)
+ predicate: ((_col0 * _col5) <> _col9) (type: boolean)
Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
@@ -1531,29 +1292,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: min(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
@@ -1569,7 +1307,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[29][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[14][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select * from part where (p_partkey*p_size) <> (select min(p_partkey) from part)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -1617,10 +1355,9 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE)
Reducer 5 <- Map 4 (SIMPLE_EDGE)
- Reducer 7 <- Map 6 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1644,26 +1381,6 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3146 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: p_name is not null (type: boolean)
- Statistics: Num rows: 26 Data size: 3146 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: p_name (type: string)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: no inputs
- Map 6
- Map Operator Tree:
- TableScan
- alias: part
Statistics: Num rows: 26 Data size: 3250 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_name is not null (type: boolean)
@@ -1688,18 +1405,16 @@ STAGE PLANS:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col0 (type: string)
- 1 _col0 (type: string)
- 2 _col2 (type: string)
- outputColumnNames: _col1, _col4, _col5
- Statistics: Num rows: 2 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
+ 1 _col2 (type: string)
+ outputColumnNames: _col1, _col2, _col3
+ Statistics: Num rows: 26 Data size: 312 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: ((_col1 + 100) < CASE WHEN (_col5 is null) THEN (null) ELSE (_col4) END) (type: boolean)
- Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ predicate: ((_col1 + 100) < CASE WHEN (_col3 is null) THEN (null) ELSE (_col2) END) (type: boolean)
+ Statistics: Num rows: 8 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 8 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
@@ -1728,32 +1443,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- keys: KEY._col0 (type: string)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- keys: _col0 (type: string)
- mode: complete
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1677 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col1) <= 1) (type: boolean)
- Statistics: Num rows: 4 Data size: 516 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 4 Data size: 516 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 4 Data size: 516 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 7
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: max(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
@@ -1798,9 +1487,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
- Reducer 6 <- Map 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1824,26 +1512,6 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part_null
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: p_type is not null (type: boolean)
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: p_type (type: string)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: part_null
Statistics: Num rows: 31 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: p_type is not null (type: boolean)
@@ -1868,23 +1536,21 @@ STAGE PLANS:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col4 (type: string)
- 1 _col0 (type: string)
- 2 _col2 (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col11, _col12
- Statistics: Num rows: 57 Data size: 35406 Basic stats: COMPLETE Column stats: NONE
+ 1 _col2 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10
+ Statistics: Num rows: 28 Data size: 17703 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (UDFToDouble(_col5) > CASE WHEN (_col12 is null) THEN (null) ELSE (_col11) END) (type: boolean)
- Statistics: Num rows: 19 Data size: 11802 Basic stats: COMPLETE Column stats: NONE
+ predicate: (UDFToDouble(_col5) > CASE WHEN (_col10 is null) THEN (null) ELSE (_col9) END) (type: boolean)
+ Statistics: Num rows: 9 Data size: 5690 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 19 Data size: 11802 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 5690 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 19 Data size: 11802 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 5690 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1893,32 +1559,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- keys: KEY._col0 (type: string)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 16 Data size: 1628 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- keys: _col0 (type: string)
- mode: complete
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 8 Data size: 814 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (sq_count_check(_col1) <= 1) (type: boolean)
- Statistics: Num rows: 2 Data size: 203 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 203 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 203 Basic stats: COMPLETE Column stats: NONE
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: avg(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
@@ -1952,8 +1592,7 @@ POSTHOOK: Input: default@part
POSTHOOK: Input: default@part_null
#### A masked pattern was here ####
192697 almond antique blue firebrick mint Manufacturer#5 Brand#52 MEDIUM BURNISHED TIN 31 LG DRUM 1789.69 ickly ir
-Warning: Shuffle Join MERGEJOIN[60][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 3' is a cross product
-Warning: Shuffle Join MERGEJOIN[61][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[27][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where p_size BETWEEN (select min(p_size) from part_null where part_null.p_type = part.p_type) AND (select max(p_size) from part_null)
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from part where p_size BETWEEN (select min(p_size) from part_null where part_null.p_type = part.p_type) AND (select max(p_size) from part_null)
@@ -1967,13 +1606,10 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 10 <- Map 9 (CUSTOM_SIMPLE_EDGE)
- Reducer 12 <- Map 11 (CUSTOM_SIMPLE_EDGE)
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 10 (CUSTOM_SIMPLE_EDGE), Reducer 2 (CUSTOM_SIMPLE_EDGE)
- Reducer 4 <- Reducer 12 (CUSTOM_SIMPLE_EDGE), Reducer 3 (CUSTOM_SIMPLE_EDGE)
- Reducer 6 <- Map 5 (SIMPLE_EDGE)
- Reducer 8 <- Map 7 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
+ Reducer 5 <- Map 4 (SIMPLE_EDGE)
+ Reducer 7 <- Map 6 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1993,47 +1629,7 @@ STAGE PLANS:
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Execution mode: llap
LLAP IO: no inputs
- Map 11
- Map Operator Tree:
- TableScan
- alias: part_null
- Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: p_size (type: int)
- outputColumnNames: p_size
- Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: max(p_size)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: int)
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: part_null
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: p_type is not null (type: boolean)
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: p_type (type: string)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 7
+ Map 4
Map Operator Tree:
TableScan
alias: part_null
@@ -2055,145 +1651,67 @@ STAGE PLANS:
value expressions: _col1 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Map 9
+ Map 6
Map Operator Tree:
TableScan
alias: part_null
Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
+ expressions: p_size (type: int)
+ outputColumnNames: p_size
Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
Group By Operator
- aggregations: count(p_partkey)
+ aggregations: max(p_size)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Reducer 10
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reducer 12
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: max(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: int)
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col4 (type: string)
- 1 _col0 (type: string)
- 2 _col2 (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col11, _col12
- Statistics: Num rows: 57 Data size: 35406 Basic stats: COMPLETE Column stats: NONE
+ 1 _col2 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10
+ Statistics: Num rows: 28 Data size: 17703 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 57 Data size: 35406 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col11 (type: int), _col12 (type: boolean)
+ Statistics: Num rows: 28 Data size: 17703 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col9 (type: int), _col10 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col11, _col12
- Statistics: Num rows: 57 Data size: 35919 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 57 Data size: 35919 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col11 (type: int), _col12 (type: boolean)
- Reducer 4
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
Left Outer Join0 to 1
keys:
0
1
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col11, _col12, _col15
- Statistics: Num rows: 57 Data size: 36204 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col12
+ Statistics: Num rows: 28 Data size: 17843 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: _col5 BETWEEN CASE WHEN (_col12 is null) THEN (null) ELSE (_col11) END AND _col15 (type: boolean)
- Statistics: Num rows: 6 Data size: 3810 Basic stats: COMPLETE Column stats: NONE
+ predicate: _col5 BETWEEN CASE WHEN (_col10 is null) THEN (null) ELSE (_col9) END AND _col12 (type: boolean)
+ Statistics: Num rows: 3 Data size: 1911 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 6 Data size: 3810 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 1911 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 6 Data size: 3810 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 1911 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 6
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- keys: KEY._col0 (type: string)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 16 Data size: 1628 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- keys: _col0 (type: string)
- mode: complete
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 8 Data size: 814 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (sq_count_check(_col1) <= 1) (type: boolean)
- Statistics: Num rows: 2 Data size: 203 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 203 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 203 Basic stats: COMPLETE Column stats: NONE
- Reducer 8
+ Reducer 5
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -2212,6 +1730,18 @@ STAGE PLANS:
Map-reduce partition columns: _col2 (type: string)
Statistics: Num rows: 15 Data size: 1575 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int), _col1 (type: boolean)
+ Reducer 7
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: max(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: int)
Stage: Stage-0
Fetch Operator
@@ -2219,8 +1749,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[60][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 3' is a cross product
-Warning: Shuffle Join MERGEJOIN[61][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[27][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where p_size BETWEEN (select min(p_size) from part_null where part_null.p_type = part.p_type) AND (select max(p_size) from part_null)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -2257,7 +1786,7 @@ POSTHOOK: Input: default@part_null
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
86428 almond aquamarine burnished black steel Manufacturer#1 Brand#12 STANDARD ANODIZED STEEL 28 WRAP BAG 1414.42 arefully
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
-Warning: Shuffle Join MERGEJOIN[60][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[29][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where p_size >= (select min(p_size) from part_null where part_null.p_type = part.p_type) AND p_retailprice <= (select max(p_retailprice) from part_null)
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from part where p_size >= (select min(p_size) from part_null where part_null.p_type = part.p_type) AND p_retailprice <= (select max(p_retailprice) from part_null)
@@ -2271,12 +1800,10 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 11 <- Map 10 (CUSTOM_SIMPLE_EDGE)
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 11 (CUSTOM_SIMPLE_EDGE), Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 9 (CUSTOM_SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
Reducer 5 <- Map 4 (SIMPLE_EDGE)
- Reducer 7 <- Map 6 (SIMPLE_EDGE)
- Reducer 9 <- Map 8 (CUSTOM_SIMPLE_EDGE)
+ Reducer 7 <- Map 6 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -2296,50 +1823,10 @@ STAGE PLANS:
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Execution mode: llap
LLAP IO: no inputs
- Map 10
- Map Operator Tree:
- TableScan
- alias: part_null
- Statistics: Num rows: 407 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: p_retailprice (type: double)
- outputColumnNames: p_retailprice
- Statistics: Num rows: 407 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: max(p_retailprice)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: double)
- Execution mode: llap
- LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: part_null
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: p_type is not null (type: boolean)
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: p_type (type: string)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 32 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 6
- Map Operator Tree:
- TableScan
- alias: part_null
Statistics: Num rows: 31 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: p_type is not null (type: boolean)
@@ -2358,61 +1845,47 @@ STAGE PLANS:
value expressions: _col1 (type: int)
Execution mode: llap
LLAP IO: no inputs
- Map 8
+ Map 6
Map Operator Tree:
TableScan
alias: part_null
- Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 407 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
- Statistics: Num rows: 814 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
+ expressions: p_retailprice (type: double)
+ outputColumnNames: p_retailprice
+ Statistics: Num rows: 407 Data size: 3256 Basic stats: COMPLETE Column stats: NONE
Group By Operator
- aggregations: count(p_partkey)
+ aggregations: max(p_retailprice)
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ value expressions: _col0 (type: double)
Execution mode: llap
LLAP IO: no inputs
- Reducer 11
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: max(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: double)
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col4 (type: string)
- 1 _col0 (type: string)
- 2 _col2 (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col11, _col12
- Statistics: Num rows: 57 Data size: 35406 Basic stats: COMPLETE Column stats: NONE
+ 1 _col2 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10
+ Statistics: Num rows: 28 Data size: 17703 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (_col5 >= CASE WHEN (_col12 is null) THEN (null) ELSE (_col11) END) (type: boolean)
- Statistics: Num rows: 19 Data size: 11802 Basic stats: COMPLETE Column stats: NONE
+ predicate: (_col5 >= CASE WHEN (_col10 is null) THEN (null) ELSE (_col9) END) (type: boolean)
+ Statistics: Num rows: 9 Data size: 5690 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 19 Data size: 11802 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 5690 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 19 Data size: 11802 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 5690 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Reducer 3
Execution mode: llap
@@ -2420,23 +1893,21 @@ STAGE PLANS:
Merge Join Operator
condition map:
Inner Join 0 to 1
- Inner Join 0 to 2
keys:
0
1
- 2
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col15
- Statistics: Num rows: 19 Data size: 12125 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12
+ Statistics: Num rows: 9 Data size: 5771 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (_col7 <= _col15) (type: boolean)
- Statistics: Num rows: 6 Data size: 3828 Basic stats: COMPLETE Column stats: NONE
+ predicate: (_col7 <= _col12) (type: boolean)
+ Statistics: Num rows: 3 Data size: 1923 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 6 Data size: 3828 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 1923 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 6 Data size: 3828 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 1923 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -2445,32 +1916,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- keys: KEY._col0 (type: string)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 16 Data size: 1628 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- keys: _col0 (type: string)
- mode: complete
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 8 Data size: 814 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (sq_count_check(_col1) <= 1) (type: boolean)
- Statistics: Num rows: 2 Data size: 203 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 203 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 203 Basic stats: COMPLETE Column stats: NONE
- Reducer 7
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: min(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
@@ -2486,29 +1931,18 @@ STAGE PLANS:
Map-reduce partition columns: _col2 (type: string)
Statistics: Num rows: 15 Data size: 1575 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int), _col1 (type: boolean)
- Reducer 9
+ Reducer 7
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
+ aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
+ Reduce Output Operator
+ sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: double)
Stage: Stage-0
Fetch Operator
@@ -2516,7 +1950,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[60][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[29][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where p_size >= (select min(p_size) from part_null where part_null.p_type = part.p_type) AND p_retailprice <= (select max(p_retailprice) from part_null)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -2553,7 +1987,7 @@ POSTHOOK: Input: default@part_null
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
86428 almond aquamarine burnished black steel Manufacturer#1 Brand#12 STANDARD ANODIZED STEEL 28 WRAP BAG 1414.42 arefully
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
-Warning: Shuffle Join MERGEJOIN[46][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select * from part where p_brand <> (select min(p_brand) from part ) AND p_size IN (select (p_size) from part p where p.p_type = part.p_type ) AND p_size <> 340
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from part where p_brand <> (select min(p_brand) from part ) AND p_size IN (select (p_size) from part p where p.p_type = part.p_type ) AND p_size <> 340
@@ -2567,11 +2001,10 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE), Reducer 7 (CUSTOM_SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 9 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE)
- Reducer 7 <- Map 6 (CUSTOM_SIMPLE_EDGE)
- Reducer 9 <- Map 8 (SIMPLE_EDGE)
+ Reducer 7 <- Map 6 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -2596,26 +2029,6 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: p_partkey (type: int)
- outputColumnNames: p_partkey
- Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count(p_partkey)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: bigint)
- Execution mode: llap
- LLAP IO: no inputs
- Map 6
- Map Operator Tree:
- TableScan
- alias: part
Statistics: Num rows: 26 Data size: 2392 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_brand (type: string)
@@ -2632,7 +2045,7 @@ STAGE PLANS:
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
- Map 8
+ Map 6
Map Operator Tree:
TableScan
alias: p
@@ -2658,15 +2071,13 @@ STAGE PLANS:
Merge Join Operator
condition map:
Inner Join 0 to 1
- Inner Join 0 to 2
keys:
0
1
- 2
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
Statistics: Num rows: 26 Data size: 20878 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: (_col3 <> _col10) (type: boolean)
+ predicate: (_col3 <> _col9) (type: boolean)
Statistics: Num rows: 26 Data size: 20878 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col4 (type: string), _col5 (type: int)
@@ -2696,29 +2107,6 @@ STAGE PLANS:
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
- mode: mergepartial
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- mode: complete
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: (sq_count_check(_col0) <= 1) (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 7
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
aggregations: min(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
@@ -2727,7 +2115,7 @@ STAGE PLANS:
sort order:
Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
- Reducer 9
+ Reducer 7
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -2751,7 +2139,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[46][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select * from part where p_brand <> (select min(p_brand) from part ) AND p_size IN (select (p_size) from part p where p.p_type = part.p_type ) AND p_size <> 340
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -2796,9 +2184,8 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
- Reducer 6 <- Map 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -2822,26 +2209,6 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p
- Statistics: Num rows: 26 Data size: 208 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: ((p_size = p_size) and (p_partkey = p_partkey)) (type: boolean)
- Statistics: Num rows: 6 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: p_partkey (type: int), p_size (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int), _col1 (type: int)
- sort order: ++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
- Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: no inputs
- Map 5
- Map Operator Tree:
- TableScan
- alias: p
Statistics: Num rows: 26 Data size: 3354 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((p_size = p_size) and (p_partkey = p_partkey)) (type: boolean)
@@ -2866,54 +2233,26 @@ STAGE PLANS:
Merge Join Operator
condition map:
Left Outer Join0 to 1
- Left Outer Join0 to 2
keys:
0 _col0 (type: int), _col5 (type: int)
- 1 _col0 (type: int), _col1 (type: int)
- 2 _col2 (type: int), _col3 (type: int)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13
- Statistics: Num rows: 1 Data size: 631 Basic stats: COMPLETE Column stats: COMPLETE
+ 1 _col2 (type: int), _col3 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10
+ Statistics: Num rows: 26 Data size: 16406 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: (UDFToLong(_col5) <> CASE WHEN (_col13 is null) THEN (0) ELSE (_col12) END) (type: boolean)
- Statistics: Num rows: 1 Data size: 631 Basic stats: COMPLETE Column stats: COMPLETE
+ predicate: (UDFToLong(_col5) <> CASE WHEN (_col10 is null) THEN (0) ELSE (_col9) END) (type: boolean)
+ Statistics: Num rows: 26 Data size: 16406 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 1 Data size: 619 Basic stats: COMPLETE Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 619 Basic stats: COMPLETE Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 4
- Execution mode: llap
- Reduce Operator
<TRUNCATED>