You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by vi...@apache.org on 2014/01/30 21:46:19 UTC
svn commit: r1562955 [13/14] - in /hive/trunk:
common/src/java/org/apache/hadoop/hive/conf/ conf/
itests/util/src/main/java/org/apache/hadoop/hive/ql/hooks/
ql/src/java/org/apache/hadoop/hive/ql/exec/
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ ql/s...
Modified: hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out Thu Jan 30 20:46:16 2014
@@ -18,46 +18,10 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_UNION (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) x1) (TOK_TABREF (TOK_TABNAME src1) y1) (= (. (TOK_TABLE_OR_COL x1) key) (. (TOK_TABLE_OR_COL y1) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x1) key) key)))) (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) x2) (TOK_TABREF (TOK_TABNAME src1) y2) (= (. (TOK_TABLE_OR_COL x2) key) (. (TOK_TABLE_OR_COL y2) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x2) key) key))))) tmp)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp) key))) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (. (TOK_TABLE_OR_COL tmp) key)))))
STAGE DEPENDENCIES:
- Stage-8 is a root stage
- Stage-2 depends on stages: Stage-8
+ Stage-2 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-8
- Map Reduce Local Work
- Alias -> Map Local Tables:
- null-subquery1:tmp-subquery1:y1
- Fetch Operator
- limit: -1
- null-subquery2:tmp-subquery2:y2
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- null-subquery1:tmp-subquery1:y1
- TableScan
- alias: y1
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
- null-subquery2:tmp-subquery2:y2
- TableScan
- alias: y2
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-2
Map Reduce
Alias -> Map Operator Tree:
@@ -133,6 +97,20 @@ STAGE PLANS:
type: string
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ null-subquery1:tmp-subquery1:y1
+ Fetch Operator
+ limit: -1
+ null-subquery2:tmp-subquery2:y2
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ null-subquery1:tmp-subquery1:y1
+ TableScan
+ alias: y1
+ null-subquery2:tmp-subquery2:y2
+ TableScan
+ alias: y2
Reduce Operator Tree:
Extract
File Output Operator
@@ -271,33 +249,11 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_UNION (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) x1) (TOK_TABREF (TOK_TABNAME src1) y1) (= (. (TOK_TABLE_OR_COL x1) key) (. (TOK_TABLE_OR_COL y1) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x1) key) key)))) (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) x2) (TOK_TABREF (TOK_TABNAME src1) y2) (= (. (TOK_TABLE_OR_COL x2) key) (. (TOK_TABLE_OR_COL y2) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x2) key) key))))) tmp)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp) key))) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (. (TOK_TABLE_OR_COL tmp) key)))))
STAGE DEPENDENCIES:
- Stage-9 is a root stage
- Stage-7 depends on stages: Stage-9
- Stage-8 depends on stages: Stage-7
- Stage-2 depends on stages: Stage-8
+ Stage-7 is a root stage
+ Stage-2 depends on stages: Stage-7
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-9
- Map Reduce Local Work
- Alias -> Map Local Tables:
- null-subquery1:tmp-subquery1:y1
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- null-subquery1:tmp-subquery1:y1
- TableScan
- alias: y1
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-7
Map Reduce
Alias -> Map Operator Tree:
@@ -330,26 +286,14 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
-
- Stage: Stage-8
- Map Reduce Local Work
- Alias -> Map Local Tables:
- null-subquery2:tmp-subquery2:y2
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- null-subquery2:tmp-subquery2:y2
- TableScan
- alias: y2
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
+ Alias -> Map Local Tables:
+ null-subquery1:tmp-subquery1:y1
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ null-subquery1:tmp-subquery1:y1
+ TableScan
+ alias: y1
Stage: Stage-2
Map Reduce
@@ -408,6 +352,14 @@ STAGE PLANS:
type: string
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ null-subquery2:tmp-subquery2:y2
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ null-subquery2:tmp-subquery2:y2
+ TableScan
+ alias: y2
Reduce Operator Tree:
Extract
File Output Operator
@@ -541,8 +493,7 @@ ABSTRACT SYNTAX TREE:
STAGE DEPENDENCIES:
Stage-4 is a root stage
- Stage-6 depends on stages: Stage-4
- Stage-2 depends on stages: Stage-6
+ Stage-2 depends on stages: Stage-4
Stage-0 is a root stage
STAGE PLANS:
@@ -594,26 +545,6 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-6
- Map Reduce Local Work
- Alias -> Map Local Tables:
- null-subquery2:tmp-subquery2:y2
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- null-subquery2:tmp-subquery2:y2
- TableScan
- alias: y2
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-2
Map Reduce
Alias -> Map Operator Tree:
@@ -671,6 +602,14 @@ STAGE PLANS:
type: string
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ null-subquery2:tmp-subquery2:y2
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ null-subquery2:tmp-subquery2:y2
+ TableScan
+ alias: y2
Reduce Operator Tree:
Extract
File Output Operator
@@ -798,41 +737,17 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) x1) (TOK_TABREF (TOK_TABNAME src1) y1) (= (. (TOK_TABLE_OR_COL x1) key) (. (TOK_TABLE_OR_COL y1) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x1) key) key)) (TOK_GROUPBY (. (TOK_TABLE_OR_COL x1) key)))) tmp1) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) x2) (TOK_TABREF (TOK_TABNAME src1) y2) (= (. (TOK_TABLE_OR_COL x2) key) (. (TOK_TABLE_OR_COL y2) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x2) key) key)) (TOK_GROUPBY (. (TOK_TABLE_OR_COL x2) key)))) tmp2) (= (. (TOK_TABLE_OR_COL tmp1) key) (. (TOK_TABLE_OR_COL tmp2) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp1) key) key) (TOK_SELEXPR (TOK_FUNCTIONSTAR count) cnt)) (TOK_GROUPBY (. (TOK_TABLE_OR_COL tmp1) key
)) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL cnt)))))
STAGE DEPENDENCIES:
- Stage-17 is a root stage
- Stage-2 depends on stages: Stage-17
- Stage-12 depends on stages: Stage-2, Stage-8 , consists of Stage-15, Stage-16, Stage-3
- Stage-15 has a backup stage: Stage-3
- Stage-10 depends on stages: Stage-15
+ Stage-2 is a root stage
+ Stage-12 depends on stages: Stage-2, Stage-8 , consists of Stage-10, Stage-11, Stage-3
+ Stage-10 has a backup stage: Stage-3
Stage-4 depends on stages: Stage-3, Stage-10, Stage-11
Stage-5 depends on stages: Stage-4
- Stage-16 has a backup stage: Stage-3
- Stage-11 depends on stages: Stage-16
+ Stage-11 has a backup stage: Stage-3
Stage-3
- Stage-18 is a root stage
- Stage-8 depends on stages: Stage-18
+ Stage-8 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-17
- Map Reduce Local Work
- Alias -> Map Local Tables:
- tmp2:y2
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- tmp2:y2
- TableScan
- alias: y2
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-2
Map Reduce
Alias -> Map Operator Tree:
@@ -874,6 +789,14 @@ STAGE PLANS:
tag: -1
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ tmp2:y2
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ tmp2:y2
+ TableScan
+ alias: y2
Reduce Operator Tree:
Group By Operator
bucketGroup: false
@@ -898,25 +821,6 @@ STAGE PLANS:
Stage: Stage-12
Conditional Operator
- Stage: Stage-15
- Map Reduce Local Work
- Alias -> Map Local Tables:
- $INTNAME
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- $INTNAME
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {_col0}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[_col0]]
- 1 [Column[_col0]]
- Position of Big Table: 0
-
Stage: Stage-10
Map Reduce
Alias -> Map Operator Tree:
@@ -957,6 +861,13 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ $INTNAME
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ $INTNAME
+ TableScan
Stage: Stage-4
Map Reduce
@@ -1028,25 +939,6 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Stage: Stage-16
- Map Reduce Local Work
- Alias -> Map Local Tables:
- $INTNAME1
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- $INTNAME1
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {_col0}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[_col0]]
- 1 [Column[_col0]]
- Position of Big Table: 1
-
Stage: Stage-11
Map Reduce
Alias -> Map Operator Tree:
@@ -1087,6 +979,13 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ $INTNAME1
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ $INTNAME1
+ TableScan
Stage: Stage-3
Map Reduce
@@ -1147,26 +1046,6 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-18
- Map Reduce Local Work
- Alias -> Map Local Tables:
- tmp1:y1
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- tmp1:y1
- TableScan
- alias: y1
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-8
Map Reduce
Alias -> Map Operator Tree:
@@ -1208,6 +1087,14 @@ STAGE PLANS:
tag: -1
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ tmp1:y1
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ tmp1:y1
+ TableScan
+ alias: y1
Reduce Operator Tree:
Group By Operator
bucketGroup: false
@@ -1312,47 +1199,11 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) x1) (TOK_TABREF (TOK_TABNAME src1) y1) (= (. (TOK_TABLE_OR_COL x1) key) (. (TOK_TABLE_OR_COL y1) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x1) key) key)) (TOK_GROUPBY (. (TOK_TABLE_OR_COL x1) key)))) tmp1) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) x2) (TOK_TABREF (TOK_TABNAME src1) y2) (= (. (TOK_TABLE_OR_COL x2) key) (. (TOK_TABLE_OR_COL y2) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x2) key) key)) (TOK_GROUPBY (. (TOK_TABLE_OR_COL x2) key)))) tmp2) (= (. (TOK_TABLE_OR_COL tmp1) key) (. (TOK_TABLE_OR_COL tmp2) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp1) key) key) (TOK_SELEXPR (TOK_FUNCTIONSTAR count) cnt)) (TOK_GROUPBY (. (TOK_TABLE_OR_COL tmp1) key
)) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL cnt)))))
STAGE DEPENDENCIES:
- Stage-9 is a root stage
- Stage-2 depends on stages: Stage-9
+ Stage-2 is a root stage
Stage-3 depends on stages: Stage-2
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-9
- Map Reduce Local Work
- Alias -> Map Local Tables:
- tmp1:y1
- Fetch Operator
- limit: -1
- tmp2:y2
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- tmp1:y1
- TableScan
- alias: y1
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
- tmp2:y2
- TableScan
- alias: y2
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-2
Map Reduce
Alias -> Map Operator Tree:
@@ -1430,6 +1281,20 @@ STAGE PLANS:
tag: 1
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ tmp1:y1
+ Fetch Operator
+ limit: -1
+ tmp2:y2
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ tmp1:y1
+ TableScan
+ alias: y1
+ tmp2:y2
+ TableScan
+ alias: y2
Reduce Operator Tree:
Demux Operator
Group By Operator
@@ -1651,16 +1516,13 @@ ABSTRACT SYNTAX TREE:
STAGE DEPENDENCIES:
Stage-7 is a root stage
- Stage-10 depends on stages: Stage-2, Stage-7 , consists of Stage-12, Stage-13, Stage-3
- Stage-12 has a backup stage: Stage-3
- Stage-8 depends on stages: Stage-12
+ Stage-10 depends on stages: Stage-2, Stage-7 , consists of Stage-8, Stage-9, Stage-3
+ Stage-8 has a backup stage: Stage-3
Stage-4 depends on stages: Stage-3, Stage-8, Stage-9
Stage-5 depends on stages: Stage-4
- Stage-13 has a backup stage: Stage-3
- Stage-9 depends on stages: Stage-13
+ Stage-9 has a backup stage: Stage-3
Stage-3
- Stage-14 is a root stage
- Stage-2 depends on stages: Stage-14
+ Stage-2 is a root stage
Stage-0 is a root stage
STAGE PLANS:
@@ -1715,25 +1577,6 @@ STAGE PLANS:
Stage: Stage-10
Conditional Operator
- Stage: Stage-12
- Map Reduce Local Work
- Alias -> Map Local Tables:
- $INTNAME
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- $INTNAME
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {_col0}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[_col0]]
- 1 [Column[_col0]]
- Position of Big Table: 0
-
Stage: Stage-8
Map Reduce
Alias -> Map Operator Tree:
@@ -1774,6 +1617,13 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ $INTNAME
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ $INTNAME
+ TableScan
Stage: Stage-4
Map Reduce
@@ -1845,25 +1695,6 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Stage: Stage-13
- Map Reduce Local Work
- Alias -> Map Local Tables:
- $INTNAME1
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- $INTNAME1
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {_col0}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[_col0]]
- 1 [Column[_col0]]
- Position of Big Table: 1
-
Stage: Stage-9
Map Reduce
Alias -> Map Operator Tree:
@@ -1904,6 +1735,13 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ $INTNAME1
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ $INTNAME1
+ TableScan
Stage: Stage-3
Map Reduce
@@ -1964,26 +1802,6 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-14
- Map Reduce Local Work
- Alias -> Map Local Tables:
- tmp2:y2
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- tmp2:y2
- TableScan
- alias: y2
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-2
Map Reduce
Alias -> Map Operator Tree:
@@ -2025,6 +1843,14 @@ STAGE PLANS:
tag: -1
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ tmp2:y2
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ tmp2:y2
+ TableScan
+ alias: y2
Reduce Operator Tree:
Group By Operator
bucketGroup: false
@@ -2129,32 +1955,11 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src1) x1)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x1) key) key)) (TOK_GROUPBY (. (TOK_TABLE_OR_COL x1) key)))) tmp1) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) x2) (TOK_TABREF (TOK_TABNAME src1) y2) (= (. (TOK_TABLE_OR_COL x2) key) (. (TOK_TABLE_OR_COL y2) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL x2) key) key)) (TOK_GROUPBY (. (TOK_TABLE_OR_COL x2) key)))) tmp2) (= (. (TOK_TABLE_OR_COL tmp1) key) (. (TOK_TABLE_OR_COL tmp2) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp1) key) key) (TOK_SELEXPR (TOK_FUNCTIONSTAR count) cnt)) (TOK_GROUPBY (. (TOK_TABLE_OR_COL tmp1) key)) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL cnt))
)))
STAGE DEPENDENCIES:
- Stage-7 is a root stage
- Stage-2 depends on stages: Stage-7
+ Stage-2 is a root stage
Stage-3 depends on stages: Stage-2
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-7
- Map Reduce Local Work
- Alias -> Map Local Tables:
- tmp2:y2
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- tmp2:y2
- TableScan
- alias: y2
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-2
Map Reduce
Alias -> Map Operator Tree:
@@ -2220,6 +2025,14 @@ STAGE PLANS:
tag: 1
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ tmp2:y2
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ tmp2:y2
+ TableScan
+ alias: y2
Reduce Operator Tree:
Demux Operator
Group By Operator
@@ -2447,31 +2260,10 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME part_table) x) (TOK_TABREF (TOK_TABNAME src1) y) (= (. (TOK_TABLE_OR_COL x) key) (. (TOK_TABLE_OR_COL y) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTIONSTAR count)))))
STAGE DEPENDENCIES:
- Stage-5 is a root stage
- Stage-2 depends on stages: Stage-5
+ Stage-2 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-5
- Map Reduce Local Work
- Alias -> Map Local Tables:
- y
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- y
- TableScan
- alias: y
- HashTable Sink Operator
- condition expressions:
- 0
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-2
Map Reduce
Alias -> Map Operator Tree:
@@ -2504,6 +2296,14 @@ STAGE PLANS:
type: bigint
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ y
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ y
+ TableScan
+ alias: y
Reduce Operator Tree:
Group By Operator
aggregations:
@@ -2583,35 +2383,12 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_UNION (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) a) (TOK_TABREF (TOK_TABNAME src) b) (= (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL b) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) key))) (TOK_GROUPBY (. (TOK_TABLE_OR_COL a) key)))) tmp) (TOK_TABREF (TOK_TABNAME src) c) (= (. (TOK_TABLE_OR_COL tmp) key) (. (TOK_TABLE_OR_COL c) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL c) key))))) (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) a) (TOK_TABREF (TOK_TABNAME src) b) (= (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL b) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) key))) (TOK_GROUPBY (. (TOK_TABLE_OR_COL a) key)))) tmp) (TOK_TABR
EF (TOK_TABNAME src) c) (= (. (TOK_TABLE_OR_COL tmp) key) (. (TOK_TABLE_OR_COL c) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL c) key)))))) x)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF))))
STAGE DEPENDENCIES:
- Stage-17 is a root stage
- Stage-2 depends on stages: Stage-17
- Stage-16 depends on stages: Stage-2, Stage-7
- Stage-4 depends on stages: Stage-16
- Stage-18 is a root stage
- Stage-7 depends on stages: Stage-18
+ Stage-2 is a root stage
+ Stage-4 depends on stages: Stage-2, Stage-7
+ Stage-7 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-17
- Map Reduce Local Work
- Alias -> Map Local Tables:
- null-subquery2:x-subquery2:tmp:a
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- null-subquery2:x-subquery2:tmp:a
- TableScan
- alias: a
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 1
-
Stage: Stage-2
Map Reduce
Alias -> Map Operator Tree:
@@ -2653,6 +2430,14 @@ STAGE PLANS:
tag: -1
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ null-subquery2:x-subquery2:tmp:a
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ null-subquery2:x-subquery2:tmp:a
+ TableScan
+ alias: a
Reduce Operator Tree:
Group By Operator
bucketGroup: false
@@ -2674,41 +2459,6 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-16
- Map Reduce Local Work
- Alias -> Map Local Tables:
- null-subquery1:x-subquery1:c
- Fetch Operator
- limit: -1
- null-subquery2:x-subquery2:c
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- null-subquery1:x-subquery1:c
- TableScan
- alias: c
- HashTable Sink Operator
- condition expressions:
- 0
- 1 {key}
- handleSkewJoin: false
- keys:
- 0 [Column[_col0]]
- 1 [Column[key]]
- Position of Big Table: 0
- null-subquery2:x-subquery2:c
- TableScan
- alias: c
- HashTable Sink Operator
- condition expressions:
- 0
- 1 {key}
- handleSkewJoin: false
- keys:
- 0 [Column[_col0]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-4
Map Reduce
Alias -> Map Operator Tree:
@@ -2778,26 +2528,20 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Local Work:
Map Reduce Local Work
-
- Stage: Stage-18
- Map Reduce Local Work
- Alias -> Map Local Tables:
- null-subquery1:x-subquery1:tmp:b
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- null-subquery1:x-subquery1:tmp:b
- TableScan
- alias: b
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
+ Alias -> Map Local Tables:
+ null-subquery1:x-subquery1:c
+ Fetch Operator
+ limit: -1
+ null-subquery2:x-subquery2:c
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ null-subquery1:x-subquery1:c
+ TableScan
+ alias: c
+ null-subquery2:x-subquery2:c
+ TableScan
+ alias: c
Stage: Stage-7
Map Reduce
@@ -2840,6 +2584,14 @@ STAGE PLANS:
tag: -1
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ null-subquery1:x-subquery1:tmp:b
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ null-subquery1:x-subquery1:tmp:b
+ TableScan
+ alias: b
Reduce Operator Tree:
Group By Operator
bucketGroup: false
Modified: hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out Thu Jan 30 20:46:16 2014
@@ -48,9 +48,6 @@ STAGE PLANS:
Stage: Stage-8
Map Reduce Local Work
Alias -> Map Local Tables:
- a
- Fetch Operator
- limit: -1
c-subquery1:a-subquery1:src13
Fetch Operator
limit: -1
@@ -58,18 +55,6 @@ STAGE PLANS:
Fetch Operator
limit: -1
Alias -> Map Local Operator Tree:
- a
- TableScan
- alias: a
- HashTable Sink Operator
- condition expressions:
- 0 {key} {value}
- 1 {key} {value}
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 1
c-subquery1:a-subquery1:src13
TableScan
alias: src13
@@ -179,6 +164,18 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ a
+ Fetch Operator
+ limit: -1
+ c-subquery1:a-subquery1:src13
+ c-subquery2:a-subquery2:src14
+ Alias -> Map Local Operator Tree:
+ a
+ TableScan
+ alias: a
+ c-subquery1:a-subquery1:src13
+ c-subquery2:a-subquery2:src14
Stage: Stage-0
Fetch Operator
Modified: hive/trunk/ql/src/test/results/clientpositive/reduce_deduplicate_exclude_join.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/reduce_deduplicate_exclude_join.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/reduce_deduplicate_exclude_join.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/reduce_deduplicate_exclude_join.q.out Thu Jan 30 20:46:16 2014
@@ -7,8 +7,7 @@ ABSTRACT SYNTAX TREE:
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4
+ Stage-3 depends on stages: Stage-2
Stage-0 is a root stage
STAGE PLANS:
@@ -49,26 +48,6 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-4
- Map Reduce Local Work
- Alias -> Map Local Tables:
- b
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- b
- TableScan
- alias: b
- HashTable Sink Operator
- condition expressions:
- 0 {_col0} {_col1}
- 1 {key} {value}
- handleSkewJoin: false
- keys:
- 0 [Column[_col0]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-3
Map Reduce
Alias -> Map Operator Tree:
@@ -107,6 +86,14 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ b
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ b
+ TableScan
+ alias: b
Stage: Stage-0
Fetch Operator
Modified: hive/trunk/ql/src/test/results/clientpositive/select_transform_hint.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/select_transform_hint.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/select_transform_hint.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/select_transform_hint.q.out Thu Jan 30 20:46:16 2014
@@ -14,31 +14,10 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src) a) (TOK_TABREF (TOK_TABNAME src) b) (= (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL b) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_HINTLIST (TOK_HINT TOK_MAPJOIN (TOK_HINTARGLIST a))) (TOK_SELEXPR (TOK_TRANSFORM (TOK_EXPLIST (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL a) value)) TOK_SERDE TOK_RECORDWRITER 'cat' TOK_SERDE TOK_RECORDREADER (TOK_ALIASLIST tkey tvalue))))))
STAGE DEPENDENCIES:
- Stage-3 is a root stage
- Stage-1 depends on stages: Stage-3
+ Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-3
- Map Reduce Local Work
- Alias -> Map Local Tables:
- a
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- a
- TableScan
- alias: a
- HashTable Sink Operator
- condition expressions:
- 0 {key} {value}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 1
-
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
@@ -79,6 +58,14 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ a
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ a
+ TableScan
+ alias: a
Stage: Stage-0
Fetch Operator
Modified: hive/trunk/ql/src/test/results/clientpositive/skewjoin.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/skewjoin.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/skewjoin.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/skewjoin.q.out Thu Jan 30 20:46:16 2014
@@ -60,9 +60,8 @@ ABSTRACT SYNTAX TREE:
STAGE DEPENDENCIES:
Stage-1 is a root stage
- Stage-5 depends on stages: Stage-1 , consists of Stage-6, Stage-0
- Stage-6
- Stage-4 depends on stages: Stage-6
+ Stage-5 depends on stages: Stage-1 , consists of Stage-4, Stage-0
+ Stage-4
Stage-0 depends on stages: Stage-1, Stage-4
Stage-2 depends on stages: Stage-0
@@ -128,25 +127,6 @@ STAGE PLANS:
Stage: Stage-5
Conditional Operator
- Stage: Stage-6
- Map Reduce Local Work
- Alias -> Map Local Tables:
- 1
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- 1
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {0_VALUE_0}
- 1 {1_VALUE_0}
- handleSkewJoin: false
- keys:
- 0 [Column[joinkey0]]
- 1 [Column[joinkey0]]
- Position of Big Table: 0
-
Stage: Stage-4
Map Reduce
Alias -> Map Operator Tree:
@@ -181,6 +161,13 @@ STAGE PLANS:
name: default.dest_j1
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ 1
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ 1
+ TableScan
Stage: Stage-0
Move Operator
@@ -701,9 +688,8 @@ ABSTRACT SYNTAX TREE:
STAGE DEPENDENCIES:
Stage-1 is a root stage
- Stage-5 depends on stages: Stage-1 , consists of Stage-6, Stage-2
- Stage-6
- Stage-4 depends on stages: Stage-6
+ Stage-5 depends on stages: Stage-1 , consists of Stage-4, Stage-2
+ Stage-4
Stage-2 depends on stages: Stage-1, Stage-4
Stage-0 is a root stage
@@ -786,25 +772,6 @@ STAGE PLANS:
Stage: Stage-5
Conditional Operator
- Stage: Stage-6
- Map Reduce Local Work
- Alias -> Map Local Tables:
- 1
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- 1
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0
- 1 {1_VALUE_0} {1_VALUE_1}
- handleSkewJoin: false
- keys:
- 0 [Column[joinkey0]]
- 1 [Column[joinkey0]]
- Position of Big Table: 0
-
Stage: Stage-4
Map Reduce
Alias -> Map Operator Tree:
@@ -845,6 +812,13 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ 1
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ 1
+ TableScan
Stage: Stage-2
Map Reduce
@@ -928,9 +902,8 @@ ABSTRACT SYNTAX TREE:
STAGE DEPENDENCIES:
Stage-1 is a root stage
- Stage-5 depends on stages: Stage-1 , consists of Stage-6, Stage-2
- Stage-6
- Stage-4 depends on stages: Stage-6
+ Stage-5 depends on stages: Stage-1 , consists of Stage-4, Stage-2
+ Stage-4
Stage-2 depends on stages: Stage-1, Stage-4
Stage-0 is a root stage
@@ -1023,25 +996,6 @@ STAGE PLANS:
Stage: Stage-5
Conditional Operator
- Stage: Stage-6
- Map Reduce Local Work
- Alias -> Map Local Tables:
- 1
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- 1
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0
- 1 {1_VALUE_0} {1_VALUE_1}
- handleSkewJoin: false
- keys:
- 0 [Column[joinkey0], Column[joinkey1]]
- 1 [Column[joinkey0], Column[joinkey1]]
- Position of Big Table: 0
-
Stage: Stage-4
Map Reduce
Alias -> Map Operator Tree:
@@ -1082,6 +1036,13 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ 1
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ 1
+ TableScan
Stage: Stage-2
Map Reduce
@@ -1173,12 +1134,10 @@ ABSTRACT SYNTAX TREE:
STAGE DEPENDENCIES:
Stage-1 is a root stage
- Stage-7 depends on stages: Stage-1 , consists of Stage-8, Stage-9, Stage-2
- Stage-8
- Stage-5 depends on stages: Stage-8
+ Stage-7 depends on stages: Stage-1 , consists of Stage-5, Stage-6, Stage-2
+ Stage-5
Stage-2 depends on stages: Stage-1, Stage-5, Stage-6
- Stage-9
- Stage-6 depends on stages: Stage-9
+ Stage-6
Stage-0 is a root stage
STAGE PLANS:
@@ -1292,43 +1251,6 @@ STAGE PLANS:
Stage: Stage-7
Conditional Operator
- Stage: Stage-8
- Map Reduce Local Work
- Alias -> Map Local Tables:
- 1
- Fetch Operator
- limit: -1
- 2
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- 1
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {0_VALUE_0}
- 1 {1_VALUE_0}
- 2
- handleSkewJoin: false
- keys:
- 0 [Column[joinkey0]]
- 1 [Column[joinkey0]]
- 2 [Column[joinkey0]]
- Position of Big Table: 0
- 2
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {0_VALUE_0}
- 1 {1_VALUE_0}
- 2
- handleSkewJoin: false
- keys:
- 0 [Column[joinkey0]]
- 1 [Column[joinkey0]]
- 2 [Column[joinkey0]]
- Position of Big Table: 0
-
Stage: Stage-5
Map Reduce
Alias -> Map Operator Tree:
@@ -1372,6 +1294,18 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ 1
+ Fetch Operator
+ limit: -1
+ 2
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ 1
+ TableScan
+ 2
+ TableScan
Stage: Stage-2
Map Reduce
@@ -1409,43 +1343,6 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Stage: Stage-9
- Map Reduce Local Work
- Alias -> Map Local Tables:
- 0
- Fetch Operator
- limit: -1
- 2
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- 0
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {0_VALUE_0}
- 1 {1_VALUE_0}
- 2
- handleSkewJoin: false
- keys:
- 0 [Column[joinkey0]]
- 1 [Column[joinkey0]]
- 2 [Column[joinkey0]]
- Position of Big Table: 1
- 2
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {0_VALUE_0}
- 1 {1_VALUE_0}
- 2
- handleSkewJoin: false
- keys:
- 0 [Column[joinkey0]]
- 1 [Column[joinkey0]]
- 2 [Column[joinkey0]]
- Position of Big Table: 1
-
Stage: Stage-6
Map Reduce
Alias -> Map Operator Tree:
@@ -1489,6 +1386,18 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ 0
+ Fetch Operator
+ limit: -1
+ 2
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ 0
+ TableScan
+ 2
+ TableScan
Stage: Stage-0
Fetch Operator
@@ -1533,31 +1442,10 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_LEFTOUTERJOIN (TOK_TABREF (TOK_TABNAME T1) k) (TOK_TABREF (TOK_TABNAME T1) v) (= (+ (. (TOK_TABLE_OR_COL k) key) 1) (. (TOK_TABLE_OR_COL v) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_HINTLIST (TOK_HINT TOK_MAPJOIN (TOK_HINTARGLIST v))) (TOK_SELEXPR (TOK_FUNCTION sum (TOK_FUNCTION hash (. (TOK_TABLE_OR_COL k) key)))) (TOK_SELEXPR (TOK_FUNCTION sum (TOK_FUNCTION hash (. (TOK_TABLE_OR_COL v) val)))))))
STAGE DEPENDENCIES:
- Stage-3 is a root stage
- Stage-1 depends on stages: Stage-3
+ Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-3
- Map Reduce Local Work
- Alias -> Map Local Tables:
- v
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- v
- TableScan
- alias: v
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1 {val}
- handleSkewJoin: false
- keys:
- 0 [GenericUDFOPPlus(Column[key], Const int 1)]
- 1 [GenericUDFBridge(Column[key])]
- Position of Big Table: 0
-
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
@@ -1600,6 +1488,14 @@ STAGE PLANS:
type: bigint
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ v
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ v
+ TableScan
+ alias: v
Reduce Operator Tree:
Group By Operator
aggregations:
Modified: hive/trunk/ql/src/test/results/clientpositive/skewjoin_noskew.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/skewjoin_noskew.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/skewjoin_noskew.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/skewjoin_noskew.q.out Thu Jan 30 20:46:16 2014
@@ -9,9 +9,8 @@ ABSTRACT SYNTAX TREE:
STAGE DEPENDENCIES:
Stage-1 is a root stage
- Stage-6 depends on stages: Stage-1 , consists of Stage-7, Stage-2
- Stage-7
- Stage-5 depends on stages: Stage-7
+ Stage-6 depends on stages: Stage-1 , consists of Stage-5, Stage-2
+ Stage-5
Stage-2 depends on stages: Stage-1, Stage-5
Stage-0 depends on stages: Stage-2
Stage-8 depends on stages: Stage-0
@@ -77,25 +76,6 @@ STAGE PLANS:
Stage: Stage-6
Conditional Operator
- Stage: Stage-7
- Map Reduce Local Work
- Alias -> Map Local Tables:
- 1
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- 1
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {0_VALUE_0} {0_VALUE_1}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[joinkey0]]
- 1 [Column[joinkey0]]
- Position of Big Table: 0
-
Stage: Stage-5
Map Reduce
Alias -> Map Operator Tree:
@@ -129,6 +109,13 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ 1
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ 1
+ TableScan
Stage: Stage-2
Map Reduce
Modified: hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_13.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_13.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_13.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_13.q.out Thu Jan 30 20:46:16 2014
@@ -257,34 +257,10 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME test_table3) a) (TOK_TABREF (TOK_TABNAME test_table4) b) (= (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL b) value)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_HINTLIST (TOK_HINT TOK_MAPJOIN (TOK_HINTARGLIST b))) (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (. (TOK_TABLE_OR_COL a) key))) (TOK_LIMIT 10)))
STAGE DEPENDENCIES:
- Stage-3 is a root stage
- Stage-1 depends on stages: Stage-3
+ Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-3
- Map Reduce Local Work
- Alias -> Map Local Tables:
- b
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- b
- TableScan
- alias: b
- Statistics:
- numRows: 500 dataSize: 5312 basicStatsState: COMPLETE colStatsState: NONE
- GatherStats: false
- HashTable Sink Operator
- condition expressions:
- 0 {key} {value}
- 1 {key} {value}
- handleSkewJoin: false
- keys:
- 0 [GenericUDFBridge(Column[key])]
- 1 [GenericUDFBridge(Column[value])]
- Position of Big Table: 0
-
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
@@ -340,6 +316,17 @@ STAGE PLANS:
type: string
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ b
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ b
+ TableScan
+ alias: b
+ Statistics:
+ numRows: 500 dataSize: 5312 basicStatsState: COMPLETE colStatsState: NONE
+ GatherStats: false
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
Modified: hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_15.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_15.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_15.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_15.q.out Thu Jan 30 20:46:16 2014
@@ -681,41 +681,10 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME test_table1) a) (TOK_TABREF (TOK_TABNAME test_table2) b) (and (= (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL b) key)) (= (. (TOK_TABLE_OR_COL a) value) (. (TOK_TABLE_OR_COL b) value))))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_HINTLIST (TOK_HINT TOK_MAPJOIN (TOK_HINTARGLIST b))) (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (. (TOK_TABLE_OR_COL a) key))) (TOK_LIMIT 10)))
STAGE DEPENDENCIES:
- Stage-3 is a root stage
- Stage-1 depends on stages: Stage-3
+ Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-3
- Map Reduce Local Work
- Alias -> Map Local Tables:
- b
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- b
- TableScan
- alias: b
- Statistics:
- numRows: 500 dataSize: 7218 basicStatsState: COMPLETE colStatsState: NONE
- GatherStats: false
- HashTable Sink Operator
- condition expressions:
- 0 {key} {key2} {value}
- 1 {key} {key2} {value}
- handleSkewJoin: false
- keys:
- 0 [Column[key], Column[value]]
- 1 [Column[key], Column[value]]
- Position of Big Table: 0
- Bucket Mapjoin Context:
- Alias Bucket Base File Name Mapping:
- b {000000_0=[000000_0], 000001_0=[000001_0], 000002_0=[000002_0], 000003_0=[000003_0], 000004_0=[000004_0], 000005_0=[000005_0], 000006_0=[000006_0], 000007_0=[000007_0], 000008_0=[000008_0], 000009_0=[000009_0], 000010_0=[000010_0], 000011_0=[000011_0], 000012_0=[000012_0], 000013_0=[000013_0], 000014_0=[000014_0], 000015_0=[000015_0]}
- Alias Bucket File Name Mapping:
-#### A masked pattern was here ####
- Alias Bucket Output File Name Mapping:
-#### A masked pattern was here ####
-
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
@@ -780,6 +749,24 @@ STAGE PLANS:
type: string
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ b
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ b
+ TableScan
+ alias: b
+ Statistics:
+ numRows: 500 dataSize: 7218 basicStatsState: COMPLETE colStatsState: NONE
+ GatherStats: false
+ Bucket Mapjoin Context:
+ Alias Bucket Base File Name Mapping:
+ b {000000_0=[000000_0], 000001_0=[000001_0], 000002_0=[000002_0], 000003_0=[000003_0], 000004_0=[000004_0], 000005_0=[000005_0], 000006_0=[000006_0], 000007_0=[000007_0], 000008_0=[000008_0], 000009_0=[000009_0], 000010_0=[000010_0], 000011_0=[000011_0], 000012_0=[000012_0], 000013_0=[000013_0], 000014_0=[000014_0], 000015_0=[000015_0]}
+ Alias Bucket File Name Mapping:
+#### A masked pattern was here ####
+ Alias Bucket Output File Name Mapping:
+#### A masked pattern was here ####
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
Modified: hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out Thu Jan 30 20:46:16 2014
@@ -70,83 +70,10 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME srcbucket_mapjoin_part_1) a) (TOK_TABREF (TOK_TABNAME srcbucket_mapjoin_part_2) b) (AND (AND (= (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL b) key)) (= (. (TOK_TABLE_OR_COL a) part) '1')) (= (. (TOK_TABLE_OR_COL b) part) '1')))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_HINTLIST (TOK_HINT TOK_MAPJOIN (TOK_HINTARGLIST b))) (TOK_SELEXPR (TOK_FUNCTIONSTAR count)))))
STAGE DEPENDENCIES:
- Stage-3 is a root stage
- Stage-1 depends on stages: Stage-3
+ Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-3
- Map Reduce Local Work
- Alias -> Map Local Tables:
- b
- Fetch Operator
- limit: -1
- Partition Description:
- Partition
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- partition values:
- part 1
- properties:
- COLUMN_STATS_ACCURATE true
- bucket_count 2
- bucket_field_name key
- columns key,value
- columns.types int:string
-#### A masked pattern was here ####
- name default.srcbucket_mapjoin_part_2
- numFiles 2
- numRows 500
- partition_columns part
- rawDataSize 5312
- serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- totalSize 5812
-#### A masked pattern was here ####
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- properties:
- bucket_count 2
- bucket_field_name key
- columns key,value
- columns.types int:string
-#### A masked pattern was here ####
- name default.srcbucket_mapjoin_part_2
- partition_columns part
- serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-#### A masked pattern was here ####
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.srcbucket_mapjoin_part_2
- name: default.srcbucket_mapjoin_part_2
- Alias -> Map Local Operator Tree:
- b
- TableScan
- alias: b
- Statistics:
- numRows: 500 dataSize: 5312 basicStatsState: COMPLETE colStatsState: NONE
- GatherStats: false
- HashTable Sink Operator
- condition expressions:
- 0
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
- Bucket Mapjoin Context:
- Alias Bucket Base File Name Mapping:
- b {part=1/000000_0=[part=1/000000_0], part=1/000001_0=[part=1/000001_0]}
- Alias Bucket File Name Mapping:
-#### A masked pattern was here ####
- Alias Bucket Output File Name Mapping:
-#### A masked pattern was here ####
-
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
@@ -191,6 +118,66 @@ STAGE PLANS:
type: bigint
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ b
+ Fetch Operator
+ limit: -1
+ Partition Description:
+ Partition
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ partition values:
+ part 1
+ properties:
+ COLUMN_STATS_ACCURATE true
+ bucket_count 2
+ bucket_field_name key
+ columns key,value
+ columns.types int:string
+#### A masked pattern was here ####
+ name default.srcbucket_mapjoin_part_2
+ numFiles 2
+ numRows 500
+ partition_columns part
+ rawDataSize 5312
+ serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 5812
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ properties:
+ bucket_count 2
+ bucket_field_name key
+ columns key,value
+ columns.types int:string
+#### A masked pattern was here ####
+ name default.srcbucket_mapjoin_part_2
+ partition_columns part
+ serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.srcbucket_mapjoin_part_2
+ name: default.srcbucket_mapjoin_part_2
+ Alias -> Map Local Operator Tree:
+ b
+ TableScan
+ alias: b
+ Statistics:
+ numRows: 500 dataSize: 5312 basicStatsState: COMPLETE colStatsState: NONE
+ GatherStats: false
+ Bucket Mapjoin Context:
+ Alias Bucket Base File Name Mapping:
+ b {part=1/000000_0=[part=1/000000_0], part=1/000001_0=[part=1/000001_0]}
+ Alias Bucket File Name Mapping:
+#### A masked pattern was here ####
+ Alias Bucket Output File Name Mapping:
+#### A masked pattern was here ####
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
Modified: hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out Thu Jan 30 20:46:16 2014
@@ -134,117 +134,10 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME srcbucket_mapjoin_part_1) a) (TOK_TABREF (TOK_TABNAME srcbucket_mapjoin_part_2) b) (AND (AND (= (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL b) key)) (TOK_FUNCTION TOK_ISNOTNULL (. (TOK_TABLE_OR_COL a) part))) (TOK_FUNCTION TOK_ISNOTNULL (. (TOK_TABLE_OR_COL b) part))))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_HINTLIST (TOK_HINT TOK_MAPJOIN (TOK_HINTARGLIST b))) (TOK_SELEXPR (TOK_FUNCTIONSTAR count)))))
STAGE DEPENDENCIES:
- Stage-3 is a root stage
- Stage-1 depends on stages: Stage-3
+ Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-3
- Map Reduce Local Work
- Alias -> Map Local Tables:
- b
- Fetch Operator
- limit: -1
- Partition Description:
- Partition
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- partition values:
- part 1
- properties:
- COLUMN_STATS_ACCURATE true
- bucket_count 2
- bucket_field_name key
- columns key,value
- columns.types int:string
-#### A masked pattern was here ####
- name default.srcbucket_mapjoin_part_2
- numFiles 2
- numRows 500
- partition_columns part
- rawDataSize 5312
- serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- totalSize 5812
-#### A masked pattern was here ####
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- properties:
- bucket_count 2
- bucket_field_name key
- columns key,value
- columns.types int:string
-#### A masked pattern was here ####
- name default.srcbucket_mapjoin_part_2
- partition_columns part
- serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-#### A masked pattern was here ####
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.srcbucket_mapjoin_part_2
- name: default.srcbucket_mapjoin_part_2
- Partition
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- partition values:
- part 2
- properties:
- COLUMN_STATS_ACCURATE true
- bucket_count 2
- bucket_field_name key
- columns key,value
- columns.types int:string
-#### A masked pattern was here ####
- name default.srcbucket_mapjoin_part_2
- numFiles 2
- numRows 500
- partition_columns part
- rawDataSize 5312
- serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- totalSize 5812
-#### A masked pattern was here ####
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- properties:
- bucket_count 2
- bucket_field_name key
- columns key,value
- columns.types int:string
-#### A masked pattern was here ####
- name default.srcbucket_mapjoin_part_2
- partition_columns part
- serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-#### A masked pattern was here ####
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.srcbucket_mapjoin_part_2
- name: default.srcbucket_mapjoin_part_2
- Alias -> Map Local Operator Tree:
- b
- TableScan
- alias: b
- Statistics:
- numRows: 1000 dataSize: 10624 basicStatsState: COMPLETE colStatsState: NONE
- GatherStats: false
- HashTable Sink Operator
- condition expressions:
- 0
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
-
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
@@ -288,6 +181,100 @@ STAGE PLANS:
type: bigint
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ b
+ Fetch Operator
+ limit: -1
+ Partition Description:
+ Partition
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ partition values:
+ part 1
+ properties:
+ COLUMN_STATS_ACCURATE true
+ bucket_count 2
+ bucket_field_name key
+ columns key,value
+ columns.types int:string
+#### A masked pattern was here ####
+ name default.srcbucket_mapjoin_part_2
+ numFiles 2
+ numRows 500
+ partition_columns part
+ rawDataSize 5312
+ serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 5812
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ properties:
+ bucket_count 2
+ bucket_field_name key
+ columns key,value
+ columns.types int:string
+#### A masked pattern was here ####
+ name default.srcbucket_mapjoin_part_2
+ partition_columns part
+ serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.srcbucket_mapjoin_part_2
+ name: default.srcbucket_mapjoin_part_2
+ Partition
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ partition values:
+ part 2
+ properties:
+ COLUMN_STATS_ACCURATE true
+ bucket_count 2
+ bucket_field_name key
+ columns key,value
+ columns.types int:string
+#### A masked pattern was here ####
+ name default.srcbucket_mapjoin_part_2
+ numFiles 2
+ numRows 500
+ partition_columns part
+ rawDataSize 5312
+ serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 5812
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ properties:
+ bucket_count 2
+ bucket_field_name key
+ columns key,value
+ columns.types int:string
+#### A masked pattern was here ####
+ name default.srcbucket_mapjoin_part_2
+ partition_columns part
+ serialization.ddl struct srcbucket_mapjoin_part_2 { i32 key, string value}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.srcbucket_mapjoin_part_2
+ name: default.srcbucket_mapjoin_part_2
+ Alias -> Map Local Operator Tree:
+ b
+ TableScan
+ alias: b
+ Statistics:
+ numRows: 1000 dataSize: 10624 basicStatsState: COMPLETE colStatsState: NONE
+ GatherStats: false
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
Modified: hive/trunk/ql/src/test/results/clientpositive/stats11.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/stats11.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/stats11.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/stats11.q.out Thu Jan 30 20:46:16 2014
@@ -284,8 +284,7 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME srcbucket_mapjoin) a) (TOK_TABREF (TOK_TABNAME srcbucket_mapjoin_part) b) (= (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL b) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_TAB (TOK_TABNAME bucketmapjoin_tmp_result))) (TOK_SELECT (TOK_HINTLIST (TOK_HINT TOK_MAPJOIN (TOK_HINTARGLIST b))) (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) key)) (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) value)) (TOK_SELEXPR (. (TOK_TABLE_OR_COL b) value))) (TOK_WHERE (= (. (TOK_TABLE_OR_COL b) ds) "2008-04-08"))))
STAGE DEPENDENCIES:
- Stage-9 is a root stage
- Stage-1 depends on stages: Stage-9
+ Stage-1 is a root stage
Stage-8 depends on stages: Stage-1 , consists of Stage-5, Stage-4, Stage-6
Stage-5
Stage-0 depends on stages: Stage-5, Stage-4, Stage-7
@@ -295,78 +294,6 @@ STAGE DEPENDENCIES:
Stage-7 depends on stages: Stage-6
STAGE PLANS:
- Stage: Stage-9
- Map Reduce Local Work
- Alias -> Map Local Tables:
- b
- Fetch Operator
- limit: -1
- Partition Description:
- Partition
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- partition values:
- ds 2008-04-08
- properties:
- COLUMN_STATS_ACCURATE true
- bucket_count 4
- bucket_field_name key
- columns key,value
- columns.types int:string
-#### A masked pattern was here ####
- name default.srcbucket_mapjoin_part
- numFiles 4
- numRows 0
- partition_columns ds
- rawDataSize 0
- serialization.ddl struct srcbucket_mapjoin_part { i32 key, string value}
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- totalSize 5812
-#### A masked pattern was here ####
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- properties:
- bucket_count 4
- bucket_field_name key
- columns key,value
- columns.types int:string
-#### A masked pattern was here ####
- name default.srcbucket_mapjoin_part
- partition_columns ds
- serialization.ddl struct srcbucket_mapjoin_part { i32 key, string value}
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-#### A masked pattern was here ####
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.srcbucket_mapjoin_part
- name: default.srcbucket_mapjoin_part
- Alias -> Map Local Operator Tree:
- b
- TableScan
- alias: b
- Statistics:
- numRows: 55 dataSize: 5812 basicStatsState: COMPLETE colStatsState: NONE
- GatherStats: false
- HashTable Sink Operator
- condition expressions:
- 0 {key} {value}
- 1 {value}
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 0
- Bucket Mapjoin Context:
- Alias Bucket Base File Name Mapping:
- b {srcbucket20.txt=[ds=2008-04-08/srcbucket20.txt, ds=2008-04-08/srcbucket22.txt], srcbucket21.txt=[ds=2008-04-08/srcbucket21.txt, ds=2008-04-08/srcbucket23.txt]}
- Alias Bucket File Name Mapping:
-#### A masked pattern was here ####
- Alias Bucket Output File Name Mapping:
-#### A masked pattern was here ####
-
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
@@ -430,6 +357,66 @@ STAGE PLANS:
MultiFileSpray: false
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ b
+ Fetch Operator
+ limit: -1
+ Partition Description:
+ Partition
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ partition values:
+ ds 2008-04-08
+ properties:
+ COLUMN_STATS_ACCURATE true
+ bucket_count 4
+ bucket_field_name key
+ columns key,value
+ columns.types int:string
+#### A masked pattern was here ####
+ name default.srcbucket_mapjoin_part
+ numFiles 4
+ numRows 0
+ partition_columns ds
+ rawDataSize 0
+ serialization.ddl struct srcbucket_mapjoin_part { i32 key, string value}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 5812
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ properties:
+ bucket_count 4
+ bucket_field_name key
+ columns key,value
+ columns.types int:string
+#### A masked pattern was here ####
+ name default.srcbucket_mapjoin_part
+ partition_columns ds
+ serialization.ddl struct srcbucket_mapjoin_part { i32 key, string value}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.srcbucket_mapjoin_part
+ name: default.srcbucket_mapjoin_part
+ Alias -> Map Local Operator Tree:
+ b
+ TableScan
+ alias: b
+ Statistics:
+ numRows: 55 dataSize: 5812 basicStatsState: COMPLETE colStatsState: NONE
+ GatherStats: false
+ Bucket Mapjoin Context:
+ Alias Bucket Base File Name Mapping:
+ b {srcbucket20.txt=[ds=2008-04-08/srcbucket20.txt, ds=2008-04-08/srcbucket22.txt], srcbucket21.txt=[ds=2008-04-08/srcbucket21.txt, ds=2008-04-08/srcbucket23.txt]}
+ Alias Bucket File Name Mapping:
+#### A masked pattern was here ####
+ Alias Bucket Output File Name Mapping:
+#### A masked pattern was here ####
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -825,8 +812,7 @@ ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME srcbucket_mapjoin) a) (TOK_TABREF (TOK_TABNAME srcbucket_mapjoin_part) b) (= (. (TOK_TABLE_OR_COL a) key) (. (TOK_TABLE_OR_COL b) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_TAB (TOK_TABNAME bucketmapjoin_tmp_result))) (TOK_SELECT (TOK_HINTLIST (TOK_HINT TOK_MAPJOIN (TOK_HINTARGLIST a))) (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) key)) (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) value)) (TOK_SELEXPR (. (TOK_TABLE_OR_COL b) value))) (TOK_WHERE (= (. (TOK_TABLE_OR_COL b) ds) "2008-04-08"))))
STAGE DEPENDENCIES:
- Stage-9 is a root stage
- Stage-1 depends on stages: Stage-9
+ Stage-1 is a root stage
Stage-7 depends on stages: Stage-1 , consists of Stage-4, Stage-3, Stage-5
Stage-4
Stage-0 depends on stages: Stage-4, Stage-3, Stage-6
@@ -836,36 +822,6 @@ STAGE DEPENDENCIES:
Stage-6 depends on stages: Stage-5
STAGE PLANS:
- Stage: Stage-9
- Map Reduce Local Work
- Alias -> Map Local Tables:
- a
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- a
- TableScan
- alias: a
- Statistics:
- numRows: 26 dataSize: 2750 basicStatsState: COMPLETE colStatsState: NONE
- GatherStats: false
- HashTable Sink Operator
- condition expressions:
- 0 {key} {value}
- 1 {value}
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[key]]
- Position of Big Table: 1
- Bucket Mapjoin Context:
- Alias Bucket Base File Name Mapping:
- a {ds=2008-04-08/srcbucket20.txt=[srcbucket20.txt], ds=2008-04-08/srcbucket21.txt=[srcbucket21.txt], ds=2008-04-08/srcbucket22.txt=[srcbucket20.txt], ds=2008-04-08/srcbucket23.txt=[srcbucket21.txt]}
- Alias Bucket File Name Mapping:
-#### A masked pattern was here ####
- Alias Bucket Output File Name Mapping:
-#### A masked pattern was here ####
-
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
@@ -934,6 +890,24 @@ STAGE PLANS:
MultiFileSpray: false
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ a
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ a
+ TableScan
+ alias: a
+ Statistics:
+ numRows: 26 dataSize: 2750 basicStatsState: COMPLETE colStatsState: NONE
+ GatherStats: false
+ Bucket Mapjoin Context:
+ Alias Bucket Base File Name Mapping:
+ a {ds=2008-04-08/srcbucket20.txt=[srcbucket20.txt], ds=2008-04-08/srcbucket21.txt=[srcbucket21.txt], ds=2008-04-08/srcbucket22.txt=[srcbucket20.txt], ds=2008-04-08/srcbucket23.txt=[srcbucket21.txt]}
+ Alias Bucket File Name Mapping:
+#### A masked pattern was here ####
+ Alias Bucket Output File Name Mapping:
+#### A masked pattern was here ####
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
Modified: hive/trunk/ql/src/test/results/clientpositive/subq_where_serialization.q.out
URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/subq_where_serialization.q.out?rev=1562955&r1=1562954&r2=1562955&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/subq_where_serialization.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/subq_where_serialization.q.out Thu Jan 30 20:46:16 2014
@@ -7,9 +7,8 @@ ABSTRACT SYNTAX TREE:
STAGE DEPENDENCIES:
Stage-1 is a root stage
- Stage-5 depends on stages: Stage-1 , consists of Stage-6, Stage-2
- Stage-6 has a backup stage: Stage-2
- Stage-4 depends on stages: Stage-6
+ Stage-5 depends on stages: Stage-1 , consists of Stage-4, Stage-2
+ Stage-4 has a backup stage: Stage-2
Stage-2
Stage-0 is a root stage
@@ -72,25 +71,6 @@ STAGE PLANS:
Stage: Stage-5
Conditional Operator
- Stage: Stage-6
- Map Reduce Local Work
- Alias -> Map Local Tables:
- $INTNAME
- Fetch Operator
- limit: -1
- Alias -> Map Local Operator Tree:
- $INTNAME
- TableScan
- HashTable Sink Operator
- condition expressions:
- 0 {key}
- 1
- handleSkewJoin: false
- keys:
- 0 [Column[key]]
- 1 [Column[_col0]]
- Position of Big Table: 0
-
Stage: Stage-4
Map Reduce
Alias -> Map Operator Tree:
@@ -127,6 +107,13 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Local Work:
Map Reduce Local Work
+ Alias -> Map Local Tables:
+ $INTNAME
+ Fetch Operator
+ limit: -1
+ Alias -> Map Local Operator Tree:
+ $INTNAME
+ TableScan
Stage: Stage-2
Map Reduce