You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Mostafa Mokhtar (JIRA)" <ji...@apache.org> on 2014/09/11 01:14:35 UTC
[jira] [Commented] (HIVE-7993) With CBO enabled Q75 fails with
RuntimeException: cannot find field _col69 from [0:_col18,...]
[ https://issues.apache.org/jira/browse/HIVE-7993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14129331#comment-14129331 ]
Mostafa Mokhtar commented on HIVE-7993:
---------------------------------------
Query generates a plan now and runs successfully
{code}
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
Edges:
Map 15 <- Map 1 (BROADCAST_EDGE), Map 24 (BROADCAST_EDGE)
Map 2 <- Map 27 (BROADCAST_EDGE), Map 29 (BROADCAST_EDGE)
Map 23 <- Map 13 (BROADCAST_EDGE), Map 19 (BROADCAST_EDGE), Map 21 (BROADCAST_EDGE), Union 10 (CONTAINS)
Map 28 <- Map 22 (BROADCAST_EDGE), Map 34 (BROADCAST_EDGE)
Map 31 <- Map 12 (BROADCAST_EDGE), Map 14 (BROADCAST_EDGE), Map 32 (BROADCAST_EDGE), Union 4 (CONTAINS)
Map 8 <- Map 20 (BROADCAST_EDGE), Map 30 (BROADCAST_EDGE)
Reducer 11 <- Union 10 (SIMPLE_EDGE)
Reducer 16 <- Map 15 (SIMPLE_EDGE), Map 33 (SIMPLE_EDGE), Union 4 (CONTAINS)
Reducer 26 <- Map 25 (SIMPLE_EDGE), Map 28 (SIMPLE_EDGE), Union 10 (CONTAINS)
Reducer 3 <- Map 17 (SIMPLE_EDGE), Map 2 (SIMPLE_EDGE), Union 4 (CONTAINS)
Reducer 5 <- Union 4 (SIMPLE_EDGE)
Reducer 6 <- Reducer 11 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
Reducer 9 <- Map 18 (SIMPLE_EDGE), Map 8 (SIMPLE_EDGE), Union 10 (CONTAINS)
DagName: mmokhtar_20140910191313_4c3247fa-60c7-47c0-b407-ee2dc6936148:8
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: item
filterExpr: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 48000 Data size: 68732712 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: i_item_sk (type: int)
sort order: +
Map-reduce partition columns: i_item_sk (type: int)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
value expressions: i_brand_id (type: int), i_class_id (type: int), i_category_id (type: int), i_manufact_id (type: int)
Execution mode: vectorized
Map 12
Map Operator Tree:
TableScan
alias: item
filterExpr: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 48000 Data size: 68732712 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: i_item_sk (type: int)
sort order: +
Map-reduce partition columns: i_item_sk (type: int)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
value expressions: i_brand_id (type: int), i_class_id (type: int), i_category_id (type: int), i_manufact_id (type: int)
Execution mode: vectorized
Map 13
Map Operator Tree:
TableScan
alias: date_dim
filterExpr: (d_date_sk is not null and (d_year = 2001)) (type: boolean)
Statistics: Num rows: 73049 Data size: 81741831 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (d_date_sk is not null and (d_year = 2001)) (type: boolean)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: d_date_sk (type: int)
sort order: +
Map-reduce partition columns: d_date_sk (type: int)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized
Map 14
Map Operator Tree:
TableScan
alias: web_returns
Statistics: Num rows: 13749816 Data size: 1237758344 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: wr_order_number (type: int), wr_item_sk (type: int)
sort order: ++
Map-reduce partition columns: wr_order_number (type: int), wr_item_sk (type: int)
Statistics: Num rows: 13749816 Data size: 1237758344 Basic stats: COMPLETE Column stats: NONE
value expressions: wr_return_quantity (type: int), wr_return_amt (type: float)
Execution mode: vectorized
Map 15
Map Operator Tree:
TableScan
alias: catalog_sales
filterExpr: (cs_item_sk is not null and cs_sold_date_sk is not null) (type: boolean)
Statistics: Num rows: 286549727 Data size: 37743959324 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: cs_item_sk is not null (type: boolean)
Statistics: Num rows: 143274864 Data size: 18871979727 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {cs_item_sk} {cs_order_number} {cs_quantity} {cs_ext_sales_price} {cs_sold_date_sk}
1 {i_brand_id} {i_class_id} {i_category_id} {i_manufact_id}
keys:
0 cs_item_sk (type: int)
1 i_item_sk (type: int)
outputColumnNames: _col14, _col16, _col17, _col22, _col33, _col44, _col46, _col48, _col50
input vertices:
1 Map 1
Statistics: Num rows: 157602352 Data size: 20759179264 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {_col14} {_col16} {_col17} {_col22} {_col44} {_col46} {_col48} {_col50}
1
keys:
0 _col33 (type: int)
1 d_date_sk (type: int)
outputColumnNames: _col14, _col16, _col17, _col22, _col44, _col46, _col48, _col50
input vertices:
1 Map 24
Statistics: Num rows: 173362592 Data size: 22835097600 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col16 (type: int), _col14 (type: int)
sort order: ++
Map-reduce partition columns: _col16 (type: int), _col14 (type: int)
Statistics: Num rows: 173362592 Data size: 22835097600 Basic stats: COMPLETE Column stats: NONE
value expressions: _col17 (type: int), _col22 (type: float), _col44 (type: int), _col46 (type: int), _col48 (type: int), _col50 (type: int)
Execution mode: vectorized
Map 17
Map Operator Tree:
TableScan
alias: store_returns
Statistics: Num rows: 55578005 Data size: 4155315616 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: sr_ticket_number (type: int), sr_item_sk (type: int)
sort order: ++
Map-reduce partition columns: sr_ticket_number (type: int), sr_item_sk (type: int)
Statistics: Num rows: 55578005 Data size: 4155315616 Basic stats: COMPLETE Column stats: NONE
value expressions: sr_return_quantity (type: int), sr_return_amt (type: float)
Execution mode: vectorized
Map 18
Map Operator Tree:
TableScan
alias: store_returns
Statistics: Num rows: 55578005 Data size: 4155315616 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: sr_ticket_number (type: int), sr_item_sk (type: int)
sort order: ++
Map-reduce partition columns: sr_ticket_number (type: int), sr_item_sk (type: int)
Statistics: Num rows: 55578005 Data size: 4155315616 Basic stats: COMPLETE Column stats: NONE
value expressions: sr_return_quantity (type: int), sr_return_amt (type: float)
Execution mode: vectorized
Map 19
Map Operator Tree:
TableScan
alias: web_returns
Statistics: Num rows: 13749816 Data size: 1237758344 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: wr_order_number (type: int), wr_item_sk (type: int)
sort order: ++
Map-reduce partition columns: wr_order_number (type: int), wr_item_sk (type: int)
Statistics: Num rows: 13749816 Data size: 1237758344 Basic stats: COMPLETE Column stats: NONE
value expressions: wr_return_quantity (type: int), wr_return_amt (type: float)
Execution mode: vectorized
Map 2
Map Operator Tree:
TableScan
alias: store_sales
filterExpr: (ss_item_sk is not null and ss_sold_date_sk is not null) (type: boolean)
Statistics: Num rows: 550076554 Data size: 47370018896 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: ss_item_sk is not null (type: boolean)
Statistics: Num rows: 275038277 Data size: 23685009448 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {ss_item_sk} {ss_ticket_number} {ss_quantity} {ss_ext_sales_price} {ss_sold_date_sk}
1 {i_brand_id} {i_class_id} {i_category_id} {i_manufact_id}
keys:
0 ss_item_sk (type: int)
1 i_item_sk (type: int)
outputColumnNames: _col1, _col8, _col9, _col14, _col22, _col33, _col35, _col37, _col39
input vertices:
1 Map 29
Statistics: Num rows: 302542112 Data size: 26053511168 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {_col1} {_col8} {_col9} {_col14} {_col33} {_col35} {_col37} {_col39}
1
keys:
0 _col22 (type: int)
1 d_date_sk (type: int)
outputColumnNames: _col1, _col8, _col9, _col14, _col33, _col35, _col37, _col39
input vertices:
1 Map 27
Statistics: Num rows: 332796320 Data size: 28658862080 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col8 (type: int), _col1 (type: int)
sort order: ++
Map-reduce partition columns: _col8 (type: int), _col1 (type: int)
Statistics: Num rows: 332796320 Data size: 28658862080 Basic stats: COMPLETE Column stats: NONE
value expressions: _col9 (type: int), _col14 (type: float), _col33 (type: int), _col35 (type: int), _col37 (type: int), _col39 (type: int)
Execution mode: vectorized
Map 20
Map Operator Tree:
TableScan
alias: date_dim
filterExpr: (d_date_sk is not null and (d_year = 2001)) (type: boolean)
Statistics: Num rows: 73049 Data size: 81741831 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (d_date_sk is not null and (d_year = 2001)) (type: boolean)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: d_date_sk (type: int)
sort order: +
Map-reduce partition columns: d_date_sk (type: int)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized
Map 21
Map Operator Tree:
TableScan
alias: item
filterExpr: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 48000 Data size: 68732712 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: i_item_sk (type: int)
sort order: +
Map-reduce partition columns: i_item_sk (type: int)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
value expressions: i_brand_id (type: int), i_class_id (type: int), i_category_id (type: int), i_manufact_id (type: int)
Execution mode: vectorized
Map 22
Map Operator Tree:
TableScan
alias: item
filterExpr: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 48000 Data size: 68732712 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: i_item_sk (type: int)
sort order: +
Map-reduce partition columns: i_item_sk (type: int)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
value expressions: i_brand_id (type: int), i_class_id (type: int), i_category_id (type: int), i_manufact_id (type: int)
Execution mode: vectorized
Map 23
Map Operator Tree:
TableScan
alias: web_sales
filterExpr: (ws_item_sk is not null and ws_sold_date_sk is not null) (type: boolean)
Filter Operator
predicate: ws_item_sk is not null (type: boolean)
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {ws_item_sk} {ws_order_number} {ws_quantity} {ws_ext_sales_price} {ws_sold_date_sk}
1 {i_brand_id} {i_class_id} {i_category_id} {i_manufact_id}
keys:
0 ws_item_sk (type: int)
1 i_item_sk (type: int)
outputColumnNames: _col2, _col16, _col17, _col22, _col33, _col44, _col46, _col48, _col50
input vertices:
1 Map 21
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {_col2} {_col16} {_col17} {_col22} {_col44} {_col46} {_col48} {_col50}
1
keys:
0 _col33 (type: int)
1 d_date_sk (type: int)
outputColumnNames: _col2, _col16, _col17, _col22, _col44, _col46, _col48, _col50
input vertices:
1 Map 13
Map Join Operator
condition map:
Left Outer Join0 to 1
condition expressions:
0 {_col17} {_col22} {_col44} {_col46} {_col48} {_col50}
1 {wr_return_quantity} {wr_return_amt}
keys:
0 _col16 (type: int), _col2 (type: int)
1 wr_order_number (type: int), wr_item_sk (type: int)
outputColumnNames: _col17, _col22, _col44, _col46, _col48, _col50, _col106, _col107
input vertices:
1 Map 19
Select Operator
expressions: 2001 (type: int), _col44 (type: int), _col46 (type: int), _col48 (type: int), _col50 (type: int), (_col17 - COALESCE(_col106,0)) (type: int), (_col22 - COALESCE(_col107,0.0)) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: int), _col6 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Group By Operator
aggregations: sum(_col5), sum(_col6)
keys: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
sort order: +++++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
value expressions: _col5 (type: bigint), _col6 (type: double)
Execution mode: vectorized
Map 24
Map Operator Tree:
TableScan
alias: date_dim
filterExpr: (d_date_sk is not null and (d_year = 2002)) (type: boolean)
Statistics: Num rows: 73049 Data size: 81741831 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (d_date_sk is not null and (d_year = 2002)) (type: boolean)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: d_date_sk (type: int)
sort order: +
Map-reduce partition columns: d_date_sk (type: int)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized
Map 25
Map Operator Tree:
TableScan
alias: catalog_returns
Statistics: Num rows: 28798881 Data size: 2942039156 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: cr_order_number (type: int), cr_item_sk (type: int)
sort order: ++
Map-reduce partition columns: cr_order_number (type: int), cr_item_sk (type: int)
Statistics: Num rows: 28798881 Data size: 2942039156 Basic stats: COMPLETE Column stats: NONE
value expressions: cr_return_quantity (type: int), cr_return_amount (type: float)
Execution mode: vectorized
Map 27
Map Operator Tree:
TableScan
alias: date_dim
filterExpr: (d_date_sk is not null and (d_year = 2002)) (type: boolean)
Statistics: Num rows: 73049 Data size: 81741831 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (d_date_sk is not null and (d_year = 2002)) (type: boolean)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: d_date_sk (type: int)
sort order: +
Map-reduce partition columns: d_date_sk (type: int)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized
Map 28
Map Operator Tree:
TableScan
alias: catalog_sales
filterExpr: (cs_item_sk is not null and cs_sold_date_sk is not null) (type: boolean)
Statistics: Num rows: 286549727 Data size: 37743959324 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: cs_item_sk is not null (type: boolean)
Statistics: Num rows: 143274864 Data size: 18871979727 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {cs_item_sk} {cs_order_number} {cs_quantity} {cs_ext_sales_price} {cs_sold_date_sk}
1 {i_brand_id} {i_class_id} {i_category_id} {i_manufact_id}
keys:
0 cs_item_sk (type: int)
1 i_item_sk (type: int)
outputColumnNames: _col14, _col16, _col17, _col22, _col33, _col44, _col46, _col48, _col50
input vertices:
1 Map 22
Statistics: Num rows: 157602352 Data size: 20759179264 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {_col14} {_col16} {_col17} {_col22} {_col44} {_col46} {_col48} {_col50}
1
keys:
0 _col33 (type: int)
1 d_date_sk (type: int)
outputColumnNames: _col14, _col16, _col17, _col22, _col44, _col46, _col48, _col50
input vertices:
1 Map 34
Statistics: Num rows: 173362592 Data size: 22835097600 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col16 (type: int), _col14 (type: int)
sort order: ++
Map-reduce partition columns: _col16 (type: int), _col14 (type: int)
Statistics: Num rows: 173362592 Data size: 22835097600 Basic stats: COMPLETE Column stats: NONE
value expressions: _col17 (type: int), _col22 (type: float), _col44 (type: int), _col46 (type: int), _col48 (type: int), _col50 (type: int)
Execution mode: vectorized
Map 29
Map Operator Tree:
TableScan
alias: item
filterExpr: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 48000 Data size: 68732712 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: i_item_sk (type: int)
sort order: +
Map-reduce partition columns: i_item_sk (type: int)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
value expressions: i_brand_id (type: int), i_class_id (type: int), i_category_id (type: int), i_manufact_id (type: int)
Execution mode: vectorized
Map 30
Map Operator Tree:
TableScan
alias: item
filterExpr: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 48000 Data size: 68732712 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (((((i_item_sk is not null and (i_category = 'Sports')) and i_brand_id is not null) and i_class_id is not null) and i_category_id is not null) and i_manufact_id is not null) (type: boolean)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: i_item_sk (type: int)
sort order: +
Map-reduce partition columns: i_item_sk (type: int)
Statistics: Num rows: 750 Data size: 1073948 Basic stats: COMPLETE Column stats: NONE
value expressions: i_brand_id (type: int), i_class_id (type: int), i_category_id (type: int), i_manufact_id (type: int)
Execution mode: vectorized
Map 31
Map Operator Tree:
TableScan
alias: web_sales
filterExpr: (ws_item_sk is not null and ws_sold_date_sk is not null) (type: boolean)
Filter Operator
predicate: ws_item_sk is not null (type: boolean)
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {ws_item_sk} {ws_order_number} {ws_quantity} {ws_ext_sales_price} {ws_sold_date_sk}
1 {i_brand_id} {i_class_id} {i_category_id} {i_manufact_id}
keys:
0 ws_item_sk (type: int)
1 i_item_sk (type: int)
outputColumnNames: _col2, _col16, _col17, _col22, _col33, _col44, _col46, _col48, _col50
input vertices:
1 Map 12
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {_col2} {_col16} {_col17} {_col22} {_col44} {_col46} {_col48} {_col50}
1
keys:
0 _col33 (type: int)
1 d_date_sk (type: int)
outputColumnNames: _col2, _col16, _col17, _col22, _col44, _col46, _col48, _col50
input vertices:
1 Map 32
Map Join Operator
condition map:
Left Outer Join0 to 1
condition expressions:
0 {_col17} {_col22} {_col44} {_col46} {_col48} {_col50}
1 {wr_return_quantity} {wr_return_amt}
keys:
0 _col16 (type: int), _col2 (type: int)
1 wr_order_number (type: int), wr_item_sk (type: int)
outputColumnNames: _col17, _col22, _col44, _col46, _col48, _col50, _col106, _col107
input vertices:
1 Map 14
Select Operator
expressions: 2002 (type: int), _col44 (type: int), _col46 (type: int), _col48 (type: int), _col50 (type: int), (_col17 - COALESCE(_col106,0)) (type: int), (_col22 - COALESCE(_col107,0.0)) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: int), _col6 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Group By Operator
aggregations: sum(_col5), sum(_col6)
keys: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
sort order: +++++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
value expressions: _col5 (type: bigint), _col6 (type: double)
Execution mode: vectorized
Map 32
Map Operator Tree:
TableScan
alias: date_dim
filterExpr: (d_date_sk is not null and (d_year = 2002)) (type: boolean)
Statistics: Num rows: 73049 Data size: 81741831 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (d_date_sk is not null and (d_year = 2002)) (type: boolean)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: d_date_sk (type: int)
sort order: +
Map-reduce partition columns: d_date_sk (type: int)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized
Map 33
Map Operator Tree:
TableScan
alias: catalog_returns
Statistics: Num rows: 28798881 Data size: 2942039156 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: cr_order_number (type: int), cr_item_sk (type: int)
sort order: ++
Map-reduce partition columns: cr_order_number (type: int), cr_item_sk (type: int)
Statistics: Num rows: 28798881 Data size: 2942039156 Basic stats: COMPLETE Column stats: NONE
value expressions: cr_return_quantity (type: int), cr_return_amount (type: float)
Execution mode: vectorized
Map 34
Map Operator Tree:
TableScan
alias: date_dim
filterExpr: (d_date_sk is not null and (d_year = 2001)) (type: boolean)
Statistics: Num rows: 73049 Data size: 81741831 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (d_date_sk is not null and (d_year = 2001)) (type: boolean)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: d_date_sk (type: int)
sort order: +
Map-reduce partition columns: d_date_sk (type: int)
Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized
Map 8
Map Operator Tree:
TableScan
alias: store_sales
filterExpr: (ss_item_sk is not null and ss_sold_date_sk is not null) (type: boolean)
Statistics: Num rows: 550076554 Data size: 47370018896 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: ss_item_sk is not null (type: boolean)
Statistics: Num rows: 275038277 Data size: 23685009448 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {ss_item_sk} {ss_ticket_number} {ss_quantity} {ss_ext_sales_price} {ss_sold_date_sk}
1 {i_brand_id} {i_class_id} {i_category_id} {i_manufact_id}
keys:
0 ss_item_sk (type: int)
1 i_item_sk (type: int)
outputColumnNames: _col1, _col8, _col9, _col14, _col22, _col33, _col35, _col37, _col39
input vertices:
1 Map 30
Statistics: Num rows: 302542112 Data size: 26053511168 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {_col1} {_col8} {_col9} {_col14} {_col33} {_col35} {_col37} {_col39}
1
keys:
0 _col22 (type: int)
1 d_date_sk (type: int)
outputColumnNames: _col1, _col8, _col9, _col14, _col33, _col35, _col37, _col39
input vertices:
1 Map 20
Statistics: Num rows: 332796320 Data size: 28658862080 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col8 (type: int), _col1 (type: int)
sort order: ++
Map-reduce partition columns: _col8 (type: int), _col1 (type: int)
Statistics: Num rows: 332796320 Data size: 28658862080 Basic stats: COMPLETE Column stats: NONE
value expressions: _col9 (type: int), _col14 (type: float), _col33 (type: int), _col35 (type: int), _col37 (type: int), _col39 (type: int)
Execution mode: vectorized
Reducer 11
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
keys: KEY._col0 (type: int), KEY._col1 (type: int), KEY._col2 (type: int), KEY._col3 (type: int), KEY._col4 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 326292384 Data size: 34644464640 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: bigint), _col6 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 326292384 Data size: 34644464640 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
sort order: ++++
Map-reduce partition columns: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
Statistics: Num rows: 326292384 Data size: 34644464640 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int), _col5 (type: bigint), _col6 (type: double)
Reducer 16
Reduce Operator Tree:
Join Operator
condition map:
Left Outer Join0 to 1
condition expressions:
0 {VALUE._col15} {VALUE._col20} {VALUE._col42} {VALUE._col44} {VALUE._col46} {VALUE._col48}
1 {VALUE._col14} {VALUE._col15}
outputColumnNames: _col17, _col22, _col44, _col46, _col48, _col50, _col109, _col110
Select Operator
expressions: 2002 (type: int), _col44 (type: int), _col46 (type: int), _col48 (type: int), _col50 (type: int), (_col17 - COALESCE(_col109,0)) (type: int), (_col22 - COALESCE(_col110,0.0)) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: int), _col6 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Group By Operator
aggregations: sum(_col5), sum(_col6)
keys: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
sort order: +++++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
value expressions: _col5 (type: bigint), _col6 (type: double)
Reducer 26
Reduce Operator Tree:
Join Operator
condition map:
Left Outer Join0 to 1
condition expressions:
0 {VALUE._col15} {VALUE._col20} {VALUE._col42} {VALUE._col44} {VALUE._col46} {VALUE._col48}
1 {VALUE._col14} {VALUE._col15}
outputColumnNames: _col17, _col22, _col44, _col46, _col48, _col50, _col109, _col110
Select Operator
expressions: 2001 (type: int), _col44 (type: int), _col46 (type: int), _col48 (type: int), _col50 (type: int), (_col17 - COALESCE(_col109,0)) (type: int), (_col22 - COALESCE(_col110,0.0)) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: int), _col6 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Group By Operator
aggregations: sum(_col5), sum(_col6)
keys: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
sort order: +++++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
value expressions: _col5 (type: bigint), _col6 (type: double)
Reducer 3
Reduce Operator Tree:
Join Operator
condition map:
Left Outer Join0 to 1
condition expressions:
0 {VALUE._col7} {VALUE._col12} {VALUE._col31} {VALUE._col33} {VALUE._col35} {VALUE._col37}
1 {VALUE._col7} {VALUE._col8}
outputColumnNames: _col9, _col14, _col33, _col35, _col37, _col39, _col91, _col92
Select Operator
expressions: 2002 (type: int), _col33 (type: int), _col35 (type: int), _col37 (type: int), _col39 (type: int), (_col9 - COALESCE(_col91,0)) (type: int), (_col14 - COALESCE(_col92,0.0)) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: int), _col6 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Group By Operator
aggregations: sum(_col5), sum(_col6)
keys: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
sort order: +++++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
value expressions: _col5 (type: bigint), _col6 (type: double)
Reducer 5
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
keys: KEY._col0 (type: int), KEY._col1 (type: int), KEY._col2 (type: int), KEY._col3 (type: int), KEY._col4 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 326292384 Data size: 34644464640 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: bigint), _col6 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 326292384 Data size: 34644464640 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
sort order: ++++
Map-reduce partition columns: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
Statistics: Num rows: 326292384 Data size: 34644464640 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int), _col5 (type: bigint), _col6 (type: double)
Reducer 6
Reduce Operator Tree:
Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {VALUE._col0} {KEY.reducesinkkey0} {KEY.reducesinkkey1} {KEY.reducesinkkey2} {KEY.reducesinkkey3} {VALUE._col1} {VALUE._col2}
1 {VALUE._col0} {KEY.reducesinkkey0} {KEY.reducesinkkey1} {KEY.reducesinkkey2} {KEY.reducesinkkey3} {VALUE._col1} {VALUE._col2}
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13
Statistics: Num rows: 358921632 Data size: 38108909568 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (((((((_col1 = _col8) and (_col2 = _col9)) and (_col3 = _col10)) and (_col4 = _col11)) and (_col0 = 2002)) and (_col7 = 2001)) and ((CAST( _col5 AS decimal(17,2)) / CAST( _col12 AS decimal(17,2))) < 0.9)) (type: boolean)
Statistics: Num rows: 1869383 Data size: 198483850 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: 2001 (type: int), 2002 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col12 (type: bigint), _col5 (type: bigint), (_col5 - _col12) (type: bigint), (_col6 - _col13) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
Statistics: Num rows: 1869383 Data size: 198483850 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col8 (type: bigint)
sort order: +
Statistics: Num rows: 1869383 Data size: 198483850 Basic stats: COMPLETE Column stats: NONE
TopN Hash Memory Usage: 0.04
value expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: int), _col6 (type: bigint), _col7 (type: bigint), _col9 (type: double)
Reducer 7
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: int), VALUE._col1 (type: int), VALUE._col2 (type: int), VALUE._col3 (type: int), VALUE._col4 (type: int), VALUE._col5 (type: int), VALUE._col6 (type: bigint), VALUE._col7 (type: bigint), KEY.reducesinkkey0 (type: bigint), VALUE._col8 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
Statistics: Num rows: 1869383 Data size: 198483850 Basic stats: COMPLETE Column stats: NONE
Limit
Number of rows: 100
Statistics: Num rows: 100 Data size: 10600 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 100 Data size: 10600 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: vectorized
Reducer 9
Reduce Operator Tree:
Join Operator
condition map:
Left Outer Join0 to 1
condition expressions:
0 {VALUE._col7} {VALUE._col12} {VALUE._col31} {VALUE._col33} {VALUE._col35} {VALUE._col37}
1 {VALUE._col7} {VALUE._col8}
outputColumnNames: _col9, _col14, _col33, _col35, _col37, _col39, _col91, _col92
Select Operator
expressions: 2001 (type: int), _col33 (type: int), _col35 (type: int), _col37 (type: int), _col39 (type: int), (_col9 - COALESCE(_col91,0)) (type: int), (_col14 - COALESCE(_col92,0.0)) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: int), _col6 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Group By Operator
aggregations: sum(_col5), sum(_col6)
keys: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
sort order: +++++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
value expressions: _col5 (type: bigint), _col6 (type: double)
Union 10
Vertex: Union 10
Union 4
Vertex: Union 4
Stage: Stage-0
Fetch Operator
limit: 100
Processor Tree:
ListSink
{code}
> With CBO enabled Q75 fails with RuntimeException: cannot find field _col69 from [0:_col18,...]
> ----------------------------------------------------------------------------------------------
>
> Key: HIVE-7993
> URL: https://issues.apache.org/jira/browse/HIVE-7993
> Project: Hive
> Issue Type: Bug
> Components: CBO
> Affects Versions: 0.14.0
> Reporter: Mostafa Mokhtar
> Assignee: Matt McCline
> Fix For: 0.14.0
>
>
> Exception
> {code}
> 2014-09-04 19:28:24,032 INFO [TezChild] org.apache.hadoop.hive.ql.exec.JoinOperator: 41 Close done
> 2014-09-04 19:28:24,033 ERROR [TezChild] org.apache.hadoop.hive.ql.exec.tez.TezProcessor: java.lang.RuntimeException: Reduce operator initialization failed
> at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:254)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:309)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:180)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:172)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:167)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.lang.RuntimeException: cannot find field _col69 from [0:_col18, 1:_col23, 2:_col45, 3:_col47, 4:_col49, 5:_col51, 6:_col108, 7:_col109]
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:410)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:147)
> at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:55)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:938)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:964)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:63)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420)
> at org.apache.hadoop.hive.ql.exec.JoinOperator.initializeOp(JoinOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
> at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:223)
> ... 13 more
> 2014-09-04 19:28:24,034 INFO [TezChild] org.apache.tez.runtime.task.TezTaskRunner: Encounted an error while executing task: attempt_1406566393272_3882_3_24_000028_0
> java.lang.RuntimeException: java.lang.RuntimeException: Reduce operator initialization failed
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:309)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:180)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:172)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:167)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.lang.RuntimeException: Reduce operator initialization failed
> at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:254)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> ... 12 more
> Caused by: java.lang.RuntimeException: cannot find field _col69 from [0:_col18, 1:_col23, 2:_col45, 3:_col47, 4:_col49, 5:_col51, 6:_col108, 7:_col109]
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:410)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:147)
> at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:55)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:938)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:964)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:63)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420)
> at org.apache.hadoop.hive.ql.exec.JoinOperator.initializeOp(JoinOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
> at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:223)
> ... 13 more
> {code}
> Query
> {code}
> WITH all_sales AS (
> SELECT d_year
> ,i_brand_id
> ,i_class_id
> ,i_category_id
> ,i_manufact_id
> ,SUM(sales_cnt) AS sales_cnt
> ,SUM(sales_amt) AS sales_amt
> FROM (SELECT d_year
> ,i_brand_id
> ,i_class_id
> ,i_category_id
> ,i_manufact_id
> ,cs_quantity - COALESCE(cr_return_quantity,0) AS sales_cnt
> ,cs_ext_sales_price - COALESCE(cr_return_amount,0.0) AS sales_amt
> FROM catalog_sales JOIN item ON i_item_sk=cs_item_sk
> JOIN date_dim ON d_date_sk=cs_sold_date_sk
> LEFT JOIN catalog_returns ON (cs_order_number=cr_order_number
> AND cs_item_sk=cr_item_sk)
> WHERE i_category='Sports'
> UNION ALL
> SELECT d_year
> ,i_brand_id
> ,i_class_id
> ,i_category_id
> ,i_manufact_id
> ,ss_quantity - COALESCE(sr_return_quantity,0) AS sales_cnt
> ,ss_ext_sales_price - COALESCE(sr_return_amt,0.0) AS sales_amt
> FROM store_sales JOIN item ON i_item_sk=ss_item_sk
> JOIN date_dim ON d_date_sk=ss_sold_date_sk
> LEFT JOIN store_returns ON (ss_ticket_number=sr_ticket_number
> AND ss_item_sk=sr_item_sk)
> WHERE i_category='Sports'
> UNION ALL
> SELECT d_year
> ,i_brand_id
> ,i_class_id
> ,i_category_id
> ,i_manufact_id
> ,ws_quantity - COALESCE(wr_return_quantity,0) AS sales_cnt
> ,ws_ext_sales_price - COALESCE(wr_return_amt,0.0) AS sales_amt
> FROM web_sales JOIN item ON i_item_sk=ws_item_sk
> JOIN date_dim ON d_date_sk=ws_sold_date_sk
> LEFT JOIN web_returns ON (ws_order_number=wr_order_number
> AND ws_item_sk=wr_item_sk)
> WHERE i_category='Sports') sales_detail
> GROUP BY d_year, i_brand_id, i_class_id, i_category_id, i_manufact_id)
> SELECT prev_yr.d_year AS prev_year
> ,curr_yr.d_year AS year
> ,curr_yr.i_brand_id
> ,curr_yr.i_class_id
> ,curr_yr.i_category_id
> ,curr_yr.i_manufact_id
> ,prev_yr.sales_cnt AS prev_yr_cnt
> ,curr_yr.sales_cnt AS curr_yr_cnt
> ,curr_yr.sales_cnt-prev_yr.sales_cnt AS sales_cnt_diff
> ,curr_yr.sales_amt-prev_yr.sales_amt AS sales_amt_diff
> FROM all_sales curr_yr, all_sales prev_yr
> WHERE curr_yr.i_brand_id=prev_yr.i_brand_id
> AND curr_yr.i_class_id=prev_yr.i_class_id
> AND curr_yr.i_category_id=prev_yr.i_category_id
> AND curr_yr.i_manufact_id=prev_yr.i_manufact_id
> AND curr_yr.d_year=2002
> AND prev_yr.d_year=2002-1
> AND CAST(curr_yr.sales_cnt AS DECIMAL(17,2))/CAST(prev_yr.sales_cnt AS DECIMAL(17,2))<0.9
> ORDER BY sales_cnt_diff
> limit 100
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)