You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Charles Chen (JIRA)" <ji...@apache.org> on 2011/08/17 02:57:27 UTC
[jira] [Created] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
-------------------------------------------------------------------------------------------------------------------
Key: HIVE-2382
URL: https://issues.apache.org/jira/browse/HIVE-2382
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.6.0
Reporter: Charles Chen
Assignee: Charles Chen
Priority: Critical
When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
Here is a simple case of this going wrong: Using
{noformat}
create table invites (id int, foo int, bar int)
{noformat}
executing the query
{noformat}
explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
{noformat}
results in
{noformat}
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
a-subquery1:b-subquery1:c
TableScan
alias: c
Filter Operator
predicate:
expr: (foo = 1)
type: boolean
Select Operator
expressions:
expr: bar
type: int
expr: foo
type: int
outputColumnNames: _col0, _col1
Union
Select Operator
expressions:
expr: _col1
type: int
expr: _col0
type: int
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
outputColumnNames: _col0, _col1
Group By Operator
bucketGroup: false
keys:
expr: _col1
type: int
expr: _col0
type: int
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: int
expr: _col1
type: int
sort order: ++
Map-reduce partition columns:
expr: _col0
type: int
expr: _col1
type: int
tag: -1
a-subquery2:b-subquery2:d
TableScan
alias: d
Filter Operator
predicate:
expr: (foo = 1)
type: boolean
Select Operator
expressions:
expr: bar
type: int
expr: foo
type: int
outputColumnNames: _col0, _col1
Union
Select Operator
expressions:
expr: _col1
type: int
expr: _col0
type: int
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
outputColumnNames: _col0, _col1
Group By Operator
bucketGroup: false
keys:
expr: _col1
type: int
expr: _col0
type: int
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: int
expr: _col1
type: int
sort order: ++
Map-reduce partition columns:
expr: _col0
type: int
expr: _col1
type: int
tag: -1
Reduce Operator Tree:
Group By Operator
bucketGroup: false
keys:
expr: KEY._col0
type: int
expr: KEY._col1
type: int
mode: mergepartial
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
outputColumnNames: _col0, _col1
File Output Operator
compressed: false
GlobalTableId: 0
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Stage: Stage-0
Fetch Operator
limit: -1
{noformat}
Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1".
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087264#comment-13087264 ]
jiraposter@reviews.apache.org commented on HIVE-2382:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1566/
-----------------------------------------------------------
(Updated 2011-08-18 20:29:54.207958)
Review request for hive.
Changes
-------
Unit tests passed
Summary
-------
https://issues.apache.org/jira/browse/HIVE-2382
This addresses bug HIVE-2382.
https://issues.apache.org/jira/browse/HIVE-2382
Diffs
-----
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1157990
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/groupby_ppd.q PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_ppd.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1566/diff
Testing (updated)
-------
Unit tests passed
Thanks,
Charles
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2382v1.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2382:
---------------------------------
Status: Patch Available (was: Open)
@Charles: Ignore my previous comment -- sorry.
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2382v1.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2382:
---------------------------------
Status: Open (was: Patch Available)
@Charles: Please make sure the patch applies cleanly with 'patch -p0'. Thanks!
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2382v1.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094343#comment-13094343 ]
Hudson commented on HIVE-2382:
------------------------------
Integrated in Hive-trunk-h0.21 #925 (See [https://builds.apache.org/job/Hive-trunk-h0.21/925/])
HIVE-2382. Invalid predicate pushdown from incorrect column expression
map for select operator generated by GROUP BY operation
(Charles Chen via jvs)
jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1163437
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
* /hive/trunk/ql/src/test/queries/clientpositive/groupby_ppd.q
* /hive/trunk/ql/src/test/results/clientpositive/groupby_ppd.q.out
* /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2382v1.patch, HIVE-2382v2.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "Charles Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Charles Chen updated HIVE-2382:
-------------------------------
Attachment: HIVE-2382v2.patch
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2382v1.patch, HIVE-2382v2.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "Charles Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Charles Chen updated HIVE-2382:
-------------------------------
Description:
When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
Here is a simple case of this going wrong: Using
{noformat}
create table invites (id int, foo int, bar int);
{noformat}
executing the query
{noformat}
explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
{noformat}
results in
{noformat}
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
a-subquery1:b-subquery1:c
TableScan
alias: c
Filter Operator
predicate:
expr: (foo = 1)
type: boolean
Select Operator
expressions:
expr: bar
type: int
expr: foo
type: int
outputColumnNames: _col0, _col1
Union
Select Operator
expressions:
expr: _col1
type: int
expr: _col0
type: int
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
outputColumnNames: _col0, _col1
Group By Operator
bucketGroup: false
keys:
expr: _col1
type: int
expr: _col0
type: int
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: int
expr: _col1
type: int
sort order: ++
Map-reduce partition columns:
expr: _col0
type: int
expr: _col1
type: int
tag: -1
a-subquery2:b-subquery2:d
TableScan
alias: d
Filter Operator
predicate:
expr: (foo = 1)
type: boolean
Select Operator
expressions:
expr: bar
type: int
expr: foo
type: int
outputColumnNames: _col0, _col1
Union
Select Operator
expressions:
expr: _col1
type: int
expr: _col0
type: int
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
outputColumnNames: _col0, _col1
Group By Operator
bucketGroup: false
keys:
expr: _col1
type: int
expr: _col0
type: int
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: int
expr: _col1
type: int
sort order: ++
Map-reduce partition columns:
expr: _col0
type: int
expr: _col1
type: int
tag: -1
Reduce Operator Tree:
Group By Operator
bucketGroup: false
keys:
expr: KEY._col0
type: int
expr: KEY._col1
type: int
mode: mergepartial
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
outputColumnNames: _col0, _col1
File Output Operator
compressed: false
GlobalTableId: 0
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Stage: Stage-0
Fetch Operator
limit: -1
{noformat}
Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
was:
When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
Here is a simple case of this going wrong: Using
{noformat}
create table invites (id int, foo int, bar int)
{noformat}
executing the query
{noformat}
explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
{noformat}
results in
{noformat}
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
a-subquery1:b-subquery1:c
TableScan
alias: c
Filter Operator
predicate:
expr: (foo = 1)
type: boolean
Select Operator
expressions:
expr: bar
type: int
expr: foo
type: int
outputColumnNames: _col0, _col1
Union
Select Operator
expressions:
expr: _col1
type: int
expr: _col0
type: int
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
outputColumnNames: _col0, _col1
Group By Operator
bucketGroup: false
keys:
expr: _col1
type: int
expr: _col0
type: int
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: int
expr: _col1
type: int
sort order: ++
Map-reduce partition columns:
expr: _col0
type: int
expr: _col1
type: int
tag: -1
a-subquery2:b-subquery2:d
TableScan
alias: d
Filter Operator
predicate:
expr: (foo = 1)
type: boolean
Select Operator
expressions:
expr: bar
type: int
expr: foo
type: int
outputColumnNames: _col0, _col1
Union
Select Operator
expressions:
expr: _col1
type: int
expr: _col0
type: int
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
outputColumnNames: _col0, _col1
Group By Operator
bucketGroup: false
keys:
expr: _col1
type: int
expr: _col0
type: int
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: int
expr: _col1
type: int
sort order: ++
Map-reduce partition columns:
expr: _col0
type: int
expr: _col1
type: int
tag: -1
Reduce Operator Tree:
Group By Operator
bucketGroup: false
keys:
expr: KEY._col0
type: int
expr: KEY._col1
type: int
mode: mergepartial
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
outputColumnNames: _col0, _col1
File Output Operator
compressed: false
GlobalTableId: 0
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Stage: Stage-0
Fetch Operator
limit: -1
{noformat}
Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1".
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "Charles Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Charles Chen updated HIVE-2382:
-------------------------------
Attachment: HIVE-2382v1.patch
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Attachments: HIVE-2382v1.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2382:
-----------------------------
Resolution: Fixed
Fix Version/s: (was: 0.8.0)
0.9.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Charles!
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2382v1.patch, HIVE-2382v2.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094102#comment-13094102 ]
jiraposter@reviews.apache.org commented on HIVE-2382:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1566/
-----------------------------------------------------------
(Updated 2011-08-30 21:07:18.651251)
Review request for hive.
Changes
-------
TestParse test output different
Summary
-------
https://issues.apache.org/jira/browse/HIVE-2382
This addresses bug HIVE-2382.
https://issues.apache.org/jira/browse/HIVE-2382
Diffs (updated)
-----
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1163362
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/groupby_ppd.q PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_ppd.q.out PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml 1163362
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml 1163362
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml 1163362
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml 1163362
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml 1163362
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml 1163362
Diff: https://reviews.apache.org/r/1566/diff
Testing
-------
Unit tests passed
Thanks,
Charles
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2382v1.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "Charles Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Charles Chen updated HIVE-2382:
-------------------------------
Fix Version/s: 0.8.0
Status: Patch Available (was: Open)
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2382v1.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093366#comment-13093366 ]
John Sichi commented on HIVE-2382:
----------------------------------
+1. Will commit when tests pass.
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2382v1.patch
>
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2382) Invalid predicate pushdown from
incorrect column expression map for select operator generated by GROUP BY
operation
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086525#comment-13086525 ]
jiraposter@reviews.apache.org commented on HIVE-2382:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1566/
-----------------------------------------------------------
Review request for hive.
Summary
-------
https://issues.apache.org/jira/browse/HIVE-2382
This addresses bug HIVE-2382.
https://issues.apache.org/jira/browse/HIVE-2382
Diffs
-----
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1157990
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/groupby_ppd.q PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_ppd.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1566/diff
Testing
-------
Thanks,
Charles
> Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2382
> URL: https://issues.apache.org/jira/browse/HIVE-2382
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Critical
>
> When a GROUP BY is specified, a select operator is added before the GROUP BY in SemanticAnalyzer.insertSelectAllPlanForGroupBy. Currently, the column expression map for this is set to the column expression map for the parent operator. This behavior is incorrect as, for example, the parent operator could rearrange the order of the columns (_col0 => _col0, _col1 => _col2, _col2 => _col1) and the new operator should not repeat this.
> The predicate pushdown optimization uses the column expression map to track which columns a filter expression refers to at different operators. This results in a filter on incorrect columns.
> Here is a simple case of this going wrong: Using
> {noformat}
> create table invites (id int, foo int, bar int);
> {noformat}
> executing the query
> {noformat}
> explain select * from (select foo, bar from (select bar, foo from invites c union all select bar, foo from invites d) b) a group by bar, foo having bar=1;
> {noformat}
> results in
> {noformat}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> a-subquery1:b-subquery1:c
> TableScan
> alias: c
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> a-subquery2:b-subquery2:d
> TableScan
> alias: d
> Filter Operator
> predicate:
> expr: (foo = 1)
> type: boolean
> Select Operator
> expressions:
> expr: bar
> type: int
> expr: foo
> type: int
> outputColumnNames: _col0, _col1
> Union
> Select Operator
> expressions:
> expr: _col1
> type: int
> expr: _col0
> type: int
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> Group By Operator
> bucketGroup: false
> keys:
> expr: _col1
> type: int
> expr: _col0
> type: int
> mode: hash
> outputColumnNames: _col0, _col1
> Reduce Output Operator
> key expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> sort order: ++
> Map-reduce partition columns:
> expr: _col0
> type: int
> expr: _col1
> type: int
> tag: -1
> Reduce Operator Tree:
> Group By Operator
> bucketGroup: false
> keys:
> expr: KEY._col0
> type: int
> expr: KEY._col1
> type: int
> mode: mergepartial
> outputColumnNames: _col0, _col1
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-0
> Fetch Operator
> limit: -1
> {noformat}
> Note that the filter is now "foo = 1", while the correct behavior is to have "bar = 1". If we remove the group by, the behavior is correct.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira