You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Rich Haase (JIRA)" <ji...@apache.org> on 2015/11/12 21:55:10 UTC
[jira] [Commented] (HIVE-11132) Queries using join and group by
produce incorrect output when hive.auto.convert.join=false and
hive.optimize.reducededuplication=true
[ https://issues.apache.org/jira/browse/HIVE-11132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002911#comment-15002911 ]
Rich Haase commented on HIVE-11132:
-----------------------------------
It's definitely a bug. I would think that back porting would make a lot of sense.
> Queries using join and group by produce incorrect output when hive.auto.convert.join=false and hive.optimize.reducededuplication=true
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-11132
> URL: https://issues.apache.org/jira/browse/HIVE-11132
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
> Affects Versions: 0.14.0, 1.0.0, 1.2.0, 1.1.0
> Reporter: Rich Haase
> Assignee: Ashutosh Chauhan
> Fix For: 2.0.0
>
> Attachments: HIVE-11132.2.patch, HIVE-11132.patch
>
>
> Queries using join and group by produce multiple output rows with the same key when hive.auto.convert.join=false and hive.optimize.reducededuplication=true. This interaction between configuration parameters is unexpected and should be well documented at the very least and should likely be considered a bug.
> e.g.
> hive> set hive.auto.convert.join = false;
> hive> set hive.optimize.reducededuplication = true;
> hive> SELECT foo.id, count(*) as factor
> > FROM foo
> > JOIN bar ON (foo.id = bar.id and foo.line_id = bar.line_id)
> > JOIN split ON (foo.id = split.id and foo.line_id = split.line_id)
> > JOIN forecast ON (foo.id = forecast.id AND foo.line_id = forecast.line_id)
> > WHERE foo.order != ‘blah’ AND foo.id = ‘XYZ'
> > GROUP BY foo.id;
> XYZ 79
> XYZ 74
> XYZ 297
> XYZ 66
> hive> set hive.auto.convert.join = true;
> hive> set hive.optimize.reducededuplication = true;
> hive> SELECT foo.id, count(*) as factor
> > FROM foo
> > JOIN bar ON (foo.id = bar.id and foo.line_id = bar.line_id)
> > JOIN split ON (foo.id = split.id and foo.line_id = split.line_id)
> > JOIN forecast ON (foo.id = forecast.id AND foo.line_id = forecast.line_id)
> > WHERE foo.order != ‘blah’ AND foo.id = ‘XYZ'
> > GROUP BY foo.id;
> XYZ 516
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)