You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "zhaolong (Jira)" <ji...@apache.org> on 2021/09/15 02:50:00 UTC
[jira] [Commented] (HIVE-25523) data loss when none bucket table
convert to bucket map join in tez
[ https://issues.apache.org/jira/browse/HIVE-25523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17415296#comment-17415296 ]
zhaolong commented on HIVE-25523:
---------------------------------
[~djaiswal] Does HIVE-20187 consider this sitution?
> data loss when none bucket table convert to bucket map join in tez
> ------------------------------------------------------------------
>
> Key: HIVE-25523
> URL: https://issues.apache.org/jira/browse/HIVE-25523
> Project: Hive
> Issue Type: Bug
> Affects Versions: 3.1.0
> Reporter: zhaolong
> Priority: Major
> Attachments: 000000_0, image-2021-09-15-10-44-43-366.png, image-2021-09-15-10-46-57-668.png
>
>
> create table wtest (equi_type varchar(765), ym string, trade_typ_code string, cons_no string, t_settle_pq decimal(38,0), fl string)
> row format delimited fields terminated by ',' stored as textfile ;
> create table wtest2 (equi_type varchar(765), ym string, trade_typ_code string, cons_no string, t_settle_pq decimal(38,0), fl string)
> row format delimited fields terminated by ',' stored as textfile ;
> load data local inpath '/opt/000000_0' into table wtest;
> load data local inpath '/opt/000000_0' into table wtest2;
> set hive.execution.engine=tez;
> select x1.equi_type, x1.ym, x1.fl, x1.z,x2.fl
> from
> (
> select equi_type,ym,count(distinct cons_no) z,fl
> from wtest
> group by equi_type,ym,fl
> ) x1,
> (
> select equi_type,ym,count(distinct cons_no) z,fl
> from wtest2
> group by equi_type,ym,fl
> ) x2
> where
> x1.equi_type = x2.equi_type and x1.fl = x2.fl and x1.ym = x2.ym;
>
> get wrong result:
> !image-2021-09-15-10-44-43-366.png!
>
> when disable mapjoin or set hive.convert.join.bucket.mapjoin.tez=false, result is correct.
> !image-2021-09-15-10-46-57-668.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)