[jira] [Updated] (HIVE-25523) data loss when convert to bucket map join in tez

["zhaolong (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

zhaolong updated HIVE-25523:
----------------------------
    Summary: data loss when convert to bucket map join in tez  (was: data loss when none bucket table convert to bucket map join in tez)

> data loss when convert to bucket map join in tez
> ------------------------------------------------
>
>                 Key: HIVE-25523
>                 URL: https://issues.apache.org/jira/browse/HIVE-25523
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.1.0
>            Reporter: zhaolong
>            Priority: Major
>         Attachments: 000000_0, image-2021-09-15-10-44-43-366.png, image-2021-09-15-10-46-57-668.png, image-2021-09-15-11-04-46-834.png
>
>
> create table wtest (equi_type varchar(765), ym string, trade_typ_code string, cons_no string, t_settle_pq decimal(38,0), fl string)
>  row format delimited fields terminated by ',' stored as textfile ;
> create table wtest2 (equi_type varchar(765), ym string, trade_typ_code string, cons_no string, t_settle_pq decimal(38,0), fl string)
>  row format delimited fields terminated by ',' stored as textfile ;
> load data local inpath '/opt/000000_0' into table wtest;
> load data local inpath '/opt/000000_0' into table wtest2;
> set hive.execution.engine=tez;
> select x1.equi_type, x1.ym, x1.fl, x1.z,x2.fl
>  from
>  (
>  select equi_type,ym,count(distinct cons_no) z,fl
>  from wtest 
>  group by equi_type,ym,fl
>  ) x1,
>  (
>  select equi_type,ym,count(distinct cons_no) z,fl
>  from wtest2 
>  group by equi_type,ym,fl
>  ) x2
>  where
>  x1.equi_type = x2.equi_type and x1.fl = x2.fl and x1.ym = x2.ym;
>  
> get wrong result:
> !image-2021-09-15-10-44-43-366.png!
>  
> when disable mapjoin or set hive.convert.join.bucket.mapjoin.tez=false, result is correct.
> !image-2021-09-15-10-46-57-668.png!
>  
> Change wtest and wtest2 to bucket table also has this problem.
> !image-2021-09-15-11-04-46-834.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)