[jira] [Comment Edited] (HIVE-22501) Stats reported multiple times during MR execution for UNION queries

["Daniel Voros (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-22501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16975148#comment-16975148 ] 

Daniel Voros edited comment on HIVE-22501 at 11/15/19 2:46 PM:
---------------------------------------------------------------

Attached patch #1 that only lets operators report their stats once.


was (Author: dvoros):
Attached patch #1 that only let's operators report their stats once.

> Stats reported multiple times during MR execution for UNION queries
> -------------------------------------------------------------------
>
>                 Key: HIVE-22501
>                 URL: https://issues.apache.org/jira/browse/HIVE-22501
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Daniel Voros
>            Assignee: Daniel Voros
>            Priority: Major
>              Labels: mapreduce
>         Attachments: HIVE-22501.1.patch
>
>
> Take the following example:
> {code}
> set hive.execution.engine=mr;
> create table tb(id string) stored as orc;
> insert into tb values('1');
> create table tb2 like tb stored as orc;
> insert into tb2 select * from tb union all select * from tb;
> {code}
> Last insert results in 2 records in the table, but {{TOTAL_TABLE_ROWS_WRITTEN}} statistic (and number of affected rows on the consolse) is 4.
> We seem to traverse the operator graph multiple times starting from every TS operator and increment the counters every time we hit the FS operator. UNION-ing the table 3 times results in 9 TOTAL_TABLE_ROWS_WRITTEN.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)