You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Zhihua Deng (Jira)" <ji...@apache.org> on 2023/07/10 07:52:00 UTC

[jira] [Created] (HIVE-27494) Deduplicate the task result that generated by more branches in union all

Zhihua Deng created HIVE-27494:
----------------------------------

             Summary: Deduplicate the task result that generated by more branches in union all
                 Key: HIVE-27494
                 URL: https://issues.apache.org/jira/browse/HIVE-27494
             Project: Hive
          Issue Type: Bug
            Reporter: Zhihua Deng
            Assignee: Zhihua Deng


HIVE-23891 adds the ability to deduplicate the task result that under the directory,

<table-dir>/<staging-dir>/_tmp.-ext-10000/<dynamic-partition-dir>/HIVE_UNION_SUBDIR_1,

but turns out to ignore taking the same action to the directory for the same query:

<table-dir>/<staging-dir>/_tmp.-ext-10000/<dynamic-partition-dir>/HIVE_UNION_SUBDIR_2.

So user may still have the same data duplication problem in multiple tez task attempts.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)