You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Zhihua Deng (Jira)" <ji...@apache.org> on 2023/07/10 07:52:00 UTC
[jira] [Created] (HIVE-27494) Deduplicate the task result that generated by more branches in union all
Zhihua Deng created HIVE-27494:
----------------------------------
Summary: Deduplicate the task result that generated by more branches in union all
Key: HIVE-27494
URL: https://issues.apache.org/jira/browse/HIVE-27494
Project: Hive
Issue Type: Bug
Reporter: Zhihua Deng
Assignee: Zhihua Deng
HIVE-23891 adds the ability to deduplicate the task result that under the directory,
<table-dir>/<staging-dir>/_tmp.-ext-10000/<dynamic-partition-dir>/HIVE_UNION_SUBDIR_1,
but turns out to ignore taking the same action to the directory for the same query:
<table-dir>/<staging-dir>/_tmp.-ext-10000/<dynamic-partition-dir>/HIVE_UNION_SUBDIR_2.
So user may still have the same data duplication problem in multiple tez task attempts.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)