You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Ashutosh Chauhan (Jira)" <ji...@apache.org> on 2020/06/23 14:25:00 UTC
[jira] [Commented] (HIVE-23735) Reducer misestimate for export
command
[ https://issues.apache.org/jira/browse/HIVE-23735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17142978#comment-17142978 ]
Ashutosh Chauhan commented on HIVE-23735:
-----------------------------------------
+1
> Reducer misestimate for export command
> --------------------------------------
>
> Key: HIVE-23735
> URL: https://issues.apache.org/jira/browse/HIVE-23735
> Project: Hive
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23735.1.wip.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L6869
> {code}
> if (dest_tab.getNumBuckets() > 0) {
> ...
> }
> {code}
> For "export" command, HS2 creates a dummy table and for this table and gets "1" as the number of buckets.
> {noformat}
> set hive.stats.autogather=false;
> export table sample_table to '/tmp/export/sampe_db/t1';
> {noformat}
> This causes issues in reducer estimates and always lands up with '1' as the number of reducer task.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)