You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Chao Sun (JIRA)" <ji...@apache.org> on 2017/04/19 21:12:41 UTC

[jira] [Created] (HIVE-16483) HoS should populate split related configurations to HiveConf

Chao Sun created HIVE-16483:
-------------------------------

             Summary: HoS should populate split related configurations to HiveConf
                 Key: HIVE-16483
                 URL: https://issues.apache.org/jira/browse/HIVE-16483
             Project: Hive
          Issue Type: Bug
          Components: Spark
            Reporter: Chao Sun
            Assignee: Chao Sun


There are several split related configurations, such as {{MAPREDMINSPLITSIZE}}, {{MAPREDMINSPLITSIZEPERNODE}}, {{MAPREDMINSPLITSIZEPERRACK}}, etc., that should be populated to HiveConf. Currently we only do this for {{MAPREDMINSPLITSIZE}}.

Without these, Spark sometimes will not merge small files for file formats such as text.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)