[jira] [Created] (HIVE-16337) HoS: use separate config for mapjoin hash table size limit rather than hive.auto.convert.join.noconditionaltask.size

["Chao Sun (JIRA)" <ji...@apache.org>]

Chao Sun created HIVE-16337:
-------------------------------

             Summary: HoS: use separate config for mapjoin hash table size limit rather than hive.auto.convert.join.noconditionaltask.size
                 Key: HIVE-16337
                 URL: https://issues.apache.org/jira/browse/HIVE-16337
             Project: Hive
          Issue Type: Improvement
          Components: Spark
            Reporter: Chao Sun


HoS currently share the same config {{hive.auto.convert.join.noconditionaltask.size}} with MR, however it interprets it differently: in MR this is the file size of the input, while in Spark it is the data size. This could cause Spark to be over conservative in many mapjoin optimizations. I suggest we use a separate config for HoS for this purpose.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)