You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Chengxiang Li (JIRA)" <ji...@apache.org> on 2014/08/11 08:31:12 UTC

[jira] [Created] (HIVE-7675) Implement native HiveMapFunction

Chengxiang Li created HIVE-7675:
-----------------------------------

             Summary: Implement native HiveMapFunction
                 Key: HIVE-7675
                 URL: https://issues.apache.org/jira/browse/HIVE-7675
             Project: Hive
          Issue Type: New Feature
          Components: Spark
            Reporter: Chengxiang Li


Currently, Hive on Spark depend on ExecMapper to execute operator logic, full stack is like: Spark FrameWork=>HiveMapFunction=>ExecMapper=>Hive operators. HiveMapFunction is just a thin wrapper of ExecMapper, this introduce several problems as following:
# ExecMapper is designed for MR single process task mode, it does not work well under Spark multi-thread task node.
# ExecMapper introduce extra API level restriction.

We need implement native HiveMapFunction, as the bridge between Spark framework and Hive operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)