You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Gunther Hagleitner (JIRA)" <ji...@apache.org> on 2015/04/02 23:24:54 UTC

[jira] [Created] (HIVE-10203) Move fragment execution onto a thread pool

Gunther Hagleitner created HIVE-10203:
-----------------------------------------

             Summary: Move fragment execution onto a thread pool
                 Key: HIVE-10203
                 URL: https://issues.apache.org/jira/browse/HIVE-10203
             Project: Hive
          Issue Type: Sub-task
            Reporter: Gunther Hagleitner


Currently we use a lot of thread local magic to make things work in llap. MRLegacyInput makes sure we use the same thread in input and processor. The execution spawns a new thread per fragment and kills it afterwards.

That's being done because of Tez/MapredContext, ExecContext and IOContext. These need to be shared for a single fragment but not across.

Way to clean this up is probably by creating a HiveInput that can handle the config + state sharing more efficiently than generic Tez code.

cc [~sseth] [~gopalv] [~vikram.dixit]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)