[jira] [Created] (TEZ-4170) RootInputInitializerManager could make use of ThreadPool from appContext

["Rajesh Balamohan (Jira)" <ji...@apache.org>]

Rajesh Balamohan created TEZ-4170:
-------------------------------------

             Summary: RootInputInitializerManager could make use of ThreadPool from appContext
                 Key: TEZ-4170
                 URL: https://issues.apache.org/jira/browse/TEZ-4170
             Project: Apache Tez
          Issue Type: Improvement
            Reporter: Rajesh Balamohan
         Attachments: Screenshot 2020-05-06 at 6.26.34 AM.png

[https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/dag/app/dag/RootInputInitializerManager.java#L106]

 

This could make use of executor from {{appContext}} instead of spinning one for every root input.

 

Also, initialization part of InputInitializer could be moved inside this thread. For e.g, in certain cases like HiveSplitGenerator, it ends up with some heavy operations which can be offloaded from blocking central dispatcher thread (e.g unpacking payloads, running kryo deserialization)

 

!Screenshot 2020-05-06 at 6.26.34 AM.png|width=972,height=740!

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)