[jira] [Created] (IMPALA-6115) Investigate scheduling options for block metadata loading threads

["bharath v (JIRA)" <ji...@apache.org>]

bharath v created IMPALA-6115:
---------------------------------

             Summary: Investigate scheduling options for block metadata loading threads
                 Key: IMPALA-6115
                 URL: https://issues.apache.org/jira/browse/IMPALA-6115
             Project: IMPALA
          Issue Type: Improvement
    Affects Versions: Impala 2.11.0
            Reporter: bharath v
            Priority: Minor


In the current implementation of IMPALA-5429, we spin up an executor pool of threads every time we load the table and close it once the load has finished/failed. We need to investigate the following options and see if they can give better throughput.

1. Single thread pool for the process lifetime to minimize the startup and teardown latency and all the block metadata loading tasks are submitted to it.
2. Clustered scheduling of threads rather than submitting them all at once.
3. Any other scheduling options that guarantee fairness to all the tables and are more performant.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)