[jira] [Created] (TAJO-982) Improve Fetcher to get multiple shuffle outputs through a request

["Hyunsik Choi (JIRA)" <ji...@apache.org>]

Hyunsik Choi created TAJO-982:
---------------------------------

             Summary: Improve Fetcher to get multiple shuffle outputs through a request
                 Key: TAJO-982
                 URL: https://issues.apache.org/jira/browse/TAJO-982
             Project: Tajo
          Issue Type: Bug
          Components: data shuffle
            Reporter: Hyunsik Choi
             Fix For: 0.9.0


Currently, Fetcher only can request at most a fetch for one shuffle output at a time. The implementation can cause performance degradation even though intermediate data is actually small.

For example, If an input data set of the first stage is big and the intermediate data is very small, QueryMaster will choose a few of nodes for next execution block. Since each node keeps limited threads for fetch, it will take a long time for the nodes for next stage to fetch all intermediate.

If Fetcher can get multiple shuffle outputs through a request, it would solve the slowness which occurs in some cases.



--
This message was sent by Atlassian JIRA
(v6.2#6252)