[jira] [Assigned] (TAJO-982) Improve Fetcher to get multiple shuffle outputs through a request

["Hyoungjun Kim (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TAJO-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyoungjun Kim reassigned TAJO-982:
----------------------------------

    Assignee: Hyoungjun Kim

> Improve Fetcher to get multiple shuffle outputs through a request
> -----------------------------------------------------------------
>
>                 Key: TAJO-982
>                 URL: https://issues.apache.org/jira/browse/TAJO-982
>             Project: Tajo
>          Issue Type: Bug
>          Components: data shuffle
>            Reporter: Hyunsik Choi
>            Assignee: Hyoungjun Kim
>             Fix For: 0.9.0
>
>
> Currently, Fetcher only can request at most a fetch for one shuffle output at a time. The implementation can cause performance degradation even though intermediate data is actually small.
> For example, If an input data set of the first stage is big and the intermediate data is very small, QueryMaster will choose a few of nodes for next execution block. Since each node keeps limited threads for fetch, it will take a long time for the nodes for next stage to fetch all intermediate.
> If Fetcher can get multiple shuffle outputs through a request, it would solve the slowness which occurs in some cases.



--
This message was sent by Atlassian JIRA
(v6.2#6252)