[jira] [Comment Edited] (TAJO-385) Refactoring TaskScheduler to assign multiple fragments

["Keuntae Park (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TAJO-385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842408#comment-13842408 ] 

Keuntae Park edited comment on TAJO-385 at 12/8/13 4:37 AM:
------------------------------------------------------------

+1 for this great idea. I think it is very important to determine the proper number of fragments assigned to one worker at a time and relationship between those fragments.


was (Author: sirpkt):
+1 for this great idea. I think it is very important to determine the number of fragments assigned to one worker at a time and relationship between those fragments.

> Refactoring TaskScheduler to assign multiple fragments
> ------------------------------------------------------
>
>                 Key: TAJO-385
>                 URL: https://issues.apache.org/jira/browse/TAJO-385
>             Project: Tajo
>          Issue Type: Improvement
>          Components: master
>    Affects Versions: 0.8-incubating
>            Reporter: Jihoon Son
>            Assignee: Jihoon Son
>
> In the current implementation, each task processes only one fragment.
> However, processing multiple fragments in a task will increase the query processing performance according to the storage layout and the user queries.
> In this issue, TaskScheduler is refactored to enable assigning multiple fragments to each task.
> Followings should be contained.
> * Schedule Fragments instead of QueryUnits in TaskScheduler
> ** The QueryUnit creation is postponed until TaskScheduler receives task requests from workers.
> ** When TaskScheduler receives task requests from workers, it dynamically creates an QueryUnit and assigns one or more fragments.
> ** The fragment scheduling should take into account the disk load balancing.



--
This message was sent by Atlassian JIRA
(v6.1#6144)