[jira] [Commented] (TAJO-367) Separate the locality information from Fragment

["Jihoon Son (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TAJO-367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838623#comment-13838623 ] 

Jihoon Son commented on TAJO-367:
---------------------------------

This issue can be solved after resolving TAJO-337.

> Separate the locality information from Fragment
> -----------------------------------------------
>
>                 Key: TAJO-367
>                 URL: https://issues.apache.org/jira/browse/TAJO-367
>             Project: Tajo
>          Issue Type: Improvement
>          Components: master, storage, worker
>            Reporter: Jihoon Son
>             Fix For: 0.8-incubating
>
>
> Fragment is designed to represent a portion of the abstracted input source.
> However, since It is currently used for the task scheduling and the task allocation, it includes the locality information as well as the abstraction of the input data.
> The locality information is used only in the task scheduling, and thus the locality information should be separated from Fragment.
> The locality information is used in the task scheduling to assign tasks to workers closes to the data regardless of the kind of the storage layer.
> To consider input data and their locality in the task scheduler, we need to design a new class including a Fragment and the locality information such as FragmentWithHost.
> In this issue, following works should be resolved.
> * Removing the host information from FileFragment
> * Creating a new class FragmentWithHost that contains an instance of the Fragment interface and the locality information consisting of hosts and disk ids
> * Refactoring SubQuery, StorageManager and TaskScheduler to use FragmentWithHost



--
This message was sent by Atlassian JIRA
(v6.1#6144)