[jira] [Commented] (YARN-2139) Add support for disk IO isolation/scheduling for containers

["Arun C Murthy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199957#comment-14199957 ] 

Arun C Murthy commented on YARN-2139:
-------------------------------------

[~ywskycn] - thanks for the design doc, it's well put together.

Some feedback:

# We shouldn't embed Linux or blkio specific semantics such as {{proportional weight division}} into YARN. We need something generic such as {{bandwidth}} which can be understood by users, supportable on heterogenous nodes in the same cluster and supportable on other platforms like Windows.
# Spindle locality or I/O parallelism is a real concern - we should probably support {{bandwidth}} and {{spindles}}.
# Spindle locality or I/O parallelism cannot be tied to HDFS. In fact, YARN should not have a dependency on HDFS at all (*smile*)! This is particularly important in light of developments like Kafka-on-YARN (KAFKA-1754) because people want to use YARN to deploy only Kafka & Storm etc. YARN-2817 helps in this regard.

Makes sense?

> Add support for disk IO isolation/scheduling for containers
> -----------------------------------------------------------
>
>                 Key: YARN-2139
>                 URL: https://issues.apache.org/jira/browse/YARN-2139
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Wei Yan
>            Assignee: Wei Yan
>         Attachments: Disk_IO_Scheduling_Design_1.pdf, Disk_IO_Scheduling_Design_2.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)