Re: [Discuss] Merge discussion for Node attribute support feature YARN-3409

[Weiwei Yang <ab...@gmail.com>]

Hi Naga

+1 for the merge. We need it for 3.2.0.
This brings a lot of value when it works with placement constraints APIs,
it will be very important for long-running jobs and service type tasks.

Thanks!
--
Weiwei

On Wed, Aug 29, 2018 at 10:30 PM Naganarasimha Garla <
naganarasimha_gr@apache.org> wrote:

> Hi All,
>
> We would like to hear your thoughts on merging “Node Attributes Support in
> YARN” branch (YARN-3409) [2] into trunk in a few weeks. The goal is to get
> it in for HADOOP 3.2.
>
> *Major work happened in this branch*
>
> YARN-6858. Attribute Manager to store and provide node attributes in RM
> YARN-7871. Support Node attributes reporting from NM to RM( distributed
> node attributes)
> YARN-7863. Modify placement constraints to support node attributes
> YARN-7875. Node Attribute store for storing and recovering attributes
>
> *Detailed Design:*
>
> Please refer [1] for detailed design document.
>
> *Testing Efforts:*
>
> We did detailed tests for the feature in the last few weeks.
> This feature will be enabled only when Node Attributes constraints are
> specified through SchedulingRequest from AM.
> Manager implementation will help to store and recover Node Attributes. This
> works with existing placement constraints.
>
> *Regarding to API stability:*
>
> All newly added @Public APIs are @Unstable.
>
> Documentation jira [3] could help to provide detailed configuration
> details. This feature works from end-to-end and we tested this in our local
> cluster. Branch code is run against trunk and tracked via [4].
>
> We would love to get your thoughts before opening a voting thread.
>
> Special thanks to a team of folks who worked hard and contributed towards
> this efforts including design discussion / patch / reviews, etc.: Weiwei
> Yang, Bibin Chundatt, Wangda Tan, Vinod Kumar Vavilappali, Konstantinos
> Karanasos, Arun Suresh, Varun Saxena, Devaraj Kavali, Lei Guo, Chong Chen.
>
> [1] :
>
> https://issues.apache.org/jira/secure/attachment/12937633/Node-Attributes-Requirements-Design-doc_v2.pdf
> [2] : https://issues.apache.org/jira/browse/YARN-3409
> [3] : https://issues.apache.org/jira/browse/YARN-7865
> [4] : https://issues.apache.org/jira/browse/YARN-8718
>
> Thanks,
> + Naga & Sunil Govindan
>


-- 
Weiwei Yang