You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by Eric Payne <er...@yahoo.com.INVALID> on 2017/12/04 20:06:12 UTC
Re: [VOTE] Merge Absolute resource configuration support in
Capacity Scheduler (YARN-5881) to trunk
+1. Thanks Sunil for the work on this branch.
Eric
From: Sunil G <su...@apache.org>
To: "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; Hdfs-dev <hd...@hadoop.apache.org>; Hadoop Common <co...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
Sent: Wednesday, November 29, 2017 7:56 PM
Subject: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk
Hi All,
Based on the discussion at [1], I'd like to start a vote to merge feature
branch
YARN-5881 to trunk. Vote will run for 7 days, ending Wednesday Dec 6 at
6:00PM PDT.
This branch adds support to configure queue capacity as absolute resource in
capacity scheduler. This will help admins who want fine control of
resources of queues.
Feature development is done at YARN-5881 [2], jenkins build is here
(YARN-7510 [3]).
All required tasks for this feature are committed. This feature changes
RM’s Capacity Scheduler only,
and we did extensive tests for the feature in the last couple of months
including performance tests.
Key points:
- The feature is turned off by default, and have to configure absolute
resource to enable same.
- Detailed documentation about how to use this feature is done as part of
[4].
- No major performance degradation is observed with this branch work. SLS
and UT performance
tests are done.
There were 11 subtasks completed for this feature.
Huge thanks to everyone who helped with reviews, commits, guidance, and
technical discussion/design, including Wangda Tan, Vinod Vavilapalli,
Rohith Sharma K S, Eric Payne .
[1] :
http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201711.mbox/%3CCACYiTuhKhF1JCtR7ZFuZSEKQ4sBvN_n_tV5GHsbJ3YeyJP%2BP4Q%40mail.gmail.com%3E
[2] : https://issues.apache.org/jira/browse/YARN-5881
[3] : https://issues.apache.org/jira/browse/YARN-7510
[4] : https://issues.apache.org/jira/browse/YARN-7533
Regards
Sunil and Wangda
Re: [VOTE] Merge Absolute resource configuration support in Capacity
Scheduler (YARN-5881) to trunk
Posted by Weiwei Yang <ch...@hotmail.com>.
+1 (non-binding)
Thanks for getting this done Sunil.
--
Weiwei
On 5 Dec 2017, 4:06 AM +0800, Eric Payne <er...@yahoo.com.invalid>, wrote:
+1. Thanks Sunil for the work on this branch.
Eric
From: Sunil G <sunilg@apache.org
To: "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; Hdfs-dev <hd...@hadoop.apache.org>; Hadoop Common <co...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org
Sent: Wednesday, November 29, 2017 7:56 PM
Subject: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk
Hi All,
Based on the discussion at [1], I'd like to start a vote to merge feature
branch
YARN-5881 to trunk. Vote will run for 7 days, ending Wednesday Dec 6 at
6:00PM PDT.
This branch adds support to configure queue capacity as absolute resource in
capacity scheduler. This will help admins who want fine control of
resources of queues.
Feature development is done at YARN-5881 [2], jenkins build is here
(YARN-7510 [3]).
All required tasks for this feature are committed. This feature changes
RM’s Capacity Scheduler only,
and we did extensive tests for the feature in the last couple of months
including performance tests.
Key points:
- The feature is turned off by default, and have to configure absolute
resource to enable same.
- Detailed documentation about how to use this feature is done as part of
[4].
- No major performance degradation is observed with this branch work. SLS
and UT performance
tests are done.
There were 11 subtasks completed for this feature.
Huge thanks to everyone who helped with reviews, commits, guidance, and
technical discussion/design, including Wangda Tan, Vinod Vavilapalli,
Rohith Sharma K S, Eric Payne .
[1] :
http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201711.mbox/%3CCACYiTuhKhF1JCtR7ZFuZSEKQ4sBvN_n_tV5GHsbJ3YeyJP%2BP4Q%40mail.gmail.com%3E
[2] : https://issues.apache.org/jira/browse/YARN-5881
[3] : https://issues.apache.org/jira/browse/YARN-7510
[4] : https://issues.apache.org/jira/browse/YARN-7533
Regards
Sunil and Wangda
Re: [VOTE] Merge Absolute resource configuration support in Capacity
Scheduler (YARN-5881) to trunk
Posted by Weiwei Yang <ch...@hotmail.com>.
+1 (non-binding)
Thanks for getting this done Sunil.
--
Weiwei
On 5 Dec 2017, 4:06 AM +0800, Eric Payne <er...@yahoo.com.invalid>, wrote:
+1. Thanks Sunil for the work on this branch.
Eric
From: Sunil G <sunilg@apache.org
To: "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; Hdfs-dev <hd...@hadoop.apache.org>; Hadoop Common <co...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org
Sent: Wednesday, November 29, 2017 7:56 PM
Subject: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk
Hi All,
Based on the discussion at [1], I'd like to start a vote to merge feature
branch
YARN-5881 to trunk. Vote will run for 7 days, ending Wednesday Dec 6 at
6:00PM PDT.
This branch adds support to configure queue capacity as absolute resource in
capacity scheduler. This will help admins who want fine control of
resources of queues.
Feature development is done at YARN-5881 [2], jenkins build is here
(YARN-7510 [3]).
All required tasks for this feature are committed. This feature changes
RM’s Capacity Scheduler only,
and we did extensive tests for the feature in the last couple of months
including performance tests.
Key points:
- The feature is turned off by default, and have to configure absolute
resource to enable same.
- Detailed documentation about how to use this feature is done as part of
[4].
- No major performance degradation is observed with this branch work. SLS
and UT performance
tests are done.
There were 11 subtasks completed for this feature.
Huge thanks to everyone who helped with reviews, commits, guidance, and
technical discussion/design, including Wangda Tan, Vinod Vavilapalli,
Rohith Sharma K S, Eric Payne .
[1] :
http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201711.mbox/%3CCACYiTuhKhF1JCtR7ZFuZSEKQ4sBvN_n_tV5GHsbJ3YeyJP%2BP4Q%40mail.gmail.com%3E
[2] : https://issues.apache.org/jira/browse/YARN-5881
[3] : https://issues.apache.org/jira/browse/YARN-7510
[4] : https://issues.apache.org/jira/browse/YARN-7533
Regards
Sunil and Wangda
Re: [VOTE] Merge Absolute resource configuration support in Capacity
Scheduler (YARN-5881) to trunk
Posted by Weiwei Yang <ch...@hotmail.com>.
+1 (non-binding)
Thanks for getting this done Sunil.
--
Weiwei
On 5 Dec 2017, 4:06 AM +0800, Eric Payne <er...@yahoo.com.invalid>, wrote:
+1. Thanks Sunil for the work on this branch.
Eric
From: Sunil G <sunilg@apache.org
To: "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; Hdfs-dev <hd...@hadoop.apache.org>; Hadoop Common <co...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org
Sent: Wednesday, November 29, 2017 7:56 PM
Subject: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk
Hi All,
Based on the discussion at [1], I'd like to start a vote to merge feature
branch
YARN-5881 to trunk. Vote will run for 7 days, ending Wednesday Dec 6 at
6:00PM PDT.
This branch adds support to configure queue capacity as absolute resource in
capacity scheduler. This will help admins who want fine control of
resources of queues.
Feature development is done at YARN-5881 [2], jenkins build is here
(YARN-7510 [3]).
All required tasks for this feature are committed. This feature changes
RM’s Capacity Scheduler only,
and we did extensive tests for the feature in the last couple of months
including performance tests.
Key points:
- The feature is turned off by default, and have to configure absolute
resource to enable same.
- Detailed documentation about how to use this feature is done as part of
[4].
- No major performance degradation is observed with this branch work. SLS
and UT performance
tests are done.
There were 11 subtasks completed for this feature.
Huge thanks to everyone who helped with reviews, commits, guidance, and
technical discussion/design, including Wangda Tan, Vinod Vavilapalli,
Rohith Sharma K S, Eric Payne .
[1] :
http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201711.mbox/%3CCACYiTuhKhF1JCtR7ZFuZSEKQ4sBvN_n_tV5GHsbJ3YeyJP%2BP4Q%40mail.gmail.com%3E
[2] : https://issues.apache.org/jira/browse/YARN-5881
[3] : https://issues.apache.org/jira/browse/YARN-7510
[4] : https://issues.apache.org/jira/browse/YARN-7533
Regards
Sunil and Wangda
Re: [VOTE] Merge Absolute resource configuration support in Capacity
Scheduler (YARN-5881) to trunk
Posted by Weiwei Yang <ch...@hotmail.com>.
+1 (non-binding)
Thanks for getting this done Sunil.
--
Weiwei
On 5 Dec 2017, 4:06 AM +0800, Eric Payne <er...@yahoo.com.invalid>, wrote:
+1. Thanks Sunil for the work on this branch.
Eric
From: Sunil G <sunilg@apache.org
To: "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; Hdfs-dev <hd...@hadoop.apache.org>; Hadoop Common <co...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org
Sent: Wednesday, November 29, 2017 7:56 PM
Subject: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk
Hi All,
Based on the discussion at [1], I'd like to start a vote to merge feature
branch
YARN-5881 to trunk. Vote will run for 7 days, ending Wednesday Dec 6 at
6:00PM PDT.
This branch adds support to configure queue capacity as absolute resource in
capacity scheduler. This will help admins who want fine control of
resources of queues.
Feature development is done at YARN-5881 [2], jenkins build is here
(YARN-7510 [3]).
All required tasks for this feature are committed. This feature changes
RM’s Capacity Scheduler only,
and we did extensive tests for the feature in the last couple of months
including performance tests.
Key points:
- The feature is turned off by default, and have to configure absolute
resource to enable same.
- Detailed documentation about how to use this feature is done as part of
[4].
- No major performance degradation is observed with this branch work. SLS
and UT performance
tests are done.
There were 11 subtasks completed for this feature.
Huge thanks to everyone who helped with reviews, commits, guidance, and
technical discussion/design, including Wangda Tan, Vinod Vavilapalli,
Rohith Sharma K S, Eric Payne .
[1] :
http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201711.mbox/%3CCACYiTuhKhF1JCtR7ZFuZSEKQ4sBvN_n_tV5GHsbJ3YeyJP%2BP4Q%40mail.gmail.com%3E
[2] : https://issues.apache.org/jira/browse/YARN-5881
[3] : https://issues.apache.org/jira/browse/YARN-7510
[4] : https://issues.apache.org/jira/browse/YARN-7533
Regards
Sunil and Wangda