You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Dong Ye <ye...@gmail.com> on 2023/01/04 07:48:20 UTC

consistency of yarn exclude file

Hi, All:

        For resource manager, can we set
yarn.resourcemanager.nodes.exclude-path to a s3 file, so all 3 resource
managers can access it. The benefit is that there is no need to sync the
exclude.xml file. If not, how to sync the file on different HA resource
managers?

Thanks.

Ref :
https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html

Re: consistency of yarn exclude file

Posted by Chris Nauroth <cn...@apache.org>.
Yes, I expect that will work (for both
yarn.resourcemanager.nodes.exclude-path and
yarn.resourcemanager.nodes.include-path), using the "s3a://..." scheme to
specify a file in an S3 bucket.

Chris Nauroth


On Tue, Jan 3, 2023 at 11:50 PM Dong Ye <ye...@gmail.com> wrote:

> Hi, All:
>
>         For resource manager, can we set
> yarn.resourcemanager.nodes.exclude-path to a s3 file, so all 3 resource
> managers can access it. The benefit is that there is no need to sync the
> exclude.xml file. If not, how to sync the file on different HA resource
> managers?
>
> Thanks.
>
> Ref :
> https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html
>

Re: consistency of yarn exclude file

Posted by Vinod Kumar Vavilapalli <vi...@cloudera.com.INVALID>.
You can do this by pushing the same file to all Resource Managers at the same time.

This is either done by (1) admins / ops via something like scp / rsync with the source file in something like git, or (b) by an installer application that keeps the source in a DB and pushes to all the nodes.

Thanks
+Vinod 

> On 04-Jan-2023, at 1:18 PM, Dong Ye <ye...@gmail.com> wrote:
> 
> Hi, All:
> 
>        For resource manager, can we set yarn.resourcemanager.nodes.exclude-path to a s3 file, so all 3 resource managers can access it. The benefit is that there is no need to sync the exclude.xml file. If not, how to sync the file on different HA resource managers?
> 
> Thanks.
> 
> Ref : https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org