You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Dong Ye <ye...@gmail.com> on 2023/01/04 07:48:20 UTC
consistency of yarn exclude file
Hi, All:
For resource manager, can we set
yarn.resourcemanager.nodes.exclude-path to a s3 file, so all 3 resource
managers can access it. The benefit is that there is no need to sync the
exclude.xml file. If not, how to sync the file on different HA resource
managers?
Thanks.
Ref :
https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html
Re: consistency of yarn exclude file
Posted by Chris Nauroth <cn...@apache.org>.
Yes, I expect that will work (for both
yarn.resourcemanager.nodes.exclude-path and
yarn.resourcemanager.nodes.include-path), using the "s3a://..." scheme to
specify a file in an S3 bucket.
Chris Nauroth
On Tue, Jan 3, 2023 at 11:50 PM Dong Ye <ye...@gmail.com> wrote:
> Hi, All:
>
> For resource manager, can we set
> yarn.resourcemanager.nodes.exclude-path to a s3 file, so all 3 resource
> managers can access it. The benefit is that there is no need to sync the
> exclude.xml file. If not, how to sync the file on different HA resource
> managers?
>
> Thanks.
>
> Ref :
> https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html
>
Re: consistency of yarn exclude file
Posted by Vinod Kumar Vavilapalli <vi...@cloudera.com.INVALID>.
You can do this by pushing the same file to all Resource Managers at the same time.
This is either done by (1) admins / ops via something like scp / rsync with the source file in something like git, or (b) by an installer application that keeps the source in a DB and pushes to all the nodes.
Thanks
+Vinod
> On 04-Jan-2023, at 1:18 PM, Dong Ye <ye...@gmail.com> wrote:
>
> Hi, All:
>
> For resource manager, can we set yarn.resourcemanager.nodes.exclude-path to a s3 file, so all 3 resource managers can access it. The benefit is that there is no need to sync the exclude.xml file. If not, how to sync the file on different HA resource managers?
>
> Thanks.
>
> Ref : https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org