You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hadoop.apache.org by ya...@gmail.com on 2021/08/04 12:48:42 UTC

setting label to node in Distributed mode

Hi,

 

I have a cluster that is defined as 'Distributed" with "config" node-labels
provider.

 

I can't see to find which cli I can use to add a label to an existing node.

 

Any guidance is appreciated.

 

Thanks,

 

Yair

Re: setting label to node in Distributed mode

Posted by Naga Pavan Kumar <na...@gmail.com>.

Add/modify node labels list to YARN

Add cluster node labels list:
Executing yarn rmadmin -addToClusterNodeLabels "label_1(exclusive=true/false),label_2(exclusive=true/false)"to add node label.
If user don’t specify “(exclusive=…)”, exclusive will be true by default.
Run yarn cluster --list-node-labels to check added node labels are visible in the cluster.
Remove node labels from YARN

Remove cluster node labels:
To remove one or more node labels, execute the following command: yarn rmadmin -removeFromClusterNodeLabels "<label>[,<label>,...]". The command argument should be a comma-separated list of node labels to remove.
It is not allowed to remove a label which has been associated with queues, i.e., one or more queues have access to this label.
To verify if specified node labels have been successfully removed, run yarn cluster --list-node-labels.
Add/modify node-to-labels mapping to YARN

Configuring nodes to labels mapping in Centralized NodeLabel setup

Executing yarn rmadmin -replaceLabelsOnNode “node1[:port]=label1 node2=label2” [-failOnUnknownNodes]. Added label1 to node1, label2 to node2. If user don’t specify port, it adds the label to all NodeManagersrunning on the node. If option -failOnUnknownNodes is set, this command will fail if specified nodes are unknown.

Sent from my iPhone

> On 05-Aug-2021, at 6:00 AM, Naganarasimha Garla <na...@apache.org> wrote:
> 
> 
> If that's the scenario then you can either use script based distributed and configure script on the node manager side or set centralised and use the yarn cli to set the attributes 
> 
> 
> Regards,
> + Naga 
> 
> 
> 
> 
>> On Thu, 5 Aug 2021, 4:17 am Yair Ogen, <ya...@gmail.com> wrote:
>> Thanks. My question was more in the sense of updating this dynamically in different periods of time during the cluster life cycle. 
>> 
>> I was looking for some cli or other means of doing this.
>> 
>> Does anything like this exist?
>> From: Naganarasimha Garla <na...@apache.org>
>> Sent: Wednesday, August 4, 2021 10:27:18 PM
>> To: yairogen@gmail.com <ya...@gmail.com>
>> Cc: common-user@hadoop.apache.org <us...@hadoop.apache.org>
>> Subject: Re: setting label to node in Distributed mode
>>  
>> Hi @Yair,
>> 
>> I think documentation is pretty detailed @https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeAttributes.html#Distributed_Node_Attributes_mapping.
>> 
>> yarn.nodemanager.node-attributes.provider.configured-node-attributes needs to be configured for your requirements...
>> 
>> Regards,
>> Naga
>> 
>> On Wed, 4 Aug 2021, 8:49 pm , <ya...@gmail.com> wrote:
>> Hi,
>> 
>>  
>> 
>> I have a cluster that is defined as ‘Distributed“ with “config” node-labels provider.
>> 
>>  
>> 
>> I can’t see to find which cli I can use to add a label to an existing node.
>> 
>>  
>> 
>> Any guidance is appreciated.
>> 
>>  
>> 
>> Thanks,
>> 
>>  
>> 
>> Yair

Re: setting label to node in Distributed mode

Posted by Naganarasimha Garla <na...@apache.org>.

If that's the scenario then you can either use script based distributed and
configure script on the node manager side or set centralised and use the
yarn cli to set the attributes


Regards,
+ Naga




On Thu, 5 Aug 2021, 4:17 am Yair Ogen, <ya...@gmail.com> wrote:

> Thanks. My question was more in the sense of updating this dynamically in
> different periods of time during the cluster life cycle.
>
> I was looking for some cli or other means of doing this.
>
> Does anything like this exist?
> ------------------------------
> *From:* Naganarasimha Garla <na...@apache.org>
> *Sent:* Wednesday, August 4, 2021 10:27:18 PM
> *To:* yairogen@gmail.com <ya...@gmail.com>
> *Cc:* common-user@hadoop.apache.org <us...@hadoop.apache.org>
> *Subject:* Re: setting label to node in Distributed mode
>
> Hi @Yair,
>
> I think documentation is pretty detailed @
> https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeAttributes.html#Distributed_Node_Attributes_mapping
> .
>
> yarn.nodemanager.node-attributes.provider.configured-node-attributes needs
> to be configured for your requirements...
>
> Regards,
> Naga
>
> On Wed, 4 Aug 2021, 8:49 pm , <ya...@gmail.com> wrote:
>
> Hi,
>
>
>
> I have a cluster that is defined as ‘Distributed“ with “config”
> node-labels provider.
>
>
>
> I can’t see to find which cli I can use to add a label to an existing node.
>
>
>
> Any guidance is appreciated.
>
>
>
> Thanks,
>
>
>
> Yair
>
>

Re: setting label to node in Distributed mode

Posted by Yair Ogen <ya...@gmail.com>.

Thanks. My question was more in the sense of updating this dynamically in different periods of time during the cluster life cycle.

I was looking for some cli or other means of doing this.

Does anything like this exist?
________________________________
From: Naganarasimha Garla <na...@apache.org>
Sent: Wednesday, August 4, 2021 10:27:18 PM
To: yairogen@gmail.com <ya...@gmail.com>
Cc: common-user@hadoop.apache.org <us...@hadoop.apache.org>
Subject: Re: setting label to node in Distributed mode

Hi @Yair,

I think documentation is pretty detailed @https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeAttributes.html#Distributed_Node_Attributes_mapping.

yarn.nodemanager.node-attributes.provider.configured-node-attributes needs to be configured for your requirements...

Regards,
Naga

On Wed, 4 Aug 2021, 8:49 pm , <ya...@gmail.com>> wrote:

Hi,

I have a cluster that is defined as ‘Distributed“ with “config” node-labels provider.

I can’t see to find which cli I can use to add a label to an existing node.

Any guidance is appreciated.

Thanks,

Yair

Re: setting label to node in Distributed mode

Posted by Naganarasimha Garla <na...@apache.org>.

Hi @Yair,

I think documentation is pretty detailed @
https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeAttributes.html#Distributed_Node_Attributes_mapping
.

yarn.nodemanager.node-attributes.provider.configured-node-attributes needs
to be configured for your requirements...

Regards,
Naga

On Wed, 4 Aug 2021, 8:49 pm , <ya...@gmail.com> wrote:

> Hi,
>
>
>
> I have a cluster that is defined as ‘Distributed“ with “config”
> node-labels provider.
>
>
>
> I can’t see to find which cli I can use to add a label to an existing node.
>
>
>
> Any guidance is appreciated.
>
>
>
> Thanks,
>
>
>
> Yair
>