[jira] [Comment Edited] (YARN-2786) Create yarn cluster CLI to enable list node labels collection

["Allen Wittenauer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208300#comment-14208300 ] 

Allen Wittenauer edited comment on YARN-2786 at 11/12/14 5:40 PM:
------------------------------------------------------------------

bq. By the time I started on HOD, those experiments were presumably over, so I never heard about them.

I'm not surprised... otherwise you'd realize what a mess having a state file for this info causes. Every single time a node gets added, every single time a node gets modified, every single time a node gets deleted, this state file is going to have to be verified.  This is in addition to the already existing includex2, excludex2, and slaves files.

bq.Further, you don't want a central list of valid labels which I disagree with. 

The part that no one has had a reasonable explanation for is where exactly these invalid labels are coming from.  If the operations teams are the ones setting up the cluster, shouldn't they know what labels they want to use?  The only conclusion I can come up with is Ambari or Cloudera Manager.  So, yes, I can see why people would need protection against them.

bq. Others like me can turn it on to avoid random-labels-bubbling-up-in-my-cluster madness.

You seem to be confusing 'random' and 'ephemeral'.  

The whole reason I'm against adding this change is that it just makes the state file situation that much worse.  Now instead of being able to control the labels either via code or configuration management, we're going to trust humans to type things in at the command line. Essentially: adding this feature *creates* the invalid label problem you are hoping to avoid!  

So in summary, again, I'm completely and totally against the ability to add labels from the command line.  It's a big flaw in an implementation full of flaws.


was (Author: aw):
bq. By the time I started on HOD, those experiments were presumably over, so I never heard about them.

I'm not surprised... otherwise you'd realize what a mess having a state file for this info causes. Every single time a node gets added, every single time a node gets modified, every single time a node gets deleted, this state file is going to have to verified.  This is in addition to the already existing includex2, excludex2, and slaves files.

bq.Further, you don't want a central list of valid labels which I disagree with. 

The part that no one has had a reasonable explanation for is where exactly these invalid labels are coming from.  If the operations teams are the ones setting up the cluster, shouldn't they know what labels they want to use?  The only conclusion I can come up with is Ambari or Cloudera Manager.  So, yes, I can see why people would need protection against them.

bq. Others like me can turn it on to avoid random-labels-bubbling-up-in-my-cluster madness.

You seem to be confusing 'random' and 'ephemeral'.  

The whole reason I'm against adding this change is that it just makes the state file situation that much worse.  Now instead of being able to control the labels either via code or configuration management, we're going to trust humans to type things in at the command line. Essentially: adding this feature *creates* the invalid label problem you are hoping to avoid!  

So in summary, again, I'm completely and totally against the ability to add labels from the command line.  It's a big flaw in an implementation full of flaws.

> Create yarn cluster CLI to enable list node labels collection
> -------------------------------------------------------------
>
>                 Key: YARN-2786
>                 URL: https://issues.apache.org/jira/browse/YARN-2786
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, client, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-2786-20141031-1.patch, YARN-2786-20141031-2.patch, YARN-2786-20141102-2.patch, YARN-2786-20141102-3.patch, YARN-2786-20141103-1-full.patch, YARN-2786-20141103-1-without-yarn.cmd.patch, YARN-2786-20141104-1-full.patch, YARN-2786-20141104-1-without-yarn.cmd.patch, YARN-2786-20141104-2-full.patch, YARN-2786-20141104-2-without-yarn.cmd.patch
>
>
> With YARN-2778, we can list node labels on existing RM nodes. But it is not enough, we should be able to: 
> 1) list node labels collection
> The command should start with "yarn cluster ...", in the future, we can add more functionality to the "yarnClusterCLI"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)