[jira] [Created] (YARN-10164) Allow NM to start even when custom resource type not defined

["Eric Payne (Jira)" <ji...@apache.org>]

Eric Payne created YARN-10164:
---------------------------------

             Summary: Allow NM to start even when custom resource type not defined
                 Key: YARN-10164
                 URL: https://issues.apache.org/jira/browse/YARN-10164
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: nodemanager
            Reporter: Eric Payne
            Assignee: Eric Payne


In the [custom resource documentation|https://hadoop.apache.org/docs/r3.2.1/hadoop-yarn/hadoop-yarn-site/ResourceModel.html], it tells you to add the number of custom resources to a property called {{yarn.nodemanager.resource-type.<resource>}} in a file called {{node-resources.xml}}.

For GPU resources, this would look something like
{code:xml}
  <property>
    <name>yarn.nodemanager.resource-type.gpu</name>
    <value>16</value>
  </property>
{code}

A corresponding config property must also be in {{resource-types.xml}} called yarn.resource-types:
{code:xml}
  <property>
    <name>yarn.resource-types</name>
    <value>gpu</value>
    <description>Custom resources to be used for scheduling. </description>
  </property>
{code}

If the yarn.nodemanager.resource-type.gpu property exists without the corresponding yarn.resource-types property, the nodemanager fails to start.

I would like the option to automatically create the node-resources.xml on all new nodes regardless of whether or not the cluster supports GPU resources so that if I deploy a GPU node into an existing cluster that does not (yet) support GPU resources, the nodemanager will at least start. Even though it doesn't support the GPU resource, the other supported resources will still be available to be used by the apps in the cluster.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org