You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "xiyang (Jira)" <ji...@apache.org> on 2022/09/27 06:30:00 UTC

[jira] [Updated] (YARN-11319) Yarn After node label is configured, Memory Used cannot be displayed in Web BI

     [ https://issues.apache.org/jira/browse/YARN-11319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

xiyang updated YARN-11319:
--------------------------
    Description: 
*After node label is configured on YARN, Memory Used in Web BI is always 0B, no matter how the program is submitted*

capacity-scheduler.xml
{code:java}
<configuration>
    <property>
        <name>yarn.scheduler.capacity.maximum-applications</name>
        <value>10000</value>
	</property>
    <property>
        <name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
        <value>0.8</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.resource-calculator</name>
        <value>org.apache.hadoop.yarn.util.resource.DominantResourceCalculator</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.queues</name>
        <value>realtime,bi,default</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.node-locality-delay</name>
        <value>40</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.capacity</name>
        <value>1</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
        <value>1</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.state</name>
        <value>RUNNING</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.capacity</name>
        <value>50</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.state</name>
        <value>RUNNING</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.acl_submit_applications</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.acl_administer_queue</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.capacity</name>
        <value>49</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.state</name>
        <value>RUNNING</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.acl_submit_applications</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.acl_administer_queue</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.queue-mappings</name>
        <value></value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
        <value>false</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.accessible-node-labels</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.accessible-node-labels</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.accessible-node-labels</name>
        <value>label02</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.accessible-node-labels</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.accessible-node-labels.label01.capacity</name>
        <value>100</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.accessible-node-labels.label02.capacity</name>
        <value>100</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.capacity</name>
        <value>1</value>
    </property>
	<property>
        <name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.maximum-capacity</name>
        <value>1</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.accessible-node-labels.label01.capacity</name>
        <value>99</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.accessible-node-labels.label02.capacity</name>
        <value>100</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default-node-label-expression</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.default-node-label-expression</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.default-node-label-expression</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.default-node-label-expression</name>
        <value>label02</value>
    </property>
</configuration> {code}
!20220927-142546.jpg!

  was:
*After node label is configured on YARN, Memory Used in Web BI is always 0B, no matter how the program is submitted*

capacity-scheduler.xml
{code:java}
<configuration>
    <property>
        <name>yarn.scheduler.capacity.maximum-applications</name>
        <value>10000</value>
	</property>
    <property>
        <name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
        <value>0.8</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.resource-calculator</name>
        <value>org.apache.hadoop.yarn.util.resource.DominantResourceCalculator</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.queues</name>
        <value>realtime,bi,default</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.node-locality-delay</name>
        <value>40</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.capacity</name>
        <value>1</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
        <value>1</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.state</name>
        <value>RUNNING</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.capacity</name>
        <value>50</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.state</name>
        <value>RUNNING</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.acl_submit_applications</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.acl_administer_queue</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.capacity</name>
        <value>49</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.state</name>
        <value>RUNNING</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.acl_submit_applications</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.acl_administer_queue</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.queue-mappings</name>
        <value></value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
        <value>false</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.accessible-node-labels</name>
        <value>*</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.accessible-node-labels</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.accessible-node-labels</name>
        <value>label02</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.accessible-node-labels</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.accessible-node-labels.label01.capacity</name>
        <value>100</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.accessible-node-labels.label02.capacity</name>
        <value>100</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.capacity</name>
        <value>1</value>
    </property>
	<property>
        <name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.maximum-capacity</name>
        <value>1</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.accessible-node-labels.label01.capacity</name>
        <value>99</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.accessible-node-labels.label02.capacity</name>
        <value>100</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default-node-label-expression</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.default.default-node-label-expression</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.bi.default-node-label-expression</name>
        <value>label01</value>
    </property>
    <property>
        <name>yarn.scheduler.capacity.root.realtime.default-node-label-expression</name>
        <value>label02</value>
    </property>
</configuration> {code}
 


> Yarn After node label is configured, Memory Used cannot be displayed in Web BI
> ------------------------------------------------------------------------------
>
>                 Key: YARN-11319
>                 URL: https://issues.apache.org/jira/browse/YARN-11319
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn-ui-v2
>    Affects Versions: 2.8.5, 3.1.3
>            Reporter: xiyang
>            Priority: Minor
>         Attachments: 20220927-142546.jpg
>
>
> *After node label is configured on YARN, Memory Used in Web BI is always 0B, no matter how the program is submitted*
> capacity-scheduler.xml
> {code:java}
> <configuration>
>     <property>
>         <name>yarn.scheduler.capacity.maximum-applications</name>
>         <value>10000</value>
> 	</property>
>     <property>
>         <name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
>         <value>0.8</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.resource-calculator</name>
>         <value>org.apache.hadoop.yarn.util.resource.DominantResourceCalculator</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.queues</name>
>         <value>realtime,bi,default</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.node-locality-delay</name>
>         <value>40</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.default.capacity</name>
>         <value>1</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
>         <value>1</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.default.state</name>
>         <value>RUNNING</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
>         <value>*</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
>         <value>*</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.realtime.capacity</name>
>         <value>50</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.realtime.state</name>
>         <value>RUNNING</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.realtime.acl_submit_applications</name>
>         <value>*</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.realtime.acl_administer_queue</name>
>         <value>*</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.bi.capacity</name>
>         <value>49</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.bi.state</name>
>         <value>RUNNING</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.bi.acl_submit_applications</name>
>         <value>*</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.bi.acl_administer_queue</name>
>         <value>*</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.queue-mappings</name>
>         <value></value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
>         <value>false</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.accessible-node-labels</name>
>         <value>*</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.default.accessible-node-labels</name>
>         <value>label01</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.realtime.accessible-node-labels</name>
>         <value>label02</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.bi.accessible-node-labels</name>
>         <value>label01</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.accessible-node-labels.label01.capacity</name>
>         <value>100</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.accessible-node-labels.label02.capacity</name>
>         <value>100</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.capacity</name>
>         <value>1</value>
>     </property>
> 	<property>
>         <name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.maximum-capacity</name>
>         <value>1</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.bi.accessible-node-labels.label01.capacity</name>
>         <value>99</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.realtime.accessible-node-labels.label02.capacity</name>
>         <value>100</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.default-node-label-expression</name>
>         <value>label01</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.default.default-node-label-expression</name>
>         <value>label01</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.bi.default-node-label-expression</name>
>         <value>label01</value>
>     </property>
>     <property>
>         <name>yarn.scheduler.capacity.root.realtime.default-node-label-expression</name>
>         <value>label02</value>
>     </property>
> </configuration> {code}
> !20220927-142546.jpg!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org