You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "xiyang (Jira)" <ji...@apache.org> on 2022/09/27 06:30:00 UTC
[jira] [Updated] (YARN-11319) Yarn After node label is configured, Memory Used cannot be displayed in Web BI
[ https://issues.apache.org/jira/browse/YARN-11319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xiyang updated YARN-11319:
--------------------------
Description:
*After node label is configured on YARN, Memory Used in Web BI is always 0B, no matter how the program is submitted*
capacity-scheduler.xml
{code:java}
<configuration>
<property>
<name>yarn.scheduler.capacity.maximum-applications</name>
<value>10000</value>
</property>
<property>
<name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
<value>0.8</value>
</property>
<property>
<name>yarn.scheduler.capacity.resource-calculator</name>
<value>org.apache.hadoop.yarn.util.resource.DominantResourceCalculator</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>realtime,bi,default</value>
</property>
<property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.state</name>
<value>RUNNING</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.capacity</name>
<value>50</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.state</name>
<value>RUNNING</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.acl_submit_applications</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.acl_administer_queue</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.capacity</name>
<value>49</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.state</name>
<value>RUNNING</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.acl_submit_applications</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.acl_administer_queue</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.queue-mappings</name>
<value></value>
</property>
<property>
<name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
<value>false</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.accessible-node-labels</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.accessible-node-labels</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.accessible-node-labels</name>
<value>label02</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.accessible-node-labels</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.accessible-node-labels.label01.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.accessible-node-labels.label02.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.capacity</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.maximum-capacity</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.accessible-node-labels.label01.capacity</name>
<value>99</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.accessible-node-labels.label02.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default-node-label-expression</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.default-node-label-expression</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.default-node-label-expression</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.default-node-label-expression</name>
<value>label02</value>
</property>
</configuration> {code}
!20220927-142546.jpg!
was:
*After node label is configured on YARN, Memory Used in Web BI is always 0B, no matter how the program is submitted*
capacity-scheduler.xml
{code:java}
<configuration>
<property>
<name>yarn.scheduler.capacity.maximum-applications</name>
<value>10000</value>
</property>
<property>
<name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
<value>0.8</value>
</property>
<property>
<name>yarn.scheduler.capacity.resource-calculator</name>
<value>org.apache.hadoop.yarn.util.resource.DominantResourceCalculator</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>realtime,bi,default</value>
</property>
<property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.state</name>
<value>RUNNING</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.capacity</name>
<value>50</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.state</name>
<value>RUNNING</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.acl_submit_applications</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.acl_administer_queue</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.capacity</name>
<value>49</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.state</name>
<value>RUNNING</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.acl_submit_applications</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.acl_administer_queue</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.queue-mappings</name>
<value></value>
</property>
<property>
<name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
<value>false</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.accessible-node-labels</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.accessible-node-labels</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.accessible-node-labels</name>
<value>label02</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.accessible-node-labels</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.accessible-node-labels.label01.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.accessible-node-labels.label02.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.capacity</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.maximum-capacity</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.accessible-node-labels.label01.capacity</name>
<value>99</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.accessible-node-labels.label02.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default-node-label-expression</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.default-node-label-expression</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.bi.default-node-label-expression</name>
<value>label01</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.realtime.default-node-label-expression</name>
<value>label02</value>
</property>
</configuration> {code}
> Yarn After node label is configured, Memory Used cannot be displayed in Web BI
> ------------------------------------------------------------------------------
>
> Key: YARN-11319
> URL: https://issues.apache.org/jira/browse/YARN-11319
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-ui-v2
> Affects Versions: 2.8.5, 3.1.3
> Reporter: xiyang
> Priority: Minor
> Attachments: 20220927-142546.jpg
>
>
> *After node label is configured on YARN, Memory Used in Web BI is always 0B, no matter how the program is submitted*
> capacity-scheduler.xml
> {code:java}
> <configuration>
> <property>
> <name>yarn.scheduler.capacity.maximum-applications</name>
> <value>10000</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
> <value>0.8</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.resource-calculator</name>
> <value>org.apache.hadoop.yarn.util.resource.DominantResourceCalculator</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.queues</name>
> <value>realtime,bi,default</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.node-locality-delay</name>
> <value>40</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.capacity</name>
> <value>1</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
> <value>1</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.state</name>
> <value>RUNNING</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
> <value>*</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
> <value>*</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.realtime.capacity</name>
> <value>50</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.realtime.state</name>
> <value>RUNNING</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.realtime.acl_submit_applications</name>
> <value>*</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.realtime.acl_administer_queue</name>
> <value>*</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.bi.capacity</name>
> <value>49</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.bi.state</name>
> <value>RUNNING</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.bi.acl_submit_applications</name>
> <value>*</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.bi.acl_administer_queue</name>
> <value>*</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.queue-mappings</name>
> <value></value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
> <value>false</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.accessible-node-labels</name>
> <value>*</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.accessible-node-labels</name>
> <value>label01</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.realtime.accessible-node-labels</name>
> <value>label02</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.bi.accessible-node-labels</name>
> <value>label01</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.accessible-node-labels.label01.capacity</name>
> <value>100</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.accessible-node-labels.label02.capacity</name>
> <value>100</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.capacity</name>
> <value>1</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.accessible-node-labels.label01.maximum-capacity</name>
> <value>1</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.bi.accessible-node-labels.label01.capacity</name>
> <value>99</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.realtime.accessible-node-labels.label02.capacity</name>
> <value>100</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default-node-label-expression</name>
> <value>label01</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.default-node-label-expression</name>
> <value>label01</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.bi.default-node-label-expression</name>
> <value>label01</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.realtime.default-node-label-expression</name>
> <value>label02</value>
> </property>
> </configuration> {code}
> !20220927-142546.jpg!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org