You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Filipp Zhinkin (JIRA)" <ji...@apache.org> on 2017/07/04 07:54:00 UTC
[jira] [Commented] (YARN-6378) Negative usedResources memory in
CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073266#comment-16073266 ]
Filipp Zhinkin commented on YARN-6378:
--------------------------------------
Seems like hadoop 2.6.0 has the same problem, at least I'm facing negative memory and vcores usage with capacity scheduler:
{code:xml}
<queue xsi:type="capacitySchedulerLeafQueueInfo">
<capacity>19.0</capacity>
<usedCapacity>-69.52686</usedCapacity>
<maxCapacity>90.0</maxCapacity>
<absoluteCapacity>19.0</absoluteCapacity>
<absoluteMaxCapacity>90.0</absoluteMaxCapacity>
<absoluteUsedCapacity>0.0</absoluteUsedCapacity>
<numApplications>10</numApplications>
<queueName>default</queueName>
<state>RUNNING</state>
<resourcesUsed>
<memory>-152576</memory>
<vCores>-41</vCores>
</resourcesUsed>
<hideReservationQueues>false</hideReservationQueues>
<nodeLabels>*</nodeLabels>
<allocatedContainers>24</allocatedContainers>
<reservedContainers>0</reservedContainers>
<pendingContainers>0</pendingContainers>
<numActiveApplications>10</numActiveApplications>
<numPendingApplications>0</numPendingApplications>
<numContainers>-41</numContainers>
<maxApplications>1900</maxApplications>
<maxApplicationsPerUser>855</maxApplicationsPerUser>
<maxActiveApplications>102</maxActiveApplications>
<maxActiveApplicationsPerUser>10</maxActiveApplicationsPerUser>
<userLimit>10</userLimit>
...
</queue>
{code}
> Negative usedResources memory in CapacityScheduler
> --------------------------------------------------
>
> Key: YARN-6378
> URL: https://issues.apache.org/jira/browse/YARN-6378
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, resourcemanager
> Affects Versions: 2.7.2
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
>
> Courtesy Thomas Nystrand, we found that on two of our clusters configured with the CapacityScheduler, usedResources occasionally becomes negative.
> e.g.
> {code}
> 2017-03-15 11:10:09,449 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: assignedContainer application attempt=appattempt_1487222361993_17177_000001 container=Container: [ContainerId: container_1487222361993_17177_01_000014, NodeId: <SOMENODE>:27249, NodeHttpAddress: <SOMENODE>:8042, Resource: <memory:6656, vCores:1>, Priority: 2, Token: null, ] queue=<somequeuename>: capacity=0.2, absoluteCapacity=0.2, usedResources=<memory:-1024, vCores:3>, usedCapacity=0.03409091, absoluteUsedCapacity=0.006818182, numApps=1, numContainers=3 clusterResource=<memory:1249280, vCores:440> type=RACK_LOCAL
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org