[jira] [Created] (YARN-11147) ResourceUsage and QueueCapacities classes provide node label iterators that are not thread safe

["András Győri (Jira)" <ji...@apache.org>]

András Győri created YARN-11147:
-----------------------------------

             Summary: ResourceUsage and QueueCapacities classes provide node label iterators that are not thread safe
                 Key: YARN-11147
                 URL: https://issues.apache.org/jira/browse/YARN-11147
             Project: Hadoop YARN
          Issue Type: Bug
          Components: capacity scheduler
            Reporter: András Győri
            Assignee: András Győri


AbstractResourceUsage#getNodePartitionsSet and QueueCapacities#getNodePartitionsSet provide keySet, a mutable view on the HashMap's keys, that is subject to change. Iterating through an iterator that is modified by an other thread at the same time results in a ConcurrentModificationException as the following stacktrace shows:
{code:java}
2022-04-28 13:21:53,692 FATAL org.apache.hadoop.yarn.event.EventDispatcher: Error in handling event type NODE_LABELS_UPDATE to the Event Dispatcher
java.util.ConcurrentModificationException
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1445)
    at java.util.HashMap$KeyIterator.next(HashMap.java:1469)
    at com.google.common.collect.Sets$1$1.computeNext(Sets.java:758)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueUtils.updateQueueStatistics(CSQueueUtils.java:236)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.updateClusterResource(ParentQueue.java:1281)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.updateNodeLabelsAndQueueResource(CapacityScheduler.java:2115)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:1900)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:169)
    at org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
    at java.lang.Thread.run(Thread.java:748)
 {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org