You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2019/03/04 18:36:00 UTC
[jira] [Comment Edited] (YARN-9328) ParentQueue#apply move log
outside writelock
[ https://issues.apache.org/jira/browse/YARN-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16783626#comment-16783626 ]
Eric Yang edited comment on YARN-9328 at 3/4/19 6:35 PM:
---------------------------------------------------------
[~Prabhu Joseph] {quote}For too frequent operation like commits or assignment of container its better to log out of write lock rt ??{quote}
The bug happens when multi-threaded programs are trying to write log concurrently. Log4j 1.x internally uses locking to initialize logging appenders, it is unlikely to help if logging statement is moved outside of the write lock. I think the workaround is to apply [the patch|https://bz.apache.org/bugzilla/attachment.cgi?id=31193&action=diff] that I generated for Log4J 1.2.x to avoid multithread initialization race conditions of logging appenders to deadlock.
was (Author: eyang):
[~Prabhu Joseph] {quote}For too frequent operation like commits or assignment of container its better to log out of write lock rt ??{quote}
The bug happens when multi-threaded programs are trying to write log concurrently. Log4j 1.x internally uses locking to initialize logging appenders, it is unlikely to help if logging statement is moved outside of the write lock. I think the workaround is to apply the patch that I generated for Log4J 1.2.x to avoid initialization of logging appenders to deadlock.
> ParentQueue#apply move log outside writelock
> --------------------------------------------
>
> Key: YARN-9328
> URL: https://issues.apache.org/jira/browse/YARN-9328
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Bibin A Chundatt
> Assignee: Prabhu Joseph
> Priority: Major
> Attachments: YARN-9328-001.patch, YARN-9328-002.patch, YARN-9328-003.patch
>
>
> {code}
> LOG.info("assignedContainer" + " queue=" + getQueueName()
> + " usedCapacity=" + getUsedCapacity() + " absoluteUsedCapacity="
> + getAbsoluteUsedCapacity() + " used=" + queueUsage.getUsed()
> + " cluster=" + cluster);
> {code}
> Logging can be done after log.. Logging could reduce performance ..
> {code}
> "Thread-16" #40 daemon prio=5 os_prio=0 tid=0x00007f181f9bb800 nid=0x125 waiting for monitor entry [0x00007ef163bab000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.log4j.Category.callAppenders(Category.java:204)
> - locked <0x00007ef2d803e2b8> (a org.apache.log4j.spi.RootLogger)
> at org.apache.log4j.Category.forcedLog(Category.java:391)
> at org.apache.log4j.Category.log(Category.java:856)
> at org.apache.commons.logging.impl.Log4JLogger.info(Log4JLogger.java:176)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.apply(ParentQueue.java:1336)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.apply(LeafQueue.java:1371)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.apply(FiCaSchedulerApp.java:665)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.tryCommit(CapacityScheduler.java:2946)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$ResourceCommitterService.run(CapacityScheduler.java:644)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org