You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Karthik Kambatla (JIRA)" <ji...@apache.org> on 2016/10/21 06:01:58 UTC
[jira] [Commented] (YARN-4882) Change the log level to DEBUG for
recovering completed applications
[ https://issues.apache.org/jira/browse/YARN-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15594187#comment-15594187 ]
Karthik Kambatla commented on YARN-4882:
----------------------------------------
The following snippet seems more complicated:
{code}
if ((oldState != getState()) &&
((event.getType() != RMAppEventType.RECOVER) ||
(recoveredFinalState != null) || LOG.isDebugEnabled())) {
String message = appID + " State change from " + oldState + " to "
+ getState() + " on event=" + event.getType();
// Log at INFO if we're not recovering or not in a terminal state.
// Log at DEBUG otherwise.
if ((event.getType() != RMAppEventType.RECOVER) ||
(recoveredFinalState != null)) {
LOG.info(message);
} else {
LOG.debug(message);
}
{code}
How about the following instead? If we want to, we could move creating of the message to a helper method called from {{LOG.info()}} or {{LOG.debug()}}.
{code}
if (oldState != getState()) {
String message = appID + " State change from " + oldState + " to "
+ getState() + " on event=" + event.getType();
// Log at INFO if we're not recovering or not in a terminal state.
// Log at DEBUG otherwise.
if (event.getType() != RMAppEventType.RECOVER ||
recoveredFinalState != null) {
LOG.info(message);
} else if (LOG.isDebugEnabled()){
LOG.debug(message);
}
{code}
> Change the log level to DEBUG for recovering completed applications
> -------------------------------------------------------------------
>
> Key: YARN-4882
> URL: https://issues.apache.org/jira/browse/YARN-4882
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Reporter: Rohith Sharma K S
> Assignee: Daniel Templeton
> Attachments: YARN-4882.001.patch
>
>
> I think for recovering completed applications no need to log as INFO, rather it can be made it as DEBUG. The problem seen from large cluster is if any issue happens during RM start up and continuously switching , then RM logs are filled with most with recovering applications only.
> There are 6 lines are logged for 1 applications as I shown in below logs, then consider RM default value for max-completed applications is 10K. So for each switch 10K*6=60K lines will be added which is not useful I feel.
> {noformat}
> 2016-03-01 10:20:59,077 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAppManager: Default priority level is set to application:application_1456298208485_21507
> 2016-03-01 10:20:59,094 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Recovering app: application_1456298208485_21507 with 1 attempts and final state = FINISHED
> 2016-03-01 10:20:59,100 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: Recovering attempt: appattempt_1456298208485_21507_000001 with final state: FINISHED
> 2016-03-01 10:20:59,107 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1456298208485_21507_000001 State change from NEW to FINISHED
> 2016-03-01 10:20:59,111 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1456298208485_21507 State change from NEW to FINISHED
> 2016-03-01 10:20:59,112 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=rohith OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1456298208485_21507
> {noformat}
> The main problem is missing important information's from the logs before RM unstable. Even though log roll back is 50 or 100, in a short period all these logs will be rolled out and all the logs contains only RM switching information that too recovering applications!!.
> I suggest at least completed applications recovery should be logged as DEBUG.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org