You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Mehran Hassani (JIRA)" <ji...@apache.org> on 2016/09/15 23:41:20 UTC
[jira] [Updated] (YARN-5642) Typos in 11 log messages
[ https://issues.apache.org/jira/browse/YARN-5642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mehran Hassani updated YARN-5642:
---------------------------------
Description:
I am conducting research on log related bugs. I tried to make a tool to fix repetitive yet simple patterns of bugs that are related to logs. Typos in log messages are one of the reoccurring bugs. Therefore, I made a tool find typos in log statements. During my experiments, I managed to find the following typos in Hadoop YARN:
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java, LOG.info("AsyncDispatcher is draining to stop igonring any new events."),
igonring should be ignoring
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/YarnAuthorizationProvider.java, LOG.info(authorizerClass.getName() + " is instiantiated."),
instiantiated should be instantiated
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/FileSystemApplicationHistoryStore.java, LOG.info("Completed reading history information of all conatiners"+ " of application attempt " + appAttemptId),
conatiners should be containers
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java, LOG.info("Neither virutal-memory nor physical-memory monitoring is " +"needed. Not running the monitor-thread"),
virutal should be virtual
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/reservation/AbstractReservationSystem.java, LOG.info("Intialized plan {} based on reservable queue {}" plan.toString() planQueueName),
Intialized should be Initialized
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java, LOG.info("Initializing " + queueName + "\n" +"capacity = " + queueCapacities.getCapacity() +" [= (float) configuredCapacity / 100 ]" + "\n" +"asboluteCapacity = " + queueCapacities.getAbsoluteCapacity() +" [= parentAbsoluteCapacity * capacity ]" + "\n" +"maxCapacity = " + queueCapacities.getMaximumCapacity() +" [= configuredMaxCapacity ]" + "\n" +"absoluteMaxCapacity = " + queueCapacities.getAbsoluteMaximumCapacity() +" [= 1.0 maximumCapacity undefined " +"(parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]" +"\n" +"userLimit = " + userLimit +" [= configuredUserLimit ]" + "\n" +"userLimitFactor = " + userLimitFactor +" [= configuredUserLimitFactor ]" + "\n" +"maxApplications = " + maxApplications +" [= configuredMaximumSystemApplicationsPerQueue or" +" (int)(configuredMaximumSystemApplications * absoluteCapacity)]" +"\n" +"maxApplicationsPerUser = " + maxApplicationsPerUser +" [= (int)(maxApplications * (userLimit / 100.0f) * " +"userLimitFactor) ]" + "\n" +"usedCapacity = " + queueCapacities.getUsedCapacity() +" [= usedResourcesMemory / " +"(clusterResourceMemory * absoluteCapacity)]" + "\n" +"absoluteUsedCapacity = " + absoluteUsedCapacity +" [= usedResourcesMemory / clusterResourceMemory]" + "\n" +"maxAMResourcePerQueuePercent = " + maxAMResourcePerQueuePercent +" [= configuredMaximumAMResourcePercent ]" + "\n" +"minimumAllocationFactor = " + minimumAllocationFactor +" [= (float)(maximumAllocationMemory - minimumAllocationMemory) / " +"maximumAllocationMemory ]" + "\n" +"maximumAllocation = " + maximumAllocation +" [= configuredMaxAllocation ]" + "\n" +"numContainers = " + numContainers +" [= currentNumContainers ]" + "\n" +"state = " + state +" [= configuredState ]" + "\n" +"acls = " + aclsString +" [= configuredAcls ]" + "\n" +"nodeLocalityDelay = " + nodeLocalityDelay + "\n" +"labels=" + labelStrBuilder.toString() + "\n" +"reservationsContinueLooking = " +reservationsContinueLooking + "\n" +"preemptionDisabled = " + getPreemptionDisabled() + "\n" +"defaultAppPriorityPerQueue = " + defaultAppPriorityPerQueue),
asbolute should be absolute
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java, LOG.info(queueName +" capacity=" + this.queueCapacities.getCapacity() +" asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +" maxCapacity=" + this.queueCapacities.getMaximumCapacity() +" asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() +" state=" + state +" acls=" + aclsString +" labels=" + labelStrBuilder.toString() + "\n" +" reservationsContinueLooking=" + reservationsContinueLooking),
asbolute should be absolute
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java, LOG.info(queueName +" capacity=" + this.queueCapacities.getCapacity() +" asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +" maxCapacity=" + this.queueCapacities.getMaximumCapacity() +" asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() +" state=" + state +" acls=" + aclsString +" labels=" + labelStrBuilder.toString() + "\n" +" reservationsContinueLooking=" + reservationsContinueLooking), asbolute should be absolute
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java, LOG.info("Calling allocate on removed " +"or non existant application " + appAttemptId),
existant should be existent
was:
I am conducting research on log related bugs. I tried to make a tool to fix repetitive yet simple patterns of bugs that are related to logs. Typos in log messages are one of the reoccurring bugs. Therefore, I made a tool find typos in log statements. During my experiments, I managed to find the following typos in Hadoop YARN:
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java, LOG.info("AsyncDispatcher is draining to stop igonring any new events."),
igonring should be ignoring
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/YarnAuthorizationProvider.java, LOG.info(authorizerClass.getName() + " is instiantiated."),
instiantiated should be instantiated
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java, LOG.info("AsyncDispatcher is draining to stop igonring any new events."),
igonring should be ignoring
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/YarnAuthorizationProvider.java, LOG.info(authorizerClass.getName() + " is instiantiated."),
instiantiated should be instantiated
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/FileSystemApplicationHistoryStore.java, LOG.info("Completed reading history information of all conatiners"+ " of application attempt " + appAttemptId),
conatiners should be containers
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java, LOG.info("Neither virutal-memory nor physical-memory monitoring is " +"needed. Not running the monitor-thread"),
virutal should be virtual
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/reservation/AbstractReservationSystem.java, LOG.info("Intialized plan {} based on reservable queue {}" plan.toString() planQueueName),
Intialized should be Initialized
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java, LOG.info("Initializing " + queueName + "\n" +"capacity = " + queueCapacities.getCapacity() +" [= (float) configuredCapacity / 100 ]" + "\n" +"asboluteCapacity = " + queueCapacities.getAbsoluteCapacity() +" [= parentAbsoluteCapacity * capacity ]" + "\n" +"maxCapacity = " + queueCapacities.getMaximumCapacity() +" [= configuredMaxCapacity ]" + "\n" +"absoluteMaxCapacity = " + queueCapacities.getAbsoluteMaximumCapacity() +" [= 1.0 maximumCapacity undefined " +"(parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]" +"\n" +"userLimit = " + userLimit +" [= configuredUserLimit ]" + "\n" +"userLimitFactor = " + userLimitFactor +" [= configuredUserLimitFactor ]" + "\n" +"maxApplications = " + maxApplications +" [= configuredMaximumSystemApplicationsPerQueue or" +" (int)(configuredMaximumSystemApplications * absoluteCapacity)]" +"\n" +"maxApplicationsPerUser = " + maxApplicationsPerUser +" [= (int)(maxApplications * (userLimit / 100.0f) * " +"userLimitFactor) ]" + "\n" +"usedCapacity = " + queueCapacities.getUsedCapacity() +" [= usedResourcesMemory / " +"(clusterResourceMemory * absoluteCapacity)]" + "\n" +"absoluteUsedCapacity = " + absoluteUsedCapacity +" [= usedResourcesMemory / clusterResourceMemory]" + "\n" +"maxAMResourcePerQueuePercent = " + maxAMResourcePerQueuePercent +" [= configuredMaximumAMResourcePercent ]" + "\n" +"minimumAllocationFactor = " + minimumAllocationFactor +" [= (float)(maximumAllocationMemory - minimumAllocationMemory) / " +"maximumAllocationMemory ]" + "\n" +"maximumAllocation = " + maximumAllocation +" [= configuredMaxAllocation ]" + "\n" +"numContainers = " + numContainers +" [= currentNumContainers ]" + "\n" +"state = " + state +" [= configuredState ]" + "\n" +"acls = " + aclsString +" [= configuredAcls ]" + "\n" +"nodeLocalityDelay = " + nodeLocalityDelay + "\n" +"labels=" + labelStrBuilder.toString() + "\n" +"reservationsContinueLooking = " +reservationsContinueLooking + "\n" +"preemptionDisabled = " + getPreemptionDisabled() + "\n" +"defaultAppPriorityPerQueue = " + defaultAppPriorityPerQueue),
asbolute should be absolute
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java, LOG.info(queueName +" capacity=" + this.queueCapacities.getCapacity() +" asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +" maxCapacity=" + this.queueCapacities.getMaximumCapacity() +" asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() +" state=" + state +" acls=" + aclsString +" labels=" + labelStrBuilder.toString() + "\n" +" reservationsContinueLooking=" + reservationsContinueLooking),
asbolute should be absolute
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java, LOG.info(queueName +" capacity=" + this.queueCapacities.getCapacity() +" asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +" maxCapacity=" + this.queueCapacities.getMaximumCapacity() +" asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() +" state=" + state +" acls=" + aclsString +" labels=" + labelStrBuilder.toString() + "\n" +" reservationsContinueLooking=" + reservationsContinueLooking), asbolute should be absolute
In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java, LOG.info("Calling allocate on removed " +"or non existant application " + appAttemptId),
existant should be existent
> Typos in 11 log messages
> -------------------------
>
> Key: YARN-5642
> URL: https://issues.apache.org/jira/browse/YARN-5642
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Mehran Hassani
> Priority: Trivial
> Labels: newbie
>
> I am conducting research on log related bugs. I tried to make a tool to fix repetitive yet simple patterns of bugs that are related to logs. Typos in log messages are one of the reoccurring bugs. Therefore, I made a tool find typos in log statements. During my experiments, I managed to find the following typos in Hadoop YARN:
> In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java, LOG.info("AsyncDispatcher is draining to stop igonring any new events."),
> igonring should be ignoring
> In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/YarnAuthorizationProvider.java, LOG.info(authorizerClass.getName() + " is instiantiated."),
> instiantiated should be instantiated
> In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/FileSystemApplicationHistoryStore.java, LOG.info("Completed reading history information of all conatiners"+ " of application attempt " + appAttemptId),
> conatiners should be containers
> In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java, LOG.info("Neither virutal-memory nor physical-memory monitoring is " +"needed. Not running the monitor-thread"),
> virutal should be virtual
> In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/reservation/AbstractReservationSystem.java, LOG.info("Intialized plan {} based on reservable queue {}" plan.toString() planQueueName),
> Intialized should be Initialized
> In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java, LOG.info("Initializing " + queueName + "\n" +"capacity = " + queueCapacities.getCapacity() +" [= (float) configuredCapacity / 100 ]" + "\n" +"asboluteCapacity = " + queueCapacities.getAbsoluteCapacity() +" [= parentAbsoluteCapacity * capacity ]" + "\n" +"maxCapacity = " + queueCapacities.getMaximumCapacity() +" [= configuredMaxCapacity ]" + "\n" +"absoluteMaxCapacity = " + queueCapacities.getAbsoluteMaximumCapacity() +" [= 1.0 maximumCapacity undefined " +"(parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]" +"\n" +"userLimit = " + userLimit +" [= configuredUserLimit ]" + "\n" +"userLimitFactor = " + userLimitFactor +" [= configuredUserLimitFactor ]" + "\n" +"maxApplications = " + maxApplications +" [= configuredMaximumSystemApplicationsPerQueue or" +" (int)(configuredMaximumSystemApplications * absoluteCapacity)]" +"\n" +"maxApplicationsPerUser = " + maxApplicationsPerUser +" [= (int)(maxApplications * (userLimit / 100.0f) * " +"userLimitFactor) ]" + "\n" +"usedCapacity = " + queueCapacities.getUsedCapacity() +" [= usedResourcesMemory / " +"(clusterResourceMemory * absoluteCapacity)]" + "\n" +"absoluteUsedCapacity = " + absoluteUsedCapacity +" [= usedResourcesMemory / clusterResourceMemory]" + "\n" +"maxAMResourcePerQueuePercent = " + maxAMResourcePerQueuePercent +" [= configuredMaximumAMResourcePercent ]" + "\n" +"minimumAllocationFactor = " + minimumAllocationFactor +" [= (float)(maximumAllocationMemory - minimumAllocationMemory) / " +"maximumAllocationMemory ]" + "\n" +"maximumAllocation = " + maximumAllocation +" [= configuredMaxAllocation ]" + "\n" +"numContainers = " + numContainers +" [= currentNumContainers ]" + "\n" +"state = " + state +" [= configuredState ]" + "\n" +"acls = " + aclsString +" [= configuredAcls ]" + "\n" +"nodeLocalityDelay = " + nodeLocalityDelay + "\n" +"labels=" + labelStrBuilder.toString() + "\n" +"reservationsContinueLooking = " +reservationsContinueLooking + "\n" +"preemptionDisabled = " + getPreemptionDisabled() + "\n" +"defaultAppPriorityPerQueue = " + defaultAppPriorityPerQueue),
> asbolute should be absolute
> In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java, LOG.info(queueName +" capacity=" + this.queueCapacities.getCapacity() +" asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +" maxCapacity=" + this.queueCapacities.getMaximumCapacity() +" asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() +" state=" + state +" acls=" + aclsString +" labels=" + labelStrBuilder.toString() + "\n" +" reservationsContinueLooking=" + reservationsContinueLooking),
> asbolute should be absolute
> In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java, LOG.info(queueName +" capacity=" + this.queueCapacities.getCapacity() +" asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +" maxCapacity=" + this.queueCapacities.getMaximumCapacity() +" asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() +" state=" + state +" acls=" + aclsString +" labels=" + labelStrBuilder.toString() + "\n" +" reservationsContinueLooking=" + reservationsContinueLooking), asbolute should be absolute
> In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java, LOG.info("Calling allocate on removed " +"or non existant application " + appAttemptId),
> existant should be existent
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org