You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by ricky l <ri...@gmail.com> on 2013/12/05 21:10:40 UTC
using yarn.nodemanager.container-monitor.resource-calculator.class
option in Yarn
Hi all,
In the hadoop-3.0.0-SNAPSHOT
I set the below option hoping that it will throttle a container that
over-utilize its resources.
<property>
<name>yarn.nodemanager.container-monitor.resource-calculator.class</name>
<value>org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin</value>
</property>
If I start a nodemanager with the above option, it is throwing an exception
shown below. The message seems to be
"ResourceCalculatorPlugin is unavailable on this system.
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
is disabled."
Can I enable the option? FYI, I am not using linux container to launch Yarn
container - I am using default "yarn.nodemanager.container-executor.class"
option.
2013-12-05 11:41:53,302 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices:
Adding auxiliary service httpshuffle, "mapreduce_shuffle"
2013-12-05 11:41:53,367 INFO org.apache.hadoop.service.AbstractService:
Service containers-monitor failed in state INITED; cause:
java.lang.RuntimeException: java.lang.InstantiationException
java.lang.RuntimeException: java.lang.InstantiationException
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
at
org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.getResourceCalculatorPlugin(ResourceCalculatorPlugin.java:120)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl.serviceInit(ContainersMonitorImpl.java:96)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:220)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:188)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:353)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:401)
Caused by: java.lang.InstantiationException
at
sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:30)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
... 11 more
2013-12-05 11:41:53,368 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
ResourceCalculatorPlugin is unavailable on this system.
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
is disabled.
2013-12-05 11:41:53,368 INFO org.apache.hadoop.service.AbstractService:
Service
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl
failed in state INITED; cause: java.lang.RuntimeException:
java.lang.InstantiationException
java.lang.RuntimeException: java.lang.InstantiationException
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
at
org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.getResourceCalculatorPlugin(ResourceCalculatorPlugin.java:120)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl.serviceInit(ContainersMonitorImpl.java:96)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:220)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:188)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:353)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:401)
Caused by: java.lang.InstantiationException
at
sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:30)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
... 11 more