You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by ricky l <ri...@gmail.com> on 2013/12/05 21:10:40 UTC
using yarn.nodemanager.container-monitor.resource-calculator.class option in Yarn

Hi all,

In the hadoop-3.0.0-SNAPSHOT

I set the below option hoping that it will throttle a container that
over-utilize its resources.

<property>

<name>yarn.nodemanager.container-monitor.resource-calculator.class</name>
    <value>org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin</value>
</property>

If I start a nodemanager with the above option, it is throwing an exception
shown below. The message seems to be
"ResourceCalculatorPlugin is unavailable on this system.
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
is disabled."

Can I enable the option? FYI, I am not using linux container to launch Yarn
container - I am using default "yarn.nodemanager.container-executor.class"
option.


2013-12-05 11:41:53,302 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices:
Adding auxiliary service httpshuffle, "mapreduce_shuffle"
2013-12-05 11:41:53,367 INFO org.apache.hadoop.service.AbstractService:
Service containers-monitor failed in state INITED; cause:
java.lang.RuntimeException: java.lang.InstantiationException
java.lang.RuntimeException: java.lang.InstantiationException
        at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
        at
org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.getResourceCalculatorPlugin(ResourceCalculatorPlugin.java:120)
        at
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl.serviceInit(ContainersMonitorImpl.java:96)
        at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
        at
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:220)
        at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
        at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:188)
        at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:353)
        at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:401)
Caused by: java.lang.InstantiationException
        at
sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:30)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
        ... 11 more
2013-12-05 11:41:53,368 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
ResourceCalculatorPlugin is unavailable on this system.
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
is disabled.
2013-12-05 11:41:53,368 INFO org.apache.hadoop.service.AbstractService:
Service
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl
failed in state INITED; cause: java.lang.RuntimeException:
java.lang.InstantiationException
java.lang.RuntimeException: java.lang.InstantiationException
        at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
        at
org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.getResourceCalculatorPlugin(ResourceCalculatorPlugin.java:120)
        at
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl.serviceInit(ContainersMonitorImpl.java:96)
        at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
        at
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:220)
        at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
        at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:188)
        at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:353)
        at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:401)
Caused by: java.lang.InstantiationException
        at
sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:30)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
        ... 11 more