You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Rob Blah <tm...@gmail.com> on 2013/08/11 12:27:10 UTC
DefaultResourceCalculator class not found, ResourceManager fails to start.
Hi
I have a strange problem, regarding missing class, the
DefaultResourceCalculator. I have a single node sandbox cluster working in
a pseudo-distributed mode. The cluster was working fine yesterday, however
today it stopped working. I was able to fix all issues except the following
problem in ResourceManager:
2013-08-11 12:12:42,425 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
starting ResourceManager
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
Class
org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
... 5 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
... 6 more
2013-08-11 12:12:42,426 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
SHUTDOWN_MSG:
I build YARN from sources, daily updated to the newest revision in trunk.
The class DefaultResourceCalculator exists and is present in YARN's
sources. I am using (currently) trunk revision 1512895. I build YARN
project with the following command:
mvn clean package -Pdist -Dtar -DskipTests
I create tar ball with the use of script provided in the sources:
dist-tar-stitching.sh (hadoop-dist/target)
regards
tmp
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Rob Blah <tm...@gmail.com>.
Problem solved. Thank you for your help.
@Ted Yu
Other issues where my mistakes. I have a dedicated script which
updates/builds/"deploys" YARN from sources. I was starting NN with the
"-upgrade" option which unsynchronized NN version, also leading to broken
DN. Quick NN format and deletion of DN data solved the issue (I am working
on a sandbox cluster, so that is not a problem). I have modified the script
to start the NN without the upgrade option.
Two quick questions:
- when should I use the NN upgrade option, should it be only used to
upgrade NN between new version (example 2.0.4 -> 2.0.5). How can I automate
this process?
- Is the "design/functionality" of my magical script correct? How can I
avoid future problems like the solved one?
YARN update script
- update src to trunk (opt)
- package YARN
- build dist (tar ball)
- unpack new_dist
- overwrite new_dist conf with prev_dist conf (this has lead to the problem
with DefaultResourceCalculator, my conf is bare minimum to work in
pseudo-distributed mode)
- start YARN
For any suggestions I would be grateful.
regards
tmp
2013/8/12 Ted Yu <yu...@gmail.com>
> Can you check the config entry
> for yarn.scheduler.capacity.resource-calculator ?
> It should point
> to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
>
> bq. I was able to fix all issues
>
> What other issues came up ?
>
> Thanks
>
>
> On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <tm...@gmail.com> wrote:
>
>> Hi again
>>
>> From a little investigation I have performed I have observed the
>> following. I assume the module responsible for this class is
>> hadoop-yarn-common.
>>
>> During RM init it crashes since it is looking for a class
>> DefaultResourceCalculator in
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
>> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
>> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
>> the RM crashes. Anybody knows how I can fix this? I would very grateful for
>> any help.
>>
>> regards
>> tmp
>>
>>
>> 2013/8/11 Rob Blah <tm...@gmail.com>
>>
>>> Hi
>>>
>>> I have a strange problem, regarding missing class, the
>>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>>> today it stopped working. I was able to fix all issues except the following
>>> problem in ResourceManager:
>>> 2013-08-11 12:12:42,425 FATAL
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>>> starting ResourceManager
>>> java.lang.RuntimeException: java.lang.RuntimeException:
>>> java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>>> at
>>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>>> ... 5 more
>>> Caused by: java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
>>> ... 6 more
>>> 2013-08-11 12:12:42,426 INFO
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
>>> SHUTDOWN_MSG:
>>>
>>> I build YARN from sources, daily updated to the newest revision in
>>> trunk. The class DefaultResourceCalculator exists and is present in YARN's
>>> sources. I am using (currently) trunk revision 1512895. I build YARN
>>> project with the following command:
>>> mvn clean package -Pdist -Dtar -DskipTests
>>> I create tar ball with the use of script provided in the sources:
>>> dist-tar-stitching.sh (hadoop-dist/target)
>>>
>>> regards
>>> tmp
>>>
>>
>>
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Rob Blah <tm...@gmail.com>.
Problem solved. Thank you for your help.
@Ted Yu
Other issues where my mistakes. I have a dedicated script which
updates/builds/"deploys" YARN from sources. I was starting NN with the
"-upgrade" option which unsynchronized NN version, also leading to broken
DN. Quick NN format and deletion of DN data solved the issue (I am working
on a sandbox cluster, so that is not a problem). I have modified the script
to start the NN without the upgrade option.
Two quick questions:
- when should I use the NN upgrade option, should it be only used to
upgrade NN between new version (example 2.0.4 -> 2.0.5). How can I automate
this process?
- Is the "design/functionality" of my magical script correct? How can I
avoid future problems like the solved one?
YARN update script
- update src to trunk (opt)
- package YARN
- build dist (tar ball)
- unpack new_dist
- overwrite new_dist conf with prev_dist conf (this has lead to the problem
with DefaultResourceCalculator, my conf is bare minimum to work in
pseudo-distributed mode)
- start YARN
For any suggestions I would be grateful.
regards
tmp
2013/8/12 Ted Yu <yu...@gmail.com>
> Can you check the config entry
> for yarn.scheduler.capacity.resource-calculator ?
> It should point
> to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
>
> bq. I was able to fix all issues
>
> What other issues came up ?
>
> Thanks
>
>
> On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <tm...@gmail.com> wrote:
>
>> Hi again
>>
>> From a little investigation I have performed I have observed the
>> following. I assume the module responsible for this class is
>> hadoop-yarn-common.
>>
>> During RM init it crashes since it is looking for a class
>> DefaultResourceCalculator in
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
>> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
>> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
>> the RM crashes. Anybody knows how I can fix this? I would very grateful for
>> any help.
>>
>> regards
>> tmp
>>
>>
>> 2013/8/11 Rob Blah <tm...@gmail.com>
>>
>>> Hi
>>>
>>> I have a strange problem, regarding missing class, the
>>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>>> today it stopped working. I was able to fix all issues except the following
>>> problem in ResourceManager:
>>> 2013-08-11 12:12:42,425 FATAL
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>>> starting ResourceManager
>>> java.lang.RuntimeException: java.lang.RuntimeException:
>>> java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>>> at
>>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>>> ... 5 more
>>> Caused by: java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
>>> ... 6 more
>>> 2013-08-11 12:12:42,426 INFO
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
>>> SHUTDOWN_MSG:
>>>
>>> I build YARN from sources, daily updated to the newest revision in
>>> trunk. The class DefaultResourceCalculator exists and is present in YARN's
>>> sources. I am using (currently) trunk revision 1512895. I build YARN
>>> project with the following command:
>>> mvn clean package -Pdist -Dtar -DskipTests
>>> I create tar ball with the use of script provided in the sources:
>>> dist-tar-stitching.sh (hadoop-dist/target)
>>>
>>> regards
>>> tmp
>>>
>>
>>
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Rob Blah <tm...@gmail.com>.
Problem solved. Thank you for your help.
@Ted Yu
Other issues where my mistakes. I have a dedicated script which
updates/builds/"deploys" YARN from sources. I was starting NN with the
"-upgrade" option which unsynchronized NN version, also leading to broken
DN. Quick NN format and deletion of DN data solved the issue (I am working
on a sandbox cluster, so that is not a problem). I have modified the script
to start the NN without the upgrade option.
Two quick questions:
- when should I use the NN upgrade option, should it be only used to
upgrade NN between new version (example 2.0.4 -> 2.0.5). How can I automate
this process?
- Is the "design/functionality" of my magical script correct? How can I
avoid future problems like the solved one?
YARN update script
- update src to trunk (opt)
- package YARN
- build dist (tar ball)
- unpack new_dist
- overwrite new_dist conf with prev_dist conf (this has lead to the problem
with DefaultResourceCalculator, my conf is bare minimum to work in
pseudo-distributed mode)
- start YARN
For any suggestions I would be grateful.
regards
tmp
2013/8/12 Ted Yu <yu...@gmail.com>
> Can you check the config entry
> for yarn.scheduler.capacity.resource-calculator ?
> It should point
> to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
>
> bq. I was able to fix all issues
>
> What other issues came up ?
>
> Thanks
>
>
> On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <tm...@gmail.com> wrote:
>
>> Hi again
>>
>> From a little investigation I have performed I have observed the
>> following. I assume the module responsible for this class is
>> hadoop-yarn-common.
>>
>> During RM init it crashes since it is looking for a class
>> DefaultResourceCalculator in
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
>> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
>> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
>> the RM crashes. Anybody knows how I can fix this? I would very grateful for
>> any help.
>>
>> regards
>> tmp
>>
>>
>> 2013/8/11 Rob Blah <tm...@gmail.com>
>>
>>> Hi
>>>
>>> I have a strange problem, regarding missing class, the
>>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>>> today it stopped working. I was able to fix all issues except the following
>>> problem in ResourceManager:
>>> 2013-08-11 12:12:42,425 FATAL
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>>> starting ResourceManager
>>> java.lang.RuntimeException: java.lang.RuntimeException:
>>> java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>>> at
>>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>>> ... 5 more
>>> Caused by: java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
>>> ... 6 more
>>> 2013-08-11 12:12:42,426 INFO
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
>>> SHUTDOWN_MSG:
>>>
>>> I build YARN from sources, daily updated to the newest revision in
>>> trunk. The class DefaultResourceCalculator exists and is present in YARN's
>>> sources. I am using (currently) trunk revision 1512895. I build YARN
>>> project with the following command:
>>> mvn clean package -Pdist -Dtar -DskipTests
>>> I create tar ball with the use of script provided in the sources:
>>> dist-tar-stitching.sh (hadoop-dist/target)
>>>
>>> regards
>>> tmp
>>>
>>
>>
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Rob Blah <tm...@gmail.com>.
Problem solved. Thank you for your help.
@Ted Yu
Other issues where my mistakes. I have a dedicated script which
updates/builds/"deploys" YARN from sources. I was starting NN with the
"-upgrade" option which unsynchronized NN version, also leading to broken
DN. Quick NN format and deletion of DN data solved the issue (I am working
on a sandbox cluster, so that is not a problem). I have modified the script
to start the NN without the upgrade option.
Two quick questions:
- when should I use the NN upgrade option, should it be only used to
upgrade NN between new version (example 2.0.4 -> 2.0.5). How can I automate
this process?
- Is the "design/functionality" of my magical script correct? How can I
avoid future problems like the solved one?
YARN update script
- update src to trunk (opt)
- package YARN
- build dist (tar ball)
- unpack new_dist
- overwrite new_dist conf with prev_dist conf (this has lead to the problem
with DefaultResourceCalculator, my conf is bare minimum to work in
pseudo-distributed mode)
- start YARN
For any suggestions I would be grateful.
regards
tmp
2013/8/12 Ted Yu <yu...@gmail.com>
> Can you check the config entry
> for yarn.scheduler.capacity.resource-calculator ?
> It should point
> to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
>
> bq. I was able to fix all issues
>
> What other issues came up ?
>
> Thanks
>
>
> On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <tm...@gmail.com> wrote:
>
>> Hi again
>>
>> From a little investigation I have performed I have observed the
>> following. I assume the module responsible for this class is
>> hadoop-yarn-common.
>>
>> During RM init it crashes since it is looking for a class
>> DefaultResourceCalculator in
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
>> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
>> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
>> the RM crashes. Anybody knows how I can fix this? I would very grateful for
>> any help.
>>
>> regards
>> tmp
>>
>>
>> 2013/8/11 Rob Blah <tm...@gmail.com>
>>
>>> Hi
>>>
>>> I have a strange problem, regarding missing class, the
>>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>>> today it stopped working. I was able to fix all issues except the following
>>> problem in ResourceManager:
>>> 2013-08-11 12:12:42,425 FATAL
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>>> starting ResourceManager
>>> java.lang.RuntimeException: java.lang.RuntimeException:
>>> java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>>> at
>>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>>> at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>>> ... 5 more
>>> Caused by: java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
>>> ... 6 more
>>> 2013-08-11 12:12:42,426 INFO
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
>>> SHUTDOWN_MSG:
>>>
>>> I build YARN from sources, daily updated to the newest revision in
>>> trunk. The class DefaultResourceCalculator exists and is present in YARN's
>>> sources. I am using (currently) trunk revision 1512895. I build YARN
>>> project with the following command:
>>> mvn clean package -Pdist -Dtar -DskipTests
>>> I create tar ball with the use of script provided in the sources:
>>> dist-tar-stitching.sh (hadoop-dist/target)
>>>
>>> regards
>>> tmp
>>>
>>
>>
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Ted Yu <yu...@gmail.com>.
Can you check the config entry
for yarn.scheduler.capacity.resource-calculator ?
It should point
to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
bq. I was able to fix all issues
What other issues came up ?
Thanks
On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <tm...@gmail.com> wrote:
> Hi again
>
> From a little investigation I have performed I have observed the
> following. I assume the module responsible for this class is
> hadoop-yarn-common.
>
> During RM init it crashes since it is looking for a class
> DefaultResourceCalculator in
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
> the RM crashes. Anybody knows how I can fix this? I would very grateful for
> any help.
>
> regards
> tmp
>
>
> 2013/8/11 Rob Blah <tm...@gmail.com>
>
>> Hi
>>
>> I have a strange problem, regarding missing class, the
>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>> today it stopped working. I was able to fix all issues except the following
>> problem in ResourceManager:
>> 2013-08-11 12:12:42,425 FATAL
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>> starting ResourceManager
>> java.lang.RuntimeException: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException: Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>> at
>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>> ... 5 more
>> Caused by: java.lang.ClassNotFoundException: Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
>> ... 6 more
>> 2013-08-11 12:12:42,426 INFO
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
>> SHUTDOWN_MSG:
>>
>> I build YARN from sources, daily updated to the newest revision in trunk.
>> The class DefaultResourceCalculator exists and is present in YARN's
>> sources. I am using (currently) trunk revision 1512895. I build YARN
>> project with the following command:
>> mvn clean package -Pdist -Dtar -DskipTests
>> I create tar ball with the use of script provided in the sources:
>> dist-tar-stitching.sh (hadoop-dist/target)
>>
>> regards
>> tmp
>>
>
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Ted Yu <yu...@gmail.com>.
Can you check the config entry
for yarn.scheduler.capacity.resource-calculator ?
It should point
to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
bq. I was able to fix all issues
What other issues came up ?
Thanks
On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <tm...@gmail.com> wrote:
> Hi again
>
> From a little investigation I have performed I have observed the
> following. I assume the module responsible for this class is
> hadoop-yarn-common.
>
> During RM init it crashes since it is looking for a class
> DefaultResourceCalculator in
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
> the RM crashes. Anybody knows how I can fix this? I would very grateful for
> any help.
>
> regards
> tmp
>
>
> 2013/8/11 Rob Blah <tm...@gmail.com>
>
>> Hi
>>
>> I have a strange problem, regarding missing class, the
>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>> today it stopped working. I was able to fix all issues except the following
>> problem in ResourceManager:
>> 2013-08-11 12:12:42,425 FATAL
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>> starting ResourceManager
>> java.lang.RuntimeException: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException: Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>> at
>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>> ... 5 more
>> Caused by: java.lang.ClassNotFoundException: Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
>> ... 6 more
>> 2013-08-11 12:12:42,426 INFO
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
>> SHUTDOWN_MSG:
>>
>> I build YARN from sources, daily updated to the newest revision in trunk.
>> The class DefaultResourceCalculator exists and is present in YARN's
>> sources. I am using (currently) trunk revision 1512895. I build YARN
>> project with the following command:
>> mvn clean package -Pdist -Dtar -DskipTests
>> I create tar ball with the use of script provided in the sources:
>> dist-tar-stitching.sh (hadoop-dist/target)
>>
>> regards
>> tmp
>>
>
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Ted Yu <yu...@gmail.com>.
Can you check the config entry
for yarn.scheduler.capacity.resource-calculator ?
It should point
to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
bq. I was able to fix all issues
What other issues came up ?
Thanks
On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <tm...@gmail.com> wrote:
> Hi again
>
> From a little investigation I have performed I have observed the
> following. I assume the module responsible for this class is
> hadoop-yarn-common.
>
> During RM init it crashes since it is looking for a class
> DefaultResourceCalculator in
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
> the RM crashes. Anybody knows how I can fix this? I would very grateful for
> any help.
>
> regards
> tmp
>
>
> 2013/8/11 Rob Blah <tm...@gmail.com>
>
>> Hi
>>
>> I have a strange problem, regarding missing class, the
>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>> today it stopped working. I was able to fix all issues except the following
>> problem in ResourceManager:
>> 2013-08-11 12:12:42,425 FATAL
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>> starting ResourceManager
>> java.lang.RuntimeException: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException: Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>> at
>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>> ... 5 more
>> Caused by: java.lang.ClassNotFoundException: Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
>> ... 6 more
>> 2013-08-11 12:12:42,426 INFO
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
>> SHUTDOWN_MSG:
>>
>> I build YARN from sources, daily updated to the newest revision in trunk.
>> The class DefaultResourceCalculator exists and is present in YARN's
>> sources. I am using (currently) trunk revision 1512895. I build YARN
>> project with the following command:
>> mvn clean package -Pdist -Dtar -DskipTests
>> I create tar ball with the use of script provided in the sources:
>> dist-tar-stitching.sh (hadoop-dist/target)
>>
>> regards
>> tmp
>>
>
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Ted Yu <yu...@gmail.com>.
Can you check the config entry
for yarn.scheduler.capacity.resource-calculator ?
It should point
to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
bq. I was able to fix all issues
What other issues came up ?
Thanks
On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <tm...@gmail.com> wrote:
> Hi again
>
> From a little investigation I have performed I have observed the
> following. I assume the module responsible for this class is
> hadoop-yarn-common.
>
> During RM init it crashes since it is looking for a class
> DefaultResourceCalculator in
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
> the RM crashes. Anybody knows how I can fix this? I would very grateful for
> any help.
>
> regards
> tmp
>
>
> 2013/8/11 Rob Blah <tm...@gmail.com>
>
>> Hi
>>
>> I have a strange problem, regarding missing class, the
>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>> today it stopped working. I was able to fix all issues except the following
>> problem in ResourceManager:
>> 2013-08-11 12:12:42,425 FATAL
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>> starting ResourceManager
>> java.lang.RuntimeException: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException: Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>> at
>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>> ... 5 more
>> Caused by: java.lang.ClassNotFoundException: Class
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>> not found
>> at
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
>> ... 6 more
>> 2013-08-11 12:12:42,426 INFO
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
>> SHUTDOWN_MSG:
>>
>> I build YARN from sources, daily updated to the newest revision in trunk.
>> The class DefaultResourceCalculator exists and is present in YARN's
>> sources. I am using (currently) trunk revision 1512895. I build YARN
>> project with the following command:
>> mvn clean package -Pdist -Dtar -DskipTests
>> I create tar ball with the use of script provided in the sources:
>> dist-tar-stitching.sh (hadoop-dist/target)
>>
>> regards
>> tmp
>>
>
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Rob Blah <tm...@gmail.com>.
Hi again
>From a little investigation I have performed I have observed the following.
I assume the module responsible for this class is hadoop-yarn-common.
During RM init it crashes since it is looking for a class
DefaultResourceCalculator in
org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
the RM crashes. Anybody knows how I can fix this? I would very grateful for
any help.
regards
tmp
2013/8/11 Rob Blah <tm...@gmail.com>
> Hi
>
> I have a strange problem, regarding missing class, the
> DefaultResourceCalculator. I have a single node sandbox cluster working in
> a pseudo-distributed mode. The cluster was working fine yesterday, however
> today it stopped working. I was able to fix all issues except the following
> problem in ResourceManager:
> 2013-08-11 12:12:42,425 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
> starting ResourceManager
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
> ... 6 more
> 2013-08-11 12:12:42,426 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
> SHUTDOWN_MSG:
>
> I build YARN from sources, daily updated to the newest revision in trunk.
> The class DefaultResourceCalculator exists and is present in YARN's
> sources. I am using (currently) trunk revision 1512895. I build YARN
> project with the following command:
> mvn clean package -Pdist -Dtar -DskipTests
> I create tar ball with the use of script provided in the sources:
> dist-tar-stitching.sh (hadoop-dist/target)
>
> regards
> tmp
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Rob Blah <tm...@gmail.com>.
Hi again
>From a little investigation I have performed I have observed the following.
I assume the module responsible for this class is hadoop-yarn-common.
During RM init it crashes since it is looking for a class
DefaultResourceCalculator in
org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
the RM crashes. Anybody knows how I can fix this? I would very grateful for
any help.
regards
tmp
2013/8/11 Rob Blah <tm...@gmail.com>
> Hi
>
> I have a strange problem, regarding missing class, the
> DefaultResourceCalculator. I have a single node sandbox cluster working in
> a pseudo-distributed mode. The cluster was working fine yesterday, however
> today it stopped working. I was able to fix all issues except the following
> problem in ResourceManager:
> 2013-08-11 12:12:42,425 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
> starting ResourceManager
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
> ... 6 more
> 2013-08-11 12:12:42,426 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
> SHUTDOWN_MSG:
>
> I build YARN from sources, daily updated to the newest revision in trunk.
> The class DefaultResourceCalculator exists and is present in YARN's
> sources. I am using (currently) trunk revision 1512895. I build YARN
> project with the following command:
> mvn clean package -Pdist -Dtar -DskipTests
> I create tar ball with the use of script provided in the sources:
> dist-tar-stitching.sh (hadoop-dist/target)
>
> regards
> tmp
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Rob Blah <tm...@gmail.com>.
Hi again
>From a little investigation I have performed I have observed the following.
I assume the module responsible for this class is hadoop-yarn-common.
During RM init it crashes since it is looking for a class
DefaultResourceCalculator in
org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
the RM crashes. Anybody knows how I can fix this? I would very grateful for
any help.
regards
tmp
2013/8/11 Rob Blah <tm...@gmail.com>
> Hi
>
> I have a strange problem, regarding missing class, the
> DefaultResourceCalculator. I have a single node sandbox cluster working in
> a pseudo-distributed mode. The cluster was working fine yesterday, however
> today it stopped working. I was able to fix all issues except the following
> problem in ResourceManager:
> 2013-08-11 12:12:42,425 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
> starting ResourceManager
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
> ... 6 more
> 2013-08-11 12:12:42,426 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
> SHUTDOWN_MSG:
>
> I build YARN from sources, daily updated to the newest revision in trunk.
> The class DefaultResourceCalculator exists and is present in YARN's
> sources. I am using (currently) trunk revision 1512895. I build YARN
> project with the following command:
> mvn clean package -Pdist -Dtar -DskipTests
> I create tar ball with the use of script provided in the sources:
> dist-tar-stitching.sh (hadoop-dist/target)
>
> regards
> tmp
>
Re: DefaultResourceCalculator class not found, ResourceManager fails
to start.
Posted by Rob Blah <tm...@gmail.com>.
Hi again
>From a little investigation I have performed I have observed the following.
I assume the module responsible for this class is hadoop-yarn-common.
During RM init it crashes since it is looking for a class
DefaultResourceCalculator in
org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
the RM crashes. Anybody knows how I can fix this? I would very grateful for
any help.
regards
tmp
2013/8/11 Rob Blah <tm...@gmail.com>
> Hi
>
> I have a strange problem, regarding missing class, the
> DefaultResourceCalculator. I have a single node sandbox cluster working in
> a pseudo-distributed mode. The cluster was working fine yesterday, however
> today it stopped working. I was able to fix all issues except the following
> problem in ResourceManager:
> 2013-08-11 12:12:42,425 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
> starting ResourceManager
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
> not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719)
> ... 6 more
> 2013-08-11 12:12:42,426 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
> SHUTDOWN_MSG:
>
> I build YARN from sources, daily updated to the newest revision in trunk.
> The class DefaultResourceCalculator exists and is present in YARN's
> sources. I am using (currently) trunk revision 1512895. I build YARN
> project with the following command:
> mvn clean package -Pdist -Dtar -DskipTests
> I create tar ball with the use of script provided in the sources:
> dist-tar-stitching.sh (hadoop-dist/target)
>
> regards
> tmp
>