You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by Markus Mock <ma...@gmail.com> on 2012/10/04 20:04:59 UTC
Problem with CloudStone4 benchmark in accumulo-1.4.1
Hi I am running into this problem with the CloudStone4 benchmark:
the mapreduce job it launches fails with:
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.accumulo.core.clien\
t.mapreduce.AccumuloOutputFormat^M
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1004)^M
at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)^\
M
at org.apache.hadoop.mapred.Task.initialize(Task.java:509)^M
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)^M
at org.apache.hadoop.mapred.Child$4.run(Child.java:266)^M
at java.security.AccessController.doPrivileged(Native Method)^M
at javax.security.auth.Subject.doAs(Subject.java:396)^M
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:12\
78)^M
at org.apache.hadoop.mapred.Child.main(Child.java:260)^M
Caused by: java.lang.ClassNotFoundException:
org.apache.accumulo.core.client.mapreduce.Accum\
uloOutputFormat^M
I added some printout to the code to see how it invokes / launches that job:
Running TeraSortIngest
about to call runner.start
running: ['/usr/lib/hadoop-0.20/accumulo/accumulo-1.4.1/bin/accumulo',
'org.apache.accumulo\
.examples.simple.mapreduce.TeraSortIngest', '-libjars',
'/usr/lib/hadoop-0.20/accumulo/accum\
ulo-1.4.1/lib/accumulo-core-1.4.1.jar,/usr/lib/hadoop-0.20/accumulo/accumulo-1.4.1/lib/libth\
rift-0.6.1.jar,/usr/lib/hadoop/accumulo/zookeeper-3.3.6/zookeeper-3.3.6.jar,/usr/lib/hadoop-\
0.20/accumulo/accumulo-1.4.1/lib/accumulo-start-1.4.1.jar',
'10000000000', '10', '10', '78',\
'78', 'CloudIngestTest', 'markus-dist-test', '<IP edited out>',
'root', 'secret', '400']^M
Is this a known issue? What would be the workaround? It seems that the
tasks cannot find the
AccumuloOutputFormat class, however, it is on the -libjars line and I
also checked the task trackers
and the jar files seem to be in the distributed cache, so any idea why
the tasks don't find them?
Is that benchmark broken?
Thanks.
-- Markus
Re: Problem with CloudStone4 benchmark in accumulo-1.4.1
Posted by Josh Elser <jo...@gmail.com>.
Libjars should be a comma-separated list whereas Java classpath is a
colon-separated list.
On 10/04/2012 07:41 PM, Ranjan Sen wrote:
> I ran this test successfully before. I have two points - why are the
> paths to the libjar items separated by commas? I thought they should
> be ':'. The other question is do you have the $ACCUMULO_HOME/lib to
> have these jars? Sorry if you have already checked this, but thought
> of writing about this.
> Ranjan
>
> ------------------------------------------------------------------------
> Date: Thu, 4 Oct 2012 17:18:59 -0400
> Subject: Re: Problem with CloudStone4 benchmark in accumulo-1.4.1
> From: eric.newton@gmail.com
> To: user@accumulo.apache.org; markus.mock@gmail.com
>
> It's not a known issue, but the benchmarks don't get the love they
> should. Please make a ticket.
>
> Thanks!
>
> -Eric
>
> On Thu, Oct 4, 2012 at 2:04 PM, Markus Mock <markus.mock@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi I am running into this problem with the CloudStone4 benchmark:
>
> the mapreduce job it launches fails with:
>
>
> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.accumulo.core.clien\
> t.mapreduce.AccumuloOutputFormat^M
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1004)^M
> at
> org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)^\
> M
> at org.apache.hadoop.mapred.Task.initialize(Task.java:509)^M
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)^M
> at org.apache.hadoop.mapred.Child$4.run(Child.java:266)^M
> at java.security.AccessController.doPrivileged(Native
> Method)^M
> at javax.security.auth.Subject.doAs(Subject.java:396)^M
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:12\
> 78)^M
> at org.apache.hadoop.mapred.Child.main(Child.java:260)^M
> Caused by: java.lang.ClassNotFoundException:
> org.apache.accumulo.core.client.mapreduce.Accum\
> uloOutputFormat^M
>
> I added some printout to the code to see how it invokes / launches
> that job:
>
>
> Running TeraSortIngest
> about to call runner.start
> running:
> ['/usr/lib/hadoop-0.20/accumulo/accumulo-1.4.1/bin/accumulo',
> 'org.apache.accumulo\
> .examples.simple.mapreduce.TeraSortIngest', '-libjars',
> '/usr/lib/hadoop-0.20/accumulo/accum\
> ulo-1.4.1/lib/accumulo-core-1.4.1.jar,/usr/lib/hadoop-0.20/accumulo/accumulo-1.4.1/lib/libth\
> rift-0.6.1.jar,/usr/lib/hadoop/accumulo/zookeeper-3.3.6/zookeeper-3.3.6.jar,/usr/lib/hadoop-\
> 0.20/accumulo/accumulo-1.4.1/lib/accumulo-start-1.4.1.jar',
> '10000000000', '10', '10', '78',\
> '78', 'CloudIngestTest', 'markus-dist-test', '<IP edited out>',
> 'root', 'secret', '400']^M
>
> Is this a known issue? What would be the workaround? It seems that the
> tasks cannot find the
> AccumuloOutputFormat class, however, it is on the -libjars line and I
> also checked the task trackers
> and the jar files seem to be in the distributed cache, so any idea why
> the tasks don't find them?
> Is that benchmark broken?
>
> Thanks.
>
> -- Markus
>
>
RE: Problem with CloudStone4 benchmark in accumulo-1.4.1
Posted by Ranjan Sen <ra...@hotmail.com>.
I ran this test successfully before. I have two points - why are the paths to the libjar items separated by commas? I thought they should be ':'. The other question is do you have the $ACCUMULO_HOME/lib to have these jars? Sorry if you have already checked this, but thought of writing about this. Ranjan
Date: Thu, 4 Oct 2012 17:18:59 -0400
Subject: Re: Problem with CloudStone4 benchmark in accumulo-1.4.1
From: eric.newton@gmail.com
To: user@accumulo.apache.org; markus.mock@gmail.com
It's not a known issue, but the benchmarks don't get the love they should. Please make a ticket.
Thanks!
-Eric
On Thu, Oct 4, 2012 at 2:04 PM, Markus Mock <ma...@gmail.com> wrote:
Hi I am running into this problem with the CloudStone4 benchmark:
the mapreduce job it launches fails with:
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.accumulo.core.clien\
t.mapreduce.AccumuloOutputFormat^M
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1004)^M
at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)^\
M
at org.apache.hadoop.mapred.Task.initialize(Task.java:509)^M
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)^M
at org.apache.hadoop.mapred.Child$4.run(Child.java:266)^M
at java.security.AccessController.doPrivileged(Native Method)^M
at javax.security.auth.Subject.doAs(Subject.java:396)^M
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:12\
78)^M
at org.apache.hadoop.mapred.Child.main(Child.java:260)^M
Caused by: java.lang.ClassNotFoundException:
org.apache.accumulo.core.client.mapreduce.Accum\
uloOutputFormat^M
I added some printout to the code to see how it invokes / launches that job:
Running TeraSortIngest
about to call runner.start
running: ['/usr/lib/hadoop-0.20/accumulo/accumulo-1.4.1/bin/accumulo',
'org.apache.accumulo\
.examples.simple.mapreduce.TeraSortIngest', '-libjars',
'/usr/lib/hadoop-0.20/accumulo/accum\
ulo-1.4.1/lib/accumulo-core-1.4.1.jar,/usr/lib/hadoop-0.20/accumulo/accumulo-1.4.1/lib/libth\
rift-0.6.1.jar,/usr/lib/hadoop/accumulo/zookeeper-3.3.6/zookeeper-3.3.6.jar,/usr/lib/hadoop-\
0.20/accumulo/accumulo-1.4.1/lib/accumulo-start-1.4.1.jar',
'10000000000', '10', '10', '78',\
'78', 'CloudIngestTest', 'markus-dist-test', '<IP edited out>',
'root', 'secret', '400']^M
Is this a known issue? What would be the workaround? It seems that the
tasks cannot find the
AccumuloOutputFormat class, however, it is on the -libjars line and I
also checked the task trackers
and the jar files seem to be in the distributed cache, so any idea why
the tasks don't find them?
Is that benchmark broken?
Thanks.
-- Markus
Re: Problem with CloudStone4 benchmark in accumulo-1.4.1
Posted by Eric Newton <er...@gmail.com>.
It's not a known issue, but the benchmarks don't get the love they should.
Please make a ticket.
Thanks!
-Eric
On Thu, Oct 4, 2012 at 2:04 PM, Markus Mock <ma...@gmail.com> wrote:
> Hi I am running into this problem with the CloudStone4 benchmark:
>
> the mapreduce job it launches fails with:
>
>
> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.accumulo.core.clien\
> t.mapreduce.AccumuloOutputFormat^M
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1004)^M
> at
> org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)^\
> M
> at org.apache.hadoop.mapred.Task.initialize(Task.java:509)^M
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)^M
> at org.apache.hadoop.mapred.Child$4.run(Child.java:266)^M
> at java.security.AccessController.doPrivileged(Native Method)^M
> at javax.security.auth.Subject.doAs(Subject.java:396)^M
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:12\
> 78)^M
> at org.apache.hadoop.mapred.Child.main(Child.java:260)^M
> Caused by: java.lang.ClassNotFoundException:
> org.apache.accumulo.core.client.mapreduce.Accum\
> uloOutputFormat^M
>
> I added some printout to the code to see how it invokes / launches that
> job:
>
>
> Running TeraSortIngest
> about to call runner.start
> running: ['/usr/lib/hadoop-0.20/accumulo/accumulo-1.4.1/bin/accumulo',
> 'org.apache.accumulo\
> .examples.simple.mapreduce.TeraSortIngest', '-libjars',
> '/usr/lib/hadoop-0.20/accumulo/accum\
>
> ulo-1.4.1/lib/accumulo-core-1.4.1.jar,/usr/lib/hadoop-0.20/accumulo/accumulo-1.4.1/lib/libth\
>
> rift-0.6.1.jar,/usr/lib/hadoop/accumulo/zookeeper-3.3.6/zookeeper-3.3.6.jar,/usr/lib/hadoop-\
> 0.20/accumulo/accumulo-1.4.1/lib/accumulo-start-1.4.1.jar',
> '10000000000', '10', '10', '78',\
> '78', 'CloudIngestTest', 'markus-dist-test', '<IP edited out>',
> 'root', 'secret', '400']^M
>
> Is this a known issue? What would be the workaround? It seems that the
> tasks cannot find the
> AccumuloOutputFormat class, however, it is on the -libjars line and I
> also checked the task trackers
> and the jar files seem to be in the distributed cache, so any idea why
> the tasks don't find them?
> Is that benchmark broken?
>
> Thanks.
>
> -- Markus
>