You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Shravan Mahankali <sh...@catalytic.com> on 2009/06/17 14:35:54 UTC
not a SequenceFile?
Hi Group,
I have trouble running couple of examples provided by Hadoop. Below are the
error messages I have from the console, could you please advise what could
be the problem and probable solution?
09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
: 1
09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
: 1
09/06/17 16:30:29 INFO mapred.JobClient: Running job: job_200906171601_0009
09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%
09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
attempt_200906171601_0009_m_000000_0, Status : FAILED
java.io.IOException: hdfs://localhost:9000/user/root/words not a
SequenceFile
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)
at
org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
eader.java:43)
at
org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
eInputFormat.java:54)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
attempt_200906171601_0009_r_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createInstance(UserDefinedValueAggregatorDescriptor.java:57)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
<init>(UserDefinedValueAggregatorDescriptor.java:76)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
atorDescriptor(ValueAggregatorJobBase.java:54)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
escriptors(ValueAggregatorJobBase.java:65)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
ec(ValueAggregatorJobBase.java:74)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
eAggregatorJobBase.java:42)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:242)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createInstance(UserDefinedValueAggregatorDescriptor.java:52)
... 10 more
Thank You,
Shravan Kumar. M
Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
-----------------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system
administrator - <ma...@catalytic.com>
netopshelpdesk@catalytic.com
Re: not a SequenceFile?
Posted by "Eason.Lee" <le...@gmail.com>.
http://www.nabble.com/ClassNotFoundException-td23441528.html<http://www.nabble.com/ClassNotFoundException-td23441528.html>
<http://www.nabble.com/ClassNotFoundException-td23441528.html>You must make
all of the required jars available to all of your tasks. You
can either install them all the tasktracker machines and setup the
tasktracker classpath to include them, or distributed them via the
distributed cache.
2009/6/18 Shravan Mahankali <sh...@catalytic.com>
> Thanks for your suggestion Eason.
>
> I think sequence file issue is resolved, how ever success is far on the
> hill
> top!!!
>
> *Below is the command I have executed in the terminal:*
> #/bin/hadoop jar AggregateWordCount.jar
> org.apache.hadoop.examples.AggregateWordCount words/*
> aggregatewordcount_output 2 textinputformat
>
> *Error:*
>
> 09/06/18 11:51:42 INFO mapred.JobClient: Task Id :
> attempt_200906181145_0005_r_000001_1, Status : FAILED
> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createInstance(UserDefinedValueAggregatorDescriptor.java:57)
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> <init>(UserDefinedValueAggregatorDescriptor.java:76)
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
> atorDescriptor(ValueAggregatorJobBase.java:54)
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
> escriptors(ValueAggregatorJobBase.java:65)
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
> ec(ValueAggregatorJobBase.java:74)
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
> eAggregatorJobBase.java:42)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:242)
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createInstance(UserDefinedValueAggregatorDescriptor.java:52)
> ... 10 more
>
>
> *However, I could see that
> org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass class is
> available in the AggregateWordCount.jar, below is the listing of this jar:
> *
>
> # jar -tvf AggregateWordCount.jar
> 0 Tue Jun 09 18:11:42 IST 2009 META-INF/
> 71 Tue Jun 09 18:11:42 IST 2009 META-INF/MANIFEST.MF
> 0 Tue Jun 09 18:10:58 IST 2009 org/
> 0 Tue Jun 09 18:10:58 IST 2009 org/apache/
> 0 Tue Jun 09 18:10:58 IST 2009 org/apache/hadoop/
> 0 Tue Jun 09 18:10:58 IST 2009 org/apache/hadoop/examples/
> 1298 Tue Jun 09 18:11:32 IST 2009
> org/apache/hadoop/examples/AggregateWordCount$WordCountPlugInClass.class
> 846 Tue Jun 09 18:11:32 IST 2009
> org/apache/hadoop/examples/AggregateWordCount.class
>
>
> Could you please advice what could be the problem here?
>
> Thank You,
> Shravan Kumar. M
> Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
> -----------------------------
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the system
> administrator - netopshelpdesk@catalytic.com
> -----Original Message-----
> From: Eason.Lee [mailto:leongfans@gmail.com]
> Sent: Thursday, June 18, 2009 11:27 AM
> To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
> Subject: Re: not a SequenceFile?
>
> you'd better run it like this:
>
> bin/hadoop jar hadoop-*-examples.jar aggregatewordcount input output
> numOfReducers *textinputformat*
> *
> *
> input textinputformat to use textinputformat instand of SequenceFile
>
> hope it is helpful!
>
> 2009/6/18 Shravan Mahankali <sh...@catalytic.com>
>
> > Hi Nick,
> >
> >
> >
> > Thanks for your response.
> >
> >
> >
> > I am very new to Hadoop. I was trying to execute the "AggregateWordCount"
> > example program provided by Hadoop in the distribution from the linux but
> > was having the issue as stated in my earlier email.
> >
> >
> >
> > I have also tried executing the MultiFetch example with no success.
> >
> >
> >
> > If this example program needs the input file to be a sequence file. how
> > should I create one, please advice?
> >
> >
> >
> > Thank You,
> >
> > Shravan Kumar. M
> >
> > Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
> >
> > -----------------------------
> >
> > This email and any files transmitted with it are confidential and
> intended
> > solely for the use of the individual or entity to whom they are
> addressed.
> > If you have received this email in error please notify the system
> > administrator - <ma...@catalytic.com>
> > netopshelpdesk@catalytic.com
> >
> > _____
> >
> > From: Nick Cen [mailto:cenyongh@gmail.com]
> > Sent: Wednesday, June 17, 2009 6:50 PM
> > To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
> > Subject: Re: not a SequenceFile?
> >
> >
> >
> > I guest you have set SequenceFileFormat as your inputformat in the
> > configuration object, but the file you provide is not a sequence file.
> >
> > 2009/6/17 Shravan Mahankali <sh...@catalytic.com>
> >
> > Hi Group,
> >
> >
> >
> > I have trouble running couple of examples provided by Hadoop. Below are
> the
> > error messages I have from the console, could you please advise what
> could
> > be the problem and probable solution?
> >
> >
> >
> > 09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to
> process
> > : 1
> >
> > 09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to
> process
> > : 1
> >
> > 09/06/17 16:30:29 INFO mapred.JobClient: Running job:
> job_200906171601_0009
> >
> > 09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%
> >
> > 09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
> > attempt_200906171601_0009_m_000000_0, Status : FAILED
> >
> > java.io.IOException: hdfs://localhost:9000/user/root/words not a
> > SequenceFile
> >
> > at
> > org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)
> >
> > at
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
> >
> > at
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)
> >
> > at
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
> > eader.java:43)
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
> > eInputFormat.java:54)
> >
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> >
> > at
> > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
> >
> >
> >
> > 09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
> > attempt_200906171601_0009_r_000000_0, Status : FAILED
> >
> > java.lang.RuntimeException: java.lang.ClassNotFoundException:
> > org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> > createInstance(UserDefinedValueAggregatorDescriptor.java:57)
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> > createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> > <init>(UserDefinedValueAggregatorDescriptor.java:76)
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
> > atorDescriptor(ValueAggregatorJobBase.java:54)
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
> > escriptors(ValueAggregatorJobBase.java:65)
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
> > ec(ValueAggregatorJobBase.java:74)
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
> > eAggregatorJobBase.java:42)
> >
> > at
> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> >
> > at
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
> >
> > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
> >
> > at
> > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
> >
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
> >
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> >
> > at java.security.AccessController.doPrivileged(Native Method)
> >
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> >
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> >
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
> >
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> >
> > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> >
> > at java.lang.Class.forName0(Native Method)
> >
> > at java.lang.Class.forName(Class.java:242)
> >
> > at
> >
> >
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> > createInstance(UserDefinedValueAggregatorDescriptor.java:52)
> >
> > ... 10 more
> >
> >
> >
> > Thank You,
> >
> > Shravan Kumar. M
> >
> > Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
> >
> > -----------------------------
> >
> > This email and any files transmitted with it are confidential and
> intended
> > solely for the use of the individual or entity to whom they are
> addressed.
> > If you have received this email in error please notify the system
> > administrator - <ma...@catalytic.com>
> > netopshelpdesk@catalytic.com
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> > http://daily.appspot.com/food/
> >
> >
>
>
RE: not a SequenceFile?
Posted by Shravan Mahankali <sh...@catalytic.com>.
Thanks for your suggestion Eason.
I think sequence file issue is resolved, how ever success is far on the hill
top!!!
*Below is the command I have executed in the terminal:*
#/bin/hadoop jar AggregateWordCount.jar
org.apache.hadoop.examples.AggregateWordCount words/*
aggregatewordcount_output 2 textinputformat
*Error:*
09/06/18 11:51:42 INFO mapred.JobClient: Task Id :
attempt_200906181145_0005_r_000001_1, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createInstance(UserDefinedValueAggregatorDescriptor.java:57)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
<init>(UserDefinedValueAggregatorDescriptor.java:76)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
atorDescriptor(ValueAggregatorJobBase.java:54)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
escriptors(ValueAggregatorJobBase.java:65)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
ec(ValueAggregatorJobBase.java:74)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
eAggregatorJobBase.java:42)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:242)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createInstance(UserDefinedValueAggregatorDescriptor.java:52)
... 10 more
*However, I could see that
org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass class is
available in the AggregateWordCount.jar, below is the listing of this jar: *
# jar -tvf AggregateWordCount.jar
0 Tue Jun 09 18:11:42 IST 2009 META-INF/
71 Tue Jun 09 18:11:42 IST 2009 META-INF/MANIFEST.MF
0 Tue Jun 09 18:10:58 IST 2009 org/
0 Tue Jun 09 18:10:58 IST 2009 org/apache/
0 Tue Jun 09 18:10:58 IST 2009 org/apache/hadoop/
0 Tue Jun 09 18:10:58 IST 2009 org/apache/hadoop/examples/
1298 Tue Jun 09 18:11:32 IST 2009
org/apache/hadoop/examples/AggregateWordCount$WordCountPlugInClass.class
846 Tue Jun 09 18:11:32 IST 2009
org/apache/hadoop/examples/AggregateWordCount.class
Could you please advice what could be the problem here?
Thank You,
Shravan Kumar. M
Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
-----------------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system
administrator - netopshelpdesk@catalytic.com
-----Original Message-----
From: Eason.Lee [mailto:leongfans@gmail.com]
Sent: Thursday, June 18, 2009 11:27 AM
To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
Subject: Re: not a SequenceFile?
you'd better run it like this:
bin/hadoop jar hadoop-*-examples.jar aggregatewordcount input output
numOfReducers *textinputformat*
*
*
input textinputformat to use textinputformat instand of SequenceFile
hope it is helpful!
2009/6/18 Shravan Mahankali <sh...@catalytic.com>
> Hi Nick,
>
>
>
> Thanks for your response.
>
>
>
> I am very new to Hadoop. I was trying to execute the "AggregateWordCount"
> example program provided by Hadoop in the distribution from the linux but
> was having the issue as stated in my earlier email.
>
>
>
> I have also tried executing the MultiFetch example with no success.
>
>
>
> If this example program needs the input file to be a sequence file. how
> should I create one, please advice?
>
>
>
> Thank You,
>
> Shravan Kumar. M
>
> Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
>
> -----------------------------
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the system
> administrator - <ma...@catalytic.com>
> netopshelpdesk@catalytic.com
>
> _____
>
> From: Nick Cen [mailto:cenyongh@gmail.com]
> Sent: Wednesday, June 17, 2009 6:50 PM
> To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
> Subject: Re: not a SequenceFile?
>
>
>
> I guest you have set SequenceFileFormat as your inputformat in the
> configuration object, but the file you provide is not a sequence file.
>
> 2009/6/17 Shravan Mahankali <sh...@catalytic.com>
>
> Hi Group,
>
>
>
> I have trouble running couple of examples provided by Hadoop. Below are
the
> error messages I have from the console, could you please advise what could
> be the problem and probable solution?
>
>
>
> 09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to
process
> : 1
>
> 09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to
process
> : 1
>
> 09/06/17 16:30:29 INFO mapred.JobClient: Running job:
job_200906171601_0009
>
> 09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%
>
> 09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
> attempt_200906171601_0009_m_000000_0, Status : FAILED
>
> java.io.IOException: hdfs://localhost:9000/user/root/words not a
> SequenceFile
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)
>
> at
>
>
org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
> eader.java:43)
>
> at
>
>
org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
> eInputFormat.java:54)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
>
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
>
>
>
> 09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
> attempt_200906171601_0009_r_000000_0, Status : FAILED
>
> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
>
> at
>
>
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createInstance(UserDefinedValueAggregatorDescriptor.java:57)
>
> at
>
>
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
>
> at
>
>
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> <init>(UserDefinedValueAggregatorDescriptor.java:76)
>
> at
>
>
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
> atorDescriptor(ValueAggregatorJobBase.java:54)
>
> at
>
>
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
> escriptors(ValueAggregatorJobBase.java:65)
>
> at
>
>
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
> ec(ValueAggregatorJobBase.java:74)
>
> at
>
>
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
> eAggregatorJobBase.java:42)
>
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
>
> at
>
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
>
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
>
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
>
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>
> at java.lang.Class.forName0(Native Method)
>
> at java.lang.Class.forName(Class.java:242)
>
> at
>
>
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createInstance(UserDefinedValueAggregatorDescriptor.java:52)
>
> ... 10 more
>
>
>
> Thank You,
>
> Shravan Kumar. M
>
> Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
>
> -----------------------------
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the system
> administrator - <ma...@catalytic.com>
> netopshelpdesk@catalytic.com
>
>
>
>
>
>
>
>
>
> --
> http://daily.appspot.com/food/
>
>
Re: not a SequenceFile?
Posted by "Eason.Lee" <le...@gmail.com>.
you'd better run it like this:
bin/hadoop jar hadoop-*-examples.jar aggregatewordcount input output
numOfReducers *textinputformat*
*
*
input textinputformat to use textinputformat instand of SequenceFile
hope it is helpful!
2009/6/18 Shravan Mahankali <sh...@catalytic.com>
> Hi Nick,
>
>
>
> Thanks for your response.
>
>
>
> I am very new to Hadoop. I was trying to execute the "AggregateWordCount"
> example program provided by Hadoop in the distribution from the linux but
> was having the issue as stated in my earlier email.
>
>
>
> I have also tried executing the MultiFetch example with no success.
>
>
>
> If this example program needs the input file to be a sequence file. how
> should I create one, please advice?
>
>
>
> Thank You,
>
> Shravan Kumar. M
>
> Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
>
> -----------------------------
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the system
> administrator - <ma...@catalytic.com>
> netopshelpdesk@catalytic.com
>
> _____
>
> From: Nick Cen [mailto:cenyongh@gmail.com]
> Sent: Wednesday, June 17, 2009 6:50 PM
> To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
> Subject: Re: not a SequenceFile?
>
>
>
> I guest you have set SequenceFileFormat as your inputformat in the
> configuration object, but the file you provide is not a sequence file.
>
> 2009/6/17 Shravan Mahankali <sh...@catalytic.com>
>
> Hi Group,
>
>
>
> I have trouble running couple of examples provided by Hadoop. Below are the
> error messages I have from the console, could you please advise what could
> be the problem and probable solution?
>
>
>
> 09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
> : 1
>
> 09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
> : 1
>
> 09/06/17 16:30:29 INFO mapred.JobClient: Running job: job_200906171601_0009
>
> 09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%
>
> 09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
> attempt_200906171601_0009_m_000000_0, Status : FAILED
>
> java.io.IOException: hdfs://localhost:9000/user/root/words not a
> SequenceFile
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)
>
> at
>
> org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
> eader.java:43)
>
> at
>
> org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
> eInputFormat.java:54)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
>
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
>
>
>
> 09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
> attempt_200906171601_0009_r_000000_0, Status : FAILED
>
> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createInstance(UserDefinedValueAggregatorDescriptor.java:57)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> <init>(UserDefinedValueAggregatorDescriptor.java:76)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
> atorDescriptor(ValueAggregatorJobBase.java:54)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
> escriptors(ValueAggregatorJobBase.java:65)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
> ec(ValueAggregatorJobBase.java:74)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
> eAggregatorJobBase.java:42)
>
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
>
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
>
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
>
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
>
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>
> at java.lang.Class.forName0(Native Method)
>
> at java.lang.Class.forName(Class.java:242)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createInstance(UserDefinedValueAggregatorDescriptor.java:52)
>
> ... 10 more
>
>
>
> Thank You,
>
> Shravan Kumar. M
>
> Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
>
> -----------------------------
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the system
> administrator - <ma...@catalytic.com>
> netopshelpdesk@catalytic.com
>
>
>
>
>
>
>
>
>
> --
> http://daily.appspot.com/food/
>
>
RE: not a SequenceFile?
Posted by Shravan Mahankali <sh...@catalytic.com>.
Hi Nick,
Thanks for your response.
I am very new to Hadoop. I was trying to execute the "AggregateWordCount"
example program provided by Hadoop in the distribution from the linux but
was having the issue as stated in my earlier email.
I have also tried executing the MultiFetch example with no success.
If this example program needs the input file to be a sequence file. how
should I create one, please advice?
Thank You,
Shravan Kumar. M
Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
-----------------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system
administrator - <ma...@catalytic.com>
netopshelpdesk@catalytic.com
_____
From: Nick Cen [mailto:cenyongh@gmail.com]
Sent: Wednesday, June 17, 2009 6:50 PM
To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
Subject: Re: not a SequenceFile?
I guest you have set SequenceFileFormat as your inputformat in the
configuration object, but the file you provide is not a sequence file.
2009/6/17 Shravan Mahankali <sh...@catalytic.com>
Hi Group,
I have trouble running couple of examples provided by Hadoop. Below are the
error messages I have from the console, could you please advise what could
be the problem and probable solution?
09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
: 1
09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
: 1
09/06/17 16:30:29 INFO mapred.JobClient: Running job: job_200906171601_0009
09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%
09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
attempt_200906171601_0009_m_000000_0, Status : FAILED
java.io.IOException: hdfs://localhost:9000/user/root/words not a
SequenceFile
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)
at
org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
eader.java:43)
at
org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
eInputFormat.java:54)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
attempt_200906171601_0009_r_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createInstance(UserDefinedValueAggregatorDescriptor.java:57)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
<init>(UserDefinedValueAggregatorDescriptor.java:76)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
atorDescriptor(ValueAggregatorJobBase.java:54)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
escriptors(ValueAggregatorJobBase.java:65)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
ec(ValueAggregatorJobBase.java:74)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
eAggregatorJobBase.java:42)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:242)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createInstance(UserDefinedValueAggregatorDescriptor.java:52)
... 10 more
Thank You,
Shravan Kumar. M
Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
-----------------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system
administrator - <ma...@catalytic.com>
netopshelpdesk@catalytic.com
--
http://daily.appspot.com/food/
Re: not a SequenceFile?
Posted by Nick Cen <ce...@gmail.com>.
I guest you have set SequenceFileFormat as your inputformat in the
configuration object, but the file you provide is not a sequence file.
2009/6/17 Shravan Mahankali <sh...@catalytic.com>
> Hi Group,
>
>
>
> I have trouble running couple of examples provided by Hadoop. Below are the
> error messages I have from the console, could you please advise what could
> be the problem and probable solution?
>
>
>
> 09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
> : 1
>
> 09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
> : 1
>
> 09/06/17 16:30:29 INFO mapred.JobClient: Running job: job_200906171601_0009
>
> 09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%
>
> 09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
> attempt_200906171601_0009_m_000000_0, Status : FAILED
>
> java.io.IOException: hdfs://localhost:9000/user/root/words not a
> SequenceFile
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)
>
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)
>
> at
>
> org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
> eader.java:43)
>
> at
>
> org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
> eInputFormat.java:54)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
>
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
>
>
>
> 09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
> attempt_200906171601_0009_r_000000_0, Status : FAILED
>
> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createInstance(UserDefinedValueAggregatorDescriptor.java:57)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> <init>(UserDefinedValueAggregatorDescriptor.java:76)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
> atorDescriptor(ValueAggregatorJobBase.java:54)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
> escriptors(ValueAggregatorJobBase.java:65)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
> ec(ValueAggregatorJobBase.java:74)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
> eAggregatorJobBase.java:42)
>
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
>
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
>
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
>
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
>
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>
> at java.lang.Class.forName0(Native Method)
>
> at java.lang.Class.forName(Class.java:242)
>
> at
>
> org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
> createInstance(UserDefinedValueAggregatorDescriptor.java:52)
>
> ... 10 more
>
>
>
> Thank You,
>
> Shravan Kumar. M
>
> Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
>
> -----------------------------
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the system
> administrator - <ma...@catalytic.com>
> netopshelpdesk@catalytic.com
>
>
>
>
>
>
--
http://daily.appspot.com/food/