You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by C G <pa...@yahoo.com> on 2007/07/27 00:00:23 UTC
Newbie question...with appropriate apologies in advance...
I've deployed hadoop-0.13.0 and successfully run some examples. Now I am trying to compile and run the examples prior to starting to develop my own code. I've managed to do little more than get a nice spew of error messages. This is no doubt due to my own ignorance and too much Python programming, but can anybody tell me how to compile the examples (WordCount.java for example) and run them?
I apologize for the basic question, which I am sure is classpath-magic related. Any assistance most appreciated. I promise to be helpful to other newbs like myself going forward.
I've tried a bunch of stuff, but it all comes down to various "can't find class X' errors. This is representative of as far as I've gotten:
% java -version
java version "1.6.0_02"
Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
% cd ${HADOOP_HOME}/hadoop-0.13.0/src/examples/org/apache/hadoop/examples
% javac -classpath ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar WordCount.java
% jar -cvf WordCount.jar *.class
% cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
% cd ${HADOOP_HOME/hadoop-0.13.0/
% bin/hadoop jar WordCount.jar WordCount input_files/text.txt output_files/results2.txt
Exception in thread "main" java.lang.NoClassDefFoundError: WordCount (wrong name: org/apache/hadoop/examples/WordCount)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
---------------------------------
Luggage? GPS? Comic books?
Check out fitting gifts for grads at Yahoo! Search.
Re: Newbie question...with appropriate apologies in advance...
Posted by Lucas Nazário dos Santos <na...@gmail.com>.
Sorry about my reply. I didn't see that now you are trying to run your own
examples, and that you alredy did successfully with the ones that come with
Hadoop.
On 7/26/07, Lucas Nazário dos Santos <na...@gmail.com> wrote:
>
> Hello.
>
> My reply is embedded with your text, marked with ###.
>
> Cheers,
> Lucas
>
>
>
> On 7/26/07, C G < parallelguy@yahoo.com> wrote:
> >
> > I've deployed hadoop-0.13.0 and successfully run some examples. Now I
> > am trying to compile and run the examples prior to starting to develop my
> > own code. I've managed to do little more than get a nice spew of error
> > messages. This is no doubt due to my own ignorance and too much Python
> > programming, but can anybody tell me how to compile the examples (
> > WordCount.java for example) and run them?
>
>
> ##########################################################
> You don't need to compile the examples. They already come compiled for you
> inside Hadoop's home folder, wrapped in the jar named
> hadoop-0.13.0-examples.jar.
> ##########################################################
>
> I apologize for the basic question, which I am sure is classpath-magic
> > related. Any assistance most appreciated. I promise to be helpful to other
> > newbs like myself going forward.
> >
> > I've tried a bunch of stuff, but it all comes down to various "can't
> > find class X' errors. This is representative of as far as I've gotten:
> > % java -version
> > java version "1.6.0_02"
> > Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
> > Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> > % cd ${HADOOP_HOME}/hadoop-0.13.0/src/examples/org/apache/hadoop/examples
> > % javac -classpath ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar
> > WordCount.java
> > % jar -cvf WordCount.jar *.class
> > % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> > % cd ${HADOOP_HOME/hadoop- 0.13.0/
> > % bin/hadoop jar WordCount.jar WordCount input_files/text.txt
> > output_files/results2.txt
> > Exception in thread "main" java.lang.NoClassDefFoundError: WordCount
> > (wrong name: org/apache/hadoop/examples/WordCount)
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> > at java.security.SecureClassLoader.defineClass(
> > SecureClassLoader.java:124)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> > at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at java.lang.ClassLoader.loadClass (ClassLoader.java:251)
> > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:247)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
>
>
> ##########################################################
> Instead of issuing
>
> % bin/hadoop jar WordCount.jar WordCount input_files/text.txt
> output_files/results2.txt
>
> try something like
>
> % bin/hadoop jar hadoop-0.13.0-examples.jar wordcount input_files
> output_files
>
> Notice that you must copy your input files to HDFS, under the folder named
> input_files (you can use bin/hadoop -put <src> <dest> command). You also
> musn't create the folder output_files, because Hadoop will create it
> automatically for you. And don't forget to run Hadoop's tasktracker
> (bin/hadoop tasktracker).
> ##########################################################
>
> ---------------------------------
> > Luggage? GPS? Comic books?
> > Check out fitting gifts for grads at Yahoo! Search.
>
>
>
Re: Newbie question...with appropriate apologies in advance...
Posted by C G <pa...@yahoo.com>.
Thanks Lucas...I did run everything that way to test and evaluate. Now I'm just trying to figure out how to build and deploy my own code.
Lucas Nazário dos Santos <na...@gmail.com> wrote: Hello.
My reply is embedded with your text, marked with ###.
Cheers,
Lucas
On 7/26/07, C G
wrote:
>
> I've deployed hadoop-0.13.0 and successfully run some examples. Now I am
> trying to compile and run the examples prior to starting to develop my own
> code. I've managed to do little more than get a nice spew of error
> messages. This is no doubt due to my own ignorance and too much Python
> programming, but can anybody tell me how to compile the examples (
> WordCount.java for example) and run them?
##########################################################
You don't need to compile the examples. They already come compiled for you
inside Hadoop's home folder, wrapped in the jar named
hadoop-0.13.0-examples.jar.
##########################################################
I apologize for the basic question, which I am sure is classpath-magic
> related. Any assistance most appreciated. I promise to be helpful to other
> newbs like myself going forward.
>
> I've tried a bunch of stuff, but it all comes down to various "can't
> find class X' errors. This is representative of as far as I've gotten:
> % java -version
> java version "1.6.0_02"
> Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
> Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> % cd ${HADOOP_HOME}/hadoop-0.13.0
> /src/examples/org/apache/hadoop/examples
> % javac -classpath ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar
> WordCount.java
> % jar -cvf WordCount.jar *.class
> % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> % cd ${HADOOP_HOME/hadoop-0.13.0/
> % bin/hadoop jar WordCount.jar WordCount input_files/text.txt
> output_files/results2.txt
> Exception in thread "main" java.lang.NoClassDefFoundError: WordCount
> (wrong name: org/apache/hadoop/examples/WordCount)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> at java.security.SecureClassLoader.defineClass(
> SecureClassLoader.java:124)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
##########################################################
Instead of issuing
% bin/hadoop jar WordCount.jar WordCount input_files/text.txt
output_files/results2.txt
try something like
% bin/hadoop jar hadoop-0.13.0-examples.jar wordcount input_files
output_files
Notice that you must copy your input files to HDFS, under the folder named
input_files (you can use bin/hadoop -put command). You also
musn't create the folder output_files, because Hadoop will create it
automatically for you. And don't forget to run Hadoop's tasktracker
(bin/hadoop tasktracker).
##########################################################
---------------------------------
> Luggage? GPS? Comic books?
> Check out fitting gifts for grads at Yahoo! Search.
---------------------------------
Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more.
Re: Newbie question...with appropriate apologies in advance...
Posted by Lucas Nazário dos Santos <na...@gmail.com>.
Hello.
My reply is embedded with your text, marked with ###.
Cheers,
Lucas
On 7/26/07, C G <pa...@yahoo.com> wrote:
>
> I've deployed hadoop-0.13.0 and successfully run some examples. Now I am
> trying to compile and run the examples prior to starting to develop my own
> code. I've managed to do little more than get a nice spew of error
> messages. This is no doubt due to my own ignorance and too much Python
> programming, but can anybody tell me how to compile the examples (
> WordCount.java for example) and run them?
##########################################################
You don't need to compile the examples. They already come compiled for you
inside Hadoop's home folder, wrapped in the jar named
hadoop-0.13.0-examples.jar.
##########################################################
I apologize for the basic question, which I am sure is classpath-magic
> related. Any assistance most appreciated. I promise to be helpful to other
> newbs like myself going forward.
>
> I've tried a bunch of stuff, but it all comes down to various "can't
> find class X' errors. This is representative of as far as I've gotten:
> % java -version
> java version "1.6.0_02"
> Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
> Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> % cd ${HADOOP_HOME}/hadoop-0.13.0
> /src/examples/org/apache/hadoop/examples
> % javac -classpath ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar
> WordCount.java
> % jar -cvf WordCount.jar *.class
> % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> % cd ${HADOOP_HOME/hadoop-0.13.0/
> % bin/hadoop jar WordCount.jar WordCount input_files/text.txt
> output_files/results2.txt
> Exception in thread "main" java.lang.NoClassDefFoundError: WordCount
> (wrong name: org/apache/hadoop/examples/WordCount)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> at java.security.SecureClassLoader.defineClass(
> SecureClassLoader.java:124)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
##########################################################
Instead of issuing
% bin/hadoop jar WordCount.jar WordCount input_files/text.txt
output_files/results2.txt
try something like
% bin/hadoop jar hadoop-0.13.0-examples.jar wordcount input_files
output_files
Notice that you must copy your input files to HDFS, under the folder named
input_files (you can use bin/hadoop -put <src> <dest> command). You also
musn't create the folder output_files, because Hadoop will create it
automatically for you. And don't forget to run Hadoop's tasktracker
(bin/hadoop tasktracker).
##########################################################
---------------------------------
> Luggage? GPS? Comic books?
> Check out fitting gifts for grads at Yahoo! Search.
Re: Newbie question...with appropriate apologies in advance...
Posted by oj...@doc.ic.ac.uk.
Hi,
I'm not sure if my original reply was any use. But here's some
background on running jar's on hadoop that may help.
Basically, the jar command runs the JarUtil class. This attempts to
unpack the jar and first looks in its Manifest file. If there is a
main class specified it will attempt to run it. If not it will look
for the argument following the word jar on the command line (in your
case Wordcount) and will attempt to run that class.
As an aside, in the *examples*.jar the mainclass is specified in the
Manifest and is a Driver program. I'd recommend having a look at the
source for this. This driver takes the next argument on the command
line and matches it to a string associated with a class. It then runs
that class. This is why bin/hadoop jar *examples*.jar pi 10 10 works
when the class is actually called ?????
So, with your own WordCount.jar, you are using the first method
described. However, the exception you get is saying you have the wrong
class name. I get this exception when I have the correct class name,
but without the correct folder structure before it, corresponding to
the package it is in. If the class is not there at all, you don't get
the "wrong name" part of the exception.
I think your hadoop is probably set up correctly, there's just some
problemt, either with the file structure inside your jar or with what
you type on the command line.
Actually, looking at how you jarred it, I think you should do the following:
jar -cvf WordCount.jar package/structure/*.class, where
package/structure is the folder structure corresponding to the package
your class is in and you are in the folder that contains the directory
package. Then to run:
bin/hadoop jar WordCount.jar package/structure/WorCount dir dir
If its not in a package, then I don't know what's going on!
Hope some of this helped.
Ollie
Quoting C G <pa...@yahoo.com>:
> I've deployed hadoop-0.13.0 and successfully run some examples. Now
> I am trying to compile and run the examples prior to starting to
> develop my own code. I've managed to do little more than get a nice
> spew of error messages. This is no doubt due to my own ignorance
> and too much Python programming, but can anybody tell me how to
> compile the examples (WordCount.java for example) and run them?
>
> I apologize for the basic question, which I am sure is
> classpath-magic related. Any assistance most appreciated. I
> promise to be helpful to other newbs like myself going forward.
>
> I've tried a bunch of stuff, but it all comes down to various
> "can't find class X' errors. This is representative of as far as
> I've gotten:
> % java -version
> java version "1.6.0_02"
> Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
> Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> % cd ${HADOOP_HOME}/hadoop-0.13.0/src/examples/org/apache/hadoop/examples
> % javac -classpath
> ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar WordCount.java
> % jar -cvf WordCount.jar *.class
> % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> % cd ${HADOOP_HOME/hadoop-0.13.0/
> % bin/hadoop jar WordCount.jar WordCount input_files/text.txt
> output_files/results2.txt
> Exception in thread "main" java.lang.NoClassDefFoundError: WordCount
> (wrong name: org/apache/hadoop/examples/WordCount)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
>
>
>
> ---------------------------------
> Luggage? GPS? Comic books?
> Check out fitting gifts for grads at Yahoo! Search.
Re: Newbie question...with appropriate apologies in advance...
Posted by oj...@doc.ic.ac.uk.
Hi, can you show the output of "%> jar tf WordCount.jar". I'm
wondering whether you've jarred it nested in any package folders that
need to be written on the command line eg.
bin/hadoop jar WordCount.jar WordCount
org/apache/hadoop/examples/WordCount dir dir
Quoting C G <pa...@yahoo.com>:
> I've deployed hadoop-0.13.0 and successfully run some examples. Now
> I am trying to compile and run the examples prior to starting to
> develop my own code. I've managed to do little more than get a nice
> spew of error messages. This is no doubt due to my own ignorance
> and too much Python programming, but can anybody tell me how to
> compile the examples (WordCount.java for example) and run them?
>
> I apologize for the basic question, which I am sure is
> classpath-magic related. Any assistance most appreciated. I
> promise to be helpful to other newbs like myself going forward.
>
> I've tried a bunch of stuff, but it all comes down to various
> "can't find class X' errors. This is representative of as far as
> I've gotten:
> % java -version
> java version "1.6.0_02"
> Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
> Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> % cd ${HADOOP_HOME}/hadoop-0.13.0/src/examples/org/apache/hadoop/examples
> % javac -classpath
> ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar WordCount.java
> % jar -cvf WordCount.jar *.class
> % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> % cd ${HADOOP_HOME/hadoop-0.13.0/
> % bin/hadoop jar WordCount.jar WordCount input_files/text.txt
> output_files/results2.txt
> Exception in thread "main" java.lang.NoClassDefFoundError: WordCount
> (wrong name: org/apache/hadoop/examples/WordCount)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
>
>
>
> ---------------------------------
> Luggage? GPS? Comic books?
> Check out fitting gifts for grads at Yahoo! Search.
RE: Newbie question...with appropriate apologies in advance...
Posted by C G <pa...@yahoo.com>.
I figured it out finally...the answer lies with the version of ant I was using.
I had installed the latest release, 1.7.0...which turns out to be incompatible with the build.xml currently used by hadoop. I installed 1.6.5 and everything appears to build as expected.
Happy happy joy joy...
C G <pa...@yahoo.com> wrote:
No love trying to build examples...any thoughts?
[cg ~]$ java -version
java version "1.5.0_12"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_12-b04)
Java HotSpot(TM) Server VM (build 1.5.0_12-b04, mixed mode)
[cg ~]$ ant -version
Apache Ant version 1.7.0 compiled on December 13 2006
[cg ~]$ cd hadoop-0.13.0
[cg hadoop-0.13.0]$ ant examples
Buildfile: build.xml
init:
BUILD FAILED
/home/cg/hadoop-0.13.0/build.xml:116: Specify at least one source--a file or resource collection.
Total time: 0 seconds
[cg hadoop-0.13.0]$
Vivek Ratan wrote:
NOt sure if I understand what you're trying to do. But,
- if you're modiying one of the examples and want to compile the modified
code, run 'ant examples' under ${HADOOP_HOME}/hadoop-0.13.0. (I assume
you're familiar with 'ant'). This will recreate the examples jar file (under
the build directory), which you can copy over the existing one. This is the
easiest way to compile your code as the root build.xml, which ant uses, sets
up all the right classpaths and other things for compilation.
- if you're creating examples of your own, you'll probably need to add your
class to org.apache.hadoop.example.ExampleDriver and then run ant.
> -----Original Message-----
> From: C G [mailto:parallelguy@yahoo.com]
> Sent: Thursday, July 26, 2007 3:00 PM
> To: hadoop-user@lucene.apache.org
> Subject: Newbie question...with appropriate apologies in advance...
>
> I've deployed hadoop-0.13.0 and successfully run some
> examples. Now I am trying to compile and run the examples
> prior to starting to develop my own code. I've managed to do
> little more than get a nice spew of error messages. This is
> no doubt due to my own ignorance and too much Python
> programming, but can anybody tell me how to compile the
> examples (WordCount.java for example) and run them?
>
> I apologize for the basic question, which I am sure is
> classpath-magic related. Any assistance most appreciated. I
> promise to be helpful to other newbs like myself going forward.
>
> I've tried a bunch of stuff, but it all comes down to
> various "can't find class X' errors. This is representative
> of as far as I've gotten:
> % java -version
> java version "1.6.0_02"
> Java(TM) SE Runtime Environment (build 1.6.0_02-b05) Java
> HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> % cd
> ${HADOOP_HOME}/hadoop-0.13.0/src/examples/org/apache/hadoop/examples
> % javac -classpath
> ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar WordCount.java
> % jar -cvf WordCount.jar *.class
> % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> % cd ${HADOOP_HOME/hadoop-0.13.0/
> % bin/hadoop jar WordCount.jar WordCount
> input_files/text.txt output_files/results2.txt Exception in
> thread "main" java.lang.NoClassDefFoundError: WordCount
> (wrong name: org/apache/hadoop/examples/WordCount)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.
> java:124)
> at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at
> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
>
>
>
> ---------------------------------
> Luggage? GPS? Comic books?
> Check out fitting gifts for grads at Yahoo! Search.
>
---------------------------------
Luggage? GPS? Comic books?
Check out fitting gifts for grads at Yahoo! Search.
---------------------------------
Luggage? GPS? Comic books?
Check out fitting gifts for grads at Yahoo! Search.
RE: Newbie question...with appropriate apologies in advance...
Posted by C G <pa...@yahoo.com>.
No love trying to build examples...any thoughts?
[cg ~]$ java -version
java version "1.5.0_12"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_12-b04)
Java HotSpot(TM) Server VM (build 1.5.0_12-b04, mixed mode)
[cg ~]$ ant -version
Apache Ant version 1.7.0 compiled on December 13 2006
[cg ~]$ cd hadoop-0.13.0
[cg hadoop-0.13.0]$ ant examples
Buildfile: build.xml
init:
BUILD FAILED
/home/cg/hadoop-0.13.0/build.xml:116: Specify at least one source--a file or resource collection.
Total time: 0 seconds
[cg hadoop-0.13.0]$
Vivek Ratan <vi...@yahoo-inc.com> wrote:
NOt sure if I understand what you're trying to do. But,
- if you're modiying one of the examples and want to compile the modified
code, run 'ant examples' under ${HADOOP_HOME}/hadoop-0.13.0. (I assume
you're familiar with 'ant'). This will recreate the examples jar file (under
the build directory), which you can copy over the existing one. This is the
easiest way to compile your code as the root build.xml, which ant uses, sets
up all the right classpaths and other things for compilation.
- if you're creating examples of your own, you'll probably need to add your
class to org.apache.hadoop.example.ExampleDriver and then run ant.
> -----Original Message-----
> From: C G [mailto:parallelguy@yahoo.com]
> Sent: Thursday, July 26, 2007 3:00 PM
> To: hadoop-user@lucene.apache.org
> Subject: Newbie question...with appropriate apologies in advance...
>
> I've deployed hadoop-0.13.0 and successfully run some
> examples. Now I am trying to compile and run the examples
> prior to starting to develop my own code. I've managed to do
> little more than get a nice spew of error messages. This is
> no doubt due to my own ignorance and too much Python
> programming, but can anybody tell me how to compile the
> examples (WordCount.java for example) and run them?
>
> I apologize for the basic question, which I am sure is
> classpath-magic related. Any assistance most appreciated. I
> promise to be helpful to other newbs like myself going forward.
>
> I've tried a bunch of stuff, but it all comes down to
> various "can't find class X' errors. This is representative
> of as far as I've gotten:
> % java -version
> java version "1.6.0_02"
> Java(TM) SE Runtime Environment (build 1.6.0_02-b05) Java
> HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> % cd
> ${HADOOP_HOME}/hadoop-0.13.0/src/examples/org/apache/hadoop/examples
> % javac -classpath
> ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar WordCount.java
> % jar -cvf WordCount.jar *.class
> % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> % cd ${HADOOP_HOME/hadoop-0.13.0/
> % bin/hadoop jar WordCount.jar WordCount
> input_files/text.txt output_files/results2.txt Exception in
> thread "main" java.lang.NoClassDefFoundError: WordCount
> (wrong name: org/apache/hadoop/examples/WordCount)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.
> java:124)
> at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at
> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
>
>
>
> ---------------------------------
> Luggage? GPS? Comic books?
> Check out fitting gifts for grads at Yahoo! Search.
>
---------------------------------
Luggage? GPS? Comic books?
Check out fitting gifts for grads at Yahoo! Search.
RE: Newbie question...with appropriate apologies in advance...
Posted by Hemanth Yamijala <yh...@yahoo-inc.com>.
Hi,
I used to get this error when using the Eclipse IDE and a setting called
the Compiler compliance level was set to lower than Java 1.5, which does
not support generics.
Even if you are not using Eclipse, I wonder if the problem is the same -
i.e. ant is somehow picking up an old version of the JDK. Can you try
ensuring that it is pointing to a proper version at build time ?
Thanks
Hemanth
On Thu, 2007-07-26 at 19:11 -0700, C G wrote:
> Vivek and everyone: Thanks for the info. I am just trying to build the examples as a cheap proxy for how to write my own code.
>
> I tried your suggestion, the result was 125 errors and a slew of warnings, here's one for example:
>
> [javac] 1. ERROR in /home/CG/hadoop-0.13.0/src/java/org/apache/hadoop/record/compiler/CGenerator.java (at line 42
> )
> [javac] void genCode(String name, ArrayList<JFile> ilist,
> [javac] ^^^^^^^^^
> [javac] The type ArrayList is not generic; it cannot be parameterized with arguments <JFile>
>
> Clearly there is something environmental I am doing wrong. I followed the configuration instructions in the getting started guide, and it all worked from the perspective of being able to run things out of the box.
>
> Is there a guide or FAQ discussing building and deploying that I missed?
>
> Any advice greatly appreciated. The map/reduce paradigm is super cool and I can't wait to get past these operational issues and start laying down some code.
>
> Thanks!
> CG
>
> Vivek Ratan <vi...@yahoo-inc.com> wrote:
> NOt sure if I understand what you're trying to do. But,
> - if you're modiying one of the examples and want to compile the modified
> code, run 'ant examples' under ${HADOOP_HOME}/hadoop-0.13.0. (I assume
> you're familiar with 'ant'). This will recreate the examples jar file (under
> the build directory), which you can copy over the existing one. This is the
> easiest way to compile your code as the root build.xml, which ant uses, sets
> up all the right classpaths and other things for compilation.
> - if you're creating examples of your own, you'll probably need to add your
> class to org.apache.hadoop.example.ExampleDriver and then run ant.
>
>
> > -----Original Message-----
> > From: C G [mailto:parallelguy@yahoo.com]
> > Sent: Thursday, July 26, 2007 3:00 PM
> > To: hadoop-user@lucene.apache.org
> > Subject: Newbie question...with appropriate apologies in advance...
> >
> > I've deployed hadoop-0.13.0 and successfully run some
> > examples. Now I am trying to compile and run the examples
> > prior to starting to develop my own code. I've managed to do
> > little more than get a nice spew of error messages. This is
> > no doubt due to my own ignorance and too much Python
> > programming, but can anybody tell me how to compile the
> > examples (WordCount.java for example) and run them?
> >
> > I apologize for the basic question, which I am sure is
> > classpath-magic related. Any assistance most appreciated. I
> > promise to be helpful to other newbs like myself going forward.
> >
> > I've tried a bunch of stuff, but it all comes down to
> > various "can't find class X' errors. This is representative
> > of as far as I've gotten:
> > % java -version
> > java version "1.6.0_02"
> > Java(TM) SE Runtime Environment (build 1.6.0_02-b05) Java
> > HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> > % cd
> > ${HADOOP_HOME}/hadoop-0.13.0/src/examples/org/apache/hadoop/examples
> > % javac -classpath
> > ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar WordCount.java
> > % jar -cvf WordCount.jar *.class
> > % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> > % cd ${HADOOP_HOME/hadoop-0.13.0/
> > % bin/hadoop jar WordCount.jar WordCount
> > input_files/text.txt output_files/results2.txt Exception in
> > thread "main" java.lang.NoClassDefFoundError: WordCount
> > (wrong name: org/apache/hadoop/examples/WordCount)
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> > at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.
> > java:124)
> > at
> > java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> > at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> > at
> > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:247)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
> >
> >
> >
> > ---------------------------------
> > Luggage? GPS? Comic books?
> > Check out fitting gifts for grads at Yahoo! Search.
> >
>
>
>
>
> ---------------------------------
> Park yourself in front of a world of choices in alternative vehicles.
> Visit the Yahoo! Auto Green Center.
RE: Newbie question...with appropriate apologies in advance...
Posted by C G <pa...@yahoo.com>.
Vivek and everyone: Thanks for the info. I am just trying to build the examples as a cheap proxy for how to write my own code.
I tried your suggestion, the result was 125 errors and a slew of warnings, here's one for example:
[javac] 1. ERROR in /home/CG/hadoop-0.13.0/src/java/org/apache/hadoop/record/compiler/CGenerator.java (at line 42
)
[javac] void genCode(String name, ArrayList<JFile> ilist,
[javac] ^^^^^^^^^
[javac] The type ArrayList is not generic; it cannot be parameterized with arguments <JFile>
Clearly there is something environmental I am doing wrong. I followed the configuration instructions in the getting started guide, and it all worked from the perspective of being able to run things out of the box.
Is there a guide or FAQ discussing building and deploying that I missed?
Any advice greatly appreciated. The map/reduce paradigm is super cool and I can't wait to get past these operational issues and start laying down some code.
Thanks!
CG
Vivek Ratan <vi...@yahoo-inc.com> wrote:
NOt sure if I understand what you're trying to do. But,
- if you're modiying one of the examples and want to compile the modified
code, run 'ant examples' under ${HADOOP_HOME}/hadoop-0.13.0. (I assume
you're familiar with 'ant'). This will recreate the examples jar file (under
the build directory), which you can copy over the existing one. This is the
easiest way to compile your code as the root build.xml, which ant uses, sets
up all the right classpaths and other things for compilation.
- if you're creating examples of your own, you'll probably need to add your
class to org.apache.hadoop.example.ExampleDriver and then run ant.
> -----Original Message-----
> From: C G [mailto:parallelguy@yahoo.com]
> Sent: Thursday, July 26, 2007 3:00 PM
> To: hadoop-user@lucene.apache.org
> Subject: Newbie question...with appropriate apologies in advance...
>
> I've deployed hadoop-0.13.0 and successfully run some
> examples. Now I am trying to compile and run the examples
> prior to starting to develop my own code. I've managed to do
> little more than get a nice spew of error messages. This is
> no doubt due to my own ignorance and too much Python
> programming, but can anybody tell me how to compile the
> examples (WordCount.java for example) and run them?
>
> I apologize for the basic question, which I am sure is
> classpath-magic related. Any assistance most appreciated. I
> promise to be helpful to other newbs like myself going forward.
>
> I've tried a bunch of stuff, but it all comes down to
> various "can't find class X' errors. This is representative
> of as far as I've gotten:
> % java -version
> java version "1.6.0_02"
> Java(TM) SE Runtime Environment (build 1.6.0_02-b05) Java
> HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> % cd
> ${HADOOP_HOME}/hadoop-0.13.0/src/examples/org/apache/hadoop/examples
> % javac -classpath
> ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar WordCount.java
> % jar -cvf WordCount.jar *.class
> % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> % cd ${HADOOP_HOME/hadoop-0.13.0/
> % bin/hadoop jar WordCount.jar WordCount
> input_files/text.txt output_files/results2.txt Exception in
> thread "main" java.lang.NoClassDefFoundError: WordCount
> (wrong name: org/apache/hadoop/examples/WordCount)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.
> java:124)
> at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at
> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
>
>
>
> ---------------------------------
> Luggage? GPS? Comic books?
> Check out fitting gifts for grads at Yahoo! Search.
>
---------------------------------
Park yourself in front of a world of choices in alternative vehicles.
Visit the Yahoo! Auto Green Center.
RE: Newbie question...with appropriate apologies in advance...
Posted by Vivek Ratan <vi...@yahoo-inc.com>.
NOt sure if I understand what you're trying to do. But,
- if you're modiying one of the examples and want to compile the modified
code, run 'ant examples' under ${HADOOP_HOME}/hadoop-0.13.0. (I assume
you're familiar with 'ant'). This will recreate the examples jar file (under
the build directory), which you can copy over the existing one. This is the
easiest way to compile your code as the root build.xml, which ant uses, sets
up all the right classpaths and other things for compilation.
- if you're creating examples of your own, you'll probably need to add your
class to org.apache.hadoop.example.ExampleDriver and then run ant.
> -----Original Message-----
> From: C G [mailto:parallelguy@yahoo.com]
> Sent: Thursday, July 26, 2007 3:00 PM
> To: hadoop-user@lucene.apache.org
> Subject: Newbie question...with appropriate apologies in advance...
>
> I've deployed hadoop-0.13.0 and successfully run some
> examples. Now I am trying to compile and run the examples
> prior to starting to develop my own code. I've managed to do
> little more than get a nice spew of error messages. This is
> no doubt due to my own ignorance and too much Python
> programming, but can anybody tell me how to compile the
> examples (WordCount.java for example) and run them?
>
> I apologize for the basic question, which I am sure is
> classpath-magic related. Any assistance most appreciated. I
> promise to be helpful to other newbs like myself going forward.
>
> I've tried a bunch of stuff, but it all comes down to
> various "can't find class X' errors. This is representative
> of as far as I've gotten:
> % java -version
> java version "1.6.0_02"
> Java(TM) SE Runtime Environment (build 1.6.0_02-b05) Java
> HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing
> % cd
> ${HADOOP_HOME}/hadoop-0.13.0/src/examples/org/apache/hadoop/examples
> % javac -classpath
> ${HADOOP_HOME}/hadoop-0.13.0/hadoop-0.13.0-core.jar WordCount.java
> % jar -cvf WordCount.jar *.class
> % cp *.jar ${HADOOP_HOME/hadoop-0.13.0/
> % cd ${HADOOP_HOME/hadoop-0.13.0/
> % bin/hadoop jar WordCount.jar WordCount
> input_files/text.txt output_files/results2.txt Exception in
> thread "main" java.lang.NoClassDefFoundError: WordCount
> (wrong name: org/apache/hadoop/examples/WordCount)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.
> java:124)
> at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at
> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
>
>
>
> ---------------------------------
> Luggage? GPS? Comic books?
> Check out fitting gifts for grads at Yahoo! Search.
>