You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "Stephen J. Barr" <st...@u.washington.edu> on 2008/03/22 02:35:59 UTC
walkthrough of developing first hadoop app from scratch
Hello,
I am working on developing my first hadoop app from scratch. It is a
Monte-Carlo simulation, and I am using the PiEstimator code from the
examples as a reference. I believe I have what I want in a .java file.
However, I couldn't find any documentation on how to make that .java
file into a .jar that I could run, and I haven't found much
documentation that is hadoop specific.
Is it basically javac MyApp.java
jar -cf MyApp
or something to that effect, or is there more to it?
Thanks! Sorry for the newbie question.
-stephen barr
Re: walkthrough of developing first hadoop app from scratch
Posted by Arun C Murthy <ar...@yahoo-inc.com>.
On Mar 21, 2008, at 6:35 PM, Stephen J. Barr wrote:
> Hello,
>
> I am working on developing my first hadoop app from scratch. It is
> a Monte-Carlo simulation, and I am using the PiEstimator code from
> the examples as a reference. I believe I have what I want in
> a .java file. However, I couldn't find any documentation on how to
> make that .java file into a .jar that I could run, and I haven't
> found much documentation that is hadoop specific.
>
> Is it basically javac MyApp.java
> jar -cf MyApp
>
Yes. That's pretty much it.
Here is more documentation to help you get started:
http://hadoop.apache.org/core/docs/current/mapred_tutorial.html
Arun
> or something to that effect, or is there more to it?
>
> Thanks! Sorry for the newbie question.
>
> -stephen barr
Re: walkthrough of developing first hadoop app from scratch
Posted by Paco NATHAN <ce...@gmail.com>.
Hi Stephen,
Here's a sample Hadoop app which has its build based on Ant:
http://code.google.com/p/ceteri-mapred/
Look in the "jyte" directory. A target called "prep.jar" simply uses
the <jar/> task in Ant to build a JAR for Hadoop to use.
Yeah, I agree that docs and discussions seem to lean more toward
systems engineering and are sparse about writing applications. A few
folks are trying to change that :)
On Fri, Mar 21, 2008 at 6:35 PM, Stephen J. Barr
<st...@u.washington.edu> wrote:
> Hello,
>
> I am working on developing my first hadoop app from scratch. It is a
> Monte-Carlo simulation, and I am using the PiEstimator code from the
> examples as a reference. I believe I have what I want in a .java file.
> However, I couldn't find any documentation on how to make that .java
> file into a .jar that I could run, and I haven't found much
> documentation that is hadoop specific.
>
> Is it basically javac MyApp.java
> jar -cf MyApp
>
> or something to that effect, or is there more to it?
>
> Thanks! Sorry for the newbie question.
>
> -stephen barr
>
>
Re: walkthrough of developing first hadoop app from scratch
Posted by Jimmy Lin <ji...@umd.edu>.
Hi Stephen et al.,
I would take advantage of the Hadoop plug-in for Eclipse to handle the
mundane aspects of putting together your job and running it on the cluster.
With respect to gentler introductions on application development, you
might want to take a look at the following:
http://www.umiacs.umd.edu/~jimmylin/cloud9/umd-hadoop-dist/cloud9-docs/index.html
Cloud9 is a MapReduce library primarily intended for teaching, which I
use in my cloud computing course (going on right now). The associated
tutorials might help you get started. Thus far it's worked well with U.
Maryland grads and undergrads, but I'd appreciate additional feedback.
Incidentally, I will be talking at the Hadoop summit next week, so if
anyone else on the list will be there, I look forward to meeting everyone!
-Jimmy
Stephen J. Barr wrote:
> Hello,
>
> I am working on developing my first hadoop app from scratch. It is a
> Monte-Carlo simulation, and I am using the PiEstimator code from the
> examples as a reference. I believe I have what I want in a .java file.
> However, I couldn't find any documentation on how to make that .java
> file into a .jar that I could run, and I haven't found much
> documentation that is hadoop specific.
>
> Is it basically javac MyApp.java
> jar -cf MyApp
>
> or something to that effect, or is there more to it?
>
> Thanks! Sorry for the newbie question.
>
> -stephen barr
>
Re: walkthrough of developing first hadoop app from scratch
Posted by "Stephen J. Barr" <st...@u.washington.edu>.
Thank you. That worked (well, it pointed out all the bugs in my code,
which is a good start.)
朱盛凯 wrote:
> Hi Stephen,
>
> You can get an example of word count, there shows how to create jar archive
> of you application codes.
>
> $ mkdir wordcount_classes
> $ javac -classpath ${HADOOP_HOME}/hadoop-${HADOOP_VERSION}-core.jar -d
> wordcount_classes WordCount.java
> $ jar -cvf /usr/joe/wordcount.jar -C wordcount_classes/ . (notice: here is a
> dot referring to current directory)
>
>
> On 3/22/08, Stephen J. Barr <st...@u.washington.edu> wrote:
>
>> Hello,
>>
>> I am working on developing my first hadoop app from scratch. It is a
>> Monte-Carlo simulation, and I am using the PiEstimator code from the
>> examples as a reference. I believe I have what I want in a .java file.
>> However, I couldn't find any documentation on how to make that .java
>> file into a .jar that I could run, and I haven't found much
>> documentation that is hadoop specific.
>>
>> Is it basically javac MyApp.java
>> jar -cf MyApp
>>
>> or something to that effect, or is there more to it?
>>
>> Thanks! Sorry for the newbie question.
>>
>> -stephen barr
>>
>>
>
>
Re: walkthrough of developing first hadoop app from scratch
Posted by 朱盛凯 <ge...@gmail.com>.
Hi Stephen,
You can get an example of word count, there shows how to create jar archive
of you application codes.
$ mkdir wordcount_classes
$ javac -classpath ${HADOOP_HOME}/hadoop-${HADOOP_VERSION}-core.jar -d
wordcount_classes WordCount.java
$ jar -cvf /usr/joe/wordcount.jar -C wordcount_classes/ . (notice: here is a
dot referring to current directory)
On 3/22/08, Stephen J. Barr <st...@u.washington.edu> wrote:
>
> Hello,
>
> I am working on developing my first hadoop app from scratch. It is a
> Monte-Carlo simulation, and I am using the PiEstimator code from the
> examples as a reference. I believe I have what I want in a .java file.
> However, I couldn't find any documentation on how to make that .java
> file into a .jar that I could run, and I haven't found much
> documentation that is hadoop specific.
>
> Is it basically javac MyApp.java
> jar -cf MyApp
>
> or something to that effect, or is there more to it?
>
> Thanks! Sorry for the newbie question.
>
> -stephen barr
>