learning yarn

[Tejas Patil <te...@gmail.com>]

Hi,
Is there any good place to kick start writing Yarn applications ? All I see
is [0] and a possibility of a book getting published by Arun Murthy in 2014

[0] :
http://hadoop.apache.org/docs/r2.0.6-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html

Thanks,
Tejas

RE: learning yarn

[Chris Riccomini <cr...@linkedin.com>]

Hey Tejas,

The best place to start is distributed shell, which you've already found. To better understand how YARN works, I recommend:

    http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html
    https://issues.apache.org/jira/secure/attachment/12486023/MapReduce_NextGen_Architecture.pdf
    http://developer.yahoo.com/blogs/hadoop/next-generation-apache-hadoop-mapreduce-scheduler-4141.html

You can play with two frameworks that are meant to help with writing YARN apps:

    Weave: https://github.com/continuuity/weave
    Kitten: https://github.com/cloudera/kitten

The next best place to start is by looking at how other people wrote their Application Masters. Spark was an early adopter, and one of the AMs that I used to learn how YARN works. Here is their AM:

    https://github.com/mesos/spark/blob/master/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala

You can also have a look at Samza's AM. The guts are located here:

    https://github.com/apache/incubator-samza/tree/master/samza-yarn/src/main/scala/org/apache/samza/job/yarn

Cheers,
Chris
________________________________________
From: Tejas Patil [tejas.patil.cs@gmail.com]
Sent: Sunday, September 01, 2013 12:40 PM
To: dev@samza.incubator.apache.org
Subject: learning yarn

Hi,
Is there any good place to kick start writing Yarn applications ? All I see
is [0] and a possibility of a book getting published by Arun Murthy in 2014

[0] :
http://hadoop.apache.org/docs/r2.0.6-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html

Thanks,
Tejas