Got Giraph 1.1.0 examples running on YARN

[John Yost <so...@gmail.com>]

Hi Everyone,

I just gotten the Giraph examples to run on YARN and I thought I would
share the details since it looks like a few people have struggled with
this. This is what I did:

1. Downloaded the latest snapshot (giraph-b218d2)
2. Built with mvn install -P=hadoop_2 -DskipTests=true
3. Executed with the following CLI entry against a Hadoop 2.2.0
pseudo-cluster with mapreduce.framework.name=yarn:

hadoop jar
/usr/local/java/giraph/giraph-1.1/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/hadoop/tiny.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hadoop/shortestpaths -w 1 -ca giraph.zkList=localhost:2181 -ca
giraph.SplitMasterWorker=true -ca mapred.job.tracker=localhost:54311 -ca
mapreduce.job.tracker=localhost:54311

Note: the parameter -ca mapred.job.tracker=localhost:54311 is crucial to
this working in cluster/pseudo-cluster mode.  Otherwise, You'll get the
following error hat Satyajit recently posted:

java.lang.IllegalArgumentException: checkLocalJobRunnerConfiguration: When
using LocalJobRunner, you cannot run in split master / worker mode since
there is only 1 task at a time!

If you actually want to run in local mode, have not figured that out as I
want to be able to run on my cluster instead.

--John.