You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@giraph.apache.org by Ahmet Emre Aladağ <em...@agmlab.com> on 2013/07/02 13:04:16 UTC

Running Giraph job inside Java code

Hi,

How can we run a Giraph job inside the java code? Using the scheme below 
didn't work.

...

     GiraphJob giraphJob = new GiraphJob(giraphConf, "LinkRank2");
giraphJob.getInternalJob().setJarByClass(LinkRankComputation.class);
     giraphJob.run(true);

...

Error in Hadoop logs:

java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.giraph.examples.LinkRank.LinkRankComputation
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:881)

I'm setting $HADOOP_CLASSPATH to the folder with giraph, giraph-examples 
jars. But it can't find the computation class.

Re: Running Giraph job inside Java code

Posted by Ahmet Emre Aladağ <em...@agmlab.com>.

Thanks, just to note down:

It seems like it was a dependency problem I had. I tried to provide my 
jars as dependencies but there was a mistake in the configuration.
Moreover, I had too remove 
giraphJob.getInternalJob().setJarByClass(LinkRankComputation.class); 
part as well.

On 07/04/2013 07:13 AM, Avery Ching wrote:
> Take a look at PageRankBenchmark, it is a stand alone java program 
> that runs Giraph jobs.

Re: Running Giraph job inside Java code

Posted by Avery Ching <ac...@apache.org>.

Take a look at PageRankBenchmark, it is a stand alone java program that 
runs Giraph jobs.

On 7/2/13 4:08 AM, Ahmet Emre Aladağ wrote:
> By the way, I have set the corresponding classes in the giraph 
> configuration.
>
>     GiraphConfiguration giraphConf = new GiraphConfiguration(config);
>
>     giraphConf.setZooKeeperConfiguration(
>             zooKeeperWatcher.getQuorum());
>     giraphConf.setComputationClass(LinkRankComputation.class);
> giraphConf.setMasterComputeClass(LinkRankVertexMasterCompute.class);
>     giraphConf.setOutEdgesClass(ByteArrayEdges.class);
> giraphConf.setVertexInputFormatClass(NutchTableEdgeInputFormat.class);
> giraphConf.setVertexOutputFormatClass(NutchTableEdgeOutputFormat.class);
>     giraphConf.setInt("giraph.pageRank.superstepCount", 40);
>     giraphConf.setWorkerConfiguration(1, 1, 100.0f);
>     giraphConf.set(TableInputFormat.INPUT_TABLE, TABLE_NAME);
>     giraphConf.set(TableOutputFormat.OUTPUT_TABLE, TABLE_NAME);

Re: Running Giraph job inside Java code

Posted by Ahmet Emre Aladağ <em...@agmlab.com>.

By the way, I have set the corresponding classes in the giraph 
configuration.

     GiraphConfiguration giraphConf = new GiraphConfiguration(config);

     giraphConf.setZooKeeperConfiguration(
             zooKeeperWatcher.getQuorum());
     giraphConf.setComputationClass(LinkRankComputation.class);
giraphConf.setMasterComputeClass(LinkRankVertexMasterCompute.class);
     giraphConf.setOutEdgesClass(ByteArrayEdges.class);
giraphConf.setVertexInputFormatClass(NutchTableEdgeInputFormat.class);
giraphConf.setVertexOutputFormatClass(NutchTableEdgeOutputFormat.class);
     giraphConf.setInt("giraph.pageRank.superstepCount", 40);
     giraphConf.setWorkerConfiguration(1, 1, 100.0f);
     giraphConf.set(TableInputFormat.INPUT_TABLE, TABLE_NAME);
     giraphConf.set(TableOutputFormat.OUTPUT_TABLE, TABLE_NAME);