You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Marco Aurelio Barbosa Fagnani Lotz <m....@stu12.qmul.ac.uk> on 2013/07/20 19:50:33 UTC
Zookeeper problem when running in Pure Yarn
Hello :)
When I run the SinglePageRankComputation example, using the following cmd line:
hadoop jar giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.0.3-alpha-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimplePageRankComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /input/input.txt -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /outPageRank -w 1 -mc org.apache.giraph.examples.SimplePageRankComputation\$SimplePageRankMasterCompute
I am getting the following error:
"13/07/20 18:37:57 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/07/20 18:37:58 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
13/07/20 18:37:58 INFO yarn.GiraphYarnClient: Final output path is: hdfs://localhost:9000/outPageRank
13/07/20 18:37:58 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
Exception in thread "main" java.lang.IllegalArgumentException: Giraph on YARN does not currentlysupport Giraph-managed ZK instances: use a standalone ZooKeeper: 'null'
at org.apache.giraph.yarn.GiraphYarnClient.checkJobLocalZooKeeperSupported(GiraphYarnClient.java:392)
at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:106)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)"
I couldn't fine anything about this Zookeeper error on google, any hints?
Since the PageRankBenchmark is not working on Yarn, what are the examples actually working on the current versions?
Oh, another thing: Does anyone knows how to remove the:
"13/07/20 18:37:57 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable"
On Hadoop Yarn 2.0.3-alpha?
Best regards,
Marco
Re: Zookeeper problem when running in Pure Yarn
Posted by Eli Reisman <ap...@gmail.com>.
Its buried in the middle of the stack trace :) sorry my bad!
You need to launch your own standalone Zookeeper instance or quorum, when
the YARN port is better integrated into mainline Giraph the plan is for
Zookeeper to be launched in its own process at the Application Master
instead of within Giraph, so we don't integrate it currently. If you don't
supply your own instance or quorum before the job run to connect to, Giraph
on YARN doesn't launch one.
On Sat, Jul 20, 2013 at 10:50 AM, Marco Aurelio Barbosa Fagnani Lotz <
m.a.b.lotz@stu12.qmul.ac.uk> wrote:
> Hello :)
>
> When I run the SinglePageRankComputation example, using the following cmd
> line:
>
> hadoop jar
> giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.0.3-alpha-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimplePageRankComputation -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /input/input.txt -of
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /outPageRank
> -w 1 -mc
> org.apache.giraph.examples.SimplePageRankComputation\$SimplePageRankMasterCompute
>
> I am getting the following error:
>
> "13/07/20 18:37:57 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> 13/07/20 18:37:58 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 13/07/20 18:37:58 INFO yarn.GiraphYarnClient: Final output path is:
> hdfs://localhost:9000/outPageRank
> 13/07/20 18:37:58 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
> Exception in thread "main" java.lang.IllegalArgumentException: Giraph on
> YARN does not currentlysupport Giraph-managed ZK instances: use a
> standalone ZooKeeper: 'null'
> at
> org.apache.giraph.yarn.GiraphYarnClient.checkJobLocalZooKeeperSupported(GiraphYarnClient.java:392)
> at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:106)
> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)"
>
>
> I couldn't fine anything about this Zookeeper error on google, any hints?
> Since the PageRankBenchmark is not working on Yarn, what are the examples
> actually working on the current versions?
>
> Oh, another thing: Does anyone knows how to remove the:
>
> "13/07/20 18:37:57 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable"
>
> On Hadoop Yarn 2.0.3-alpha?
>
> Best regards,
> Marco
>
Re: Missing classes SendMessageToAllCache / SendWorkerOneToAllMessagesRequest
Posted by Avery Ching <ac...@apache.org>.
This should be fixed now.
On 7/20/13 12:20 PM, Avery Ching wrote:
>
> My bad. I am out but will fix in a few hours.
>
> On Jul 20, 2013 11:02 AM, "Christian Krause" <me@ckrause.org
> <ma...@ckrause.org>> wrote:
>
> Hi,
> I get these compile errors. Could it be that some classes are missing?
>
> Cheers,
> Christian
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile
> (default-compile) on project giraph-core: Compilation failure:
> Compilation failure:
> [ERROR]
> /home/christian/giraph-git/giraph-core/target/munged/main/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java:[24,30]
> cannot find symbol
> [ERROR] symbol: class SendMessageToAllCache
> [ERROR] location: package org.apache.giraph.comm
> [ERROR]
> /home/christian/giraph-git/giraph-core/target/munged/main/org/apache/giraph/comm/requests/RequestType.java:[41,5]
> cannot find symbol
> [ERROR] symbol: class SendWorkerOneToAllMessagesRequest
> [ERROR] location: class org.apache.giraph.comm.requests.RequestType
> [ERROR]
> /home/christian/giraph-git/giraph-core/target/munged/main/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java:[132,13]
> cannot find symbol
> [ERROR] symbol: class SendMessageToAllCache
> [ERROR] location: class
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor<I,V,E>
>
Missing classes SendMessageToAllCache / SendWorkerOneToAllMessagesRequest
Posted by Christian Krause <me...@ckrause.org>.
Hi,
I get these compile errors. Could it be that some classes are missing?
Cheers,
Christian
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.0:compile
(default-compile) on project giraph-core: Compilation failure:
Compilation failure:
[ERROR]
/home/christian/giraph-git/giraph-core/target/munged/main/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java:[24,30]
cannot find symbol
[ERROR] symbol: class SendMessageToAllCache
[ERROR] location: package org.apache.giraph.comm
[ERROR]
/home/christian/giraph-git/giraph-core/target/munged/main/org/apache/giraph/comm/requests/RequestType.java:[41,5]
cannot find symbol
[ERROR] symbol: class SendWorkerOneToAllMessagesRequest
[ERROR] location: class org.apache.giraph.comm.requests.RequestType
[ERROR]
/home/christian/giraph-git/giraph-core/target/munged/main/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java:[132,13]
cannot find symbol
[ERROR] symbol: class SendMessageToAllCache
[ERROR] location: class
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor<I,V,E>