You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@gora.apache.org by Chahinez Benkoussas <me...@gmail.com> on 2014/12/01 18:03:05 UTC
Failing to run Giraph with Gora
Hi!,
I've stored a simple graph into HBase and now I want to run Giraph on the
Graph by the intermediate of Gora.
I'm following this tutorial : http://giraph.apache.org/gora.html
The commande line that I'm using is:
hadoop/bin/hadoop jar $GIRAPH_EXAMPLES_JAR org.apache.giraph.GiraphRunner \
-files
/usr/local/giraph/giraph-gora/conf/gora.properties,/usr/local/giraph/giraph-gora/conf/gora-hbase-mapping.xml,/usr/local/hbase/conf/hbase-site.xml
\
-Dio.serializations=org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.JavaSerialization
\
-Dgiraph.gora.datastore.class=org.apache.gora.hbase.store.HBaseStore \
-Dgiraph.gora.key.class=java.lang.String \
-Dgiraph.gora.persistent.class=org.apache.giraph.io.gora.generated.GEdge \
-Dgiraph.gora.start.key=0 \
-Dgiraph.gora.end.key=10 \
-Dgiraph.gora.keys.factory.class=org.apache.giraph.io.gora.utils.KeyFactory
\
-Dgiraph.gora.output.datastore.class=org.apache.gora.hbase.store.HBaseStore
\
-Dgiraph.gora.output.key.class=java.lang.String \
-Dgiraph.gora.output.persistent.class=org.apache.giraph.io.gora.generated.GEdgeResult
\
-libjars $GIRAPH_GORA_JAR,$GORA_HBASE_JAR,$HBASE_JAR \
org.apache.giraph.examples.SimpleShortestPathsComputation \
-eif org.apache.giraph.io.gora.GoraGEdgeEdgeInputFormat \
-eof org.apache.giraph.io.gora.GoraGEdgeEdgeOutputFormat \
-w 1
and I have this error:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hadoop-2.4.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/giraph/giraph-gora/target/giraph-gora-1.1.0-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/12/01 17:46:33 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/12/01 17:46:33 INFO utils.ConfigurationUtils: No vertex input format
specified. Ensure your InputFormat does not require one.
14/12/01 17:46:33 INFO utils.ConfigurationUtils: No vertex output format
specified. Ensure your OutputFormat does not require one.
14/12/01 17:46:33 INFO utils.ConfigurationUtils: No input path for edge
data was specified. Ensure your InputFormat does not require one.
14/12/01 17:46:33 INFO Configuration.deprecation:
mapreduce.job.counters.limit is deprecated. Instead, use
mapreduce.job.counters.max
14/12/01 17:46:33 INFO Configuration.deprecation: mapred.job.map.memory.mb
is deprecated. Instead, use mapreduce.map.memory.mb
14/12/01 17:46:33 INFO Configuration.deprecation:
mapred.job.reduce.memory.mb is deprecated. Instead, use
mapreduce.reduce.memory.mb
14/12/01 17:46:33 INFO Configuration.deprecation:
mapred.map.tasks.speculative.execution is deprecated. Instead, use
mapreduce.map.speculative
14/12/01 17:46:33 INFO Configuration.deprecation:
mapreduce.user.classpath.first is deprecated. Instead, use
mapreduce.job.user.classpath.first
14/12/01 17:46:33 INFO Configuration.deprecation: mapred.map.max.attempts
is deprecated. Instead, use mapreduce.map.maxattempts
14/12/01 17:46:33 INFO job.GiraphJob: run: Since checkpointing is disabled
(default), do not allow any task retries (setting mapred.map.max.attempts =
0, old value = 4)
14/12/01 17:46:33 INFO Configuration.deprecation: mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
14/12/01 17:46:34 INFO client.RMProxy: Connecting to ResourceManager at
localhost/127.0.0.1:8050
14/12/01 17:46:36 INFO mapreduce.JobSubmitter: number of splits:2
14/12/01 17:46:36 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1415962409090_0006
14/12/01 17:46:36 INFO impl.YarnClientImpl: Submitted application
application_1415962409090_0006
14/12/01 17:46:36 INFO mapreduce.Job: The url to track the job:
http://up12.in.revues.org:8088/proxy/application_1415962409090_0006/
14/12/01 17:46:36 INFO job.GiraphJob: Tracking URL:
http://up12.in.revues.org:8088/proxy/application_1415962409090_0006/
14/12/01 17:46:36 INFO job.GiraphJob: Waiting for resources... Job will
start only when it gets all 2 mappers
14/12/01 17:46:46 INFO mapreduce.Job: Running job: job_1415962409090_0006
14/12/01 17:46:46 INFO mapreduce.Job: Job job_1415962409090_0006 running in
uber mode : false
14/12/01 17:46:46 INFO mapreduce.Job: map 0% reduce 0%
14/12/01 17:46:46 INFO mapreduce.Job: Job job_1415962409090_0006 failed
with state FAILED due to: Application application_1415962409090_0006 failed
2 times due to AM Container for appattempt_1415962409090_0006_000002 exited
with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
14/12/01 17:46:46 INFO mapreduce.Job: Counters: 0
Ican't see where is the problem.
Heeeelp!
Bests,
Chahinez
Re: Failing to run Giraph with Gora
Posted by Renato Marroquín Mogrovejo <re...@gmail.com>.
Hi there,
Could you check the logs in the application containers? They probably have
a more detailed error message that we can look at.
Renato M.
2014-12-01 18:03 GMT+01:00 Chahinez Benkoussas <me...@gmail.com>:
> Hi!,
>
> I've stored a simple graph into HBase and now I want to run Giraph on the
> Graph by the intermediate of Gora.
>
> I'm following this tutorial : http://giraph.apache.org/gora.html
>
> The commande line that I'm using is:
>
> hadoop/bin/hadoop jar $GIRAPH_EXAMPLES_JAR org.apache.giraph.GiraphRunner \
> -files
> /usr/local/giraph/giraph-gora/conf/gora.properties,/usr/local/giraph/giraph-gora/conf/gora-hbase-mapping.xml,/usr/local/hbase/conf/hbase-site.xml
> \
>
> -Dio.serializations=org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.JavaSerialization
> \
> -Dgiraph.gora.datastore.class=org.apache.gora.hbase.store.HBaseStore \
> -Dgiraph.gora.key.class=java.lang.String \
>
> -Dgiraph.gora.persistent.class=org.apache.giraph.io.gora.generated.GEdge \
> -Dgiraph.gora.start.key=0 \
> -Dgiraph.gora.end.key=10 \
>
> -Dgiraph.gora.keys.factory.class=org.apache.giraph.io.gora.utils.KeyFactory
> \
>
> -Dgiraph.gora.output.datastore.class=org.apache.gora.hbase.store.HBaseStore
> \
> -Dgiraph.gora.output.key.class=java.lang.String \
>
> -Dgiraph.gora.output.persistent.class=org.apache.giraph.io.gora.generated.GEdgeResult
> \
> -libjars $GIRAPH_GORA_JAR,$GORA_HBASE_JAR,$HBASE_JAR \
> org.apache.giraph.examples.SimpleShortestPathsComputation \
> -eif org.apache.giraph.io.gora.GoraGEdgeEdgeInputFormat \
> -eof org.apache.giraph.io.gora.GoraGEdgeEdgeOutputFormat \
> -w 1
>
> and I have this error:
>
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop-2.4.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/giraph/giraph-gora/target/giraph-gora-1.1.0-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 14/12/01 17:46:33 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 14/12/01 17:46:33 INFO utils.ConfigurationUtils: No vertex input format
> specified. Ensure your InputFormat does not require one.
> 14/12/01 17:46:33 INFO utils.ConfigurationUtils: No vertex output format
> specified. Ensure your OutputFormat does not require one.
> 14/12/01 17:46:33 INFO utils.ConfigurationUtils: No input path for edge
> data was specified. Ensure your InputFormat does not require one.
> 14/12/01 17:46:33 INFO Configuration.deprecation:
> mapreduce.job.counters.limit is deprecated. Instead, use
> mapreduce.job.counters.max
> 14/12/01 17:46:33 INFO Configuration.deprecation: mapred.job.map.memory.mb
> is deprecated. Instead, use mapreduce.map.memory.mb
> 14/12/01 17:46:33 INFO Configuration.deprecation:
> mapred.job.reduce.memory.mb is deprecated. Instead, use
> mapreduce.reduce.memory.mb
> 14/12/01 17:46:33 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 14/12/01 17:46:33 INFO Configuration.deprecation:
> mapreduce.user.classpath.first is deprecated. Instead, use
> mapreduce.job.user.classpath.first
> 14/12/01 17:46:33 INFO Configuration.deprecation: mapred.map.max.attempts
> is deprecated. Instead, use mapreduce.map.maxattempts
> 14/12/01 17:46:33 INFO job.GiraphJob: run: Since checkpointing is disabled
> (default), do not allow any task retries (setting mapred.map.max.attempts =
> 0, old value = 4)
> 14/12/01 17:46:33 INFO Configuration.deprecation: mapred.job.tracker is
> deprecated. Instead, use mapreduce.jobtracker.address
> 14/12/01 17:46:34 INFO client.RMProxy: Connecting to ResourceManager at
> localhost/127.0.0.1:8050
> 14/12/01 17:46:36 INFO mapreduce.JobSubmitter: number of splits:2
> 14/12/01 17:46:36 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1415962409090_0006
> 14/12/01 17:46:36 INFO impl.YarnClientImpl: Submitted application
> application_1415962409090_0006
> 14/12/01 17:46:36 INFO mapreduce.Job: The url to track the job:
> http://up12.in.revues.org:8088/proxy/application_1415962409090_0006/
> 14/12/01 17:46:36 INFO job.GiraphJob: Tracking URL:
> http://up12.in.revues.org:8088/proxy/application_1415962409090_0006/
> 14/12/01 17:46:36 INFO job.GiraphJob: Waiting for resources... Job will
> start only when it gets all 2 mappers
> 14/12/01 17:46:46 INFO mapreduce.Job: Running job: job_1415962409090_0006
> 14/12/01 17:46:46 INFO mapreduce.Job: Job job_1415962409090_0006 running
> in uber mode : false
> 14/12/01 17:46:46 INFO mapreduce.Job: map 0% reduce 0%
> 14/12/01 17:46:46 INFO mapreduce.Job: Job job_1415962409090_0006 failed
> with state FAILED due to: Application application_1415962409090_0006 failed
> 2 times due to AM Container for appattempt_1415962409090_0006_000002 exited
> with exitCode: 1 due to: Exception from container-launch:
> org.apache.hadoop.util.Shell$ExitCodeException:
> org.apache.hadoop.util.Shell$ExitCodeException:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
> at org.apache.hadoop.util.Shell.run(Shell.java:418)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
> Container exited with a non-zero exit code 1
> .Failing this attempt.. Failing the application.
> 14/12/01 17:46:46 INFO mapreduce.Job: Counters: 0
>
>
>
> Ican't see where is the problem.
> Heeeelp!
>
> Bests,
> Chahinez
>
Re: Failing to run Giraph with Gora
Posted by Renato Marroquín Mogrovejo <re...@gmail.com>.
Hi there,
Could you check the logs in the application containers? They probably have
a more detailed error message that we can look at.
Renato M.
2014-12-01 18:03 GMT+01:00 Chahinez Benkoussas <me...@gmail.com>:
> Hi!,
>
> I've stored a simple graph into HBase and now I want to run Giraph on the
> Graph by the intermediate of Gora.
>
> I'm following this tutorial : http://giraph.apache.org/gora.html
>
> The commande line that I'm using is:
>
> hadoop/bin/hadoop jar $GIRAPH_EXAMPLES_JAR org.apache.giraph.GiraphRunner \
> -files
> /usr/local/giraph/giraph-gora/conf/gora.properties,/usr/local/giraph/giraph-gora/conf/gora-hbase-mapping.xml,/usr/local/hbase/conf/hbase-site.xml
> \
>
> -Dio.serializations=org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.JavaSerialization
> \
> -Dgiraph.gora.datastore.class=org.apache.gora.hbase.store.HBaseStore \
> -Dgiraph.gora.key.class=java.lang.String \
>
> -Dgiraph.gora.persistent.class=org.apache.giraph.io.gora.generated.GEdge \
> -Dgiraph.gora.start.key=0 \
> -Dgiraph.gora.end.key=10 \
>
> -Dgiraph.gora.keys.factory.class=org.apache.giraph.io.gora.utils.KeyFactory
> \
>
> -Dgiraph.gora.output.datastore.class=org.apache.gora.hbase.store.HBaseStore
> \
> -Dgiraph.gora.output.key.class=java.lang.String \
>
> -Dgiraph.gora.output.persistent.class=org.apache.giraph.io.gora.generated.GEdgeResult
> \
> -libjars $GIRAPH_GORA_JAR,$GORA_HBASE_JAR,$HBASE_JAR \
> org.apache.giraph.examples.SimpleShortestPathsComputation \
> -eif org.apache.giraph.io.gora.GoraGEdgeEdgeInputFormat \
> -eof org.apache.giraph.io.gora.GoraGEdgeEdgeOutputFormat \
> -w 1
>
> and I have this error:
>
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop-2.4.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/giraph/giraph-gora/target/giraph-gora-1.1.0-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 14/12/01 17:46:33 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 14/12/01 17:46:33 INFO utils.ConfigurationUtils: No vertex input format
> specified. Ensure your InputFormat does not require one.
> 14/12/01 17:46:33 INFO utils.ConfigurationUtils: No vertex output format
> specified. Ensure your OutputFormat does not require one.
> 14/12/01 17:46:33 INFO utils.ConfigurationUtils: No input path for edge
> data was specified. Ensure your InputFormat does not require one.
> 14/12/01 17:46:33 INFO Configuration.deprecation:
> mapreduce.job.counters.limit is deprecated. Instead, use
> mapreduce.job.counters.max
> 14/12/01 17:46:33 INFO Configuration.deprecation: mapred.job.map.memory.mb
> is deprecated. Instead, use mapreduce.map.memory.mb
> 14/12/01 17:46:33 INFO Configuration.deprecation:
> mapred.job.reduce.memory.mb is deprecated. Instead, use
> mapreduce.reduce.memory.mb
> 14/12/01 17:46:33 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 14/12/01 17:46:33 INFO Configuration.deprecation:
> mapreduce.user.classpath.first is deprecated. Instead, use
> mapreduce.job.user.classpath.first
> 14/12/01 17:46:33 INFO Configuration.deprecation: mapred.map.max.attempts
> is deprecated. Instead, use mapreduce.map.maxattempts
> 14/12/01 17:46:33 INFO job.GiraphJob: run: Since checkpointing is disabled
> (default), do not allow any task retries (setting mapred.map.max.attempts =
> 0, old value = 4)
> 14/12/01 17:46:33 INFO Configuration.deprecation: mapred.job.tracker is
> deprecated. Instead, use mapreduce.jobtracker.address
> 14/12/01 17:46:34 INFO client.RMProxy: Connecting to ResourceManager at
> localhost/127.0.0.1:8050
> 14/12/01 17:46:36 INFO mapreduce.JobSubmitter: number of splits:2
> 14/12/01 17:46:36 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1415962409090_0006
> 14/12/01 17:46:36 INFO impl.YarnClientImpl: Submitted application
> application_1415962409090_0006
> 14/12/01 17:46:36 INFO mapreduce.Job: The url to track the job:
> http://up12.in.revues.org:8088/proxy/application_1415962409090_0006/
> 14/12/01 17:46:36 INFO job.GiraphJob: Tracking URL:
> http://up12.in.revues.org:8088/proxy/application_1415962409090_0006/
> 14/12/01 17:46:36 INFO job.GiraphJob: Waiting for resources... Job will
> start only when it gets all 2 mappers
> 14/12/01 17:46:46 INFO mapreduce.Job: Running job: job_1415962409090_0006
> 14/12/01 17:46:46 INFO mapreduce.Job: Job job_1415962409090_0006 running
> in uber mode : false
> 14/12/01 17:46:46 INFO mapreduce.Job: map 0% reduce 0%
> 14/12/01 17:46:46 INFO mapreduce.Job: Job job_1415962409090_0006 failed
> with state FAILED due to: Application application_1415962409090_0006 failed
> 2 times due to AM Container for appattempt_1415962409090_0006_000002 exited
> with exitCode: 1 due to: Exception from container-launch:
> org.apache.hadoop.util.Shell$ExitCodeException:
> org.apache.hadoop.util.Shell$ExitCodeException:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
> at org.apache.hadoop.util.Shell.run(Shell.java:418)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
> Container exited with a non-zero exit code 1
> .Failing this attempt.. Failing the application.
> 14/12/01 17:46:46 INFO mapreduce.Job: Counters: 0
>
>
>
> Ican't see where is the problem.
> Heeeelp!
>
> Bests,
> Chahinez
>