You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Zeyuan Xiang <xi...@gmail.com> on 2013/08/05 15:06:21 UTC
Running Example Error: Child Error
I used Hadoop 1.1 which is a real cluster with four slave node and I build
success.
When I refer to reference "Shortest Paths", step by step, I got an error as
follows:
* **java.lang.Throwable: Child Error *
* at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242)
Caused by: java.io.IOException: Task process exit with nonzero
status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229)*
* *
*what's more, all the example produce this same error.......[?]*
*And I find someone also encounter this problem, but it seems not solved*
The command:
----------------------------------------------------------------------------------------------------------
hadoop jar
/usr/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation
-vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip shortest/shortest.json
-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op outShort -w 1
----------------------------------------------------------------------------------------------------------
Running info:
----------------------------------------------------------------------------------------------------------
13/08/02 10:10:27 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
13/08/02 10:10:27 INFO job.GiraphJob: run: Since checkpointing is disabled
(default), do not allow any task retries (setting mapred.map.max.attempts =
0, old value = 4)
13/08/02 10:10:33 INFO mapred.JobClient: Running job: job_201308020915_0002
13/08/02 10:10:34 INFO mapred.JobClient: map 0% reduce 0%
13/08/02 10:10:51 INFO mapred.JobClient: map 50% reduce 0%
13/08/02 10:10:54 INFO mapred.JobClient: map 100% reduce 0%
13/08/02 10:10:55 INFO mapred.JobClient: map 50% reduce 0%
13/08/02 10:21:42 INFO mapred.JobClient: map 0% reduce 0%
13/08/02 10:21:42 INFO mapred.JobClient: Job complete: job_201308020915_0002
13/08/02 10:21:42 INFO mapred.JobClient: Counters: 6
13/08/02 10:21:42 INFO mapred.JobClient: Job Counters
13/08/02 10:21:42 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=658594
13/08/02 10:21:42 INFO mapred.JobClient: Total time spent by all
reduces waiting after reserving slots (ms)=0
13/08/02 10:21:42 INFO mapred.JobClient: Total time spent by all maps
waiting after reserving slots (ms)=0
13/08/02 10:21:42 INFO mapred.JobClient: Launched map tasks=2
13/08/02 10:21:42 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
13/08/02 10:21:42 INFO mapred.JobClient: Failed map tasks=1
----------------------------------------------------------------------------------------------------------
Logs:
----------------------------------------------------------------------------------------------------------
2013-08-02 10:10:50,721 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from attempt_201308020915_0002_m_000000_0: java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.TaskInProgress:
TaskInProgress task_201308020915_0002_m_000000 has failed 0 times.
2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.JobTracker: Removing
task 'attempt_201308020915_0002_m_000000_0'
2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.JobTracker: Adding
task (TASK_CLEANUP) 'attempt_201308020915_0002_m_000000_0' to tip
task_201308020915_0002_m_000000, for tracker
'tracker_Slave3.Hadoop:Slave3.Hadoop/127.0.0.1:49172'
2013-08-02 10:10:54,070 INFO org.apache.hadoop.mapred.TaskInProgress:
TaskInProgress task_201308020915_0002_m_000000 has failed 1 times.
2013-08-02 10:10:54,071 INFO org.apache.hadoop.mapred.JobTracker: Removing
task 'attempt_201308020915_0002_m_000000_0'
2013-08-02 10:11:40,700 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from attempt_201308020915_0002_m_000001_0: java.lang.IllegalStateException:
run: Caught an unrecoverable exception exists: Failed to check
/_hadoopBsp/job_201308020915_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
after 3 tries!
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.IllegalStateException: exists: Failed to check
/_hadoopBsp/job_201308020915_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
after 3 tries!
at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt.java:369)
at
org.apache.giraph.worker.BspServiceWorker.startSuperstep(BspServiceWorker.java:686)
at
org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:482)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:246)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
... 7 more
2013-08-02 10:21:41,001 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from attempt_201308020915_0002_m_000001_0: Task
attempt_201308020915_0002_m_000001_0 failed to report status for 600
seconds. Killing!
-----------------------------------------------------------------------------------------------------
I also use "mvn -Phadoop_1.0 clean package" and "mvn
-pHadoop=non_secure", but it did not take effect. Can anybody give me some
advice?
Thank you very much !!!
And is ZooKeeper necessary to Giraph, do I need install it?
Best regards,
Your sincerely Allan Xiang.
Re: Running Example Error: Child Error
Posted by Zeyuan Xiang <xi...@gmail.com>.
Update, this is my error log.
It seems the that the error:
Channel failed with remote address null
java.net.ConnectException: Connection refused
But I didn't know how to solve it. Anybody know? And my hosts file is right.
*syslog logs*
INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Connect attempt 1 of 10 max trying to connect to Slave3.Hadoop:22181
with poll msecs = 3000
INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Connected to Slave3.Hadoop/127.0.0.1:22181!
INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Creating my filestamp
_bsp/_defaultZkManagerDir/job_201308091003_0002/_zkServer/Slave3.Hadoop
0
INFO org.apache.giraph.graph.GraphTaskManager: setup: Chosen to run ZooKeeper...
INFO org.apache.giraph.graph.GraphTaskManager: setup: Starting up
BspServiceMaster (master thread)...
INFO org.apache.giraph.bsp.BspService: BspService: Connecting to
ZooKeeper with job job_201308091003_0002, 0 on Slave3.Hadoop:22181
INFO org.apache.zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27
GMT
INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=Slave3.Hadoop
INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_37
INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.home=/usr/java/jdk1.6.0_37/jre
INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.class.path=/home/hadoop/hadoop-1.1.1/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201308091003_0002/jars/classes:/home/hadoop/hadoop-1.1.1/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201308091003_0002/jars:/home/hadoop/hadoop-1.1.1/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201308091003_0002/attempt_201308091003_0002_m_000000_0/work:/home/hadoop/hadoop-1.1.1/libexec/../conf:/usr/java/jdk1.6.0_37/lib/tools.jar:/home/hadoop/hadoop-1.1.1/libexec/..:/home/hadoop/hadoop-1.1.1/libexec/../hadoop-core-1.1.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/asm-3.2.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/aspectjrt-1.6.11.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/aspectjtools-1.6.11.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-beanutils-1.7.0.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-beanutils-core-1.8.0.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-cli-1.2.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-codec-1.4.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-collections-3.2.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-configuration-1.6.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-daemon-1.0.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-digester-1.8.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-el-1.0.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-httpclient-3.0.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-io-2.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-lang-2.4.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-logging-1.1.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-logging-api-1.0.4.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-math-2.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/commons-net-3.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/core-3.1.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/hadoop-capacity-scheduler-1.1.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/hadoop-fairscheduler-1.1.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/hadoop-thriftfs-1.1.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/hsqldb-1.8.0.10.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jackson-core-asl-1.8.8.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jasper-compiler-5.5.12.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jasper-runtime-5.5.12.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jdeb-0.8.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jersey-core-1.8.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jersey-json-1.8.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jersey-server-1.8.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jets3t-0.6.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jetty-6.1.26.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jetty-util-6.1.26.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jsch-0.1.42.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/junit-4.5.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/kfs-0.2.2.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/log4j-1.2.15.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/mockito-all-1.8.5.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/oro-2.0.8.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/servlet-api-2.5-20081211.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/slf4j-api-1.4.3.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/slf4j-log4j12-1.4.3.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/xmlenc-0.52.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jsp-2.1/jsp-2.1.jar:/home/hadoop/hadoop-1.1.1/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
2013-08-09 10:20:58,718 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.library.path=/home/hadoop/hadoop-1.1.1/libexec/../lib/native/Linux-amd64-64:/home/hadoop/hadoop-1.1.1/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201308091003_0002/attempt_201308091003_0002_m_000000_0/work
INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/home/hadoop/hadoop-1.1.1/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201308091003_0002/attempt_201308091003_0002_m_000000_0/work/tmp
INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.version=2.6.32-71.29.1.el6.x86_64
INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=hadoop
INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/hadoop
INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.dir=/home/hadoop/hadoop-1.1.1/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201308091003_0002/attempt_201308091003_0002_m_000000_0/work
INFO org.apache.zookeeper.ZooKeeper: Initiating client connection,
connectString=Slave3.Hadoop:22181 sessionTimeout=60000
watcher=org.apache.giraph.master.BspServiceMaster@64df83e5
INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to
server Slave3.Hadoop/127.0.0.1:22181
INFO org.apache.zookeeper.ClientCnxn: Socket connection established to
Slave3.Hadoop/127.0.0.1:22181, initiating session
INFO org.apache.zookeeper.ClientCnxn: Session establishment complete
on server Slave3.Hadoop/127.0.0.1:22181, sessionid =
0x14060e06ed10000, negotiated timeout = 600000
INFO org.apache.giraph.bsp.BspService: process: Asynchronous
connection complete.
INFO org.apache.giraph.graph.GraphTaskManager: map: No need to do
anything when not a worker
INFO org.apache.giraph.graph.GraphTaskManager: cleanup: Starting for
MASTER_ZOOKEEPER_ONLY
INFO org.apache.giraph.master.BspServiceMaster: becomeMaster: First
child is '/_hadoopBsp/job_201308091003_0002/_masterElectionDir/Slave3.Hadoop_00000000000'
and my bid is '/_hadoopBsp/job_201308091003_0002/_masterElectionDir/Slave3.Hadoop_00000000000'
INFO org.apache.giraph.comm.netty.NettyServer: NettyServer: Using
execution handler with 8 threads after requestFrameDecoder.
INFO org.apache.giraph.comm.netty.NettyServer: start: Started server
communication server: Slave3.Hadoop/127.0.0.1:30000 with up to 16
threads on bind attempt 0 with sendBufferSize = 32768
receiveBufferSize = 524288 backlog = 1
INFO org.apache.giraph.comm.netty.NettyClient: NettyClient: Using
execution handler with 8 threads after requestEncoder.
INFO org.apache.giraph.master.BspServiceMaster: becomeMaster: I am now
the master!
INFO org.apache.giraph.bsp.BspService: process:
applicationAttemptChanged signaled
WARN org.apache.giraph.bsp.BspService: process: Unknown and
unprocessed event
(path=/_hadoopBsp/job_201308091003_0002/_applicationAttemptsDir/0/_superstepDir,
type=NodeChildrenChanged, state=SyncConnected)
INFO org.apache.giraph.io.formats.GiraphFileInputFormat: Total input
paths to process : 1
WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native
library not loaded
INFO org.apache.giraph.master.BspServiceMaster:
generateVertexInputSplits: Got 1 input splits for 1 input threads
INFO org.apache.giraph.master.BspServiceMaster:
createVertexInputSplits: Starting to write input split data to
zookeeper with 1 threads
INFO org.apache.giraph.master.BspServiceMaster:
createVertexInputSplits: Done writing input split data to zookeeper
INFO org.apache.giraph.comm.netty.NettyClient: Using Netty without
authentication.
WARN org.apache.giraph.comm.netty.NettyClient: connectAllAddresses:
Future failed to connect with Slave2.Hadoop/10.141.201.46:30001 with 0
failures because of java.net.ConnectException: Connection refused
INFO org.apache.giraph.comm.netty.NettyClient: Using Netty without
authentication.
INFO org.apache.giraph.comm.netty.NettyClient: connectAllAddresses:
Successfully added 0 connections, (0 total connected) 1 failed, 1
failures total.
WARN org.apache.giraph.comm.netty.NettyClient: connectAllAddresses:
Future failed to connect with Slave2.Hadoop/10.141.201.46:30001 with 1
failures because of java.net.ConnectException: Connection refused
INFO org.apache.giraph.comm.netty.NettyClient: Using Netty without
authentication.
INFO org.apache.giraph.comm.netty.NettyClient: connectAllAddresses:
Successfully added 0 connections, (0 total connected) 1 failed, 2
failures total.
WARN org.apache.giraph.comm.netty.handler.ResponseClientHandler:
exceptionCaught: Channel failed with remote address null
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:404)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:366)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:282)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
On Tue, Aug 6, 2013 at 12:36 PM, Zeyuan Xiang <xi...@gmail.com>wrote:
> Hi, I didn't install Zookeeper, is it necessary? (The example tutorials
> didn't show it.)
> On my master logs, I only find these about Zookeeper:
> *Caused by: java.lang.IllegalStateException: exists: Failed to check
> /_hadoopBsp/job_201308051938_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
> after 3 tries!*
> * at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt
> .java:369)*
> * at
> org.apache.giraph.worker.BspServiceWorker.startSuperstep(BspServiceWorker.java:686)
> *
> * at
> org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:482)
> *
> * at
> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:246)
> *
> * at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)*
>
> Thank you for your reply!
>
> Best Regards,
> Your sincerely Xiang.
>
>
> On Tue, Aug 6, 2013 at 12:05 PM, Avery Ching <ac...@apache.org> wrote:
>
>> Does the master log show ZooKeeper starting okay?
>>
>>
>> On 8/5/13 6:06 AM, Zeyuan Xiang wrote:
>>
>> I used Hadoop 1.1 which is a real cluster with four slave node and I
>> build success.
>>
>> When I refer to reference "Shortest Paths", step by step, I got an error
>> as follows:
>> * **java.lang.Throwable: Child Error *
>> * at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242)
>> Caused by: java.io.IOException: Task process exit with nonzero
>> status of 1.
>> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229)*
>> * *
>> *what's more, all the example produce this same error.......[?]*
>> *And I find someone also encounter this problem, but it seems not solved
>> *
>> The command:
>>
>> ----------------------------------------------------------------------------------------------------------
>> hadoop jar
>> /usr/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation
>> -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> -vip shortest/shortest.json
>> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat
>> -op outShort -w 1
>>
>> ----------------------------------------------------------------------------------------------------------
>>
>> Running info:
>>
>> ----------------------------------------------------------------------------------------------------------
>> 13/08/02 10:10:27 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 13/08/02 10:10:27 INFO job.GiraphJob: run: Since checkpointing is
>> disabled (default), do not allow any task retries (setting
>> mapred.map.max.attempts = 0, old value = 4)
>> 13/08/02 10:10:33 INFO mapred.JobClient: Running job:
>> job_201308020915_0002
>> 13/08/02 10:10:34 INFO mapred.JobClient: map 0% reduce 0%
>> 13/08/02 10:10:51 INFO mapred.JobClient: map 50% reduce 0%
>> 13/08/02 10:10:54 INFO mapred.JobClient: map 100% reduce 0%
>> 13/08/02 10:10:55 INFO mapred.JobClient: map 50% reduce 0%
>> 13/08/02 10:21:42 INFO mapred.JobClient: map 0% reduce 0%
>> 13/08/02 10:21:42 INFO mapred.JobClient: Job complete:
>> job_201308020915_0002
>> 13/08/02 10:21:42 INFO mapred.JobClient: Counters: 6
>> 13/08/02 10:21:42 INFO mapred.JobClient: Job Counters
>> 13/08/02 10:21:42 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=658594
>> 13/08/02 10:21:42 INFO mapred.JobClient: Total time spent by all
>> reduces waiting after reserving slots (ms)=0
>> 13/08/02 10:21:42 INFO mapred.JobClient: Total time spent by all maps
>> waiting after reserving slots (ms)=0
>> 13/08/02 10:21:42 INFO mapred.JobClient: Launched map tasks=2
>> 13/08/02 10:21:42 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
>> 13/08/02 10:21:42 INFO mapred.JobClient: Failed map tasks=1
>>
>> ----------------------------------------------------------------------------------------------------------
>>
>> Logs:
>>
>> ----------------------------------------------------------------------------------------------------------
>> 2013-08-02 10:10:50,721 INFO org.apache.hadoop.mapred.TaskInProgress:
>> Error from attempt_201308020915_0002_m_000000_0: java.lang.Throwable: Child
>> Error
>> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
>> Caused by: java.io.IOException: Task process exit with nonzero status of
>> 1.
>> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>>
>> 2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.TaskInProgress:
>> TaskInProgress task_201308020915_0002_m_000000 has failed 0 times.
>> 2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.JobTracker:
>> Removing task 'attempt_201308020915_0002_m_000000_0'
>> 2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.JobTracker: Adding
>> task (TASK_CLEANUP) 'attempt_201308020915_0002_m_000000_0' to tip
>> task_201308020915_0002_m_000000, for tracker
>> 'tracker_Slave3.Hadoop:Slave3.Hadoop/127.0.0.1:49172'
>> 2013-08-02 10:10:54,070 INFO org.apache.hadoop.mapred.TaskInProgress:
>> TaskInProgress task_201308020915_0002_m_000000 has failed 1 times.
>> 2013-08-02 10:10:54,071 INFO org.apache.hadoop.mapred.JobTracker:
>> Removing task 'attempt_201308020915_0002_m_000000_0'
>> 2013-08-02 10:11:40,700 INFO org.apache.hadoop.mapred.TaskInProgress:
>> Error from attempt_201308020915_0002_m_000001_0:
>> java.lang.IllegalStateException: run: Caught an unrecoverable exception
>> exists: Failed to check
>> /_hadoopBsp/job_201308020915_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
>> after 3 tries!
>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:396)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
>> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>> Caused by: java.lang.IllegalStateException: exists: Failed to check
>> /_hadoopBsp/job_201308020915_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
>> after 3 tries!
>> at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt.java:369)
>> at
>> org.apache.giraph.worker.BspServiceWorker.startSuperstep(BspServiceWorker.java:686)
>> at
>> org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:482)
>> at
>> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:246)
>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
>> ... 7 more
>>
>> 2013-08-02 10:21:41,001 INFO org.apache.hadoop.mapred.TaskInProgress:
>> Error from attempt_201308020915_0002_m_000001_0: Task
>> attempt_201308020915_0002_m_000001_0 failed to report status for 600
>> seconds. Killing!
>>
>> -----------------------------------------------------------------------------------------------------
>>
>> I also use "mvn -Phadoop_1.0 clean package" and "mvn
>> -pHadoop=non_secure", but it did not take effect. Can anybody give me some
>> advice?
>> Thank you very much !!!
>>
>> And is ZooKeeper necessary to Giraph, do I need install it?
>>
>> Best regards,
>> Your sincerely Allan Xiang.
>>
>>
>>
>
Re: Running Example Error: Child Error
Posted by Zeyuan Xiang <xi...@gmail.com>.
Hi, I didn't install Zookeeper, is it necessary? (The example tutorials
didn't show it.)
On my master logs, I only find these about Zookeeper:
*Caused by: java.lang.IllegalStateException: exists: Failed to check
/_hadoopBsp/job_201308051938_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
after 3 tries!*
* at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt.java:369)*
* at
org.apache.giraph.worker.BspServiceWorker.startSuperstep(BspServiceWorker.java:686)
*
* at
org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:482)*
* at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:246)*
* at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)*
Thank you for your reply!
Best Regards,
Your sincerely Xiang.
On Tue, Aug 6, 2013 at 12:05 PM, Avery Ching <ac...@apache.org> wrote:
> Does the master log show ZooKeeper starting okay?
>
>
> On 8/5/13 6:06 AM, Zeyuan Xiang wrote:
>
> I used Hadoop 1.1 which is a real cluster with four slave node and I
> build success.
>
> When I refer to reference "Shortest Paths", step by step, I got an error
> as follows:
> * **java.lang.Throwable: Child Error *
> * at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242)
> Caused by: java.io.IOException: Task process exit with nonzero
> status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229)*
> * *
> *what's more, all the example produce this same error.......[?]*
> *And I find someone also encounter this problem, but it seems not solved*
> The command:
>
> ----------------------------------------------------------------------------------------------------------
> hadoop jar
> /usr/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation
> -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip shortest/shortest.json
> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat
> -op outShort -w 1
>
> ----------------------------------------------------------------------------------------------------------
>
> Running info:
>
> ----------------------------------------------------------------------------------------------------------
> 13/08/02 10:10:27 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 13/08/02 10:10:27 INFO job.GiraphJob: run: Since checkpointing is disabled
> (default), do not allow any task retries (setting mapred.map.max.attempts =
> 0, old value = 4)
> 13/08/02 10:10:33 INFO mapred.JobClient: Running job:
> job_201308020915_0002
> 13/08/02 10:10:34 INFO mapred.JobClient: map 0% reduce 0%
> 13/08/02 10:10:51 INFO mapred.JobClient: map 50% reduce 0%
> 13/08/02 10:10:54 INFO mapred.JobClient: map 100% reduce 0%
> 13/08/02 10:10:55 INFO mapred.JobClient: map 50% reduce 0%
> 13/08/02 10:21:42 INFO mapred.JobClient: map 0% reduce 0%
> 13/08/02 10:21:42 INFO mapred.JobClient: Job complete:
> job_201308020915_0002
> 13/08/02 10:21:42 INFO mapred.JobClient: Counters: 6
> 13/08/02 10:21:42 INFO mapred.JobClient: Job Counters
> 13/08/02 10:21:42 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=658594
> 13/08/02 10:21:42 INFO mapred.JobClient: Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/08/02 10:21:42 INFO mapred.JobClient: Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/08/02 10:21:42 INFO mapred.JobClient: Launched map tasks=2
> 13/08/02 10:21:42 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> 13/08/02 10:21:42 INFO mapred.JobClient: Failed map tasks=1
>
> ----------------------------------------------------------------------------------------------------------
>
> Logs:
>
> ----------------------------------------------------------------------------------------------------------
> 2013-08-02 10:10:50,721 INFO org.apache.hadoop.mapred.TaskInProgress:
> Error from attempt_201308020915_0002_m_000000_0: java.lang.Throwable: Child
> Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>
> 2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.TaskInProgress:
> TaskInProgress task_201308020915_0002_m_000000 has failed 0 times.
> 2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201308020915_0002_m_000000_0'
> 2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.JobTracker: Adding
> task (TASK_CLEANUP) 'attempt_201308020915_0002_m_000000_0' to tip
> task_201308020915_0002_m_000000, for tracker
> 'tracker_Slave3.Hadoop:Slave3.Hadoop/127.0.0.1:49172'
> 2013-08-02 10:10:54,070 INFO org.apache.hadoop.mapred.TaskInProgress:
> TaskInProgress task_201308020915_0002_m_000000 has failed 1 times.
> 2013-08-02 10:10:54,071 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201308020915_0002_m_000000_0'
> 2013-08-02 10:11:40,700 INFO org.apache.hadoop.mapred.TaskInProgress:
> Error from attempt_201308020915_0002_m_000001_0:
> java.lang.IllegalStateException: run: Caught an unrecoverable exception
> exists: Failed to check
> /_hadoopBsp/job_201308020915_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
> after 3 tries!
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.IllegalStateException: exists: Failed to check
> /_hadoopBsp/job_201308020915_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
> after 3 tries!
> at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt.java:369)
> at
> org.apache.giraph.worker.BspServiceWorker.startSuperstep(BspServiceWorker.java:686)
> at
> org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:482)
> at
> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:246)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
> ... 7 more
>
> 2013-08-02 10:21:41,001 INFO org.apache.hadoop.mapred.TaskInProgress:
> Error from attempt_201308020915_0002_m_000001_0: Task
> attempt_201308020915_0002_m_000001_0 failed to report status for 600
> seconds. Killing!
>
> -----------------------------------------------------------------------------------------------------
>
> I also use "mvn -Phadoop_1.0 clean package" and "mvn
> -pHadoop=non_secure", but it did not take effect. Can anybody give me some
> advice?
> Thank you very much !!!
>
> And is ZooKeeper necessary to Giraph, do I need install it?
>
> Best regards,
> Your sincerely Allan Xiang.
>
>
>
Re: Running Example Error: Child Error
Posted by Avery Ching <ac...@apache.org>.
Does the master log show ZooKeeper starting okay?
On 8/5/13 6:06 AM, Zeyuan Xiang wrote:
> I used Hadoop 1.1which is a real cluster with four slave node and I
> build success.
> When I refer to reference "Shortest Paths", step by step, I got an
> error as follows:
> ***java.lang.Throwable: Child Error *
> * at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242)
> Caused by: java.io.IOException: Task process exit with nonzero
> status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229)*
> **
> *what's more, all the example produce this same error.......*
> *And I find someone also encounter this problem, but it seems not solved*
> The command:
> ----------------------------------------------------------------------------------------------------------
> hadoop jar
> /usr/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation
> -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip shortest/shortest.json
> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat
> -op outShort -w 1
> ----------------------------------------------------------------------------------------------------------
>
> Running info:
> ----------------------------------------------------------------------------------------------------------
> 13/08/02 10:10:27 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 13/08/02 10:10:27 INFO job.GiraphJob: run: Since checkpointing is
> disabled (default), do not allow any task retries (setting
> mapred.map.max.attempts = 0, old value = 4)
> 13/08/02 10:10:33 INFO mapred.JobClient: Running job:
> job_201308020915_0002
> 13/08/02 10:10:34 INFO mapred.JobClient: map 0% reduce 0%
> 13/08/02 10:10:51 INFO mapred.JobClient: map 50% reduce 0%
> 13/08/02 10:10:54 INFO mapred.JobClient: map 100% reduce 0%
> 13/08/02 10:10:55 INFO mapred.JobClient: map 50% reduce 0%
> 13/08/02 10:21:42 INFO mapred.JobClient: map 0% reduce 0%
> 13/08/02 10:21:42 INFO mapred.JobClient: Job complete:
> job_201308020915_0002
> 13/08/02 10:21:42 INFO mapred.JobClient: Counters: 6
> 13/08/02 10:21:42 INFO mapred.JobClient: Job Counters
> 13/08/02 10:21:42 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=658594
> 13/08/02 10:21:42 INFO mapred.JobClient: Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/08/02 10:21:42 INFO mapred.JobClient: Total time spent by all
> maps waiting after reserving slots (ms)=0
> 13/08/02 10:21:42 INFO mapred.JobClient: Launched map tasks=2
> 13/08/02 10:21:42 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> 13/08/02 10:21:42 INFO mapred.JobClient: Failed map tasks=1
> ----------------------------------------------------------------------------------------------------------
>
> Logs:
> ----------------------------------------------------------------------------------------------------------
> 2013-08-02 10:10:50,721 INFO org.apache.hadoop.mapred.TaskInProgress:
> Error from attempt_201308020915_0002_m_000000_0: java.lang.Throwable:
> Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status
> of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>
> 2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.TaskInProgress:
> TaskInProgress task_201308020915_0002_m_000000 has failed 0 times.
> 2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.JobTracker:
> Removing task 'attempt_201308020915_0002_m_000000_0'
> 2013-08-02 10:10:53,765 INFO org.apache.hadoop.mapred.JobTracker:
> Adding task (TASK_CLEANUP) 'attempt_201308020915_0002_m_000000_0' to
> tip task_201308020915_0002_m_000000, for tracker
> 'tracker_Slave3.Hadoop:Slave3.Hadoop/127.0.0.1:49172
> <http://127.0.0.1:49172/>'
> 2013-08-02 10:10:54,070 INFO org.apache.hadoop.mapred.TaskInProgress:
> TaskInProgress task_201308020915_0002_m_000000 has failed 1 times.
> 2013-08-02 10:10:54,071 INFO org.apache.hadoop.mapred.JobTracker:
> Removing task 'attempt_201308020915_0002_m_000000_0'
> 2013-08-02 10:11:40,700 INFO org.apache.hadoop.mapred.TaskInProgress:
> Error from attempt_201308020915_0002_m_000001_0:
> java.lang.IllegalStateException: run: Caught an unrecoverable
> exception exists: Failed to check
> /_hadoopBsp/job_201308020915_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
> after 3 tries!
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.IllegalStateException: exists: Failed to check
> /_hadoopBsp/job_201308020915_0002/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions
> after 3 tries!
> at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt.java:369)
> at
> org.apache.giraph.worker.BspServiceWorker.startSuperstep(BspServiceWorker.java:686)
> at
> org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:482)
> at
> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:246)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
> ... 7 more
>
> 2013-08-02 10:21:41,001 INFO org.apache.hadoop.mapred.TaskInProgress:
> Error from attempt_201308020915_0002_m_000001_0: Task
> attempt_201308020915_0002_m_000001_0 failed to report status for 600
> seconds. Killing!
> -----------------------------------------------------------------------------------------------------
>
> I also use "mvn -Phadoop_1.0 clean package" and "mvn
> -pHadoop=non_secure", but it did not take effect. Can anybody give me
> some advice?
> Thank you very much !!!
>
> And is ZooKeeper necessary to Giraph, do I need install it?
>
> Best regards,
> Your sincerely Allan Xiang.