You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Rafal Wojdyla (JIRA)" <ji...@apache.org> on 2013/12/01 15:34:35 UTC
[jira] [Updated] (GIRAPH-806) Application goes into infinite loop
after success, waits for: Waiting for
org.apache.giraph.utils.ProgressableUtils
[ https://issues.apache.org/jira/browse/GIRAPH-806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rafal Wojdyla updated GIRAPH-806:
---------------------------------
Description:
Run example:
{noformat}
hadoop jar giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0.2.0.6.0-76-jar-with-dependencies.jar org.apache.giraph.GiraphRunner -D giraph.zkList="zkNode.net:2181" -D yarn.app.mapreduce.am.log.level=DEBUG -D mapreduce.map.log.level=DEBUG -D mapreduce.reduce.log.level=DEBUG org.apache.giraph.examples.SimplePageRankComputation -w 1 -yh 1024 -yj ./giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0.2.0.6.0-76-jar-with-dependencies.jar -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -mc org.apache.giraph.examples.SimplePageRankComputation\$SimplePageRankMasterCompute -vip /user/rav/giraph/input/giraph.in -op /user/rav/giraph/output
{noformat}
After application successfully computer result it goes to infinite loop of:
{noformat}
2013-11-28 20:09:51,764 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: setJobState: {"_stateKey":"FINISHED","_applicationAttemptKey":-1,"_superstepKey":-1} on superstep 31
2013-11-28 20:09:51,821 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanup: Notifying master its okay to cleanup with /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir/0_master
2013-11-28 20:09:51,829 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanUpZooKeeper: Node /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir already exists, no need to create.
2013-11-28 20:09:51,831 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanUpZooKeeper: Got 1 of 2 desired children from /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir
2013-11-28 20:09:51,831 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanedUpZooKeeper: Waiting for the children of /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir to change since only got 1 nodes.
2013-11-28 20:09:51,909 INFO [main-EventThread] org.apache.giraph.bsp.BspService: process: cleanedUpChildrenChanged signaled
2013-11-28 20:09:51,910 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanUpZooKeeper: Got 2 of 2 desired children from /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir
2013-11-28 20:09:51,911 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanupZooKeeper: Removing the following path and all children - /_hadoopBsp/job_1385571160481_11413 from ZooKeeper list zkNode.net:2181
2013-11-28 20:09:52,329 INFO [main-EventThread] org.apache.giraph.bsp.BspService: process: masterElectionChildrenChanged signaled
2013-11-28 20:09:52,379 INFO [main-EventThread] org.apache.giraph.bsp.BspService: process: cleanedUpChildrenChanged signaled
2013-11-28 20:09:52,447 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanup: Removed HDFS checkpoint directory (_bsp/_checkpoints//job_1385571160481_11413) with return = false since the job Giraph: org.apache.giraph.examples.SimplePageRankComputation succeeded
2013-11-28 20:09:52,449 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.comm.netty.NettyClient: stop: reached wait threshold, 1 connections closed, releasing NettyClient.bootstrap resources now.
2013-11-28 20:09:52,454 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.comm.netty.NettyServer: stop: Halting netty server
2013-11-28 20:10:52,463 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:11:52,463 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:12:52,463 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:13:52,464 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:14:52,464 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:15:52,464 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:16:52,465 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:17:52,465 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:18:52,465 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:19:52,466 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:20:52,466 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:21:52,466 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:22:52,467 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
{noformat}
was:
Run example:
{noformat}
hadoop jar giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0.2.0.6.0-76-jar-with-dependencies.jar org.apache.giraph.GiraphRunnerD giraph.zkList="zkNode.net:2181" -D yarn.app.mapreduce.am.log.level=DEBUG -D mapreduce.map.log.level=DEBUG -D mapreduce.reduce.log.level=DEBUG org.apache.giraph.examples.SimplePageRankComputation -w 1 -yh 1024 -yj ./giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0.2.0.6.0-76-jar-with-dependencies.jar -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -mc org.apache.giraph.examples.SimplePageRankComputation\$SimplePageRankMasterCompute -vip /user/rav/giraph/input/giraph.in -op /user/rav/giraph/output
{noformat}
After application successfully computer result it goes to infinite loop of:
{noformat}
2013-11-28 20:09:51,764 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: setJobState: {"_stateKey":"FINISHED","_applicationAttemptKey":-1,"_superstepKey":-1} on superstep 31
2013-11-28 20:09:51,821 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanup: Notifying master its okay to cleanup with /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir/0_master
2013-11-28 20:09:51,829 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanUpZooKeeper: Node /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir already exists, no need to create.
2013-11-28 20:09:51,831 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanUpZooKeeper: Got 1 of 2 desired children from /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir
2013-11-28 20:09:51,831 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanedUpZooKeeper: Waiting for the children of /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir to change since only got 1 nodes.
2013-11-28 20:09:51,909 INFO [main-EventThread] org.apache.giraph.bsp.BspService: process: cleanedUpChildrenChanged signaled
2013-11-28 20:09:51,910 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanUpZooKeeper: Got 2 of 2 desired children from /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir
2013-11-28 20:09:51,911 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanupZooKeeper: Removing the following path and all children - /_hadoopBsp/job_1385571160481_11413 from ZooKeeper list zkNode.net:2181
2013-11-28 20:09:52,329 INFO [main-EventThread] org.apache.giraph.bsp.BspService: process: masterElectionChildrenChanged signaled
2013-11-28 20:09:52,379 INFO [main-EventThread] org.apache.giraph.bsp.BspService: process: cleanedUpChildrenChanged signaled
2013-11-28 20:09:52,447 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanup: Removed HDFS checkpoint directory (_bsp/_checkpoints//job_1385571160481_11413) with return = false since the job Giraph: org.apache.giraph.examples.SimplePageRankComputation succeeded
2013-11-28 20:09:52,449 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.comm.netty.NettyClient: stop: reached wait threshold, 1 connections closed, releasing NettyClient.bootstrap resources now.
2013-11-28 20:09:52,454 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.comm.netty.NettyServer: stop: Halting netty server
2013-11-28 20:10:52,463 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:11:52,463 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:12:52,463 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:13:52,464 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:14:52,464 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:15:52,464 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:16:52,465 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:17:52,465 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:18:52,465 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:19:52,466 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:20:52,466 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:21:52,466 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
2013-11-28 20:22:52,467 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
{noformat}
> Application goes into infinite loop after success, waits for: Waiting for org.apache.giraph.utils.ProgressableUtils
> -------------------------------------------------------------------------------------------------------------------
>
> Key: GIRAPH-806
> URL: https://issues.apache.org/jira/browse/GIRAPH-806
> Project: Giraph
> Issue Type: Bug
> Components: examples, mapreduce
> Affects Versions: 1.1.0
> Reporter: Rafal Wojdyla
>
> Run example:
> {noformat}
> hadoop jar giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0.2.0.6.0-76-jar-with-dependencies.jar org.apache.giraph.GiraphRunner -D giraph.zkList="zkNode.net:2181" -D yarn.app.mapreduce.am.log.level=DEBUG -D mapreduce.map.log.level=DEBUG -D mapreduce.reduce.log.level=DEBUG org.apache.giraph.examples.SimplePageRankComputation -w 1 -yh 1024 -yj ./giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0.2.0.6.0-76-jar-with-dependencies.jar -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -mc org.apache.giraph.examples.SimplePageRankComputation\$SimplePageRankMasterCompute -vip /user/rav/giraph/input/giraph.in -op /user/rav/giraph/output
> {noformat}
> After application successfully computer result it goes to infinite loop of:
> {noformat}
> 2013-11-28 20:09:51,764 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: setJobState: {"_stateKey":"FINISHED","_applicationAttemptKey":-1,"_superstepKey":-1} on superstep 31
> 2013-11-28 20:09:51,821 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanup: Notifying master its okay to cleanup with /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir/0_master
> 2013-11-28 20:09:51,829 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanUpZooKeeper: Node /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir already exists, no need to create.
> 2013-11-28 20:09:51,831 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanUpZooKeeper: Got 1 of 2 desired children from /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir
> 2013-11-28 20:09:51,831 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanedUpZooKeeper: Waiting for the children of /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir to change since only got 1 nodes.
> 2013-11-28 20:09:51,909 INFO [main-EventThread] org.apache.giraph.bsp.BspService: process: cleanedUpChildrenChanged signaled
> 2013-11-28 20:09:51,910 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanUpZooKeeper: Got 2 of 2 desired children from /_hadoopBsp/job_1385571160481_11413/_cleanedUpDir
> 2013-11-28 20:09:51,911 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanupZooKeeper: Removing the following path and all children - /_hadoopBsp/job_1385571160481_11413 from ZooKeeper list zkNode.net:2181
> 2013-11-28 20:09:52,329 INFO [main-EventThread] org.apache.giraph.bsp.BspService: process: masterElectionChildrenChanged signaled
> 2013-11-28 20:09:52,379 INFO [main-EventThread] org.apache.giraph.bsp.BspService: process: cleanedUpChildrenChanged signaled
> 2013-11-28 20:09:52,447 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.master.BspServiceMaster: cleanup: Removed HDFS checkpoint directory (_bsp/_checkpoints//job_1385571160481_11413) with return = false since the job Giraph: org.apache.giraph.examples.SimplePageRankComputation succeeded
> 2013-11-28 20:09:52,449 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.comm.netty.NettyClient: stop: reached wait threshold, 1 connections closed, releasing NettyClient.bootstrap resources now.
> 2013-11-28 20:09:52,454 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.comm.netty.NettyServer: stop: Halting netty server
> 2013-11-28 20:10:52,463 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:11:52,463 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:12:52,463 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:13:52,464 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:14:52,464 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:15:52,464 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:16:52,465 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:17:52,465 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:18:52,465 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:19:52,466 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:20:52,466 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:21:52,466 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> 2013-11-28 20:22:52,467 INFO [org.apache.giraph.master.MasterThread] org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for org.apache.giraph.utils.ProgressableUtils$ExecutorServiceWaitable@2f368c5d
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)