You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Avery Ching (JIRA)" <ji...@apache.org> on 2012/09/28 22:58:09 UTC
[jira] [Created] (GIRAPH-351) Fail job early when there is no input
Avery Ching created GIRAPH-351:
----------------------------------
Summary: Fail job early when there is no input
Key: GIRAPH-351
URL: https://issues.apache.org/jira/browse/GIRAPH-351
Project: Giraph
Issue Type: Improvement
Reporter: Avery Ching
Right now we get a cryptic error message when we know that there is no input!
2012-09-28 13:45:20,410 INFO org.apache.giraph.graph.BspServiceMaster: generateInputSplits: Got 0 input splits for 100 workers
2012-09-28 13:45:20,410 WARN org.apache.giraph.graph.BspServiceMaster: createInputSplits: Number of inputSplits=0 < 100=number of healthy processes, some workers will be not used
But the job keeps on going and the workers get the exception
java.lang.IllegalStateException: setup: loadVertices failed due to KeeperException
at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:628)
at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:370)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:570)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:632)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:171)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /_hadoopBsp/job_201209271814.325_0001/_inputSplitDir
at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
at org.apache.giraph.zk.ZooKeeperExt.getChildrenExt(ZooKeeperExt.java:273)
at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:225)
at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:320)
at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:607)
... 5 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-351) Fail job early when there is no
input
Posted by "Alessandro Presta (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468989#comment-13468989 ]
Alessandro Presta commented on GIRAPH-351:
------------------------------------------
+1, this has bugged me for a while too :)
> Fail job early when there is no input
> -------------------------------------
>
> Key: GIRAPH-351
> URL: https://issues.apache.org/jira/browse/GIRAPH-351
> Project: Giraph
> Issue Type: Improvement
> Reporter: Avery Ching
> Assignee: Avery Ching
> Attachments: GIRAPH-351.2.patch, GIRAPH-351.patch
>
>
> Right now we get a cryptic error message when we know that there is no input!
> 2012-09-28 13:45:20,410 INFO org.apache.giraph.graph.BspServiceMaster: generateInputSplits: Got 0 input splits for 100 workers
> 2012-09-28 13:45:20,410 WARN org.apache.giraph.graph.BspServiceMaster: createInputSplits: Number of inputSplits=0 < 100=number of healthy processes, some workers will be not used
> But the job keeps on going and the workers get the exception
> java.lang.IllegalStateException: setup: loadVertices failed due to KeeperException
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:628)
> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:370)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:570)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:632)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:171)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /_hadoopBsp/job_201209271814.325_0001/_inputSplitDir
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> at org.apache.giraph.zk.ZooKeeperExt.getChildrenExt(ZooKeeperExt.java:273)
> at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:225)
> at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:320)
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:607)
> ... 5 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-351) Fail job early when there is no
input
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468994#comment-13468994 ]
Hudson commented on GIRAPH-351:
-------------------------------
Integrated in Giraph-trunk-Commit #222 (See [https://builds.apache.org/job/Giraph-trunk-Commit/222/])
GIRAPH-351: Fail job early when there is no input (aching via ereisman) (Revision 1393836)
Result = FAILURE
ereisman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393836
Files :
* /giraph/trunk/CHANGELOG
* /giraph/trunk/src/main/java/org/apache/giraph/graph/BspServiceMaster.java
> Fail job early when there is no input
> -------------------------------------
>
> Key: GIRAPH-351
> URL: https://issues.apache.org/jira/browse/GIRAPH-351
> Project: Giraph
> Issue Type: Improvement
> Reporter: Avery Ching
> Assignee: Avery Ching
> Fix For: 0.2.0
>
> Attachments: GIRAPH-351.2.patch, GIRAPH-351.patch
>
>
> Right now we get a cryptic error message when we know that there is no input!
> 2012-09-28 13:45:20,410 INFO org.apache.giraph.graph.BspServiceMaster: generateInputSplits: Got 0 input splits for 100 workers
> 2012-09-28 13:45:20,410 WARN org.apache.giraph.graph.BspServiceMaster: createInputSplits: Number of inputSplits=0 < 100=number of healthy processes, some workers will be not used
> But the job keeps on going and the workers get the exception
> java.lang.IllegalStateException: setup: loadVertices failed due to KeeperException
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:628)
> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:370)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:570)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:632)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:171)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /_hadoopBsp/job_201209271814.325_0001/_inputSplitDir
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> at org.apache.giraph.zk.ZooKeeperExt.getChildrenExt(ZooKeeperExt.java:273)
> at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:225)
> at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:320)
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:607)
> ... 5 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-351) Fail job early when there is no input
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching updated GIRAPH-351:
-------------------------------
Attachment: GIRAPH-351.patch
Here is a fix.
2012-09-28 14:11:10,686 INFO org.apache.giraph.graph.BspServiceMaster: generateInputSplits: Got 0 input splits for 100 workers
2012-09-28 14:11:10,686 WARN org.apache.giraph.graph.BspServiceMaster: createInputSplits: Number of inputSplits=0 < 100=number of healthy processes, some workers will be not used
2012-09-28 14:11:10,686 FATAL org.apache.giraph.graph.BspServiceMaster: createInputSplits: Failing job due to 0 input splits, check input {classname}!
2012-09-28 14:11:10,686 FATAL org.apache.giraph.graph.BspServiceMaster: failJob: Killing job job_201209271814.337_0001
> Fail job early when there is no input
> -------------------------------------
>
> Key: GIRAPH-351
> URL: https://issues.apache.org/jira/browse/GIRAPH-351
> Project: Giraph
> Issue Type: Improvement
> Reporter: Avery Ching
> Attachments: GIRAPH-351.patch
>
>
> Right now we get a cryptic error message when we know that there is no input!
> 2012-09-28 13:45:20,410 INFO org.apache.giraph.graph.BspServiceMaster: generateInputSplits: Got 0 input splits for 100 workers
> 2012-09-28 13:45:20,410 WARN org.apache.giraph.graph.BspServiceMaster: createInputSplits: Number of inputSplits=0 < 100=number of healthy processes, some workers will be not used
> But the job keeps on going and the workers get the exception
> java.lang.IllegalStateException: setup: loadVertices failed due to KeeperException
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:628)
> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:370)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:570)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:632)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:171)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /_hadoopBsp/job_201209271814.325_0001/_inputSplitDir
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> at org.apache.giraph.zk.ZooKeeperExt.getChildrenExt(ZooKeeperExt.java:273)
> at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:225)
> at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:320)
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:607)
> ... 5 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-351) Fail job early when there is no
input
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467240#comment-13467240 ]
Eli Reisman commented on GIRAPH-351:
------------------------------------
Did this go stale or am I an idiot? Both? Here's what happened:
{code}
~/apache/commit/trunk$ patch -p0 < ../GIRAPH-351.patch
patching file src/main/java/org/apache/giraph/graph/BspServiceMaster.java
Hunk #1 FAILED at 531.
1 out of 1 hunk FAILED -- saving rejects to file src/main/java/org/apache/giraph/graph/BspServiceMaster.java.rej
{code}
> Fail job early when there is no input
> -------------------------------------
>
> Key: GIRAPH-351
> URL: https://issues.apache.org/jira/browse/GIRAPH-351
> Project: Giraph
> Issue Type: Improvement
> Reporter: Avery Ching
> Assignee: Avery Ching
> Attachments: GIRAPH-351.patch
>
>
> Right now we get a cryptic error message when we know that there is no input!
> 2012-09-28 13:45:20,410 INFO org.apache.giraph.graph.BspServiceMaster: generateInputSplits: Got 0 input splits for 100 workers
> 2012-09-28 13:45:20,410 WARN org.apache.giraph.graph.BspServiceMaster: createInputSplits: Number of inputSplits=0 < 100=number of healthy processes, some workers will be not used
> But the job keeps on going and the workers get the exception
> java.lang.IllegalStateException: setup: loadVertices failed due to KeeperException
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:628)
> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:370)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:570)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:632)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:171)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /_hadoopBsp/job_201209271814.325_0001/_inputSplitDir
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> at org.apache.giraph.zk.ZooKeeperExt.getChildrenExt(ZooKeeperExt.java:273)
> at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:225)
> at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:320)
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:607)
> ... 5 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-351) Fail job early when there is no input
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching updated GIRAPH-351:
-------------------------------
Attachment: GIRAPH-351.2.patch
No, you are right, this just went stale. =) Here's a rebase.
> Fail job early when there is no input
> -------------------------------------
>
> Key: GIRAPH-351
> URL: https://issues.apache.org/jira/browse/GIRAPH-351
> Project: Giraph
> Issue Type: Improvement
> Reporter: Avery Ching
> Assignee: Avery Ching
> Attachments: GIRAPH-351.2.patch, GIRAPH-351.patch
>
>
> Right now we get a cryptic error message when we know that there is no input!
> 2012-09-28 13:45:20,410 INFO org.apache.giraph.graph.BspServiceMaster: generateInputSplits: Got 0 input splits for 100 workers
> 2012-09-28 13:45:20,410 WARN org.apache.giraph.graph.BspServiceMaster: createInputSplits: Number of inputSplits=0 < 100=number of healthy processes, some workers will be not used
> But the job keeps on going and the workers get the exception
> java.lang.IllegalStateException: setup: loadVertices failed due to KeeperException
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:628)
> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:370)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:570)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:632)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:171)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /_hadoopBsp/job_201209271814.325_0001/_inputSplitDir
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> at org.apache.giraph.zk.ZooKeeperExt.getChildrenExt(ZooKeeperExt.java:273)
> at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:225)
> at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:320)
> at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:607)
> ... 5 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira