You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hama.apache.org by Apache Wiki <wi...@apache.org> on 2012/08/30 15:28:26 UTC
[Hama Wiki] Update of "WriteHamaGraphFile" by thomasjungblut
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.
The "WriteHamaGraphFile" page has been changed by thomasjungblut:
http://wiki.apache.org/hama/WriteHamaGraphFile?action=diff&rev1=6&rev2=7
}}}
to your Hama cluster.
- '''Troubleshooting'''
-
- If your job does not execute, your cluster may not have enough resources (task slots).
- Symptoms may look like this in the bsp master log:
- {{{
- 2012-05-27 20:00:51,228 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
- 2012-05-27 20:00:51,288 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 16
- 2012-05-27 20:00:51,305 INFO org.apache.hama.bsp.JobInProgress: Job is initialized.
- 2012-05-27 20:00:51,313 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Scheduling of job Pagerank could not be done successfully. Killing it!
- 2012-05-27 20:01:08,334 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 16
- 2012-05-27 20:01:08,339 INFO org.apache.hama.bsp.JobInProgress: Job is initialized.
- 2012-05-27 20:01:08,340 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Scheduling of job Pagerank could not be done successfully. Killing it!
- }}}
-
- This was run on a 8 slot cluster, but it required 16 slots because of 64m chunk size of HDFS.
- Either you can reupload the file with higher chunksize so the slots match the blocks or you can increase the slots in your Hama cluster.
-
If you sort the result descending by pagerank you can see the following top 10 sites:
{{{
@@ -250, +233 @@
Note that you can decode the indices you may see with the [[http://users.on.net/~henry/pagerank/titles-sorted.zip|page titels]] files.
+ '''Troubleshooting'''
+
+ If your job does not execute, your cluster may not have enough resources (task slots).
+ Symptoms may look like this in the bsp master log:
+ {{{
+ 2012-05-27 20:00:51,228 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
+ 2012-05-27 20:00:51,288 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 16
+ 2012-05-27 20:00:51,305 INFO org.apache.hama.bsp.JobInProgress: Job is initialized.
+ 2012-05-27 20:00:51,313 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Scheduling of job Pagerank could not be done successfully. Killing it!
+ 2012-05-27 20:01:08,334 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 16
+ 2012-05-27 20:01:08,339 INFO org.apache.hama.bsp.JobInProgress: Job is initialized.
+ 2012-05-27 20:01:08,340 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Scheduling of job Pagerank could not be done successfully. Killing it!
+ }}}
+
+ This was run on a 8 slot cluster, but it required 16 slots because of 64m chunk size of HDFS.
+ Either you can reupload the file with higher chunksize so the slots match the blocks or you can increase the slots in your Hama cluster.
+