You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon (JIRA)" <ji...@apache.org> on 2014/02/03 02:44:09 UTC
[jira] [Commented] (HAMA-862) Error partitioning the input path
[ https://issues.apache.org/jira/browse/HAMA-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13889157#comment-13889157 ]
Edward J. Yoon commented on HAMA-862:
-------------------------------------
If configured num of tasks exceeds max tasks capacity, Hama will run using available resources.
{code}
edward@edward-VirtualBox:~/workspace/hama-trunk$ bin/hama jar examples/target/hama-examples-0.7.0-SNAPSHOT.jar pi 20
14/02/03 10:37:27 WARN bsp.BSPJobClient: The configured number of tasks has exceeded the maximum allowed. Job will run with 3 tasks.
14/02/03 10:37:27 INFO bsp.BSPJobClient: Running job: job_201402031036_0002
14/02/03 10:37:30 INFO bsp.BSPJobClient: Current supersteps number: 0
{code}
Hama will throw the execption message, only when the num of splits (blocks) exceeds max tasks capacity.
> Error partitioning the input path
> ---------------------------------
>
> Key: HAMA-862
> URL: https://issues.apache.org/jira/browse/HAMA-862
> Project: Hama
> Issue Type: Bug
> Components: bsp core
> Affects Versions: 0.6.3
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.7.0
>
> Attachments: clean.patch
>
>
> It seems error occurs when block files num > desired tasks num.
> {code}
> edward@edward-VirtualBox:~/workspace/hama-0.6.3$ bin/hama jar examples/target/hama-examples-0.6.3.jar gen fastgen 7000000 3 trunk/7000000-3 3
> ..
> edward@edward-VirtualBox:~/workspace/hama-trunk$ ~/hadoop-1.2.1/bin/hadoop dfs -ls /user/edward/trunk/7000000-3
> Found 3 items
> -rw-r--r-- 3 edward supergroup 101439512 2014-01-27 17:06 /user/edward/trunk/7000000-3/part-00000
> -rw-r--r-- 3 edward supergroup 102565269 2014-01-27 17:06 /user/edward/trunk/7000000-3/part-00001
> -rw-r--r-- 3 edward supergroup 102561984 2014-01-27 17:06 /user/edward/trunk/7000000-3/part-00002
> edward@edward-VirtualBox:~/workspace/hama-0.6.3$ bin/hama jar examples/target/hama-examples-0.6.3.jar pagerank trunk/7000000-3 /tmp/result 3
> 14/01/27 17:07:43 INFO bsp.FileInputFormat: Total input paths to process : 3
> 14/01/27 17:07:43 INFO bsp.FileInputFormat: Total input paths to process : 3
> 14/01/27 17:07:43 INFO bsp.BSPJobClient: Running job: job_201401271704_0007
> 14/01/27 17:07:43 INFO bsp.BSPJobClient: Job failed.
> 14/01/27 17:07:43 ERROR bsp.BSPJobClient: Error partitioning the input path.
> java.io.IOException: Runtime partition failed for the job.
> at org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465)
> at org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333)
> at org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293)
> at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:228)
> at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203)
> at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:235)
> at org.apache.hama.examples.PageRank.main(PageRank.java:164)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hama.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> at org.apache.hama.util.ProgramDriver.driver(ProgramDriver.java:139)
> at org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:45)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hama.util.RunJar.main(RunJar.java:146)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)