[jira] [Commented] (HAMA-970) Exception can occur if the size of splits is bigger than numBSPTasks

["Behroz Sikander (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HAMA-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045728#comment-15045728 ] 

Behroz Sikander commented on HAMA-970:
--------------------------------------

I am trying to solve a problem whose solution is mentioned above but I cannot get it to work. In my Hama job, I have a small file (~70MB) that I want to divide among all of my tasks except the Master task. For example, If I have 4 tasks, then task-0 will always be selected as a Master and remaining tasks will be slaves. In this example, Master task should not get any chunk and the 70 MB file should be divided only among the 3 slaves tasks. 

How can I solve this problem ? 

Previously in my Input Paths, I was adding 2 files, one empty file and one 70 MB file. This is working but Hama only opens up 2 tasks, one for empty file (which becomes the master) and one for 70 MB file (which becomes my only slave). Now, since I want to divide the 70 MB file into 4-5 tasks if I try to do this solution, I get an exception.



> Exception can occur if the size of splits is bigger than numBSPTasks
> --------------------------------------------------------------------
>
>                 Key: HAMA-970
>                 URL: https://issues.apache.org/jira/browse/HAMA-970
>             Project: Hama
>          Issue Type: Bug
>          Components: bsp core
>    Affects Versions: 0.7.0
>            Reporter: JongYoon Lim
>            Priority: Trivial
>         Attachments: HAMA-970.patch
>
>
> In JonInProgress, it's possble to get Exception in initTasks(). 
> {code:java}
> this.tasks = new TaskInProgress[numBSPTasks];
> for (int i = 0; i < splits.length; i++) {
>   tasks[i] = new TaskInProgress(getJobID(), this.jobFile.toString(), splits[i], this.conf, this, i);
> }
> {code}
> I'm not sure that *numBSPTask* is always bigger than *splits.length*. 
> So, I think it's better to use bigger value to assign the *tasks* array. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)