You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "ZhuGuanyin (JIRA)" <ji...@apache.org> on 2008/08/22 12:42:44 UTC
[jira] Created: (HADOOP-4001) TaskTracker's Memory resource should
be considered when tasktracker asks for new task
TaskTracker's Memory resource should be considered when tasktracker asks for new task
-------------------------------------------------------------------------------------
Key: HADOOP-4001
URL: https://issues.apache.org/jira/browse/HADOOP-4001
Project: Hadoop Core
Issue Type: New Feature
Components: mapred
Affects Versions: 0.17.2
Reporter: ZhuGuanyin
Priority: Minor
Fix For: 0.19.0
Currently, taskTracker only considers enough free disk space left when it asks for new task, memory resource should be considered too, or it may works badly in SMP environment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4001) TaskTracker's Memory resource
should be considered when tasktracker asks for new task
Posted by "ZhuGuanyin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624647#action_12624647 ]
ZhuGuanyin commented on HADOOP-4001:
------------------------------------
Currently, taskTracker only considers enough free disk space left when it asks for new task, memory resource should be considered too, or it may works badly in SMP environment.
Let's consider a scenario: one smp tasktracker with 8 cpu and 8 GB memory, and mapred.tasktracker.tasks.maximum=7. It has running 5 tasks while each task consumed 1.5GB memory, so it would asks for 2 new tasks which would make things go badly.
We should add free memory check just like
askForNewTask = enoughFreeMem(localMinMemStart); in tasktracker.java just like
askForNewTask = enoughFreeSpace(localMinSpaceStart);
I will attach a patch for this feature as soon as possible.
> TaskTracker's Memory resource should be considered when tasktracker asks for new task
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-4001
> URL: https://issues.apache.org/jira/browse/HADOOP-4001
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: ZhuGuanyin
> Priority: Minor
> Fix For: 0.19.0
>
>
> Currently, taskTracker only considers enough free disk space left when it asks for new task, memory resource should be considered too, or it may works badly in SMP environment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4001) TaskTracker's Memory resource should
be considered when tasktracker asks for new task
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-4001:
--------------------------------
Fix Version/s: (was: 0.19.0)
Please mark it as a blocker for 0.19 if required
> TaskTracker's Memory resource should be considered when tasktracker asks for new task
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-4001
> URL: https://issues.apache.org/jira/browse/HADOOP-4001
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: ZhuGuanyin
> Priority: Minor
>
> Currently, taskTracker only considers enough free disk space left when it asks for new task, memory resource should be considered too, or it may works badly in SMP environment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4001) TaskTracker's Memory resource
should be considered when tasktracker asks for new task
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624648#action_12624648 ]
Vinod Kumar Vavilapalli commented on HADOOP-4001:
-------------------------------------------------
Please see HADOOP-3759 and HADOOP-3581.
> TaskTracker's Memory resource should be considered when tasktracker asks for new task
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-4001
> URL: https://issues.apache.org/jira/browse/HADOOP-4001
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: ZhuGuanyin
> Priority: Minor
> Fix For: 0.19.0
>
>
> Currently, taskTracker only considers enough free disk space left when it asks for new task, memory resource should be considered too, or it may works badly in SMP environment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.