You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2006/12/06 20:44:21 UTC
[jira] Created: (HADOOP-787) job tracker should limit the number of
completed/failed jobs in RAM
job tracker should limit the number of completed/failed jobs in RAM
-------------------------------------------------------------------
Key: HADOOP-787
URL: http://issues.apache.org/jira/browse/HADOOP-787
Project: Hadoop
Issue Type: Bug
Components: mapred
Affects Versions: 0.9.0
Reporter: Owen O'Malley
Assigned To: Arun C Murthy
Fix For: 0.10.0
The job tracker currently runs out of memory when a user runs a lot of small jobs back to back. To bound the amount of RAM that the job tracker consumes recording completed/failed jobs, I think that the JobTracker should limit the number of "remembered" jobs per a user to jobtracker.max.remembered.task.per.user (default 100).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-787) job tracker should limit the number
of completed/failed jobs in RAM
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-787?page=comments#action_12456206 ]
Owen O'Malley commented on HADOOP-787:
--------------------------------------
They are already saved persistently in the job history logs, which are restored on demand.
> job tracker should limit the number of completed/failed jobs in RAM
> -------------------------------------------------------------------
>
> Key: HADOOP-787
> URL: http://issues.apache.org/jira/browse/HADOOP-787
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.0
> Reporter: Owen O'Malley
> Assigned To: Arun C Murthy
> Fix For: 0.10.0
>
>
> The job tracker currently runs out of memory when a user runs a lot of small jobs back to back. To bound the amount of RAM that the job tracker consumes recording completed/failed jobs, I think that the JobTracker should limit the number of "remembered" jobs per a user to jobtracker.max.remembered.task.per.user (default 100).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HADOOP-787) job tracker should limit the number
of completed/failed jobs in RAM
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy resolved HADOOP-787.
----------------------------------
Resolution: Fixed
Fixed as a part of HADOOP-815.
> job tracker should limit the number of completed/failed jobs in RAM
> -------------------------------------------------------------------
>
> Key: HADOOP-787
> URL: https://issues.apache.org/jira/browse/HADOOP-787
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.0
> Reporter: Owen O'Malley
> Assigned To: Arun C Murthy
>
> The job tracker currently runs out of memory when a user runs a lot of small jobs back to back. To bound the amount of RAM that the job tracker consumes recording completed/failed jobs, I think that the JobTracker should limit the number of "remembered" jobs per a user to jobtracker.max.remembered.task.per.user (default 100).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-787) job tracker should limit the number
of completed/failed jobs in RAM
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-787?page=comments#action_12456199 ]
Runping Qi commented on HADOOP-787:
-----------------------------------
I think the jobtracker should save the old jobs to DFS and read them back when they are needed.
> job tracker should limit the number of completed/failed jobs in RAM
> -------------------------------------------------------------------
>
> Key: HADOOP-787
> URL: http://issues.apache.org/jira/browse/HADOOP-787
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.0
> Reporter: Owen O'Malley
> Assigned To: Arun C Murthy
> Fix For: 0.10.0
>
>
> The job tracker currently runs out of memory when a user runs a lot of small jobs back to back. To bound the amount of RAM that the job tracker consumes recording completed/failed jobs, I think that the JobTracker should limit the number of "remembered" jobs per a user to jobtracker.max.remembered.task.per.user (default 100).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira