You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Michael Bieniosek (JIRA)" <ji...@apache.org> on 2007/07/19 19:31:06 UTC
[jira] Created: (HADOOP-1636) constant should be user-configurable:
MAX_COMPLETE_USER_JOBS_IN_MEMORY
constant should be user-configurable: MAX_COMPLETE_USER_JOBS_IN_MEMORY
----------------------------------------------------------------------
Key: HADOOP-1636
URL: https://issues.apache.org/jira/browse/HADOOP-1636
Project: Hadoop
Issue Type: Bug
Components: mapred
Affects Versions: 0.13.0
Reporter: Michael Bieniosek
In JobTracker.java: static final int MAX_COMPLETE_USER_JOBS_IN_MEMORY = 100;
This should be configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1636) constant should be
user-configurable: MAX_COMPLETE_USER_JOBS_IN_MEMORY
Posted by "Michael Bieniosek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12514076 ]
Michael Bieniosek edited comment on HADOOP-1636 at 7/19/07 7:02 PM:
--------------------------------------------------------------------
This patch creates a new configurable variable mapred.jobtracker.completeuserjobs.maximum, which defaults to 100 (the current hard-coded value).
When this many jobs are completed (failed or succeeded), hadoop deletes finished jobs from memory, making them accessible only through the information-poor jobhistory page. This limit is supposedly per user, but I submit all jobs as the same user. (this is the current behavior, which is unchanged by my patch)
I have tested this patch, and it seems to work.
was:
This patch creates a new configurable variable mapred.jobtracker.completeuserjobs.maximum, which defaults to 100 (the current hard-coded value).
When this many jobs are completed (failed or succeeded), hadoop deletes finished jobs from memory, making them accessible only through the information-poor jobhistory page. This limit is supposedly per user, but I submit all jobs as the same user.
I have tested this patch, and it seems to work.
> constant should be user-configurable: MAX_COMPLETE_USER_JOBS_IN_MEMORY
> ----------------------------------------------------------------------
>
> Key: HADOOP-1636
> URL: https://issues.apache.org/jira/browse/HADOOP-1636
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.13.0
> Reporter: Michael Bieniosek
> Attachments: configure-max-completed-jobs.patch
>
>
> In JobTracker.java: static final int MAX_COMPLETE_USER_JOBS_IN_MEMORY = 100;
> This should be configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1636) constant should be user-configurable:
MAX_COMPLETE_USER_JOBS_IN_MEMORY
Posted by "Michael Bieniosek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Bieniosek updated HADOOP-1636:
--------------------------------------
Attachment: configure-max-completed-jobs.patch
This patch creates a new configurable variable mapred.jobtracker.completeuserjobs.maximum, which defaults to 100 (the current hard-coded value).
When this many jobs are completed (failed or succeeded), hadoop deletes finished jobs from memory, making them accessible only through the information-poor jobhistory page. This limit is supposedly per user, but I submit all jobs as the same user.
I have tested this patch, and it seems to work.
> constant should be user-configurable: MAX_COMPLETE_USER_JOBS_IN_MEMORY
> ----------------------------------------------------------------------
>
> Key: HADOOP-1636
> URL: https://issues.apache.org/jira/browse/HADOOP-1636
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.13.0
> Reporter: Michael Bieniosek
> Attachments: configure-max-completed-jobs.patch
>
>
> In JobTracker.java: static final int MAX_COMPLETE_USER_JOBS_IN_MEMORY = 100;
> This should be configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1636) constant should be
user-configurable: MAX_COMPLETE_USER_JOBS_IN_MEMORY
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12514096 ]
Hadoop QA commented on HADOOP-1636:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12362190/configure-max-completed-jobs.patch applied and successfully tested against trunk revision r557790.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/440/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/440/console
> constant should be user-configurable: MAX_COMPLETE_USER_JOBS_IN_MEMORY
> ----------------------------------------------------------------------
>
> Key: HADOOP-1636
> URL: https://issues.apache.org/jira/browse/HADOOP-1636
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.13.0
> Reporter: Michael Bieniosek
> Attachments: configure-max-completed-jobs.patch
>
>
> In JobTracker.java: static final int MAX_COMPLETE_USER_JOBS_IN_MEMORY = 100;
> This should be configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1636) constant should be user-configurable:
MAX_COMPLETE_USER_JOBS_IN_MEMORY
Posted by "Michael Bieniosek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Bieniosek updated HADOOP-1636:
--------------------------------------
Status: Patch Available (was: Open)
> constant should be user-configurable: MAX_COMPLETE_USER_JOBS_IN_MEMORY
> ----------------------------------------------------------------------
>
> Key: HADOOP-1636
> URL: https://issues.apache.org/jira/browse/HADOOP-1636
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.13.0
> Reporter: Michael Bieniosek
>
> In JobTracker.java: static final int MAX_COMPLETE_USER_JOBS_IN_MEMORY = 100;
> This should be configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1636) constant should be user-configurable:
MAX_COMPLETE_USER_JOBS_IN_MEMORY
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1636:
----------------------------------
Resolution: Fixed
Fix Version/s: 0.15.0
Assignee: Michael Bieniosek
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Michael!
> constant should be user-configurable: MAX_COMPLETE_USER_JOBS_IN_MEMORY
> ----------------------------------------------------------------------
>
> Key: HADOOP-1636
> URL: https://issues.apache.org/jira/browse/HADOOP-1636
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.13.0
> Reporter: Michael Bieniosek
> Assignee: Michael Bieniosek
> Fix For: 0.15.0
>
> Attachments: configure-max-completed-jobs.patch
>
>
> In JobTracker.java: static final int MAX_COMPLETE_USER_JOBS_IN_MEMORY = 100;
> This should be configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.