You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Alex Kozlov (JIRA)" <ji...@apache.org> on 2010/05/07 01:23:48 UTC
[jira] Created: (MAPREDUCE-1763) JobHistory should enable history
collection after a timeout or some other event
JobHistory should enable history collection after a timeout or some other event
-------------------------------------------------------------------------------
Key: MAPREDUCE-1763
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1763
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobtracker
Affects Versions: 0.20.1
Reporter: Alex Kozlov
Priority: Critical
If you search for disableHistory in JobHistory.java, one can discover that it is enabled only at the initialization time. There are two instances where job history can be disabled:
* if it fails to initialize the the output directories
* If it fails to create a single job history file
There are a few problems with that. One is that there is no way to revert the flag even if the original problem goes away. Second, these cases should probably be handled separately. The result of which is that once the job history file creation fails, the job history mechanism becomes disabled and there is no way to switch it back.
One simple solution is to have a timeout after which we can try to enable the job history collection. Another is to have a more granular job history control per job.
Alex K
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAPREDUCE-1763) JobHistory should enable history
collection after a timeout or some other event
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu resolved MAPREDUCE-1763.
------------------------------------------------
Resolution: Duplicate
Will not be an issue after MAPREDUCE-1699
> JobHistory should enable history collection after a timeout or some other event
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1763
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1763
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Alex Kozlov
> Priority: Critical
>
> If you search for disableHistory in JobHistory.java, one can discover that it is enabled only at the initialization time. There are two instances where job history can be disabled:
> * if it fails to initialize the the output directories
> * If it fails to create a single job history file
> There are a few problems with that. One is that there is no way to revert the flag even if the original problem goes away. Second, these cases should probably be handled separately. The result of which is that once the job history file creation fails, the job history mechanism becomes disabled and there is no way to switch it back.
> One simple solution is to have a timeout after which we can try to enable the job history collection. Another is to have a more granular job history control per job.
> Alex K
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.