You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya R (JIRA)" <ji...@apache.org> on 2009/07/02 08:00:48 UTC
[jira] Created: (MAPREDUCE-693) Job files not moved to "done"
subdirectory after JT restart
Job files not moved to "done" subdirectory after JT restart
-----------------------------------------------------------
Key: MAPREDUCE-693
URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.20.1
Reporter: Ramya R
Priority: Minor
Fix For: 0.20.1
After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the job files fail to be moved to the "done" subdirectory.
The exact scenario to reproduce this issue is:
* Submit a job
* Restart JT before anything is written to the job files
* Kill the job
* The old job files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-693) Conf files not moved to "done"
subdirectory after JT restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated MAPREDUCE-693:
---------------------------------
Attachment: (was: MAPREDUCE-693-v1.2-branch-0.20.patch)
> Conf files not moved to "done" subdirectory after JT restart
> ------------------------------------------------------------
>
> Key: MAPREDUCE-693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: MAPREDUCE-693-v1.1-branch-0.20.patch
>
>
> After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
> The exact scenario to reproduce this issue is:
> * Submit a job
> * Restart JT before anything is written to the job files
> * Kill the job
> * The old conf files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-693) Conf files not moved to "done"
subdirectory after JT restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated MAPREDUCE-693:
---------------------------------
Attachment: MAPREDUCE-693-v1.2-branch-0.20.patch
> Conf files not moved to "done" subdirectory after JT restart
> ------------------------------------------------------------
>
> Key: MAPREDUCE-693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: MAPREDUCE-693-v1.1-branch-0.20.patch, MAPREDUCE-693-v1.2-branch-0.20.patch
>
>
> After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
> The exact scenario to reproduce this issue is:
> * Submit a job
> * Restart JT before anything is written to the job files
> * Kill the job
> * The old conf files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-693) Conf files not moved to "done"
subdirectory after JT restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated MAPREDUCE-693:
---------------------------------
Attachment: MAPREDUCE-693-v1.2-branch-0.20.patch
Oops. Uploaded the wrong patch. Here is the patch for branch 0.20 not to be committed.
> Conf files not moved to "done" subdirectory after JT restart
> ------------------------------------------------------------
>
> Key: MAPREDUCE-693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: MAPREDUCE-693-v1.1-branch-0.20.patch
>
>
> After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
> The exact scenario to reproduce this issue is:
> * Submit a job
> * Restart JT before anything is written to the job files
> * Kill the job
> * The old conf files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-693) Conf files not moved to "done"
subdirectory after JT restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728247#action_12728247 ]
Amar Kamat commented on MAPREDUCE-693:
--------------------------------------
Note that this is a feature and MAPREDUCE-11 should fix this issue in the right way. Thoughts?
> Conf files not moved to "done" subdirectory after JT restart
> ------------------------------------------------------------
>
> Key: MAPREDUCE-693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: MAPREDUCE-693-v1.1-branch-0.20.patch
>
>
> After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
> The exact scenario to reproduce this issue is:
> * Submit a job
> * Restart JT before anything is written to the job files
> * Kill the job
> * The old conf files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAPREDUCE-693) Conf files not moved to "done"
subdirectory after JT restart
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley resolved MAPREDUCE-693.
-------------------------------------
Resolution: Cannot Reproduce
Fix Version/s: (was: 0.20.1)
> Conf files not moved to "done" subdirectory after JT restart
> ------------------------------------------------------------
>
> Key: MAPREDUCE-693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Priority: Minor
> Attachments: MAPREDUCE-693-v1.1-branch-0.20.patch, MAPREDUCE-693-v1.2-branch-0.20.patch
>
>
> After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
> The exact scenario to reproduce this issue is:
> * Submit a job
> * Restart JT before anything is written to the job files
> * Kill the job
> * The old conf files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-693) Conf files not moved to "done"
subdirectory after JT restart
Posted by "Ramya R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ramya R updated MAPREDUCE-693:
------------------------------
Description:
After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
The exact scenario to reproduce this issue is:
* Submit a job
* Restart JT before anything is written to the job files
* Kill the job
* The old conf files remain in the history folder and fail to be moved to "done" subdirectory
was:
After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the job files fail to be moved to the "done" subdirectory.
The exact scenario to reproduce this issue is:
* Submit a job
* Restart JT before anything is written to the job files
* Kill the job
* The old job files remain in the history folder and fail to be moved to "done" subdirectory
Summary: Conf files not moved to "done" subdirectory after JT restart (was: Job files not moved to "done" subdirectory after JT restart)
> Conf files not moved to "done" subdirectory after JT restart
> ------------------------------------------------------------
>
> Key: MAPREDUCE-693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.20.1
>
>
> After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
> The exact scenario to reproduce this issue is:
> * Submit a job
> * Restart JT before anything is written to the job files
> * Kill the job
> * The old conf files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-693) Conf files not moved to "done"
subdirectory after JT restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726351#action_12726351 ]
Amar Kamat commented on MAPREDUCE-693:
--------------------------------------
bq. The old conf files remain in the history folder and fail to be moved to "done" subdirectory
There is no need to move the conf file to the done folder. In this case the job is run as a new job and hence a new conf file is created for this job. The jobhistory file gets deleted as it is required for recovery (checkpoint process). The conf file is doesnt play any role in the recovery process. Here is what is happening
# jobtracker starts with id _id1_
# job job1 is submitted and creates history file hostname_id1_job1_user_jobname and conf file as hostname_id1_job1_conf.xml
# jobtracker restart with id _id2_
# jobtracker tries to recover the job. There are 2 possibilities here :
## If the job-initialization thread inits the job before the recovery-manager picks up the job for recovery then the new filename would be hostname_id1_job1_user_jobname.recover and the conf file would be hostname_id1_job1_conf.xml. In such a case there wont be any garbage left in the history folder.
## If the recovery-manager picks up the job first before the init-thread then it will assume that there is nothing to recover and will delete hostname_id1_job1_user_jobname (leaving hostname_id1_job1_conf.xml). When the job inits, it will take a new filename i.e hostname_id2_job1_user_jobname and hostname_id2_job1_conf.xml. Only in this case the conf file ( hostname_id1_job1_conf.xml) is left behind in the history folder.
AFAIK this is a timing issue. I think a proper fix for all this corner cases is MAPREDUCE-11. Thoughts?
> Conf files not moved to "done" subdirectory after JT restart
> ------------------------------------------------------------
>
> Key: MAPREDUCE-693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.20.1
>
>
> After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
> The exact scenario to reproduce this issue is:
> * Submit a job
> * Restart JT before anything is written to the job files
> * Kill the job
> * The old conf files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-693) Conf files not moved to "done"
subdirectory after JT restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated MAPREDUCE-693:
---------------------------------
Attachment: MAPREDUCE-693-v1.1-branch-0.20.patch
Attaching a fix for branch 0.20 not to be committed.
> Conf files not moved to "done" subdirectory after JT restart
> ------------------------------------------------------------
>
> Key: MAPREDUCE-693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: MAPREDUCE-693-v1.1-branch-0.20.patch
>
>
> After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
> The exact scenario to reproduce this issue is:
> * Submit a job
> * Restart JT before anything is written to the job files
> * Kill the job
> * The old conf files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-693) Conf files not moved to "done"
subdirectory after JT restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728493#action_12728493 ]
Amar Kamat commented on MAPREDUCE-693:
--------------------------------------
Used this patch and the branch 0.20 patch for MAPREDUCE-683 and now TestJobTrackerRestart passes.
> Conf files not moved to "done" subdirectory after JT restart
> ------------------------------------------------------------
>
> Key: MAPREDUCE-693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: MAPREDUCE-693-v1.1-branch-0.20.patch, MAPREDUCE-693-v1.2-branch-0.20.patch
>
>
> After MAPREDUCE-516, when a job is submitted and the JT is restarted (before job files have been written) and the job is killed after recovery, the conf files fail to be moved to the "done" subdirectory.
> The exact scenario to reproduce this issue is:
> * Submit a job
> * Restart JT before anything is written to the job files
> * Kill the job
> * The old conf files remain in the history folder and fail to be moved to "done" subdirectory
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.