You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Karam Singh (JIRA)" <ji...@apache.org> on 2010/02/26 09:50:27 UTC
[jira] Created: (MAPREDUCE-1540) Sometime JobTracker holds stale
refrence of JobInProgress.
Sometime JobTracker holds stale refrence of JobInProgress.
----------------------------------------------------------
Key: MAPREDUCE-1540
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1540
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.20.2
Reporter: Karam Singh
Ran random writer, sort and sort validate job. Checked the jmap -histo:live and verified that there is no reference of JobInProgress after Jobs are retired
Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed all the job when jobs got retired. again checked jmap -histo:live for JobInProgress for JT process found 2 references were there.
Found this while doing snaity testing of 1316
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-1540) Sometimes JobTracker holds stale
refrence of JobInProgress even after Job gets retired
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat reassigned MAPREDUCE-1540:
-------------------------------------
Assignee: Amar Kamat
> Sometimes JobTracker holds stale refrence of JobInProgress even after Job gets retired
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1540
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1540
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2
> Reporter: Karam Singh
> Assignee: Amar Kamat
>
> Ran random writer, sort and sort validate job. Checked the jmap -histo:live and verified that there is no reference of JobInProgress after Jobs are retired
> Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed all the job when jobs got retired. again checked jmap -histo:live for JobInProgress for JT process found 2 references were there.
> Found this while doing sanity testing of 1316
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAPREDUCE-1540) Sometimes JobTracker holds stale
refrence of JobInProgress even after Job gets retired
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat resolved MAPREDUCE-1540.
-----------------------------------
Resolution: Cannot Reproduce
This doesnt occur in 0.20, 0.21 and trunk.
> Sometimes JobTracker holds stale refrence of JobInProgress even after Job gets retired
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1540
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1540
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2
> Reporter: Karam Singh
> Assignee: Amar Kamat
>
> Ran random writer, sort and sort validate job. Checked the jmap -histo:live and verified that there is no reference of JobInProgress after Jobs are retired
> Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed all the job when jobs got retired. again checked jmap -histo:live for JobInProgress for JT process found 2 references were there.
> Found this while doing sanity testing of 1316
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1540) Sometimes JobTracker holds stale
refrence of JobInProgress even after Job gets retired
Posted by "Karam Singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karam Singh updated MAPREDUCE-1540:
-----------------------------------
Summary: Sometimes JobTracker holds stale refrence of JobInProgress even after Job gets retired (was: Sometime JobTracker holds stale refrence of JobInProgress.)
> Sometimes JobTracker holds stale refrence of JobInProgress even after Job gets retired
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1540
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1540
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2
> Reporter: Karam Singh
>
> Ran random writer, sort and sort validate job. Checked the jmap -histo:live and verified that there is no reference of JobInProgress after Jobs are retired
> Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed all the job when jobs got retired. again checked jmap -histo:live for JobInProgress for JT process found 2 references were there.
> Found this while doing snaity testing of 1316
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1540) Sometimes JobTracker holds stale
refrence of JobInProgress even after Job gets retired
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838774#action_12838774 ]
Amar Kamat commented on MAPREDUCE-1540:
---------------------------------------
What actually happened was that these jobs failed during submission itself. As a result, _job.fail()_ was invoked to cleanup the job files on the local-fs, which inturn invokes JobTracker.finalizeJob(). JobTracker.finalizeJob() goes ahead and adds them to _userToJobsMap_. Note that the JIP is not present in the _jobs_ structure of the JobTracker but somehow found its way in _userToJobsMap_. Later the _RetireJobs_ thread runs and tries to claim these jobs. But the condition, _minConditionToRetire()_, waits until the history file gets copied, which in this case never happens as the history file was never created. As a result of this, the jobs will be held up in the memory forever. Note that these jobs are not initialized and hence wont affect JobTracker's memory.
> Sometimes JobTracker holds stale refrence of JobInProgress even after Job gets retired
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1540
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1540
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2
> Reporter: Karam Singh
>
> Ran random writer, sort and sort validate job. Checked the jmap -histo:live and verified that there is no reference of JobInProgress after Jobs are retired
> Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed all the job when jobs got retired. again checked jmap -histo:live for JobInProgress for JT process found 2 references were there.
> Found this while doing sanity testing of 1316
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1540) Sometimes JobTracker holds stale
refrence of JobInProgress even after Job gets retired
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838932#action_12838932 ]
Arun C Murthy commented on MAPREDUCE-1540:
------------------------------------------
Thanks for the quick update Amar.
> Sometimes JobTracker holds stale refrence of JobInProgress even after Job gets retired
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1540
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1540
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2
> Reporter: Karam Singh
> Assignee: Amar Kamat
>
> Ran random writer, sort and sort validate job. Checked the jmap -histo:live and verified that there is no reference of JobInProgress after Jobs are retired
> Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed all the job when jobs got retired. again checked jmap -histo:live for JobInProgress for JT process found 2 references were there.
> Found this while doing sanity testing of 1316
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1540) Sometimes JobTracker holds stale
refrence of JobInProgress even after Job gets retired
Posted by "Karam Singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karam Singh updated MAPREDUCE-1540:
-----------------------------------
Description:
Ran random writer, sort and sort validate job. Checked the jmap -histo:live and verified that there is no reference of JobInProgress after Jobs are retired
Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed all the job when jobs got retired. again checked jmap -histo:live for JobInProgress for JT process found 2 references were there.
Found this while doing sanity testing of 1316
was:
Ran random writer, sort and sort validate job. Checked the jmap -histo:live and verified that there is no reference of JobInProgress after Jobs are retired
Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed all the job when jobs got retired. again checked jmap -histo:live for JobInProgress for JT process found 2 references were there.
Found this while doing snaity testing of 1316
> Sometimes JobTracker holds stale refrence of JobInProgress even after Job gets retired
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1540
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1540
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2
> Reporter: Karam Singh
>
> Ran random writer, sort and sort validate job. Checked the jmap -histo:live and verified that there is no reference of JobInProgress after Jobs are retired
> Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed all the job when jobs got retired. again checked jmap -histo:live for JobInProgress for JT process found 2 references were there.
> Found this while doing sanity testing of 1316
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.