You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2010/01/05 02:52:54 UTC
[jira] Created: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
---------------------------------------------------------------------------------
Key: MAPREDUCE-1354
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Devaraj Das
It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Dick King (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dick King reassigned MAPREDUCE-1354:
------------------------------------
Assignee: Dick King (was: Arun C Murthy)
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848459#action_12848459 ]
Hadoop QA commented on MAPREDUCE-1354:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12439506/mapreduce-1354--2010-03-10.patch
against trunk revision 925561.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/47/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/47/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/47/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/47/console
This message is automatically generated.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Attachment: MAPREDUCE-1354_yhadoop20.patch
Illustrative patch for yhadoop20, not to be committed.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Environment: Incremental enhancements to the JobTracker include a no-lock version of JT.getTaskCompletion events, no lock on the JT while doing i/o during job-submission and several fixes to cut down configuration parsing during heartbeat-handling.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Environment: Incremental enhancements to the JobTracker include a no-lock version of JT.getTaskCompletion events, no lock on the JT while doing i/o during job-submission and several fixes to cut down configuration parsing during heartbeat-handling.
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Dmytro Molkov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12869333#action_12869333 ]
Dmytro Molkov commented on MAPREDUCE-1354:
------------------------------------------
Is there any particular reason that only getTaskCompletionEvents dropped the synchronized modifier, but all other job access methods like getCleanupTaskReports, getSetupTaskReports, etc are still syncrhonized, while effectively they are doing a very similar kind of access?
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Dick King (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848354#action_12848354 ]
Dick King commented on MAPREDUCE-1354:
--------------------------------------
The submitted patch is designed to apply to Trunk.
The 2/25 patch applies to Release 0.20
-dk
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834368#action_12834368 ]
Hemanth Yamijala commented on MAPREDUCE-1354:
---------------------------------------------
I went over the capacity scheduler changes and changes in JobConf. I also went over the JT changes a little bit (more thorough review was done by Amareshwari). The changes look fine to me as far as I can see.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Attachment: MAPREDUCE-1354_yhadoop20.patch
As per Rajesh's suggestion, I've fixed JT.getTaskCompletionEvents to be completely lock-free on the JT by making JT.jobs a synchronized map. However, I've left JT.(running|completed|failed)Jobs broken as currently to cut down risk.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Dick King (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864387#action_12864387 ]
Dick King commented on MAPREDUCE-1354:
--------------------------------------
The [changes suggested above|https://issues.apache.org/jira/browse/MAPREDUCE-1354?focusedCommentId=12834139&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12834139] seem good to me, but since the changes are orthogonal to the code of this JIRA I would rather make them a separate change.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Hadoop Flags: [Reviewed]
Issue Type: Improvement (was: Bug)
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834139#action_12834139 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1354:
----------------------------------------------------
Some more optimizations that can be done(though these do not effect scheduling code path):
* JobInProgress.getJobConf().getUser() is called from JobTracker.submitJob() and QueueManager.hasAccess(). Those calls can also be changed to job.getUser().
* JobInProgress.finishedMaps() and finishedReduces() are synchronized; they are called from jobqueue_details.jsp which iterates through all jobs. If any job is in initialization, this page doesn't come up until the initialization finishes.
* JobTracker.setJobPriority() should also check whether the job is initialized or not.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1354:
-----------------------------------------------
Status: Patch Available (was: Open)
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Attachment: MAPREDUCE-1354_yhadoop20.patch
Updated patch to fix Math.ceil via integer division in the CapacityScheduler.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834144#action_12834144 ]
Hemanth Yamijala commented on MAPREDUCE-1354:
---------------------------------------------
bq. It has to be on "synchronized (jobs) {"
Sigh ! Access to the jobs data structure is completely messed up. I think for this patch we should *retain* the inconsistency, but reduce the scope of its damage - which I suppose is what Arun has done.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Rajesh Balamohan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834136#action_12834136 ]
Rajesh Balamohan commented on MAPREDUCE-1354:
---------------------------------------------
In the latest patch, getTaskCompletionEvents is using synchronized(this) I believe.
It has to be on "synchronized (jobs) {"
public TaskCompletionEvent[] getTaskCompletionEvents(
JobID jobid, int fromEventId, int maxEvents) throws IOException{
JobInProgress job = null;
synchronized (jobs) {
job = this.jobs.get(jobid);
}
if (null != job) {
return isJobInited(job) ?
job.getTaskCompletionEvents(fromEventId, maxEvents) :
TaskCompletionEvent.EMPTY_ARRAY;
}
return completedJobStatusStore.readJobTaskCompletionEvents(jobid, fromEventId, maxEvents);
}
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Environment: (was: Incremental enhancements to the JobTracker include a no-lock version of JT.getTaskCompletion events, no lock on the JT while doing i/o during job-submission and several fixes to cut down configuration parsing during heartbeat-handling.)
Release Note: Incremental enhancements to the JobTracker include a no-lock version of JT.getTaskCompletion events, no lock on the JT while doing i/o during job-submission and several fixes to cut down configuration parsing during heartbeat-handling.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834354#action_12834354 ]
Arun C Murthy commented on MAPREDUCE-1354:
------------------------------------------
Some discussion has been happening here: MAPREDUCE-1495. I'm going to merge these issues since they affect the same code on similar lines.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832437#action_12832437 ]
Hemanth Yamijala commented on MAPREDUCE-1354:
---------------------------------------------
One thing that was noticed was that the getCounters call in JobInProgress is synchronized. The wrapper call to getCounters in Jobtracker acquires a lock on the JT and then calls JobInProgress.getCounters. The problem is that if the job is being initialized under initTasks, then the jobtracker lock can get held up. We saw an instance of this on our clusters. To avoid this case, one solution could be to check if the job being queried is inited. This pattern is used in getTaskCompletionEvents.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Component/s: jobtracker
Priority: Critical (was: Major)
Assignee: Arun C Murthy
Issue Type: Bug (was: Improvement)
JobTracker.submitJob also forks a DU and writes to it's local-disk while holding the lock.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12867445#action_12867445 ]
Hadoop QA commented on MAPREDUCE-1354:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12444432/mapreduce-1354--2010-05-13.patch
against trunk revision 944082.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/187/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/187/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/187/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/187/console
This message is automatically generated.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Dick King (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dick King updated MAPREDUCE-1354:
---------------------------------
Attachment: mapreduce-1354--2010-05-13.patch
I honored the last two comments by Amareshwari [and ignored the one he invited me to ginore] and this is the patch, but as I write this, {{trunk}} does not compile, so I'm not resubmitting this patch just yet.
Rather than taking the Big Lock, I chose to turn {{nextJobId}} into an {{AtomicInteger}} .
I agree that the {{ugi == null}} test is dead.
When {{trunk}} comes to build I'll test this patch and Submit it.
-dk
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12850078#action_12850078 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1354:
----------------------------------------------------
Shall we do the optimizations suggested at [comment|https://issues.apache.org/jira/browse/MAPREDUCE-1354?focusedCommentId=12834139&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12834139] as part of the trunk patch?
Or raise a jira to do the same?
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Attachment: MAPREDUCE-1354_yhadoop20.patch
Updated patch to cache JobConf.getMemoryFor{Map|Reduce}Task.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated MAPREDUCE-1354:
----------------------------------------
Attachment: mr-1354-y20.patch
More updated patch for earlier version of hadoop. Not for commit here.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Dick King (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12869259#action_12869259 ]
Dick King commented on MAPREDUCE-1354:
--------------------------------------
The regression failure flagged by Hudson, {{TestJobStatusPersistency}} , does not repeat, and is hugely unlikely to have been caused by this patch.
There is no new test because this patch fixes an extremely narrow race condition and that race cannot be induced artificially.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Attachment: MAPREDUCE-1354_yhadoop20.patch
Updated, fixed the framework to use the existing hasSpeculative{Maps|Reduces} rather than fetch it from the JobConf each time.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12873553#action_12873553 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1354:
----------------------------------------------------
bq. The changes suggested above seem good to me, but since the changes are orthogonal to the code of this JIRA I would rather make them a separate change.
Raised MAPREDUCE-1825 for this.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Attachment: MAPREDUCE-1354_yhadoop20.patch
Updated patch to incorporate MAPREDUCE-1495 alongwith fixing pretty much all uses of JobConf inside the scheduling code in JIP and the CS.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Attachment: MAPREDUCE-1354_yhadoop20.patch
Updated patch.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1354:
-----------------------------------------------
Status: Open (was: Patch Available)
Patch looks fine.
Canceling patch to submit for hudson, as trunk compiles now.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated MAPREDUCE-1354:
---------------------------------
Fix Version/s: 0.22.0
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Fix For: 0.22.0
>
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Dick King (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dick King updated MAPREDUCE-1354:
---------------------------------
Attachment: mapreduce-1354--2010-03-10.patch
After studying this to see if it looked race-free, I checked this with an extensive gridmix run.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832468#action_12832468 ]
Amar Kamat commented on MAPREDUCE-1354:
---------------------------------------
Job initialization (job.split localization) can also take up considerable amount of time. Hence we should avoid access to any getter calls to JobInProgress while the initialization is in progress. Following are the other methods that first lock the JobTracker and then JobInProgress potentially locking up the JobTracker during the job initialization.
- getMapTaskReports()
- getReduceTaskReports()
- getCleanupTaskReports()
- getSetupTaskReports()
- getTaskCompletionEvents()
- getTaskDiagnostics()
- setJobPriority()
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834078#action_12834078 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1354:
----------------------------------------------------
Some comments on the patch:
1. JobInProgress constructor calls methods like JobTracker.getSystemDirectoryForJob(). This method is called with JobTracker lock sometimes, JobTracker lock followed by JobInProgress lock sometimes and without any lock in this case. I think this should not effect any, but we should verify all the locking orders for all the back calls from JobInProgress constructor to JobTracker.
2. Unused import for org.apache.tools.ant.taskdefs.condition.HasMethod in JobInProgress
3. Variable JobTracker.EMPTY_TASK_DIAGNOSTICS is not used anywhere.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Status: Resolved (was: Patch Available)
Resolution: Fixed
I just committed this. Thanks Dick!
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Dick King (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dick King updated MAPREDUCE-1354:
---------------------------------
Status: Patch Available (was: Open)
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834174#action_12834174 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1354:
----------------------------------------------------
Other than the above optimizations I suggested in my previous comment, the code changes look fine.
I verified all the accesses to JobInProgress.conf and JobInProgress.getJobConf(); verified that synchronized methods in JobInProgress are not called if the job is not initialized;
JobInProgress constructor calls jobtracker.getJobTrackerMachine(), jobtracker.getInfoPort(), jobtracker.getSystemDirectoryForJob(), jobtracker.getNumTaskCacheLevels(). All these are getters and their values are initialized in JobTracker's constructor. So, calling them without JobTracker lock is fine. JobInProgress constructor calls jobtracker.getInstrumentation().addPrepJob() also. The method addPrepJob() is a synchronized method in implementation of Instrumentation. Thus, calling JobInProgress constructor withour JobTracker lock is fine.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-1354:
-------------------------------------
Summary: Incremental enhancements to the JobTracker for better scalability (was: Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses)
Subsuming MAPREDUCE-1495 by incorporating it.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Rajesh Balamohan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834149#action_12834149 ]
Rajesh Balamohan commented on MAPREDUCE-1354:
---------------------------------------------
Plz ignore the previous comment. Had a discussion with Hemanth and will try out synchronized(jobs) on trunk.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832914#action_12832914 ]
Arun C Murthy commented on MAPREDUCE-1354:
------------------------------------------
Some more things which came up during more investigations: Math.ceil is fairly expensive, we use it in 2 places. One we can do away by using the cached value of JobInProgress.slotsPer{Map|Reduce}, the other with using rint (thanks to Chris for the suggestion).
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833070#action_12833070 ]
Todd Lipcon commented on MAPREDUCE-1354:
----------------------------------------
Arun: I don't see the ceil to rint changes in this patch. Did you upload the right one?
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12865672#action_12865672 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1354:
----------------------------------------------------
Some comments on the patch:
* patch would need changes in TaskDataView.java for corresponding computeNumSlotsPerReduce changes in 0.20 patch. But the same changes are present in MAPREDUCE-1533. So, this comment can be ignored as well.
* Any reason for making JobTracker.getNewJobID() un-synchrozied? I see deprecated getNewJobId() still synchronized.
* In JobTracker.submitJob(), the following code change
{code}
+ if (ugi == null) {
+ ugi = UserGroupInformation.getCurrentUser();
+ }
{code}
is not needed. ugi would never be null.
Other changes look fine to me.
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the
JobTracker for better scalability
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870434#action_12870434 ]
Hudson commented on MAPREDUCE-1354:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #326 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/326/])
> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Reporter: Devaraj Das
> Assignee: Dick King
> Priority: Critical
> Attachments: mapreduce-1354--2010-03-10.patch, mapreduce-1354--2010-05-13.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to
not lock the JobTracker during the HDFS accesses
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796483#action_12796483 ]
Devaraj Das commented on MAPREDUCE-1354:
----------------------------------------
bq. We should see if we can avoid taking the lock altogether.
This may require major changes.. Maybe we should just make the locking more granular within the method for the time being.
> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Devaraj Das
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.