You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Karthik Kambatla (JIRA)" <ji...@apache.org> on 2012/06/28 22:36:44 UTC
[jira] [Created] (MAPREDUCE-4385) FairScheduler.maxTasksToAssign()
should check for fairscheduler.assignmultiple.maps <
TaskTracker.availableSlots
Karthik Kambatla created MAPREDUCE-4385:
-------------------------------------------
Summary: FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
Key: MAPREDUCE-4385
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 1.0.3
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4385) FairScheduler.maxTasksToAssign()
should check for fairscheduler.assignmultiple.maps <
TaskTracker.availableSlots
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4385:
----------------------------------------
Attachment: (was: MR-4385.patch)
> FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4385
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4385.patch
>
>
> FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
> These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4385) FairScheduler.maxTasksToAssign()
should check for fairscheduler.assignmultiple.maps <
TaskTracker.availableSlots
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated MAPREDUCE-4385:
------------------------------------------
Issue Type: Bug (was: Improvement)
> FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4385
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4385.patch
>
>
> FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
> These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4385)
FairScheduler.maxTasksToAssign() should check for
fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13405422#comment-13405422 ]
Hadoop QA commented on MAPREDUCE-4385:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12534461/MR-4385.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2540//console
This message is automatically generated.
> FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4385
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4385.patch
>
>
> FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
> These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4385) FairScheduler.maxTasksToAssign()
should check for fairscheduler.assignmultiple.maps <
TaskTracker.availableSlots
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4385:
----------------------------------------
Attachment: MR-4385.patch
Uploading a straight-forward patch that returns the minimum of availableSlots() and the number of tasks requested as per config file.
Testing:
- Didn't add additional tests
- Ran TestFairScheduler
> FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4385
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4385.patch
>
>
> FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
> These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4385)
FairScheduler.maxTasksToAssign() should check for
fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404207#comment-13404207 ]
Hadoop QA commented on MAPREDUCE-4385:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533888/MR-4385.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2534//console
This message is automatically generated.
> FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4385
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4385.patch
>
>
> FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
> These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4385)
FairScheduler.maxTasksToAssign() should check for
fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407271#comment-13407271 ]
Alejandro Abdelnur commented on MAPREDUCE-4385:
-----------------------------------------------
+1
> FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4385
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4385.patch
>
>
> FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
> These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4385) FairScheduler.maxTasksToAssign()
should check for fairscheduler.assignmultiple.maps <
TaskTracker.availableSlots
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated MAPREDUCE-4385:
------------------------------------------
Resolution: Fixed
Fix Version/s: 1.2.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Karthik. Committed to branch-1.
> FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4385
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 1.2.0
>
> Attachments: MR-4385.patch
>
>
> FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
> These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4385) FairScheduler.maxTasksToAssign()
should check for fairscheduler.assignmultiple.maps <
TaskTracker.availableSlots
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4385:
----------------------------------------
Status: Patch Available (was: Open)
> FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4385
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4385.patch
>
>
> FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
> These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4385) FairScheduler.maxTasksToAssign()
should check for fairscheduler.assignmultiple.maps <
TaskTracker.availableSlots
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4385:
----------------------------------------
Attachment: MR-4385.patch
Added TestFairScheduler.testMaxTasksToAssign()
> FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps < TaskTracker.availableSlots
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4385
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4385
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4385.patch, MR-4385.patch
>
>
> FairScheduler.maxTasksToAssign() can potentially return a value greater than the available slots. Currently, we rely on canAssignMaps()/canAssignReduces() to reject such requests.
> These additional calls can be avoided by check against the available slots in maxTasksToAssign().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira