You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/01/19 07:07:43 UTC
[jira] Created: (MAPREDUCE-2271) TestSetupTaskScheduling failing in
trunk
TestSetupTaskScheduling failing in trunk
----------------------------------------
Key: MAPREDUCE-2271
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.23.0
Reporter: Todd Lipcon
Priority: Blocker
This test case is failing in trunk after the commit of MAPREDUCE-2207
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2271) TestSetupTaskScheduling failing
in trunk
Posted by "Liyin Liang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987020#action_12987020 ]
Liyin Liang commented on MAPREDUCE-2271:
----------------------------------------
Hi Todd,
I think the test case testNumSlotsUsedForTaskCleanup is supposed to check that one task-cleanup task only need one slot even for high RAM jobs. This test case create a fake high RAM job with one map task and one reduce task. Each task require 2 slots. Then check that each heartbeat will schedule one task-cleanup task which need only one slot. So it need't to create dummy tracker status with FAILED_UNCLEAN tasks.
The result of the change in MAPREDUCE-2207 is that task-cleanup tasks can't be scheduled to trackers with FAILED_UNCLEAN tasks to report during heartbeat, no matter the task failed on which tracker. This cause none task-cleanup task will be scheduled during heartbeat in the test case. The following code:
{code:}
List<Task> tasks = jobTracker.getSetupAndCleanupTasks(ttStatus);
{code}
will always return *null*, only if ttStatus has tasks with FAILED_UNCLEAN status.
> TestSetupTaskScheduling failing in trunk
> ----------------------------------------
>
> Key: MAPREDUCE-2271
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Liyin Liang
> Priority: Blocker
> Attachments: 2271-1.diff
>
>
> This test case is failing in trunk after the commit of MAPREDUCE-2207
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2271) TestSetupTaskScheduling failing
in trunk
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985881#action_12985881 ]
Todd Lipcon commented on MAPREDUCE-2271:
----------------------------------------
Sorry if I'm being dense, but I don't quite follow the logic of this test. Could you help me understand what's supposed to be going on here?
It seems createJob() already makes the two cleanup tasks with status as FAILED_UNCLEAN from that tracker. Is the change in MAPREDUCE-2207 only such that the cleanup-tasks won't be rescheduled in response to the same heartbeat that marked them failed? So in this test we get them scheduled because it's a new heartbeat with no task statuses, even though it previously had failed on that tracker?
> TestSetupTaskScheduling failing in trunk
> ----------------------------------------
>
> Key: MAPREDUCE-2271
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Attachments: 2271-1.diff
>
>
> This test case is failing in trunk after the commit of MAPREDUCE-2207
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2271) TestSetupTaskScheduling failing in
trunk
Posted by "Liyin Liang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liyin Liang updated MAPREDUCE-2271:
-----------------------------------
Attachment: 2271-1.diff
With [MAPREDUCE-2207|https://issues.apache.org/jira/browse/MAPREDUCE-2207], a tracker can't get any task-cleanup-task, if it has tasks with _FAILED_UNCLEAN_ state. The _testNumSlotsUsedForTaskCleanup_ of _TestSetupTaskScheduling_ creates a dummy tracker status with two _FAILED_UNCLEAN_ tasks to report. So the jobtracker return null when call _getSetupAndCleanupTasks_ with this tracker status.
I think it's useless to add task status to the tracker status in that test case, because the job already has two task-setup-tasks to schedule and the job's two tasks's status are _FAILED_UNCLEAN_. In other words, the job's tasks status need not to be updated.
So we can just remove _addNewTaskStatus_ codes as 2271-1.diff.
> TestSetupTaskScheduling failing in trunk
> ----------------------------------------
>
> Key: MAPREDUCE-2271
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Attachments: 2271-1.diff
>
>
> This test case is failing in trunk after the commit of MAPREDUCE-2207
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2271) TestSetupTaskScheduling failing in
trunk
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-2271:
-----------------------------------
Fix Version/s: 0.23.0
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
> TestSetupTaskScheduling failing in trunk
> ----------------------------------------
>
> Key: MAPREDUCE-2271
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Liyin Liang
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: 2271-1.diff
>
>
> This test case is failing in trunk after the commit of MAPREDUCE-2207
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2271) TestSetupTaskScheduling failing in
trunk
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-2271:
-----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed this to trunk, thanks Liyin!
> TestSetupTaskScheduling failing in trunk
> ----------------------------------------
>
> Key: MAPREDUCE-2271
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Liyin Liang
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: 2271-1.diff
>
>
> This test case is failing in trunk after the commit of MAPREDUCE-2207
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-2271) TestSetupTaskScheduling failing
in trunk
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon reassigned MAPREDUCE-2271:
--------------------------------------
Assignee: Liyin Liang
> TestSetupTaskScheduling failing in trunk
> ----------------------------------------
>
> Key: MAPREDUCE-2271
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Liyin Liang
> Priority: Blocker
> Attachments: 2271-1.diff
>
>
> This test case is failing in trunk after the commit of MAPREDUCE-2207
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2271) TestSetupTaskScheduling failing
in trunk
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016989#comment-13016989 ]
Hudson commented on MAPREDUCE-2271:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> TestSetupTaskScheduling failing in trunk
> ----------------------------------------
>
> Key: MAPREDUCE-2271
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Liyin Liang
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: 2271-1.diff
>
>
> This test case is failing in trunk after the commit of MAPREDUCE-2207
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2271) TestSetupTaskScheduling failing
in trunk
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987154#action_12987154 ]
Hudson commented on MAPREDUCE-2271:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #585 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/585/])
MAPREDUCE-2271. Fix TestSetupTaskScheduling failure on trunk. Contributed by Liyin Liang.
> TestSetupTaskScheduling failing in trunk
> ----------------------------------------
>
> Key: MAPREDUCE-2271
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Liyin Liang
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: 2271-1.diff
>
>
> This test case is failing in trunk after the commit of MAPREDUCE-2207
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.