You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Arun C Murthy (JIRA)" <ji...@apache.org> on 2008/12/16 10:26:44 UTC
[jira] Created: (HADOOP-4880) Improvements to TestJobTrackerRestart
Improvements to TestJobTrackerRestart
-------------------------------------
Key: HADOOP-4880
URL: https://issues.apache.org/jira/browse/HADOOP-4880
Project: Hadoop Core
Issue Type: Improvement
Components: mapred, test
Reporter: Arun C Murthy
TestJobTrackerRestart could use the following improvements:
# Remove the one minute 'wait' - this is really bad for test-cases
# It assumes that a particular ordering of job-scheduling based on current behaviour of JobQueueTaskScheduler assigning a single task per heartbeat (pre HADOOP-3136) and thus a particular job completion order, which is incorrect and won't work with all schedulers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4880) Improvements to TestJobTrackerRestart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4880:
-------------------------------
Attachment: HADOOP-4880-v1.1.patch
Attaching a new patch that removes {{testRecoveryWithMultipleJobs()}} and added the check for changed job priority in {{testTaskEventsAndReportsWithRecovery()}}. Testing in progress.
> Improvements to TestJobTrackerRestart
> -------------------------------------
>
> Key: HADOOP-4880
> URL: https://issues.apache.org/jira/browse/HADOOP-4880
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred, test
> Reporter: Arun C Murthy
> Assignee: Amar Kamat
> Attachments: HADOOP-4880-v1.1.patch, HADOOP-4880-v1.patch
>
>
> TestJobTrackerRestart could use the following improvements:
> # Remove the one minute 'wait' - this is really bad for test-cases
> # It assumes that a particular ordering of job-scheduling based on current behaviour of JobQueueTaskScheduler assigning a single task per heartbeat (pre HADOOP-3136) and thus a particular job completion order, which is incorrect and won't work with all schedulers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4880) Improvements to
TestJobTrackerRestart
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12658008#action_12658008 ]
Devaraj Das commented on HADOOP-4880:
-------------------------------------
The patch still assumes that job completions are ordered. I think we should remove this assumption. Also, if this particular testcase testRecoveryWithMultipleJobs may actually be removed. The only additional thing it checks for is the priority change of a job after a restart. That might be moved to some other testcase.
> Improvements to TestJobTrackerRestart
> -------------------------------------
>
> Key: HADOOP-4880
> URL: https://issues.apache.org/jira/browse/HADOOP-4880
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred, test
> Reporter: Arun C Murthy
> Assignee: Amar Kamat
> Attachments: HADOOP-4880-v1.patch
>
>
> TestJobTrackerRestart could use the following improvements:
> # Remove the one minute 'wait' - this is really bad for test-cases
> # It assumes that a particular ordering of job-scheduling based on current behaviour of JobQueueTaskScheduler assigning a single task per heartbeat (pre HADOOP-3136) and thus a particular job completion order, which is incorrect and won't work with all schedulers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4880) Improvements to TestJobTrackerRestart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4880:
-------------------------------
Attachment: HADOOP-4880-v1.patch
Attaching a patch that solves the issue. Note that {{TestJobTrackerRestart}} also requires HADOOP-4879 to work correctly.
> Improvements to TestJobTrackerRestart
> -------------------------------------
>
> Key: HADOOP-4880
> URL: https://issues.apache.org/jira/browse/HADOOP-4880
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred, test
> Reporter: Arun C Murthy
> Assignee: Amar Kamat
> Attachments: HADOOP-4880-v1.patch
>
>
> TestJobTrackerRestart could use the following improvements:
> # Remove the one minute 'wait' - this is really bad for test-cases
> # It assumes that a particular ordering of job-scheduling based on current behaviour of JobQueueTaskScheduler assigning a single task per heartbeat (pre HADOOP-3136) and thus a particular job completion order, which is incorrect and won't work with all schedulers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4880) Improvements to TestJobTrackerRestart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4880:
-------------------------------
Attachment: HADOOP-4880-v1.1.patch
> Improvements to TestJobTrackerRestart
> -------------------------------------
>
> Key: HADOOP-4880
> URL: https://issues.apache.org/jira/browse/HADOOP-4880
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred, test
> Reporter: Arun C Murthy
> Assignee: Amar Kamat
> Attachments: HADOOP-4880-v1.1.patch, HADOOP-4880-v1.patch
>
>
> TestJobTrackerRestart could use the following improvements:
> # Remove the one minute 'wait' - this is really bad for test-cases
> # It assumes that a particular ordering of job-scheduling based on current behaviour of JobQueueTaskScheduler assigning a single task per heartbeat (pre HADOOP-3136) and thus a particular job completion order, which is incorrect and won't work with all schedulers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4880) Improvements to
TestJobTrackerRestart
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das resolved HADOOP-4880.
---------------------------------
Resolution: Fixed
Fix Version/s: 0.20.0
Hadoop Flags: [Reviewed]
I just committed this. Thanks, Amar!
> Improvements to TestJobTrackerRestart
> -------------------------------------
>
> Key: HADOOP-4880
> URL: https://issues.apache.org/jira/browse/HADOOP-4880
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred, test
> Reporter: Arun C Murthy
> Assignee: Amar Kamat
> Fix For: 0.20.0
>
> Attachments: HADOOP-4880-v1.1.patch, HADOOP-4880-v1.patch
>
>
> TestJobTrackerRestart could use the following improvements:
> # Remove the one minute 'wait' - this is really bad for test-cases
> # It assumes that a particular ordering of job-scheduling based on current behaviour of JobQueueTaskScheduler assigning a single task per heartbeat (pre HADOOP-3136) and thus a particular job completion order, which is incorrect and won't work with all schedulers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4880) Improvements to
TestJobTrackerRestart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat reassigned HADOOP-4880:
----------------------------------
Assignee: Amar Kamat
> Improvements to TestJobTrackerRestart
> -------------------------------------
>
> Key: HADOOP-4880
> URL: https://issues.apache.org/jira/browse/HADOOP-4880
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred, test
> Reporter: Arun C Murthy
> Assignee: Amar Kamat
>
> TestJobTrackerRestart could use the following improvements:
> # Remove the one minute 'wait' - this is really bad for test-cases
> # It assumes that a particular ordering of job-scheduling based on current behaviour of JobQueueTaskScheduler assigning a single task per heartbeat (pre HADOOP-3136) and thus a particular job completion order, which is incorrect and won't work with all schedulers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4880) Improvements to
TestJobTrackerRestart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656931#action_12656931 ]
Amar Kamat commented on HADOOP-4880:
------------------------------------
HADOOP-4220 works on the test optimizations.
> Improvements to TestJobTrackerRestart
> -------------------------------------
>
> Key: HADOOP-4880
> URL: https://issues.apache.org/jira/browse/HADOOP-4880
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred, test
> Reporter: Arun C Murthy
>
> TestJobTrackerRestart could use the following improvements:
> # Remove the one minute 'wait' - this is really bad for test-cases
> # It assumes that a particular ordering of job-scheduling based on current behaviour of JobQueueTaskScheduler assigning a single task per heartbeat (pre HADOOP-3136) and thus a particular job completion order, which is incorrect and won't work with all schedulers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4880) Improvements to TestJobTrackerRestart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4880:
-------------------------------
Attachment: (was: HADOOP-4880-v1.1.patch)
> Improvements to TestJobTrackerRestart
> -------------------------------------
>
> Key: HADOOP-4880
> URL: https://issues.apache.org/jira/browse/HADOOP-4880
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred, test
> Reporter: Arun C Murthy
> Assignee: Amar Kamat
> Attachments: HADOOP-4880-v1.1.patch, HADOOP-4880-v1.patch
>
>
> TestJobTrackerRestart could use the following improvements:
> # Remove the one minute 'wait' - this is really bad for test-cases
> # It assumes that a particular ordering of job-scheduling based on current behaviour of JobQueueTaskScheduler assigning a single task per heartbeat (pre HADOOP-3136) and thus a particular job completion order, which is incorrect and won't work with all schedulers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.