You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amar Kamat (JIRA)" <ji...@apache.org> on 2009/05/11 14:51:45 UTC
[jira] Created: (HADOOP-5801) JobTracker should refresh the hosts
list upon recovery
JobTracker should refresh the hosts list upon recovery
------------------------------------------------------
Key: HADOOP-5801
URL: https://issues.apache.org/jira/browse/HADOOP-5801
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Amar Kamat
Assignee: Amar Kamat
If the hosts file is changes across restart then it should be refreshed upon recovery so that the excluded hosts are lost and the maps are re-executed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5801) JobTracker should refresh the hosts
list upon recovery
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-5801:
-------------------------------
Release Note: The mapred-hosts list can change across restarts. Once the jobtracker recovers, it detects all the tasktracker there are there in the history. If the hosts list is changed, then the jobtracker will still have the tasktracker (data) internally but will disallow the tracker when it contacts. As a result, the jobtracker will have to wait for the tracker to timeout in order to re-execute the tasks. This patch simply refreshes the node list upon recovery so that the invalid trackers are lost immediately.
> JobTracker should refresh the hosts list upon recovery
> ------------------------------------------------------
>
> Key: HADOOP-5801
> URL: https://issues.apache.org/jira/browse/HADOOP-5801
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: 5801-0.20.patch, HADOOP-5801-v1.2.patch
>
>
> If the hosts file is changes across restart then it should be refreshed upon recovery so that the excluded hosts are lost and the maps are re-executed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5801) JobTracker should refresh the hosts
list upon recovery
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708366#action_12708366 ]
Amar Kamat commented on HADOOP-5801:
------------------------------------
ant tests passed on my box.
> JobTracker should refresh the hosts list upon recovery
> ------------------------------------------------------
>
> Key: HADOOP-5801
> URL: https://issues.apache.org/jira/browse/HADOOP-5801
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Attachments: HADOOP-5801-v1.2.patch
>
>
> If the hosts file is changes across restart then it should be refreshed upon recovery so that the excluded hosts are lost and the maps are re-executed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5801) JobTracker should refresh the hosts
list upon recovery
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718602#action_12718602 ]
Hudson commented on HADOOP-5801:
--------------------------------
Integrated in Hadoop-trunk #863 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/863/])
> JobTracker should refresh the hosts list upon recovery
> ------------------------------------------------------
>
> Key: HADOOP-5801
> URL: https://issues.apache.org/jira/browse/HADOOP-5801
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: 5801-0.20.patch, HADOOP-5801-v1.2.patch
>
>
> If the hosts file is changes across restart then it should be refreshed upon recovery so that the excluded hosts are lost and the maps are re-executed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5801) JobTracker should refresh the hosts
list upon recovery
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-5801:
------------------------------------
Attachment: 5801-0.20.patch
Attached example for 0.20 not to be committed.
> JobTracker should refresh the hosts list upon recovery
> ------------------------------------------------------
>
> Key: HADOOP-5801
> URL: https://issues.apache.org/jira/browse/HADOOP-5801
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: 5801-0.20.patch, HADOOP-5801-v1.2.patch
>
>
> If the hosts file is changes across restart then it should be refreshed upon recovery so that the excluded hosts are lost and the maps are re-executed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-5801) JobTracker should refresh the hosts
list upon recovery
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das resolved HADOOP-5801.
---------------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
Hadoop Flags: [Reviewed]
I just committed this. Thanks, Amar!
> JobTracker should refresh the hosts list upon recovery
> ------------------------------------------------------
>
> Key: HADOOP-5801
> URL: https://issues.apache.org/jira/browse/HADOOP-5801
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: HADOOP-5801-v1.2.patch
>
>
> If the hosts file is changes across restart then it should be refreshed upon recovery so that the excluded hosts are lost and the maps are re-executed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5801) JobTracker should refresh the hosts
list upon recovery
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-5801:
-------------------------------
Attachment: HADOOP-5801-v1.2.patch
Attaching a patch that refreshes after the recovery is done. Result of test-patch
{code}
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
{code}
Running ant test now.
> JobTracker should refresh the hosts list upon recovery
> ------------------------------------------------------
>
> Key: HADOOP-5801
> URL: https://issues.apache.org/jira/browse/HADOOP-5801
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Attachments: HADOOP-5801-v1.2.patch
>
>
> If the hosts file is changes across restart then it should be refreshed upon recovery so that the excluded hosts are lost and the maps are re-executed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.