You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amar Kamat (JIRA)" <ji...@apache.org> on 2009/02/18 12:03:01 UTC
[jira] Created: (HADOOP-5278) Finish time of a TIP is incorrectly
logged to the jobhistory upon jobtracker restart
Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart
------------------------------------------------------------------------------------
Key: HADOOP-5278
URL: https://issues.apache.org/jira/browse/HADOOP-5278
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Amar Kamat
Upon recovery, the jobtracker replays the attempt information to the JIP. Upon seeing the attempt-end info, the JIP _completes_ the TIP (which sets the finish time to current time) and logs it to the new jobhistory file (the _.recover_ file). After seeing the task(tip) finish line, the recovery manager changes the finish time to correct finish time but the new jobhistory file still contains the wrong value for the finish time seen while completing the TIP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5278) Finish time of a TIP is incorrectly
logged to the jobhistory upon jobtracker restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12674570#action_12674570 ]
Amar Kamat commented on HADOOP-5278:
------------------------------------
Ravi actually found this while testing HADOOP-4191.
> Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-5278
> URL: https://issues.apache.org/jira/browse/HADOOP-5278
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
>
> Upon recovery, the jobtracker replays the attempt information to the JIP. Upon seeing the attempt-end info, the JIP _completes_ the TIP (which sets the finish time to current time) and logs it to the new jobhistory file (the _.recover_ file). After seeing the task(tip) finish line, the recovery manager changes the finish time to correct finish time but the new jobhistory file still contains the wrong value for the finish time seen while completing the TIP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5278) Finish time of a TIP is incorrectly
logged to the jobhistory upon jobtracker restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675879#action_12675879 ]
Amar Kamat commented on HADOOP-5278:
------------------------------------
Result of test-patch :
{code}
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
{code}
> Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-5278
> URL: https://issues.apache.org/jira/browse/HADOOP-5278
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Attachments: HADOOP-5278-v1.0.patch
>
>
> Upon recovery, the jobtracker replays the attempt information to the JIP. Upon seeing the attempt-end info, the JIP _completes_ the TIP (which sets the finish time to current time) and logs it to the new jobhistory file (the _.recover_ file). After seeing the task(tip) finish line, the recovery manager changes the finish time to correct finish time but the new jobhistory file still contains the wrong value for the finish time seen while completing the TIP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-5278) Finish time of a TIP is incorrectly
logged to the jobhistory upon jobtracker restart
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das resolved HADOOP-5278.
---------------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
Assignee: Amar Kamat
Hadoop Flags: [Reviewed]
I just committed this. Thanks, Amar!
> Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-5278
> URL: https://issues.apache.org/jira/browse/HADOOP-5278
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: HADOOP-5278-v1.0.patch
>
>
> Upon recovery, the jobtracker replays the attempt information to the JIP. Upon seeing the attempt-end info, the JIP _completes_ the TIP (which sets the finish time to current time) and logs it to the new jobhistory file (the _.recover_ file). After seeing the task(tip) finish line, the recovery manager changes the finish time to correct finish time but the new jobhistory file still contains the wrong value for the finish time seen while completing the TIP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5278) Finish time of a TIP is incorrectly
logged to the jobhistory upon jobtracker restart
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677032#action_12677032 ]
Hudson commented on HADOOP-5278:
--------------------------------
Integrated in Hadoop-trunk #766 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/766/])
> Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-5278
> URL: https://issues.apache.org/jira/browse/HADOOP-5278
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: HADOOP-5278-v1.0.patch
>
>
> Upon recovery, the jobtracker replays the attempt information to the JIP. Upon seeing the attempt-end info, the JIP _completes_ the TIP (which sets the finish time to current time) and logs it to the new jobhistory file (the _.recover_ file). After seeing the task(tip) finish line, the recovery manager changes the finish time to correct finish time but the new jobhistory file still contains the wrong value for the finish time seen while completing the TIP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5278) Finish time of a TIP is incorrectly
logged to the jobhistory upon jobtracker restart
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681388#action_12681388 ]
Devaraj Das commented on HADOOP-5278:
-------------------------------------
I committed this to the 0.20 branch as well.
> Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-5278
> URL: https://issues.apache.org/jira/browse/HADOOP-5278
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: HADOOP-5278-v1.0.patch
>
>
> Upon recovery, the jobtracker replays the attempt information to the JIP. Upon seeing the attempt-end info, the JIP _completes_ the TIP (which sets the finish time to current time) and logs it to the new jobhistory file (the _.recover_ file). After seeing the task(tip) finish line, the recovery manager changes the finish time to correct finish time but the new jobhistory file still contains the wrong value for the finish time seen while completing the TIP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5278) Finish time of a TIP is incorrectly
logged to the jobhistory upon jobtracker restart
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-5278:
-------------------------------
Attachment: HADOOP-5278-v1.0.patch
Attaching a patch the add another update line that overwrites the wrong finish-time with the correct finish-time.
> Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-5278
> URL: https://issues.apache.org/jira/browse/HADOOP-5278
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Attachments: HADOOP-5278-v1.0.patch
>
>
> Upon recovery, the jobtracker replays the attempt information to the JIP. Upon seeing the attempt-end info, the JIP _completes_ the TIP (which sets the finish time to current time) and logs it to the new jobhistory file (the _.recover_ file). After seeing the task(tip) finish line, the recovery manager changes the finish time to correct finish time but the new jobhistory file still contains the wrong value for the finish time seen while completing the TIP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5278) Finish time of a TIP is incorrectly
logged to the jobhistory upon jobtracker restart
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675270#action_12675270 ]
Ravi Gummadi commented on HADOOP-5278:
--------------------------------------
I tested this patch with HADOOP-4191 and I see that correct value of finish time of task(tip) is logged to history file with this patch.
+1
> Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-5278
> URL: https://issues.apache.org/jira/browse/HADOOP-5278
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Attachments: HADOOP-5278-v1.0.patch
>
>
> Upon recovery, the jobtracker replays the attempt information to the JIP. Upon seeing the attempt-end info, the JIP _completes_ the TIP (which sets the finish time to current time) and logs it to the new jobhistory file (the _.recover_ file). After seeing the task(tip) finish line, the recovery manager changes the finish time to correct finish time but the new jobhistory file still contains the wrong value for the finish time seen while completing the TIP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5278) Finish time of a TIP is incorrectly
logged to the jobhistory upon jobtracker restart
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681745#action_12681745 ]
Hudson commented on HADOOP-5278:
--------------------------------
Integrated in Hadoop-trunk #778 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/778/])
. Moved the comment about 5278 to branch-0.20 section.
> Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-5278
> URL: https://issues.apache.org/jira/browse/HADOOP-5278
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: HADOOP-5278-v1.0.patch
>
>
> Upon recovery, the jobtracker replays the attempt information to the JIP. Upon seeing the attempt-end info, the JIP _completes_ the TIP (which sets the finish time to current time) and logs it to the new jobhistory file (the _.recover_ file). After seeing the task(tip) finish line, the recovery manager changes the finish time to correct finish time but the new jobhistory file still contains the wrong value for the finish time seen while completing the TIP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.