You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Lohit Vijayarenu (JIRA)" <ji...@apache.org> on 2008/07/19 02:03:31 UTC
[jira] Created: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
NameNode does not save image if different dfs.name.dir have different checkpoint stamps
---------------------------------------------------------------------------------------
Key: HADOOP-3795
URL: https://issues.apache.org/jira/browse/HADOOP-3795
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.19.0
Reporter: Lohit Vijayarenu
There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Lohit Vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lohit Vijayarenu updated HADOOP-3795:
-------------------------------------
Description: There is a case where namenode does not save image file during startup even if their checkpoint times are different. (was: There is a case where namenode does not save image file during startup even if their checkpoint times are different. I tested this manually for the case described above, and it forces saving of good image back on all dfs.name.dir)
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Lohit Vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lohit Vijayarenu updated HADOOP-3795:
-------------------------------------
Attachment: HADOOP-3795-1.patch
Is is OK to say that checkpoint would be same across all dirs? If so, here is a patch which forces saving of image, if there is a difference in checkpoint times.
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624769#action_12624769 ]
Hudson commented on HADOOP-3795:
--------------------------------
Integrated in Hadoop-trunk #581 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/])
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Assignee: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch, HADOOP-3795-1.patch, HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615431#action_12615431 ]
Raghu Angadi commented on HADOOP-3795:
--------------------------------------
+1. Minor : Hadoop's prefered style is to keep single line block in braces. If you do add braces, could you move the comment inside the braces.
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch, HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated HADOOP-3795:
----------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks lohit.
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Assignee: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch, HADOOP-3795-1.patch, HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Lohit Vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614921#action_12614921 ]
Lohit Vijayarenu commented on HADOOP-3795:
------------------------------------------
Here is what happened.
- There were 2 dfs.name.dir directories each with valid image and edits, but different checkpoint times
- The dir which had the latest checkpoint time had latest image and zero edits. The other directory had older image with non zero entries in edits
- So, while starting, after validating consistency of all dfs.name.dir, namenode started with latest image but ignored saving since needToSave was never set in any of the conditions.
This could be bad, sometimes. In cases if the other directory had edits.new and good directory didnt have, then checkpoint by secondary namenode would fail.
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-3795) NameNode does not save
image if different dfs.name.dir have different checkpoint stamps
Posted by "Lohit Vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614926#action_12614926 ]
lohit edited comment on HADOOP-3795 at 7/18/08 5:49 PM:
-------------------------------------------------------------------
Is is OK to say that checkpoint would be same across all dirs? If so, here is a patch which forces saving of image, if there is a difference in checkpoint times. I tested this manually for the case described above, and it forces saving of good image back on all dfs.name.dir
was (Author: lohit):
Is is OK to say that checkpoint would be same across all dirs? If so, here is a patch which forces saving of image, if there is a difference in checkpoint times.
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Lohit Vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lohit Vijayarenu updated HADOOP-3795:
-------------------------------------
Description: There is a case where namenode does not save image file during startup even if their checkpoint times are different. I tested this manually for the case described above, and it forces saving of good image back on all dfs.name.dir (was: There is a case where namenode does not save image file during startup even if their checkpoint times are different.)
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different. I tested this manually for the case described above, and it forces saving of good image back on all dfs.name.dir
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Lohit Vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lohit Vijayarenu updated HADOOP-3795:
-------------------------------------
Attachment: HADOOP-3795-1.patch
Thanks Raghu, updated patch.
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch, HADOOP-3795-1.patch, HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar reassigned HADOOP-3795:
-------------------------------------
Assignee: Lohit Vijayarenu
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Assignee: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch, HADOOP-3795-1.patch, HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Lohit Vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lohit Vijayarenu updated HADOOP-3795:
-------------------------------------
Attachment: HADOOP-3795-1.patch
Updated patch, with comments, and change to compare _checkpointTime != Long.MIN_VALUE_ instead of _latestSD != null_
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch, HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615471#action_12615471 ]
Hadoop QA commented on HADOOP-3795:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12386573/HADOOP-3795-1.patch
against trunk revision 678577.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2917/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2917/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2917/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2917/console
This message is automatically generated.
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch, HADOOP-3795-1.patch, HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3795) NameNode does not save image if
different dfs.name.dir have different checkpoint stamps
Posted by "Lohit Vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lohit Vijayarenu updated HADOOP-3795:
-------------------------------------
Status: Patch Available (was: Open)
> NameNode does not save image if different dfs.name.dir have different checkpoint stamps
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3795
> URL: https://issues.apache.org/jira/browse/HADOOP-3795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-3795-1.patch, HADOOP-3795-1.patch
>
>
> There is a case where namenode does not save image file during startup even if their checkpoint times are different.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.