You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Kihwal Lee (JIRA)" <ji...@apache.org> on 2012/09/12 01:06:07 UTC
[jira] [Created] (HDFS-3922) 0.22 and 0.23 namenode throws away
blocks under construction on restart
Kihwal Lee created HDFS-3922:
--------------------------------
Summary: 0.22 and 0.23 namenode throws away blocks under construction on restart
Key: HDFS-3922
URL: https://issues.apache.org/jira/browse/HDFS-3922
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 0.20.203.0, 0.22.1
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 0.22.1, 0.23.4
When reading edits on startup, namenode may throw away blocks under construction. This is because the file inode is turned into a "under construction" one, but nothing is done to the last block.
With append/hsync, this is not acceptable because it may drop sync'ed partial blocks. In branch 2 and trunk, HDFS-1623 (HA) fixed this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3922) 0.22 and 0.23 namenode throws away
blocks under construction on restart
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455300#comment-13455300 ]
Kihwal Lee commented on HDFS-3922:
----------------------------------
I manually ran test-patch for 0.23. There were actually no additional javadoc warnings introduced by the patch.
{noformat}
-1 overall.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 7 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version ) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
{noformat}
> 0.22 and 0.23 namenode throws away blocks under construction on restart
> -----------------------------------------------------------------------
>
> Key: HDFS-3922
> URL: https://issues.apache.org/jira/browse/HDFS-3922
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.1, 0.23.3
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
> Attachments: hdfs-3922.branch-023.patch.txt
>
>
> When reading edits on startup, namenode may throw away blocks under construction. This is because the file inode is turned into a "under construction" one, but nothing is done to the last block.
> With append/hsync, this is not acceptable because it may drop sync'ed partial blocks. In branch 2 and trunk, HDFS-1623 (HA) fixed this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3922) 0.22 and 0.23 namenode throws away
blocks under construction on restart
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HDFS-3922:
----------------------------------
Fix Version/s: (was: 0.23.4)
(was: 0.22.1)
Kihwal, I just reset Fix Version/s field. It is set when a fix is committed to a branch with the corresponding release number.
If you intend it for certain releases, add those release numbers to Target Version/s field.
> 0.22 and 0.23 namenode throws away blocks under construction on restart
> -----------------------------------------------------------------------
>
> Key: HDFS-3922
> URL: https://issues.apache.org/jira/browse/HDFS-3922
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.203.0, 0.22.1
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
>
> When reading edits on startup, namenode may throw away blocks under construction. This is because the file inode is turned into a "under construction" one, but nothing is done to the last block.
> With append/hsync, this is not acceptable because it may drop sync'ed partial blocks. In branch 2 and trunk, HDFS-1623 (HA) fixed this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3922) 0.22 and 0.23 namenode throws away
blocks under construction on restart
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated HDFS-3922:
-----------------------------
Attachment: hdfs-3922.branch-023.patch.txt
Posted a patch for branch-0.23.
> 0.22 and 0.23 namenode throws away blocks under construction on restart
> -----------------------------------------------------------------------
>
> Key: HDFS-3922
> URL: https://issues.apache.org/jira/browse/HDFS-3922
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.203.0, 0.22.1
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
> Attachments: hdfs-3922.branch-023.patch.txt
>
>
> When reading edits on startup, namenode may throw away blocks under construction. This is because the file inode is turned into a "under construction" one, but nothing is done to the last block.
> With append/hsync, this is not acceptable because it may drop sync'ed partial blocks. In branch 2 and trunk, HDFS-1623 (HA) fixed this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3922) 0.22 and 0.23 namenode throws away
blocks under construction on restart
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465578#comment-13465578 ]
Hudson commented on HDFS-3922:
------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #388 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/388/])
HDFS-3922. namenode throws away blocks under construction on restart (Kihwal Lee via daryn) (Revision 1391150)
Result = UNSTABLE
daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1391150
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
> 0.22 and 0.23 namenode throws away blocks under construction on restart
> -----------------------------------------------------------------------
>
> Key: HDFS-3922
> URL: https://issues.apache.org/jira/browse/HDFS-3922
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.1, 0.23.3
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
> Fix For: 0.23.4
>
> Attachments: hdfs-3922.branch-023.patch.txt
>
>
> When reading edits on startup, namenode may throw away blocks under construction. This is because the file inode is turned into a "under construction" one, but nothing is done to the last block.
> With append/hsync, this is not acceptable because it may drop sync'ed partial blocks. In branch 2 and trunk, HDFS-1623 (HA) fixed this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3922) 0.22 and 0.23 namenode throws away
blocks under construction on restart
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated HDFS-3922:
-----------------------------
Status: Patch Available (was: Open)
> 0.22 and 0.23 namenode throws away blocks under construction on restart
> -----------------------------------------------------------------------
>
> Key: HDFS-3922
> URL: https://issues.apache.org/jira/browse/HDFS-3922
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.203.0, 0.22.1
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
> Attachments: hdfs-3922.branch-023.patch.txt
>
>
> When reading edits on startup, namenode may throw away blocks under construction. This is because the file inode is turned into a "under construction" one, but nothing is done to the last block.
> With append/hsync, this is not acceptable because it may drop sync'ed partial blocks. In branch 2 and trunk, HDFS-1623 (HA) fixed this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3922) 0.22 and 0.23 namenode throws away
blocks under construction on restart
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455209#comment-13455209 ]
Hadoop QA commented on HDFS-3922:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12545035/hdfs-3922.branch-023.patch.txt
against trunk revision .
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3184//console
This message is automatically generated.
> 0.22 and 0.23 namenode throws away blocks under construction on restart
> -----------------------------------------------------------------------
>
> Key: HDFS-3922
> URL: https://issues.apache.org/jira/browse/HDFS-3922
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.203.0, 0.22.1
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
> Attachments: hdfs-3922.branch-023.patch.txt
>
>
> When reading edits on startup, namenode may throw away blocks under construction. This is because the file inode is turned into a "under construction" one, but nothing is done to the last block.
> With append/hsync, this is not acceptable because it may drop sync'ed partial blocks. In branch 2 and trunk, HDFS-1623 (HA) fixed this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3922) 0.22 and 0.23 namenode throws away
blocks under construction on restart
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated HDFS-3922:
-----------------------------
Target Version/s: 0.23.4
Affects Version/s: (was: 0.20.203.0)
0.23.3
> 0.22 and 0.23 namenode throws away blocks under construction on restart
> -----------------------------------------------------------------------
>
> Key: HDFS-3922
> URL: https://issues.apache.org/jira/browse/HDFS-3922
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.1, 0.23.3
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
> Attachments: hdfs-3922.branch-023.patch.txt
>
>
> When reading edits on startup, namenode may throw away blocks under construction. This is because the file inode is turned into a "under construction" one, but nothing is done to the last block.
> With append/hsync, this is not acceptable because it may drop sync'ed partial blocks. In branch 2 and trunk, HDFS-1623 (HA) fixed this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3922) 0.22 and 0.23 namenode throws away
blocks under construction on restart
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-3922:
------------------------------
Resolution: Fixed
Fix Version/s: 0.23.4
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I've committed to branch-23. Thanks Kihwal!
I'll let the branch-22 maintainers decide if they want to integrate this change.
> 0.22 and 0.23 namenode throws away blocks under construction on restart
> -----------------------------------------------------------------------
>
> Key: HDFS-3922
> URL: https://issues.apache.org/jira/browse/HDFS-3922
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.1, 0.23.3
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
> Fix For: 0.23.4
>
> Attachments: hdfs-3922.branch-023.patch.txt
>
>
> When reading edits on startup, namenode may throw away blocks under construction. This is because the file inode is turned into a "under construction" one, but nothing is done to the last block.
> With append/hsync, this is not acceptable because it may drop sync'ed partial blocks. In branch 2 and trunk, HDFS-1623 (HA) fixed this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira