You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2012/11/12 21:39:12 UTC
[jira] [Created] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Konstantin Shvachko created HDFS-4179:
-----------------------------------------
Summary: BackupNode: allow reads, fix checkpointing, safeMode
Key: HDFS-4179
URL: https://issues.apache.org/jira/browse/HDFS-4179
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 2.0.2-alpha
Reporter: Konstantin Shvachko
Assignee: Konstantin Shvachko
BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500990#comment-13500990 ]
Hudson commented on HDFS-4179:
------------------------------
Integrated in Hadoop-Yarn-trunk #42 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/42/])
HDFS-4179. BackupNode: allow reads, fix checkpointing, safeMode. (shv) (Revision 1411508)
Result = SUCCESS
shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411508
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestBackupNode.java
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 2.0.3-alpha
>
> Attachments: BNAllowReads-2.0.patch, BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499327#comment-13499327 ]
Konstantin Shvachko commented on HDFS-4179:
-------------------------------------------
> I was complaining that the general community doesn't seem interested in maintaining the code.
Just to remind, this is the exact wording you used:
["Konstantin: ... Even if you are using the BN in an older version or a private fork, it is clear that you aren't maintaining it in current releases."|https://issues.apache.org/jira/browse/HDFS-4114?focusedCommentId=13488914&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13488914]
I disagree with you on both issues of a single committer contribution and the reasons we do code reviews. But I would rather stick to the essence of the jira that is fixing the BackupNode.
You did not answer my question. But if you are not interested in helping, then what are these lottery and Hawaii all about.
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499132#comment-13499132 ]
Todd Lipcon commented on HDFS-4179:
-----------------------------------
I was complaining that the general community doesn't seem interested in maintaining the code. If only a single committer is interested in the code, that doesn't seem like enough justification -- isn't that part of the reason we do code reviews? It ensures that the bus factor on the project is greater than 1 -- if you end up winning the lottery and retiring to Hawaii, there should be at least one other committer who thinks the BackupNode is a good idea and wants to help maintain it.
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498989#comment-13498989 ]
Konstantin Shvachko commented on HDFS-4179:
-------------------------------------------
I plan to commit this over weekend if there are no suggestions. Need to make progress in the evolution as Eli suggested in HDFS-4114.
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501178#comment-13501178 ]
Hudson commented on HDFS-4179:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #1263 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1263/])
HDFS-4179. BackupNode: allow reads, fix checkpointing, safeMode. (shv) (Revision 1411508)
Result = FAILURE
shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411508
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestBackupNode.java
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 2.0.3-alpha
>
> Attachments: BNAllowReads-2.0.patch, BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499370#comment-13499370 ]
Todd Lipcon commented on HDFS-4179:
-----------------------------------
OK, to be clear: I'm not interested in reviewing it, because I don't see what the BN offers aside from other already-existing approaches. So, I can't personally devote time to it.
And I don't think "no one wants to review this" is valid reason to commit without a review.
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495691#comment-13495691 ]
Hadoop QA commented on HDFS-4179:
---------------------------------
{color:green}+1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12553178/BNAllowReads.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3487//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3487//console
This message is automatically generated.
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499045#comment-13499045 ]
Konstantin Shvachko commented on HDFS-4179:
-------------------------------------------
You were complaining that I am not maintaining BackupNode code.
Are you willing to help my maintenance effort by reviewing this trivial patch?
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500619#comment-13500619 ]
Konstantin Boudnik commented on HDFS-4179:
------------------------------------------
+1 patch looks good. Tests are passing on the trunk.
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496688#comment-13496688 ]
Konstantin Shvachko commented on HDFS-4179:
-------------------------------------------
Could anybody please review this.
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499024#comment-13499024 ]
Todd Lipcon commented on HDFS-4179:
-----------------------------------
As you know, the policy for trunk is review-then-commit. Unless there is someone willing to +1, I don't think you can commit it without an exception to the policy. If you think an exception is warranted, we should discuss it on the mailing list first in order to set out some guidelines for when you don't need RTC on trunk.
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HDFS-4179:
--------------------------------------
Status: Patch Available (was: Open)
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501159#comment-13501159 ]
Hudson commented on HDFS-4179:
------------------------------
Integrated in Hadoop-Hdfs-trunk #1232 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1232/])
HDFS-4179. BackupNode: allow reads, fix checkpointing, safeMode. (shv) (Revision 1411508)
Result = SUCCESS
shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411508
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestBackupNode.java
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 2.0.3-alpha
>
> Attachments: BNAllowReads-2.0.patch, BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4179) BackupNode: allow reads, fix
checkpointing, safeMode
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HDFS-4179:
--------------------------------------
Attachment: BNAllowReads.patch
The patch
- makes BN accept read commands, but not CN.
- Checkpointer first uploads all files then loads them in memory.
- Safemode on BN should be set to unreachable threshold. Also there was an assert triggered by doConsistencyCheck() for BN.
- Added a test case that verifies reads for BN and CN.
> BackupNode: allow reads, fix checkpointing, safeMode
> ----------------------------------------------------
>
> Key: HDFS-4179
> URL: https://issues.apache.org/jira/browse/HDFS-4179
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: BNAllowReads.patch
>
>
> BackupNode should be allowed to accept read command. Needs some adjustments in checkpointing and with safe mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira