You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2012/06/27 03:49:43 UTC
[jira] [Created] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Todd Lipcon created HDFS-3573:
---------------------------------
Summary: Supply NamespaceInfo when instantiating JournalManagers
Key: HDFS-3573
URL: https://issues.apache.org/jira/browse/HDFS-3573
Project: Hadoop HDFS
Issue Type: Sub-task
Components: name-node
Affects Versions: 3.0.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Minor
Attachments: hdfs-3573.txt
Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402059#comment-13402059 ]
Hadoop QA commented on HDFS-3573:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533587/hdfs-3573.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 javadoc. The javadoc tool appears to have generated 2 warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal:
org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
org.apache.hadoop.hdfs.server.namenode.TestGenericJournalConf
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2710//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2710//console
This message is automatically generated.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HDFS-3573:
------------------------------
Attachment: hdfs-3573.txt
Addressed ATM's style nit above. I'll commit this momentarily since the only change was whitespace.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HDFS-3573:
------------------------------
Attachment: hdfs-3573.txt
New patch fixes the two test cases that failed above:
- when iterating over in-progress logs, don't look at lastTxId until the log has been validated. Without this fix, selectInputStreams() was skipping over in-progress logs, which caused TestBootstrapStandby to fail.
- BootstrapStandby needs to supply namespace info and init the edit logs explicitly now - it used to be initted in FSImage's constructor, but it's now an explicit step.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454029#comment-13454029 ]
Todd Lipcon commented on HDFS-3573:
-----------------------------------
Hey Uma. I was planning on backporting this and related patches to branch-2 after QJM has bee merged to trunk and backported. But, feel free to do it earlier if you'd like it for BKJM.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0
>
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated HDFS-3573:
-----------------------------
Attachment: 0001-HDFS-3573-for-branch-2.patch
Patch for branch-2 very similar to trunk, but distributed upgrade is removed, as per HDFS-2686
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: 0001-HDFS-3573-for-branch-2.patch, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407007#comment-13407007 ]
Hudson commented on HDFS-3573:
------------------------------
Integrated in Hadoop-Hdfs-trunk #1094 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1094/])
HDFS-3573. Supply NamespaceInfo when instantiating JournalManagers. Contributed by Todd Lipcon. (Revision 1356388)
Result = FAILURE
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1356388
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileJournalManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0
>
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403608#comment-13403608 ]
Aaron T. Myers commented on HDFS-3573:
--------------------------------------
One tiny nit, this can all reasonably be on one line:
{code}
+ private static final String DUMMY_URI =
+ "dummy://test";
{code}
+1 once this is addressed.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487668#comment-13487668 ]
Hudson commented on HDFS-3573:
------------------------------
Integrated in Hadoop-Yarn-trunk #22 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/22/])
Moved HDFS-3573 entry in CHANGES.txt from trunk to 2.0.3-alpha section (Revision 1403740)
Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1403740
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: 0001-HDFS-3573-for-branch-2.patch, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487789#comment-13487789 ]
Hudson commented on HDFS-3573:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #1242 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1242/])
Moved HDFS-3573 entry in CHANGES.txt from trunk to 2.0.3-alpha section (Revision 1403740)
Result = FAILURE
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1403740
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: 0001-HDFS-3573-for-branch-2.patch, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486575#comment-13486575 ]
Uma Maheswara Rao G commented on HDFS-3573:
-------------------------------------------
Thanks Ivan, +1. I will commit it to branch-2.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: 0001-HDFS-3573-for-branch-2.patch, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402720#comment-13402720 ]
Hadoop QA commented on HDFS-3573:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533721/hdfs-3573.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal:
org.apache.hadoop.hdfs.TestDatanodeBlockScanner
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2715//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2715//console
This message is automatically generated.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HDFS-3573:
------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0
>
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated HDFS-3573:
-----------------------------
Fix Version/s: 2.0.3-alpha
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401881#comment-13401881 ]
Hadoop QA commented on HDFS-3573:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533581/hdfs-3573.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javac. The patch appears to cause the build to fail.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2706//console
This message is automatically generated.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HDFS-3573:
------------------------------
Status: Patch Available (was: Open)
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HDFS-3573:
------------------------------
Attachment: hdfs-3573.txt
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487724#comment-13487724 ]
Hudson commented on HDFS-3573:
------------------------------
Integrated in Hadoop-Hdfs-trunk #1212 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1212/])
Moved HDFS-3573 entry in CHANGES.txt from trunk to 2.0.3-alpha section (Revision 1403740)
Result = FAILURE
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1403740
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: 0001-HDFS-3573-for-branch-2.patch, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486577#comment-13486577 ]
Uma Maheswara Rao G commented on HDFS-3573:
-------------------------------------------
I have just committed this to branch-2.
Committed revision 1403594.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: 0001-HDFS-3573-for-branch-2.patch, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407130#comment-13407130 ]
Hudson commented on HDFS-3573:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #1127 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1127/])
HDFS-3573. Supply NamespaceInfo when instantiating JournalManagers. Contributed by Todd Lipcon. (Revision 1356388)
Result = SUCCESS
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1356388
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileJournalManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0
>
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HDFS-3573:
------------------------------
Attachment: hdfs-3573.txt
Had to add a shim in the BKJM constructor so its tests would compile. I also added a TODO as requested by ivank in an offline conversation for him to make use of this improvement within BKJM.
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486979#comment-13486979 ]
Hudson commented on HDFS-3573:
------------------------------
Integrated in Hadoop-trunk-Commit #2943 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/2943/])
Moved HDFS-3573 entry in CHANGES.txt from trunk to 2.0.3-alpha section (Revision 1403740)
Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1403740
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: 0001-HDFS-3573-for-branch-2.patch, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3573) Supply NamespaceInfo when
instantiating JournalManagers
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453905#comment-13453905 ]
Uma Maheswara Rao G commented on HDFS-3573:
-------------------------------------------
Todd, any reason for not porting this to branch-2 ?
I plan to port some of the BKJM improvements. But they are depending on this change from trunk.
Thanks a lot,
Uma
> Supply NamespaceInfo when instantiating JournalManagers
> -------------------------------------------------------
>
> Key: HDFS-3573
> URL: https://issues.apache.org/jira/browse/HDFS-3573
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.0.0
>
> Attachments: hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt, hdfs-3573.txt
>
>
> Currently, the JournalManagers are instantiated before the NamespaceInfo is loaded from local storage directories. This is problematic since the JM may want to verify that the storage info associated with the journal matches the NN which is starting up (eg to prevent an operator accidentally configuring two clusters against the same remote journal storage). This JIRA rejiggers the initialization sequence so that the JMs receive NamespaceInfo as a constructor argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira