You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Eli Collins (Created) (JIRA)" <ji...@apache.org> on 2012/02/05 00:47:53 UTC
[jira] [Created] (HDFS-2894) HA: disable 2NN when HA is enabled
HA: disable 2NN when HA is enabled
----------------------------------
Key: HDFS-2894
URL: https://issues.apache.org/jira/browse/HDFS-2894
Project: Hadoop HDFS
Issue Type: Sub-task
Components: ha
Affects Versions: HA branch (HDFS-1623)
Reporter: Eli Collins
Assignee: Eli Collins
The SecondaryNameNode should log a message and refuse to start if HA is enabled since the StandbyNode checkpoints by default and IIRC we have not yet enabled the ability to have multiple checkpointers in the NN.
On the HA branch the 2NN does not currently start from start-dfs.sh because getconf -secondarynamenodes claims the http-address is not configured even though it is, though this seems like a bug, in branch 23 getconf will correctly return localhost:50090.
{noformat}
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>localhost:50090</value>
</property>
</configuration>
hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs getconf -secondarynamenodes
Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2894) HA: automatically determine the
nameservice Id if only one nameservice is configured
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201261#comment-13201261 ]
Hudson commented on HDFS-2894:
------------------------------
Integrated in Hadoop-Hdfs-HAbranch-build #70 (See [https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/70/])
HDFS-2894. HA: automatically determine the nameservice Id if only one nameservice is configured. Contributed by Eli Collins
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240917
Files :
* /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-1623.txt
* /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
* /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
* /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUtil.java
* /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHAConfiguration.java
> HA: automatically determine the nameservice Id if only one nameservice is configured
> ------------------------------------------------------------------------------------
>
> Key: HDFS-2894
> URL: https://issues.apache.org/jira/browse/HDFS-2894
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-2894.txt, hdfs-2894.txt
>
>
> The SecondaryNameNode should log a message and refuse to start if HA is enabled since the StandbyNode checkpoints by default and IIRC we have not yet enabled the ability to have multiple checkpointers in the NN.
> On the HA branch the 2NN does not currently start from start-dfs.sh because getconf -secondarynamenodes claims the http-address is not configured even though it is, though this seems like a bug, in branch 23 getconf will correctly return localhost:50090.
> {noformat}
> <property>
> <name>dfs.namenode.secondary.http-address</name>
> <value>localhost:50090</value>
> </property>
> </configuration>
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs getconf -secondarynamenodes
> Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2894) HA: automatically determine the
nameservice Id if only one nameservice is configured
Posted by "Eli Collins (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-2894:
------------------------------
Attachment: hdfs-2894.txt
Updated patch. I tested non-HA federated config, but updated DFSTestUtil in this patch to explicitly check HAUtil.isHaEnabled in both the federated and HA + federated tests. Also turned the 2NN error into an exception.
> HA: automatically determine the nameservice Id if only one nameservice is configured
> ------------------------------------------------------------------------------------
>
> Key: HDFS-2894
> URL: https://issues.apache.org/jira/browse/HDFS-2894
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-2894.txt, hdfs-2894.txt
>
>
> The SecondaryNameNode should log a message and refuse to start if HA is enabled since the StandbyNode checkpoints by default and IIRC we have not yet enabled the ability to have multiple checkpointers in the NN.
> On the HA branch the 2NN does not currently start from start-dfs.sh because getconf -secondarynamenodes claims the http-address is not configured even though it is, though this seems like a bug, in branch 23 getconf will correctly return localhost:50090.
> {noformat}
> <property>
> <name>dfs.namenode.secondary.http-address</name>
> <value>localhost:50090</value>
> </property>
> </configuration>
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs getconf -secondarynamenodes
> Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2894) HA: disable 2NN when HA is enabled
Posted by "Eli Collins (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200951#comment-13200951 ]
Eli Collins commented on HDFS-2894:
-----------------------------------
The bug here is that in the HA branch we're now treating {{dfs.namenode.secondary.http-address}} like the other configs, eg expecting that it have both a nameservice and namenode prefix.
The 2NN has a check to shutdown if HA is enabled but that check fails since DFSUtil#getSecondaryNameServiceId is used to determine the nameservice, and it can't determine the nameservice when the config is specified w/o a nameservice prefix, so it incorrectly thinks HA is not enabled.
The 2NN should check if it thinks HA is enabled for the *NN*, not the 2NN, since the 2NN http-address config option is independent of HA (it might not be set at all).
We also need to test that:
# The 2NN fails to start if the {{dfs.namenode.secondary.http-address}} is undefined
# Or specified with a nameservice prefix (but not namenode ID)
# Or specified with a nameservice prefix and a namenode ID (incorrect config, user error)
# getconf -secondarynamenode should work when {{dfs.namenode.secondary.http-address}} is specified either with or without a nameservice ID prefix
Also, the 2NN should stop cleanly if HA is enabled, currently even if it shutdowns as intended then it will NPE in Daemon.java.
> HA: disable 2NN when HA is enabled
> ----------------------------------
>
> Key: HDFS-2894
> URL: https://issues.apache.org/jira/browse/HDFS-2894
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
>
> The SecondaryNameNode should log a message and refuse to start if HA is enabled since the StandbyNode checkpoints by default and IIRC we have not yet enabled the ability to have multiple checkpointers in the NN.
> On the HA branch the 2NN does not currently start from start-dfs.sh because getconf -secondarynamenodes claims the http-address is not configured even though it is, though this seems like a bug, in branch 23 getconf will correctly return localhost:50090.
> {noformat}
> <property>
> <name>dfs.namenode.secondary.http-address</name>
> <value>localhost:50090</value>
> </property>
> </configuration>
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs getconf -secondarynamenodes
> Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2894) HA: automatically determine the
nameservice Id if only one nameservice is configured
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201113#comment-13201113 ]
Todd Lipcon commented on HDFS-2894:
-----------------------------------
+1
> HA: automatically determine the nameservice Id if only one nameservice is configured
> ------------------------------------------------------------------------------------
>
> Key: HDFS-2894
> URL: https://issues.apache.org/jira/browse/HDFS-2894
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-2894.txt, hdfs-2894.txt
>
>
> The SecondaryNameNode should log a message and refuse to start if HA is enabled since the StandbyNode checkpoints by default and IIRC we have not yet enabled the ability to have multiple checkpointers in the NN.
> On the HA branch the 2NN does not currently start from start-dfs.sh because getconf -secondarynamenodes claims the http-address is not configured even though it is, though this seems like a bug, in branch 23 getconf will correctly return localhost:50090.
> {noformat}
> <property>
> <name>dfs.namenode.secondary.http-address</name>
> <value>localhost:50090</value>
> </property>
> </configuration>
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs getconf -secondarynamenodes
> Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2894) HA: disable 2NN when HA is enabled
Posted by "Eli Collins (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-2894:
------------------------------
Attachment: hdfs-2894.txt
Patch attached. Turns out the real issue was not automatically determining the nameservice Id when only one is listed in the config, and therefore thinking that HA is not enabled when it is (and a nameservice ID is not explicitly given).
I tested the following in a build run from a tarball:
# The 2NN starts and stops cleanly in a non-HA config
# The 2NN shutsdown cleanly when HA is enabled with the appropriate error message in the log
> HA: disable 2NN when HA is enabled
> ----------------------------------
>
> Key: HDFS-2894
> URL: https://issues.apache.org/jira/browse/HDFS-2894
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-2894.txt
>
>
> The SecondaryNameNode should log a message and refuse to start if HA is enabled since the StandbyNode checkpoints by default and IIRC we have not yet enabled the ability to have multiple checkpointers in the NN.
> On the HA branch the 2NN does not currently start from start-dfs.sh because getconf -secondarynamenodes claims the http-address is not configured even though it is, though this seems like a bug, in branch 23 getconf will correctly return localhost:50090.
> {noformat}
> <property>
> <name>dfs.namenode.secondary.http-address</name>
> <value>localhost:50090</value>
> </property>
> </configuration>
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs getconf -secondarynamenodes
> Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2894) HA: automatically determine the
nameservice Id if only one nameservice is configured
Posted by "Eli Collins (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-2894:
------------------------------
Summary: HA: automatically determine the nameservice Id if only one nameservice is configured (was: HA: disable 2NN when HA is enabled)
> HA: automatically determine the nameservice Id if only one nameservice is configured
> ------------------------------------------------------------------------------------
>
> Key: HDFS-2894
> URL: https://issues.apache.org/jira/browse/HDFS-2894
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-2894.txt
>
>
> The SecondaryNameNode should log a message and refuse to start if HA is enabled since the StandbyNode checkpoints by default and IIRC we have not yet enabled the ability to have multiple checkpointers in the NN.
> On the HA branch the 2NN does not currently start from start-dfs.sh because getconf -secondarynamenodes claims the http-address is not configured even though it is, though this seems like a bug, in branch 23 getconf will correctly return localhost:50090.
> {noformat}
> <property>
> <name>dfs.namenode.secondary.http-address</name>
> <value>localhost:50090</value>
> </property>
> </configuration>
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs getconf -secondarynamenodes
> Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-2894) HA: automatically determine the
nameservice Id if only one nameservice is configured
Posted by "Eli Collins (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins resolved HDFS-2894.
-------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Thanks for the review Todd. I've committed this.
> HA: automatically determine the nameservice Id if only one nameservice is configured
> ------------------------------------------------------------------------------------
>
> Key: HDFS-2894
> URL: https://issues.apache.org/jira/browse/HDFS-2894
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-2894.txt, hdfs-2894.txt
>
>
> The SecondaryNameNode should log a message and refuse to start if HA is enabled since the StandbyNode checkpoints by default and IIRC we have not yet enabled the ability to have multiple checkpointers in the NN.
> On the HA branch the 2NN does not currently start from start-dfs.sh because getconf -secondarynamenodes claims the http-address is not configured even though it is, though this seems like a bug, in branch 23 getconf will correctly return localhost:50090.
> {noformat}
> <property>
> <name>dfs.namenode.secondary.http-address</name>
> <value>localhost:50090</value>
> </property>
> </configuration>
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs getconf -secondarynamenodes
> Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2894) HA: automatically determine the
nameservice Id if only one nameservice is configured
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201089#comment-13201089 ]
Todd Lipcon commented on HDFS-2894:
-----------------------------------
- Did you test the non-HA federated config as well?
- Rather than adding the new "shutdown" and "isShutdown" boolean, I think it's better to just change the "Cannot use SecondaryNameNode in an HA cluster" error message to be a thrown exception -- it's an exceptional condition after all.
> HA: automatically determine the nameservice Id if only one nameservice is configured
> ------------------------------------------------------------------------------------
>
> Key: HDFS-2894
> URL: https://issues.apache.org/jira/browse/HDFS-2894
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-2894.txt
>
>
> The SecondaryNameNode should log a message and refuse to start if HA is enabled since the StandbyNode checkpoints by default and IIRC we have not yet enabled the ability to have multiple checkpointers in the NN.
> On the HA branch the 2NN does not currently start from start-dfs.sh because getconf -secondarynamenodes claims the http-address is not configured even though it is, though this seems like a bug, in branch 23 getconf will correctly return localhost:50090.
> {noformat}
> <property>
> <name>dfs.namenode.secondary.http-address</name>
> <value>localhost:50090</value>
> </property>
> </configuration>
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs getconf -secondarynamenodes
> Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira