You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Eli Collins (Created) (JIRA)" <ji...@apache.org> on 2012/02/09 00:15:02 UTC
[jira] [Created] (HDFS-2918) HA: dfsadmin should failover like
other clients
HA: dfsadmin should failover like other clients
-----------------------------------------------
Key: HDFS-2918
URL: https://issues.apache.org/jira/browse/HDFS-2918
Project: Hadoop HDFS
Issue Type: Sub-task
Components: ha
Affects Versions: HA branch (HDFS-1623)
Reporter: Eli Collins
dfsadmin currently always uses the first namenode rather than failing over. It should failover like other clients, unless fs specifies a specific namenode.
{noformat}
hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs haadmin -failover nn1 nn2
Failover from nn1 to nn2 successful
# nn2 is 8022
hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode enter
Safe mode is ON
hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -safemode get
Safe mode is OFF
hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode get
Safe mode is ON
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2918) HA: Update HA docs to cover dfsadmin
Posted by "Eli Collins (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-2918:
------------------------------
Summary: HA: Update HA docs to cover dfsadmin (was: HA: dfsadmin should failover like other clients)
Yup, I filed HDFS-2922 for that with a patch. We need to update the HA docs to cover the behavior of dfsadmin (eg which operations failover, which don't and why, which should be performed only on the active, which are OK to do on the standby). Once we can name the specific namenode IDs in dfsadmin (HDFS-2916) I think it will be more clear / easy for users. Re-purposing this jira to update the docs.
> HA: Update HA docs to cover dfsadmin
> ------------------------------------
>
> Key: HDFS-2918
> URL: https://issues.apache.org/jira/browse/HDFS-2918
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
>
> dfsadmin currently always uses the first namenode rather than failing over. It should failover like other clients, unless fs specifies a specific namenode.
> {noformat}
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs haadmin -failover nn1 nn2
> Failover from nn1 to nn2 successful
> # nn2 is 8022
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode enter
> Safe mode is ON
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -safemode get
> Safe mode is OFF
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode get
> Safe mode is ON
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2918) HA: dfsadmin should failover like
other clients
Posted by "Uma Maheswara Rao G (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204242#comment-13204242 ]
Uma Maheswara Rao G commented on HDFS-2918:
-------------------------------------------
Hi Eli,
I spent some time on this issue. I dont see any problem with DFSAdmin.
{noformat}
2012-02-09 09:11:29,282 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(106)) - Exception while invoking getStats of class org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB after 0 *fail over attempts. Trying to fail over immediately.*
Configured Capacity: 190744240128 (177.64 GB)
Present Capacity: 74658306048 (69.53 GB)
DFS Remaining: 74658299904 (69.53 GB)
DFS Used: 6144 (6 KB)
DFS Used%: 0%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
-------------------------------------------------
Datanodes available: 3 (3 total, 0 dead)
{noformat}
The problem with the API categeory from Namenode. Currently safemode API are allowing on standby also. That is tyhe rason it won't do any failover. Otherwise DFSAdmin uses just not FileSystem object to invoke APIs. So, it should be able to do failover normally if we configure HA related configurations.
Addressing TODO in NameNodeRPCServer should solve this issue.
{quote}
// TODO:HA decide on OperationCategory for this
{quote}
> HA: dfsadmin should failover like other clients
> -----------------------------------------------
>
> Key: HDFS-2918
> URL: https://issues.apache.org/jira/browse/HDFS-2918
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
>
> dfsadmin currently always uses the first namenode rather than failing over. It should failover like other clients, unless fs specifies a specific namenode.
> {noformat}
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs haadmin -failover nn1 nn2
> Failover from nn1 to nn2 successful
> # nn2 is 8022
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode enter
> Safe mode is ON
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -safemode get
> Safe mode is OFF
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode get
> Safe mode is ON
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2918) HA: dfsadmin should failover like
other clients
Posted by "Eli Collins (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins reassigned HDFS-2918:
---------------------------------
Assignee: Eli Collins
> HA: dfsadmin should failover like other clients
> -----------------------------------------------
>
> Key: HDFS-2918
> URL: https://issues.apache.org/jira/browse/HDFS-2918
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
>
> dfsadmin currently always uses the first namenode rather than failing over. It should failover like other clients, unless fs specifies a specific namenode.
> {noformat}
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs haadmin -failover nn1 nn2
> Failover from nn1 to nn2 successful
> # nn2 is 8022
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode enter
> Safe mode is ON
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -safemode get
> Safe mode is OFF
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode get
> Safe mode is ON
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2918) HA: Update HA docs to cover dfsadmin
Posted by "Aaron T. Myers (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HDFS-2918:
---------------------------------
Target Version/s: 0.24.0 (was: HA branch (HDFS-1623))
Affects Version/s: (was: HA branch (HDFS-1623))
0.24.0
Converting to top-level issue with commit of HDFS-1623.
> HA: Update HA docs to cover dfsadmin
> ------------------------------------
>
> Key: HDFS-2918
> URL: https://issues.apache.org/jira/browse/HDFS-2918
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: 0.24.0
> Reporter: Eli Collins
> Assignee: Eli Collins
>
> dfsadmin currently always uses the first namenode rather than failing over. It should failover like other clients, unless fs specifies a specific namenode.
> {noformat}
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs haadmin -failover nn1 nn2
> Failover from nn1 to nn2 successful
> # nn2 is 8022
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode enter
> Safe mode is ON
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -safemode get
> Safe mode is OFF
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode get
> Safe mode is ON
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2918) HA: Update HA docs to cover dfsadmin
Posted by "Aaron T. Myers (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HDFS-2918:
---------------------------------
Issue Type: Improvement (was: Sub-task)
Parent: (was: HDFS-1623)
> HA: Update HA docs to cover dfsadmin
> ------------------------------------
>
> Key: HDFS-2918
> URL: https://issues.apache.org/jira/browse/HDFS-2918
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ha
> Affects Versions: 0.24.0
> Reporter: Eli Collins
> Assignee: Eli Collins
>
> dfsadmin currently always uses the first namenode rather than failing over. It should failover like other clients, unless fs specifies a specific namenode.
> {noformat}
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs haadmin -failover nn1 nn2
> Failover from nn1 to nn2 successful
> # nn2 is 8022
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode enter
> Safe mode is ON
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -safemode get
> Safe mode is OFF
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode get
> Safe mode is ON
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2918) HA: Update HA docs to cover dfsadmin
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins reassigned HDFS-2918:
---------------------------------
Assignee: (was: Eli Collins)
> HA: Update HA docs to cover dfsadmin
> ------------------------------------
>
> Key: HDFS-2918
> URL: https://issues.apache.org/jira/browse/HDFS-2918
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ha
> Affects Versions: 0.24.0
> Reporter: Eli Collins
>
> dfsadmin currently always uses the first namenode rather than failing over. It should failover like other clients, unless fs specifies a specific namenode.
> {noformat}
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs haadmin -failover nn1 nn2
> Failover from nn1 to nn2 successful
> # nn2 is 8022
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode enter
> Safe mode is ON
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -safemode get
> Safe mode is OFF
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode get
> Safe mode is ON
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2918) HA: Update HA docs to cover dfsadmin
Posted by "Uma Maheswara Rao G (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204326#comment-13204326 ]
Uma Maheswara Rao G commented on HDFS-2918:
-------------------------------------------
Ok, thanks Eli.
> HA: Update HA docs to cover dfsadmin
> ------------------------------------
>
> Key: HDFS-2918
> URL: https://issues.apache.org/jira/browse/HDFS-2918
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Eli Collins
> Assignee: Eli Collins
>
> dfsadmin currently always uses the first namenode rather than failing over. It should failover like other clients, unless fs specifies a specific namenode.
> {noformat}
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs haadmin -failover nn1 nn2
> Failover from nn1 to nn2 successful
> # nn2 is 8022
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode enter
> Safe mode is ON
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -safemode get
> Safe mode is OFF
> hadoop-0.24.0-SNAPSHOT $ ./bin/hdfs dfsadmin -fs localhost:8022 -safemode get
> Safe mode is ON
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira