[jira] [Commented] (HDFS-2839) HA: Remove need for client configuration of nameservice ID

["Sanjay Radia (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227729#comment-13227729 ] 

Sanjay Radia commented on HDFS-2839:
------------------------------------


Lets look at the bigger picture of the various connections to the NN, the requirements and possible solutions.
*Requirements*
# A DfsClient must  be redirected  when failover occurs
# Http client must  be redirected  when failover occurs
# Delegation tokens
** delegation tokens should work for the new active, even if issued by previous active
** renewal - renewer must be able to renew at the new active transparently
# Cross cluster access must be supported
# URI compatibility when transitioning nonHA-to-HA and HA-to-nonHA:  A URI to NN that contains a name (not ip address) should continue to work.
# Solution must work in
** Experimental seeting where DNS changes may not be permitted and also
** Production setting where the customer wants to use DNS to locate NNs.


*Solutions*
*Solution A: IP Failover*
* Ip Failover satisfies all requirements as long as clients (DFSClient, Token renewer etc)  reconnect on failure 

*Solution B:*
* NN has a logical name that is authority in URI (hdfs://logicalName/path)
* Logical name  is mapped to Active and Standby’s IP (issues discussed below)

* *Addressing the requirements in Solution B*
*# DNS Client uses the multiple IPs
*# HTTP - HTTP Proxy or DNS Round robin with standby redirecting
*# Delegation token
*** Delegation token secret is shared with standby
*** Renewer – logical name is CanonicalService name and a proxy provider object maps to the IPs. 
*# Cross cluster – The logical to IP mapping must be available across clusters
*# The non-HA NN’s DNS name is  the logical name of NN.
*# Supply 2 resolvers – config file resolver and DNS resolver. Later we can add ZK resolver.


*What is the Logical Name?*

Since the logical name is being exposed it should be the volumeName. Note this is distinct from NameServiceId since the NameServiceId was supposed to allow a name server to store multiple Volumes.

* Proposal: Add VolumeName to config
** Currently a  1-1 mapping between Volume Name and NameServiceId since a NameService can host only one volume.
** Why do this now? the logical name is being exposed via the URI -  we should select the right terminology now
** How are Volume names mapped to IPs? (See below)



*Is the notion of Logical name used when using the IP failover Solution*
        Yes – the LogicalName is the DNS name that maps to a single IP which failsover


*Where is the LogicalName -to-IPs Mapping Stored?*
* DNS-Resolver - DNS-Name to IP or IPs (single IP in case of IP-Failover)
* ConfigFile-resolver - the mapping in the config file - this config file will need to be be available in all clusters, for all clusters to allow cross cluster access.
This configuration file will be in addition to the client side mount tables. May want to consider having a single conf file that resolves mount points as well as logical names?
* Zookeeper-resolver (Volume management at some point should move to Zookeeper )
            Scalability of zookeeper may need to be addressed.



                
> HA: Remove need for client configuration of nameservice ID
> ----------------------------------------------------------
>
>                 Key: HDFS-2839
>                 URL: https://issues.apache.org/jira/browse/HDFS-2839
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: ha, hdfs client, name-node
>    Affects Versions: 0.24.0
>            Reporter: Jitendra Nath Pandey
>            Assignee: Jitendra Nath Pandey
>
> The fully qualified path from an ha cluster, won't be usable from a different cluster that doesn't know about that particular namespace id.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira