[jira] [Created] (HBASE-15860) Improvements for HBASE-14280 - Fixing Bulkload for HDFS HA Clusters

["Wellington Chevreuil (JIRA)" <ji...@apache.org>]

Wellington Chevreuil created HBASE-15860:
--------------------------------------------

             Summary: Improvements for HBASE-14280 - Fixing Bulkload for HDFS HA Clusters
                 Key: HBASE-15860
                 URL: https://issues.apache.org/jira/browse/HBASE-15860
             Project: HBase
          Issue Type: Improvement
          Components: util
    Affects Versions: 1.0.0
            Reporter: Wellington Chevreuil
            Priority: Minor


HBASE-14280 introduced fix for bulkload failures when referring a remote cluster name service id if "bulkloading" from a HA cluster.

HBASE-14280 solution on *FSHDFSUtils.getNNAddresses* was to invoke *DFSUtil.getNNServiceRpcAddressesForCluster* instead of *DFSUtil.getNNServiceRpcAddresses*. This works for hadoop 2.6 and above.

Proposed change here is to use "*DFSUtil.getRpcAddressesForNameserviceId*" instead, which already returns only addresses for specific nameservice informed. This is available since hadoop 2.4.

Sample proposal on FSHDFSUtils.getNNAddresses:
...
{noformat}
     String nameServiceId = serviceName.split(":")[1];
        if (dfsUtilClazz == null) {
          dfsUtilClazz = Class.forName("org.apache.hadoop.hdfs.DFSUtil");
        }
        if (getNNAddressesMethod == null) {
          getNNAddressesMethod =
              dfsUtilClazz.getMethod("getRpcAddressesForNameserviceId", Configuration.class,
                String.class, String.class);
        }

        Map<String, InetSocketAddress> nnMap =
            (Map<String, InetSocketAddress>) getNNAddressesMethod
                        .invoke(null, conf, nameServiceId, null);
        for (Map.Entry<String, InetSocketAddress> e2 : nnMap.entrySet()) {
            InetSocketAddress addr = e2.getValue();
            addresses.add(addr);
        }
...
{noformat}

Will also add test conditions for *FSHDFSUtils.isSameHdfs* to verify scenario when multiple name service ids are defined.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)