[jira] [Created] (PIG-4814) AvroStorage does not take namenode HA as part of schema file url

["Daniel Dai (JIRA)" <ji...@apache.org>]

Daniel Dai created PIG-4814:
-------------------------------

             Summary: AvroStorage does not take namenode HA as part of schema file url
                 Key: PIG-4814
                 URL: https://issues.apache.org/jira/browse/PIG-4814
             Project: Pig
          Issue Type: Bug
            Reporter: Daniel Dai
            Assignee: Daniel Dai
             Fix For: 0.16.0


The following script fail:
{code}
in = LOAD 'recordsOfStringArrays.avro' USING AvroStorage('', '-d -f hdfs://ml-ha/user/aeckstein/notower-rw/streamRunwaysInputSchema.avsc');
{code}

Where ml-ha is the configured HA name for namenode.

The reason is AvroStorage creates a new Configuration object in the backend, which missing the namenode HA configuration in hdfs-site.xml:
{code}
<property>
  <name>dfs.client.failover.proxy.provider.ml-ha</name>
  <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
{code}

It should use the configuration passed from frontend rather than construct a new one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)