[jira] [Created] (DRILL-6520) Setting fs.defaultFS in core-site.xml causes startup crash

["Paul Rogers (JIRA)" <ji...@apache.org>]

Paul Rogers created DRILL-6520:
----------------------------------

             Summary: Setting fs.defaultFS in core-site.xml causes startup crash
                 Key: DRILL-6520
                 URL: https://issues.apache.org/jira/browse/DRILL-6520
             Project: Apache Drill
          Issue Type: Bug
    Affects Versions: 1.13.0
            Reporter: Paul Rogers


In a Hadoop HDFS environment, the docs. say to set the HDFS configuration in the storage plugin config.

However, when Drill runs on an existing HDFS cluster, one must often provide more than the simple HDFS URL. Particularly in a secure cluster, other configuration settings are also needed. At present, these must be copied out of the HDFS config files into the Drill storage plugin config, and the two must be updated in tandem. Clearly less than ideal.

In experimenting with better integration, I added my HDFS {{core-site.xml}} file to the class path, hoping Drill would pick up the settings from the Hadoop config files that I had placed on the class path. Instead, Drill failed to start.

{noformat}
Exception in thread "main" java.lang.NullPointerException
        at org.apache.drill.exec.coord.zk.ZKClusterCoordinator.update(ZKClusterCoordinator.java:218)
        at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:228)
        at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:401)
        at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:372)
        at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:368)
{noformat}

Modify the {{core-site.xml}} file to comment out the {{fs.defaultFS}} property.

{noformat}
<configuration>
<!--
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://myNameNode:8020</value>
  </property>
-->
</configuration>
{noformat}

Now Drill starts fine.

The "myNameNode" is not the real name. But, the actual value here is identical to the value set in the dfs "connection" property, which works fine by itself without {{core-site.xml}}.

Expected that Drill would allow the common Hadoop pattern of setting {{fs.defaultFS}} in {{core-site.xml}}. At the very least, Drill should not crash when this (or any other) property is set in {{core-site.xml}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)