[jira] [Resolved] (METAMODEL-219) HdfsResource's FileSystem should be createable based on custom core-site.xml and hdfs-site.xml files

["Kasper Sørensen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/METAMODEL-219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kasper Sørensen resolved METAMODEL-219.
---------------------------------------
    Resolution: Fixed
      Assignee: Kasper Sørensen

Fixed via https://github.com/apache/metamodel/pull/78

> HdfsResource's FileSystem should be createable based on custom core-site.xml and hdfs-site.xml files
> ----------------------------------------------------------------------------------------------------
>
>                 Key: METAMODEL-219
>                 URL: https://issues.apache.org/jira/browse/METAMODEL-219
>             Project: Apache MetaModel
>          Issue Type: Improvement
>    Affects Versions: 4.4.1
>            Reporter: Kasper Sørensen
>            Assignee: Kasper Sørensen
>
> Today we always assume a very vanilla HDFS/Hadoop configuration which is based on defaults and overriding the one property {code}fs.defaultFS{code}. This behaviour is very limiting for the use of HdfsResource since many other custom properties may be needed.
> Instead we should offer a way where users of the class can specify a directory with the Hadoop configuration files. This could eventually also allow for a more flexibly style of representing the HDFS path, namely without the hostname and port (which would simply be dynamically loaded from the configuration file), like this:
> {code}
> hdfs:///path/to/file
> {code}
> How the configuration directory is provided should probably also be flexible. I would suggest that we offer three ways:
> (1) a new constructor option to provide the directory, and
> (2) a system property that will be checked, and
> (3) an environmental variable that will be checked (I suggest HADOOP_CONF_DIR and YARN_CONF_DIR to replicate common environmental variable names for this).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)