[jira] [Commented] (METAMODEL-220) Add dynamic scheme name to HdfsResource

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METAMODEL-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051704#comment-15051704 ] 

ASF GitHub Bot commented on METAMODEL-220:
------------------------------------------

GitHub user kaspersorensen opened a pull request:

    https://github.com/apache/metamodel/pull/79

    HdfsResource with non-"hdfs" scheme

    Suggested fix for METAMODEL-220.
    
    I based this PR on the branch that is represented in PR #78, so please review that one first. There is only the last commit in the branch that adds to the work in #78.
    
    In this PR I've added a scheme to the HdfsResource. To make the scheme dynamic I also needed to improve the URI parsing, which anyways ended up being a very good improvement.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kaspersorensen/metamodel METAMODEL-220-hadoop-filesystem-other-schemes

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/metamodel/pull/79.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #79
    
----
commit 14b63cdc262d40b0d6469de6cfc12103aa6239e3
Author: Kasper Sørensen <i....@gmail.com>
Date:   2015-12-10T20:59:29Z

    Moved HDFS resource stream classes to separate files.

commit 1e5dfe34d65cb9880a8ccd0e6adaa3a765640527
Author: Kasper Sørensen <i....@gmail.com>
Date:   2015-12-10T21:06:09Z

    Improved FileHelper.safeClose(...) method by using AutoCloseable
    
    ... which is now super-interface for Closeable, Connection, Statement,
    ResultSet and more.

commit 7b5fb0c09c5dac7af3a2f2ff0e2aef8e0bbf2012
Author: Kasper Sørensen <i....@gmail.com>
Date:   2015-12-10T21:24:39Z

    METAMODEL-219: Added loading of core-site.xml and hdfs-site.xml

commit 28537a1a0f5d40f67a64f902cdf3a32f0dfd5d3f
Author: Kasper Sørensen <i....@gmail.com>
Date:   2015-12-10T21:45:42Z

    METAMODEL-220: Added dynamic schemes to HdfsResource

----


> Add dynamic scheme name to HdfsResource
> ---------------------------------------
>
>                 Key: METAMODEL-220
>                 URL: https://issues.apache.org/jira/browse/METAMODEL-220
>             Project: Apache MetaModel
>          Issue Type: Improvement
>    Affects Versions: 4.4.1
>            Reporter: Kasper Sørensen
>
> We should offer a way to use other scheme's than "hdfs" for Hadoop FileSystem based resources.
> NOTE: This issue is blocked by/dependent on METAMODEL-219 because the file system configuration would need to be loaded from configuration files.
> I have heard of at least a few variants that supposedly support the same API:
>  * OpenStack Swift (swift://...)
>  * Amazon EMR FS (emrfs://...)
>  * MapR file system (maprfs://...)
> And there are probably more.
> Today "hdfs://" is hardcoded into HdfsResource and clearly it should be a field which the user can select a different value for. I suggest to keep it simply as a string and to also add a couple of constants in the class to make it easy for users to pick between the usual candidates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)