You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by "Markus Weimer (JIRA)" <ji...@apache.org> on 2015/08/06 00:18:05 UTC

[jira] [Commented] (REEF-492) Implement FileSystem using (HDFS) commands

    [ https://issues.apache.org/jira/browse/REEF-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14659072#comment-14659072 ] 

Markus Weimer commented on REEF-492:
------------------------------------

I filled [#339|https://github.com/apache/incubator-reef/pull/339] with an initial implementation. Note that it *does not* implement any of the {{Stream}} APIs of {{IFileSystem}} as they are impossible to make safe with external commands.

> Implement FileSystem using (HDFS) commands
> ------------------------------------------
>
>                 Key: REEF-492
>                 URL: https://issues.apache.org/jira/browse/REEF-492
>             Project: REEF
>          Issue Type: Sub-task
>          Components: REEF.NET
>            Reporter: Markus Weimer
>            Assignee: Markus Weimer
>
> With {{IFileSystem}} defined and implemented for local file access, the next step is to support distributed file systems like HDFS. As there is no native HDFS support in .NET, a simple approach is to implement {{IFileSystem}} using external commands, e.g. {{hadoop fs -ls}} and friends. Ideally, the set of commands used should be configurable to allow for different / future file systems to be supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)