You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Stephen Sisk (JIRA)" <ji...@apache.org> on 2017/04/20 16:01:04 UTC

[jira] [Created] (BEAM-2030) Implement beam FileSystem's copy()

Stephen Sisk created BEAM-2030:
----------------------------------

             Summary: Implement beam FileSystem's copy()
                 Key: BEAM-2030
                 URL: https://issues.apache.org/jira/browse/BEAM-2030
             Project: Beam
          Issue Type: Sub-task
          Components: sdk-java-extensions
            Reporter: Stephen Sisk
            Assignee: Davor Bonaci


Beam's FileSystem has a copy() command, however I can't find a good analog in Hadoop's FileSystem. https://hadoop.apache.org/docs/r2.8.0/api/org/apache/hadoop/fs/FileSystem.html shows lots of copy to/from local files, but no "copy between these two arbitrary paths".

cc [~davor] [~dhalperi@google.com] did either of you have thoughts about this? I don't think that it makes sense to have beam stream data from one node just so it can write it back to another node. (it could be an extension method, but I'd want to make it obvious that it's the inefficient version of things)

My default answer here is to throw an unimplemented exception on copy or to remove it from the BFS interface altogether.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)