[jira] [Commented] (BEAM-110) Use SDK implementation for HadoopIO in contrib/hadoop

["Amit Sela (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15689495#comment-15689495 ] 

Amit Sela commented on BEAM-110:
--------------------------------

[~davor] I think that concerning IOs it's clear that "direct translation" is (mostly) not feasible due to the complexity and variety of IOs.
The Spark runner supports the Source API, and besides data locality optimizations (pending proper HDFS support) I don't see reason to keep this.
I didn't remove this code yet (though it;s unused), but once we have HDFS support I plan to.

Can I close this ticket ? 

> Use SDK implementation for HadoopIO in contrib/hadoop
> -----------------------------------------------------
>
>                 Key: BEAM-110
>                 URL: https://issues.apache.org/jira/browse/BEAM-110
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-spark
>            Reporter: Amit Sela
>
> The Spark runner currently uses it's own implementation for HadoopIO but It would be better if it used the SDK's implementation so it could be used with any runner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)