[jira] [Resolved] (AVRO-1244) Provide a SeekableInput implementation for FileSystem retrieved output streams

["Harsh J (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AVRO-1244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Harsh J resolved AVRO-1244.
---------------------------

    Resolution: Invalid

I missed {{FsInput}} in avro-mapred. It already implements this! :)
                
> Provide a SeekableInput implementation for FileSystem retrieved output streams
> ------------------------------------------------------------------------------
>
>                 Key: AVRO-1244
>                 URL: https://issues.apache.org/jira/browse/AVRO-1244
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>            Reporter: Harsh J
>            Priority: Minor
>
> To use the DFW#appendTo API, one needs to pass a SeekableInput interface object. Avro provides a usable utility for files that can be represented by a File object, but in the Hadoop land, HDFS and other FSes can't be represented via a File object and need a longer route to implement this interface.
> We can add a simple HadoopSeekableFSInput or so that can take Hadoop provided objects and wrap it into a SeekableInput interface ready for passing to Avro.
> I propose something of the following type:
> {code}
> public static class HadoopSeekableFSInput implements SeekableInput {
>     FSDataInputStream in;
>     long length;
>  
>     public SeekableFSInput(FSDataInputStream in, long length) {
>       this.in = in;
>       this.length = length;
>     }
>  
>     public void close() throws IOException {
>       in.close();
>     }
>  
>     public void seek(long p) throws IOException {
>       in.seek(p);
>     }
>  
>     public long tell() throws IOException {
>       return in.getPos();
>     }
>  
>     public long length() throws IOException {
>       return length;
>     }
>  
>     public int read(byte[] b, int off, int len) throws IOException {
>       return in.read(b, off, len);
>     }
>   }
> {code}
> The above can be constructed by users via a simple call such as {{new HadoopSeekableFSInput(fs.open(filePath), fs.getFileStatus(filePath).getLen())}}.
> Ideally this class should belong in the avro core module but that strictly does not depend on Hadoop-Common today, and hence somewhere else may be more suitable.
> This lets users write Avro-append code such as https://gist.github.com/QwertyManiac/4724582 more easily.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira