[jira] [Commented] (AVRO-1035) Add the possibility to append to existing avro files

["Michael Malak (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573711#comment-13573711 ] 

Michael Malak commented on AVRO-1035:
-------------------------------------

harsh@cloudera.com has provided example code on how to accomplish HDFS Avro append at https://gist.github.com/QwertyManiac/4724582
                
> Add the possibility to append to existing avro files  
> ------------------------------------------------------
>
>                 Key: AVRO-1035
>                 URL: https://issues.apache.org/jira/browse/AVRO-1035
>             Project: Avro
>          Issue Type: New Feature
>            Reporter: Vyacheslav Zholudev
>
> Currently it is not possible to append to avro files that were written and closed. 
> Here is a Scott Carey's reply on the mailing list:
> {quote}
> It is not possible without modifying DataFileWriter. Please open a JIRA
> ticket.  
> It could not simply append to an OutputStream, since it must either:
> * Seek to the start to validate the schemas match and find the sync
> marker, or
> * Trust that the schemas match and find the sync marker from the last block
> DataFileWriter cannot refer to Hadoop classes such as FileSystem, but we
> could add something to the mapred module that takes a Path and FileSystem
> and returns
> something that implemements an interface that DataFileWriter can append
> to.  This would be something that is both a
> http://avro.apache.org/docs/1.6.2/api/java/org/apache/avro/file/SeekableInp
> ut.html
> and an OutputStream, or has both an InputStream from the start of the
> existing file and an OutputStream at the end.
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira