You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by "kulkarni.swarnim@gmail.com" <ku...@gmail.com> on 2013/07/10 18:35:44 UTC
Flushing data to an avro file
Hello,
I am using the DataFileWriter to create an avro file and was wondering is
doing a DataFileWriter#close the only way of committing the data to the
file? I tried doing a DataFileWriter#flush but no dice. I wanted a
capability where I can keep appending to an avro file without the need to
close it and reopen it every single time.
Thanks,
Re: Flushing data to an avro file
Posted by Doug Cutting <cu...@apache.org>.
The flush of the wrapper stream should also flush its underlying stream.
Doug
On Wed, Jul 10, 2013 at 9:52 AM, kulkarni.swarnim@gmail.com
<ku...@gmail.com> wrote:
> Looking at the source for flush on the DataFileWriter[1], it looks like we
> do not flush the actual outputstream that we are writing data to. Should we
> possibly have an out.flush() in there as well?
>
> [1]
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.avro/avro/1.7.4/org/apache/avro/file/DataFileWriter.java#DataFileWriter.flush%28%29
>
>
> On Wed, Jul 10, 2013 at 11:41 AM, Doug Cutting <cu...@apache.org> wrote:
>>
>> On Wed, Jul 10, 2013 at 9:35 AM, kulkarni.swarnim@gmail.com
>> <ku...@gmail.com> wrote:
>> > I tried doing a DataFileWriter#flush but no dice.
>>
>> That should work. How did it fail?
>>
>> Doug
>
>
>
>
> --
> Swarnim
Re: Flushing data to an avro file
Posted by "kulkarni.swarnim@gmail.com" <ku...@gmail.com>.
Looking at the source for flush on the DataFileWriter[1], it looks like we
do not flush the actual outputstream that we are writing data to. Should we
possibly have an out.flush() in there as well?
[1]
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.avro/avro/1.7.4/org/apache/avro/file/DataFileWriter.java#DataFileWriter.flush%28%29
On Wed, Jul 10, 2013 at 11:41 AM, Doug Cutting <cu...@apache.org> wrote:
> On Wed, Jul 10, 2013 at 9:35 AM, kulkarni.swarnim@gmail.com
> <ku...@gmail.com> wrote:
> > I tried doing a DataFileWriter#flush but no dice.
>
> That should work. How did it fail?
>
> Doug
>
--
Swarnim
Re: Flushing data to an avro file
Posted by Doug Cutting <cu...@apache.org>.
On Wed, Jul 10, 2013 at 9:35 AM, kulkarni.swarnim@gmail.com
<ku...@gmail.com> wrote:
> I tried doing a DataFileWriter#flush but no dice.
That should work. How did it fail?
Doug