Hadoop - flush() files

[Arvind Sharma <ar...@yahoo.com>]

Hi,

I was wondering if anyone here have stared using (or has been using) the newer Hadoop versions (0-20.1 ??? ) - which provides API for flushing out any open files on the HDFS ?

Are there any known issues I should be aware of ?

Thanks!
Arvind



      

Re: Hadoop - flush() files

[Aaron Kimball <aa...@cloudera.com>]

sync doesn't work. See http://issues.apache.org/jira/browse/HDFS-200
- Aaron


On Tue, Aug 18, 2009 at 8:33 AM, Arvind Sharma <ar...@yahoo.com> wrote:

> Just checking again :-)
>
> I have a setup where I am using Hadoop0-19.2 and the data files are kept
> open for a long time. I want them to be sync'd to the HDFS now and then, to
> avoid any data loss.
>
> In one of the last HUG, somebody mentioned  FSDataOutputStream.sync()
> method. But there were some known issues with that.
>
> Has anyone experienced any problem while using the sync() method ?
>
> Arvind
>
>
>
>
> ________________________________
>
>
> Hi,
>
> I was wondering if anyone here have stared using (or has been using) the
> newer Hadoop versions (0-20.1 ??? ) - which provides API for flushing out
> any open files on the HDFS ?
>
> Are there any known issues I should be aware of ?
>
> Thanks!
> Arvind
>
>
>
>

Re: Hadoop - flush() files

[Arvind Sharma <ar...@yahoo.com>]

Just checking again :-)

I have a setup where I am using Hadoop0-19.2 and the data files are kept open for a long time. I want them to be sync'd to the HDFS now and then, to avoid any data loss.

In one of the last HUG, somebody mentioned  FSDataOutputStream.sync() method. But there were some known issues with that.  

Has anyone experienced any problem while using the sync() method ? 

Arvind




________________________________


Hi,

I was wondering if anyone here have stared using (or has been using) the newer Hadoop versions (0-20.1 ??? ) - which provides API for flushing out any open files on the HDFS ?

Are there any known issues I should be aware of ?

Thanks!
Arvind