You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Hemant Bhanawat <he...@vmware.com> on 2013/07/09 09:25:23 UTC

Regarding hsync

Hi, 

I am currently working on hadoop version 2.0.*. 

Currently, hsync does not update the file size on namenode. So, if my process dies after calling hsync but before calling file close, the file is left with an inconsistent file size. I would like to fix this file size. Is there a way to do that? A workaround that I have come across is to open the file stream in append mode and close it. This fixes the file size on the namenode. Is it a reliable solution? 

Thanks, 
Hemant

Re: Regarding hsync

Posted by Jing Zhao <ji...@hortonworks.com>.
Hi Hemant,

    HDFS-4213 (https://issues.apache.org/jira/browse/HDFS-4213) may be
the one you're looking for. In general, an hsync which can update the
file size on NN is enabled. You may call
"hsync(EnumSet.of(SyncFlag.UPDATE_LENGTH))" to do that.

Thanks,
-Jing

On Thu, Jul 11, 2013 at 1:38 AM, Hemant Bhanawat <he...@vmware.com> wrote:
> Hi,
>
> Any help?
>
> Thanks in advance,
> Hemant
>
> ----- Original Message -----
>
> From: "Hemant Bhanawat" <he...@vmware.com>
> To: hdfs-dev@hadoop.apache.org
> Sent: Tuesday, July 9, 2013 12:55:23 PM
> Subject: Regarding hsync
>
> Hi,
>
> I am currently working on hadoop version 2.0.*.
>
> Currently, hsync does not update the file size on namenode. So, if my process dies after calling hsync but before calling file close, the file is left with an inconsistent file size. I would like to fix this file size. Is there a way to do that? A workaround that I have come across is to open the file stream in append mode and close it. This fixes the file size on the namenode. Is it a reliable solution?
>
> Thanks,
> Hemant
>

Re: Regarding hsync

Posted by Hemant Bhanawat <he...@vmware.com>.
Hi, 

Any help? 

Thanks in advance, 
Hemant 

----- Original Message -----

From: "Hemant Bhanawat" <he...@vmware.com> 
To: hdfs-dev@hadoop.apache.org 
Sent: Tuesday, July 9, 2013 12:55:23 PM 
Subject: Regarding hsync 

Hi, 

I am currently working on hadoop version 2.0.*. 

Currently, hsync does not update the file size on namenode. So, if my process dies after calling hsync but before calling file close, the file is left with an inconsistent file size. I would like to fix this file size. Is there a way to do that? A workaround that I have come across is to open the file stream in append mode and close it. This fixes the file size on the namenode. Is it a reliable solution? 

Thanks, 
Hemant