HDFS: how to append

[Cagdas Gerede <ca...@gmail.com>]

The HDFS documentation says it is possible to append to an HDFS file.

In org.apache.hadoop.dfs.DistributedFileSystem class,
there is no method to open an existing file for writing (there are
methods for reading).
Only similar methods are "create" methods which return FSDataOutputStream.
When I look at FSDataOutputStream class, it seems there is no "append"
method, and
all "write" methods overwrite an existing file or return an error if
such a file exists.

Does anybody know how to append to a file in HDFS?
I appreciate your help.
Thanks,

Cagdas

RE: HDFS: how to append

[dhruba Borthakur <dh...@yahoo-inc.com>]

HDFS files, once created, cannot be modified in any way. Appends to HDFS
files will probably be supported in a future release in the next couple
of months.

Thanks,
dhruba

-----Original Message-----
From: Cagdas Gerede [mailto:cagdas.gerede@gmail.com] 
Sent: Tuesday, March 18, 2008 9:53 AM
To: core-user@hadoop.apache.org
Subject: HDFS: how to append

The HDFS documentation says it is possible to append to an HDFS file.

In org.apache.hadoop.dfs.DistributedFileSystem class,
there is no method to open an existing file for writing (there are
methods for reading).
Only similar methods are "create" methods which return
FSDataOutputStream.
When I look at FSDataOutputStream class, it seems there is no "append"
method, and
all "write" methods overwrite an existing file or return an error if
such a file exists.

Does anybody know how to append to a file in HDFS?
I appreciate your help.
Thanks,

Cagdas