You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Wasim Bari <wa...@msn.com> on 2008/11/13 20:40:13 UTC
Anything like RandomAccessFile in Hadoop FS ?
Hi,
Is there any Utility for Hadoop files which can work same as RandomAccessFile in Java ?
Thanks,
Wasim
Re: Anything like RandomAccessFile in Hadoop FS ?
Posted by Dhruba Borthakur <dh...@gmail.com>.
One can open a file and then seek to an offset and then start reading
from there. For writing, one can write only to the end of an existing
file using FileSystem.append().
hope this helps,
dhruba
On Thu, Nov 13, 2008 at 1:24 PM, Tsz Wo (Nicholas), Sze
<s2...@yahoo.com> wrote:
> Append is going to be available in 0.19 (not yet released). There are new FileSystem APIs for append, e.g.
>
> //FileSysetm.java
> public abstract FSDataOutputStream append(Path f, int bufferSize,
> Progressable progress) throws IOException;
>
> Nicholas Sze
>
>
>
>
> ----- Original Message ----
>> From: Bryan Duxbury <br...@rapleaf.com>
>> To: Wasim Bari <wa...@msn.com>
>> Cc: hadoop-user@lucene.apache.org
>> Sent: Thursday, November 13, 2008 1:11:57 PM
>> Subject: Re: Anything like RandomAccessFile in Hadoop FS ?
>>
>> I'm not sure off hand. Maybe someone else can point you in the right direction?
>>
>> On Nov 13, 2008, at 1:09 PM, Wasim Bari wrote:
>>
>> > Hi,
>> > Thanks for reply.
>> >
>> > HDFS supports append file. How can I do this ?
>> > I tried to look API under fileSystem create method but couldn't find.
>> >
>> > Thanks for ur help.
>> >
>> > Wasim
>> >
>> > --------------------------------------------------
>> > From: "Bryan Duxbury"
>> > Sent: Thursday, November 13, 2008 9:48 PM
>> > To:
>> > Subject: Re: Anything like RandomAccessFile in Hadoop FS ?
>> >
>> >> If you mean a file where you can write anywhere, then no. HDFS is streaming
>> only. If you want to read from anywhere, then no problem - just use seek() and
>> then read.
>> >> On Nov 13, 2008, at 11:40 AM, Wasim Bari wrote:
>> >>> Hi,
>> >>> Is there any Utility for Hadoop files which can work same as
>> RandomAccessFile in Java ?
>> >>> Thanks,
>> >>>
>> >>> Wasim
>
>
Re: Anything like RandomAccessFile in Hadoop FS ?
Posted by "Tsz Wo (Nicholas), Sze" <s2...@yahoo.com>.
Append is going to be available in 0.19 (not yet released). There are new FileSystem APIs for append, e.g.
//FileSysetm.java
public abstract FSDataOutputStream append(Path f, int bufferSize,
Progressable progress) throws IOException;
Nicholas Sze
----- Original Message ----
> From: Bryan Duxbury <br...@rapleaf.com>
> To: Wasim Bari <wa...@msn.com>
> Cc: hadoop-user@lucene.apache.org
> Sent: Thursday, November 13, 2008 1:11:57 PM
> Subject: Re: Anything like RandomAccessFile in Hadoop FS ?
>
> I'm not sure off hand. Maybe someone else can point you in the right direction?
>
> On Nov 13, 2008, at 1:09 PM, Wasim Bari wrote:
>
> > Hi,
> > Thanks for reply.
> >
> > HDFS supports append file. How can I do this ?
> > I tried to look API under fileSystem create method but couldn't find.
> >
> > Thanks for ur help.
> >
> > Wasim
> >
> > --------------------------------------------------
> > From: "Bryan Duxbury"
> > Sent: Thursday, November 13, 2008 9:48 PM
> > To:
> > Subject: Re: Anything like RandomAccessFile in Hadoop FS ?
> >
> >> If you mean a file where you can write anywhere, then no. HDFS is streaming
> only. If you want to read from anywhere, then no problem - just use seek() and
> then read.
> >> On Nov 13, 2008, at 11:40 AM, Wasim Bari wrote:
> >>> Hi,
> >>> Is there any Utility for Hadoop files which can work same as
> RandomAccessFile in Java ?
> >>> Thanks,
> >>>
> >>> Wasim
Re: Anything like RandomAccessFile in Hadoop FS ?
Posted by Bryan Duxbury <br...@rapleaf.com>.
I'm not sure off hand. Maybe someone else can point you in the right
direction?
On Nov 13, 2008, at 1:09 PM, Wasim Bari wrote:
> Hi,
> Thanks for reply.
>
> HDFS supports append file. How can I do this ?
> I tried to look API under fileSystem create method but couldn't find.
>
> Thanks for ur help.
>
> Wasim
>
> --------------------------------------------------
> From: "Bryan Duxbury" <br...@rapleaf.com>
> Sent: Thursday, November 13, 2008 9:48 PM
> To: <co...@hadoop.apache.org>
> Subject: Re: Anything like RandomAccessFile in Hadoop FS ?
>
>> If you mean a file where you can write anywhere, then no. HDFS is
>> streaming only. If you want to read from anywhere, then no problem
>> - just use seek() and then read.
>> On Nov 13, 2008, at 11:40 AM, Wasim Bari wrote:
>>> Hi,
>>> Is there any Utility for Hadoop files which can work same
>>> as RandomAccessFile in Java ?
>>> Thanks,
>>>
>>> Wasim
Re: Anything like RandomAccessFile in Hadoop FS ?
Posted by Bryan Duxbury <br...@rapleaf.com>.
If you mean a file where you can write anywhere, then no. HDFS is
streaming only. If you want to read from anywhere, then no problem -
just use seek() and then read.
On Nov 13, 2008, at 11:40 AM, Wasim Bari wrote:
> Hi,
> Is there any Utility for Hadoop files which can work same as
> RandomAccessFile in Java ?
> Thanks,
>
> Wasim