You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Vivi Lang <sq...@gmail.com> on 2012/10/26 22:49:20 UTC
A question about reading one block
Hi guys,
I am writing a program which need the data of several specific blocks (I
know the blockinfo of those blocks), if I use the Hadoop 0.20, is there
any funtion which can help me to retrieve the content of block?
Thanks,
Vivi
Re: A question about reading one block
Posted by Harsh J <ha...@cloudera.com>.
Vivi,
There is no inbuilt function to do this, no. You'd have to write the
utility yourself.
On Thu, Nov 1, 2012 at 3:01 AM, Vivi Lang <sq...@gmail.com> wrote:
> Please ignore the previous email which I did not describe the problem
> clearly.
>
> On Wed, Oct 31, 2012 at 3:05 PM, Vivi Lang <sq...@gmail.com> wrote:
>
>> Hi, thks for the replying.
>>
>> Is there any function can help me to create a InputStream for a certain
>> Block which stored in a certain datanode
>> Thanks,
>>
>> On Sat, Oct 27, 2012 at 9:41 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>>> Vivi,
>>>
>>> Assuming you know how to get the block info out of a file's metadata,
>>> opening a file for read (with FileSystem.open API, etc.) with an
>>> offset and length set to to match that of a block, will open and read
>>> only that block.
>>>
>>> On Sat, Oct 27, 2012 at 2:19 AM, Vivi Lang <sq...@gmail.com> wrote:
>>> > Hi guys,
>>> >
>>> > I am writing a program which need the data of several specific blocks (I
>>> > know the blockinfo of those blocks), if I use the Hadoop 0.20, is there
>>> > any funtion which can help me to retrieve the content of block?
>>> >
>>> > Thanks,
>>> > Vivi
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>>
--
Harsh J
Re: A question about reading one block
Posted by Vivi Lang <sq...@gmail.com>.
Please ignore the previous email which I did not describe the problem
clearly.
On Wed, Oct 31, 2012 at 3:05 PM, Vivi Lang <sq...@gmail.com> wrote:
> Hi, thks for the replying.
>
> Is there any function can help me to create a InputStream for a certain
> Block which stored in a certain datanode
> Thanks,
>
> On Sat, Oct 27, 2012 at 9:41 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Vivi,
>>
>> Assuming you know how to get the block info out of a file's metadata,
>> opening a file for read (with FileSystem.open API, etc.) with an
>> offset and length set to to match that of a block, will open and read
>> only that block.
>>
>> On Sat, Oct 27, 2012 at 2:19 AM, Vivi Lang <sq...@gmail.com> wrote:
>> > Hi guys,
>> >
>> > I am writing a program which need the data of several specific blocks (I
>> > know the blockinfo of those blocks), if I use the Hadoop 0.20, is there
>> > any funtion which can help me to retrieve the content of block?
>> >
>> > Thanks,
>> > Vivi
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: A question about reading one block
Posted by Vivi Lang <sq...@gmail.com>.
Hi, thks for the replying.
Is there any function can help me to create a InputStream for a certain
Block which stored in a certain datanode
Thanks,
On Sat, Oct 27, 2012 at 9:41 PM, Harsh J <ha...@cloudera.com> wrote:
> Vivi,
>
> Assuming you know how to get the block info out of a file's metadata,
> opening a file for read (with FileSystem.open API, etc.) with an
> offset and length set to to match that of a block, will open and read
> only that block.
>
> On Sat, Oct 27, 2012 at 2:19 AM, Vivi Lang <sq...@gmail.com> wrote:
> > Hi guys,
> >
> > I am writing a program which need the data of several specific blocks (I
> > know the blockinfo of those blocks), if I use the Hadoop 0.20, is there
> > any funtion which can help me to retrieve the content of block?
> >
> > Thanks,
> > Vivi
>
>
>
> --
> Harsh J
>
Re: A question about reading one block
Posted by Harsh J <ha...@cloudera.com>.
Vivi,
Assuming you know how to get the block info out of a file's metadata,
opening a file for read (with FileSystem.open API, etc.) with an
offset and length set to to match that of a block, will open and read
only that block.
On Sat, Oct 27, 2012 at 2:19 AM, Vivi Lang <sq...@gmail.com> wrote:
> Hi guys,
>
> I am writing a program which need the data of several specific blocks (I
> know the blockinfo of those blocks), if I use the Hadoop 0.20, is there
> any funtion which can help me to retrieve the content of block?
>
> Thanks,
> Vivi
--
Harsh J