You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Florin P <fl...@yahoo.com> on 2011/07/28 14:33:15 UTC
How to obtain block information of a HDFS file?
Hello!
I would like to know how to obtain for a file (via Java API) the following information
- how many blocks occupies in the HDFS
-which are the machines where the blocks were replicated
I look forward for your answers.
Thank you,
Florin
Re: How to obtain block information of a HDFS file?
Posted by Harsh J <ha...@cloudera.com>.
Florin,
Right, all that info is gettable via the FileStatus objects for a given Path.
On Fri, Jul 29, 2011 at 11:55 AM, Florin P <fl...@yahoo.com> wrote:
> Hello!
> Thank you for your answers. I'm using 0.20 API.
> My request are per file, and I would like to have the same results that are displayed in the when you are using the web interface for browsing the file system (brosweBlock.jsp page).
> Regards,
> Florin
>
> P.S. My results should look like this
>
>
> filename filesize blocks machine_block_replication
>
> --- On Thu, 7/28/11, Harsh J <ha...@cloudera.com> wrote:
>
>> From: Harsh J <ha...@cloudera.com>
>> Subject: Re: How to obtain block information of a HDFS file?
>> To: hdfs-user@hadoop.apache.org
>> Date: Thursday, July 28, 2011, 10:10 AM
>> Hello,
>>
>> What version of Hadoop are you using?
>>
>> On Thu, Jul 28, 2011 at 6:03 PM, Florin P <fl...@yahoo.com>
>> wrote:
>> > Hello!
>> > I would like to know how to obtain for a file (via
>> Java API) the following information
>> > - how many blocks occupies in the HDFS
>>
>> Is this request per file? or overall? [If overall, I think
>> metrics are
>> your best bet]
>>
>> > -which are the machines where the blocks were
>> replicated
>>
>> If this request is per-file, then
>> FileSystem#getBlockLocations(…) API
>> can help you. Just request it over the entire size of the
>> file
>> (FileStatus information can be got via
>> FileSystem#getFileStatus(…))
>>
>> --
>> Harsh J
>>
>
--
Harsh J
Re: How to obtain block information of a HDFS file?
Posted by Florin P <fl...@yahoo.com>.
Hello!
Thank you for your answers. I'm using 0.20 API.
My request are per file, and I would like to have the same results that are displayed in the when you are using the web interface for browsing the file system (brosweBlock.jsp page).
Regards,
Florin
P.S. My results should look like this
filename filesize blocks machine_block_replication
--- On Thu, 7/28/11, Harsh J <ha...@cloudera.com> wrote:
> From: Harsh J <ha...@cloudera.com>
> Subject: Re: How to obtain block information of a HDFS file?
> To: hdfs-user@hadoop.apache.org
> Date: Thursday, July 28, 2011, 10:10 AM
> Hello,
>
> What version of Hadoop are you using?
>
> On Thu, Jul 28, 2011 at 6:03 PM, Florin P <fl...@yahoo.com>
> wrote:
> > Hello!
> > I would like to know how to obtain for a file (via
> Java API) the following information
> > - how many blocks occupies in the HDFS
>
> Is this request per file? or overall? [If overall, I think
> metrics are
> your best bet]
>
> > -which are the machines where the blocks were
> replicated
>
> If this request is per-file, then
> FileSystem#getBlockLocations(…) API
> can help you. Just request it over the entire size of the
> file
> (FileStatus information can be got via
> FileSystem#getFileStatus(…))
>
> --
> Harsh J
>
Re: How to obtain block information of a HDFS file?
Posted by Harsh J <ha...@cloudera.com>.
Hello,
What version of Hadoop are you using?
On Thu, Jul 28, 2011 at 6:03 PM, Florin P <fl...@yahoo.com> wrote:
> Hello!
> I would like to know how to obtain for a file (via Java API) the following information
> - how many blocks occupies in the HDFS
Is this request per file? or overall? [If overall, I think metrics are
your best bet]
> -which are the machines where the blocks were replicated
If this request is per-file, then FileSystem#getBlockLocations(…) API
can help you. Just request it over the entire size of the file
(FileStatus information can be got via FileSystem#getFileStatus(…))
--
Harsh J