Where are the meta data on HDFS ?

[tienduc_dinh <ti...@yahoo.com>]

hi everyone,

I got a question, maybe you can help me.

- how can we get the meta data of a file on HDFS ? 

For example:  If I have a file with e.g. 2 GB on HDFS, this file is split
into many chunks and these chunks are distributed on many nodes. Is there
any trick to know, which chunks belong to that file ?

Any help will be appreciated, thanks lots.

Tien Duc Dinh
-- 
View this message in context: http://www.nabble.com/Where-are-the-meta-data-on-HDFS---tp21634677p21634677.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Re: Where are the meta data on HDFS ?

[Rasit OZDAS <ra...@gmail.com>]

Hi Tien,

Configuration config = new Configuration(true);
config.addResource(new Path("/etc/hadoop-0.19.0/conf/hadoop-site.xml"));

FileSystem fileSys = FileSystem.get(config);
BlockLocation[] locations = fileSys.getFileBlockLocations(.....

I copied some lines of my code, it can also help if you prefer using the
API.
It has other useful features (methods) as well.
http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/fs/FileSystem.html


2009/1/24 tienduc_dinh <ti...@yahoo.com>

>
> that's what I needed !
>
> Thank you so much.
> --
> View this message in context:
> http://www.nabble.com/Where-are-the-meta-data-on-HDFS---tp21634677p21644206.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>


-- 
M. Raşit ÖZDAŞ

Re: Where are the meta data on HDFS ?

[tienduc_dinh <ti...@yahoo.com>]

that's what I needed !

Thank you so much.
-- 
View this message in context: http://www.nabble.com/Where-are-the-meta-data-on-HDFS---tp21634677p21644206.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Re: Where are the meta data on HDFS ?

[Peeyush Bishnoi <pe...@yahoo-inc.com>]

Hello Tien ,

There is tool in Hadoop DFS i.e fsck . I hope this will help you and
serve your purpose very well.

For e.g:
$HADOOP_HOME/bin/hadoop fsck <filename/directorie path> -files -blocks
-locations 

The above tool will display the blocks/chunks of files , locations where
this blocks/chunks of files are located. Also it will display other
useful information for files and directories. 

For more information on fsck , just refer this URL :
http://hadoop.apache.org/core/docs/r0.19.0/hdfs_user_guide.html#fsck 


Thanks ,
---
Peeyush

On Fri, 2009-01-23 at 15:24 -0800, tienduc_dinh wrote:

> hi everyone,
> 
> I got a question, maybe you can help me.
> 
> - how can we get the meta data of a file on HDFS ? 
> 
> For example:  If I have a file with e.g. 2 GB on HDFS, this file is split
> into many chunks and these chunks are distributed on many nodes. Is there
> any trick to know, which chunks belong to that file ?
> 
> Any help will be appreciated, thanks lots.
> 
> Tien Duc Dinh