You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by 一凡 李 <zh...@yahoo.com.cn> on 2013/06/04 06:49:25 UTC
how to locate the replicas of a file in HDFS?
Hi,
Could you tell me how to locate where store each replica of a file in HDFS?
Correctly speaking, if I create a file in HDFS(replicate factor:3),how to find the DataNodes which store its each block and replicas?
Best Wishes,
Yifan
____________________
Re: how to locate the replicas of a file in HDFS?
Posted by Azuryy Yu <az...@gmail.com>.
ClientProtocol namenode = DFSClient.createNamenode(conf);
HdfsFileStatus hfs = namenode.getFileInfo("your_hdfs_file_name");
LocatedBlocks lbs = namenode.getBlockLocations("your_hdfs_file_name", 0,
hfs.getLen());
for (LocatedBlock lb : lbs.getLocatedBlocks()) {
DatanodeInfo[] info = lb.getLocations() ;
//you can get data node name or address here.
}
On Tue, Jun 4, 2013 at 2:02 PM, Mahmood Naderan <nt...@yahoo.com>wrote:
> >hadoop fsck mytext.txt -files -locations -blocks
>
>
> I expect something like a tag which is attached to each block (say block
> X) that shows the position of the replicated block of X. The method you
> mentioned is a user level task. Am I right?
>
> Regards,
> Mahmood*
> *
>
> ------------------------------
> *From:* Rahul Bhattacharjee <ra...@gmail.com>
> *To:* "user@hadoop.apache.org" <us...@hadoop.apache.org>; 一凡 李 <
> zhuazhua_box@yahoo.com.cn>
> *Sent:* Tuesday, June 4, 2013 9:34 AM
> *Subject:* Re: how to locate the replicas of a file in HDFS?
>
> hadoop fsck mytext.txt -files -locations -blocks
>
> Thanks,
> Rahul
>
>
>
> On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
>
> Hi,
>
> Could you tell me how to locate where store each replica of a file in HDFS?
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas?
>
>
> Best Wishes,
> Yifan
> ____________________
>
>
>
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Azuryy Yu <az...@gmail.com>.
ClientProtocol namenode = DFSClient.createNamenode(conf);
HdfsFileStatus hfs = namenode.getFileInfo("your_hdfs_file_name");
LocatedBlocks lbs = namenode.getBlockLocations("your_hdfs_file_name", 0,
hfs.getLen());
for (LocatedBlock lb : lbs.getLocatedBlocks()) {
DatanodeInfo[] info = lb.getLocations() ;
//you can get data node name or address here.
}
On Tue, Jun 4, 2013 at 2:02 PM, Mahmood Naderan <nt...@yahoo.com>wrote:
> >hadoop fsck mytext.txt -files -locations -blocks
>
>
> I expect something like a tag which is attached to each block (say block
> X) that shows the position of the replicated block of X. The method you
> mentioned is a user level task. Am I right?
>
> Regards,
> Mahmood*
> *
>
> ------------------------------
> *From:* Rahul Bhattacharjee <ra...@gmail.com>
> *To:* "user@hadoop.apache.org" <us...@hadoop.apache.org>; 一凡 李 <
> zhuazhua_box@yahoo.com.cn>
> *Sent:* Tuesday, June 4, 2013 9:34 AM
> *Subject:* Re: how to locate the replicas of a file in HDFS?
>
> hadoop fsck mytext.txt -files -locations -blocks
>
> Thanks,
> Rahul
>
>
>
> On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
>
> Hi,
>
> Could you tell me how to locate where store each replica of a file in HDFS?
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas?
>
>
> Best Wishes,
> Yifan
> ____________________
>
>
>
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Azuryy Yu <az...@gmail.com>.
ClientProtocol namenode = DFSClient.createNamenode(conf);
HdfsFileStatus hfs = namenode.getFileInfo("your_hdfs_file_name");
LocatedBlocks lbs = namenode.getBlockLocations("your_hdfs_file_name", 0,
hfs.getLen());
for (LocatedBlock lb : lbs.getLocatedBlocks()) {
DatanodeInfo[] info = lb.getLocations() ;
//you can get data node name or address here.
}
On Tue, Jun 4, 2013 at 2:02 PM, Mahmood Naderan <nt...@yahoo.com>wrote:
> >hadoop fsck mytext.txt -files -locations -blocks
>
>
> I expect something like a tag which is attached to each block (say block
> X) that shows the position of the replicated block of X. The method you
> mentioned is a user level task. Am I right?
>
> Regards,
> Mahmood*
> *
>
> ------------------------------
> *From:* Rahul Bhattacharjee <ra...@gmail.com>
> *To:* "user@hadoop.apache.org" <us...@hadoop.apache.org>; 一凡 李 <
> zhuazhua_box@yahoo.com.cn>
> *Sent:* Tuesday, June 4, 2013 9:34 AM
> *Subject:* Re: how to locate the replicas of a file in HDFS?
>
> hadoop fsck mytext.txt -files -locations -blocks
>
> Thanks,
> Rahul
>
>
>
> On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
>
> Hi,
>
> Could you tell me how to locate where store each replica of a file in HDFS?
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas?
>
>
> Best Wishes,
> Yifan
> ____________________
>
>
>
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Azuryy Yu <az...@gmail.com>.
ClientProtocol namenode = DFSClient.createNamenode(conf);
HdfsFileStatus hfs = namenode.getFileInfo("your_hdfs_file_name");
LocatedBlocks lbs = namenode.getBlockLocations("your_hdfs_file_name", 0,
hfs.getLen());
for (LocatedBlock lb : lbs.getLocatedBlocks()) {
DatanodeInfo[] info = lb.getLocations() ;
//you can get data node name or address here.
}
On Tue, Jun 4, 2013 at 2:02 PM, Mahmood Naderan <nt...@yahoo.com>wrote:
> >hadoop fsck mytext.txt -files -locations -blocks
>
>
> I expect something like a tag which is attached to each block (say block
> X) that shows the position of the replicated block of X. The method you
> mentioned is a user level task. Am I right?
>
> Regards,
> Mahmood*
> *
>
> ------------------------------
> *From:* Rahul Bhattacharjee <ra...@gmail.com>
> *To:* "user@hadoop.apache.org" <us...@hadoop.apache.org>; 一凡 李 <
> zhuazhua_box@yahoo.com.cn>
> *Sent:* Tuesday, June 4, 2013 9:34 AM
> *Subject:* Re: how to locate the replicas of a file in HDFS?
>
> hadoop fsck mytext.txt -files -locations -blocks
>
> Thanks,
> Rahul
>
>
>
> On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
>
> Hi,
>
> Could you tell me how to locate where store each replica of a file in HDFS?
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas?
>
>
> Best Wishes,
> Yifan
> ____________________
>
>
>
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Mahmood Naderan <nt...@yahoo.com>.
>hadoop fsck mytext.txt -files -locations -blocks
I expect something like a tag which is attached to each block (say block X) that shows the position of the replicated block of X. The method you mentioned is a user level task. Am I right?
Regards,
Mahmood
________________________________
From: Rahul Bhattacharjee <ra...@gmail.com>
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>; 一凡 李 <zh...@yahoo.com.cn>
Sent: Tuesday, June 4, 2013 9:34 AM
Subject: Re: how to locate the replicas of a file in HDFS?
hadoop fsck mytext.txt -files -locations -blocks
Thanks,
Rahul
On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
Hi,
>
>
>Could you tell me how to locate where store each replica of a file in HDFS?
>
>
>Correctly speaking, if I create a file in HDFS(replicate factor:3),how to find the DataNodes which store its each block and replicas?
>
>
>
>Best Wishes,
>Yifan
>____________________
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Mahmood Naderan <nt...@yahoo.com>.
>hadoop fsck mytext.txt -files -locations -blocks
I expect something like a tag which is attached to each block (say block X) that shows the position of the replicated block of X. The method you mentioned is a user level task. Am I right?
Regards,
Mahmood
________________________________
From: Rahul Bhattacharjee <ra...@gmail.com>
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>; 一凡 李 <zh...@yahoo.com.cn>
Sent: Tuesday, June 4, 2013 9:34 AM
Subject: Re: how to locate the replicas of a file in HDFS?
hadoop fsck mytext.txt -files -locations -blocks
Thanks,
Rahul
On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
Hi,
>
>
>Could you tell me how to locate where store each replica of a file in HDFS?
>
>
>Correctly speaking, if I create a file in HDFS(replicate factor:3),how to find the DataNodes which store its each block and replicas?
>
>
>
>Best Wishes,
>Yifan
>____________________
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Mahmood Naderan <nt...@yahoo.com>.
>hadoop fsck mytext.txt -files -locations -blocks
I expect something like a tag which is attached to each block (say block X) that shows the position of the replicated block of X. The method you mentioned is a user level task. Am I right?
Regards,
Mahmood
________________________________
From: Rahul Bhattacharjee <ra...@gmail.com>
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>; 一凡 李 <zh...@yahoo.com.cn>
Sent: Tuesday, June 4, 2013 9:34 AM
Subject: Re: how to locate the replicas of a file in HDFS?
hadoop fsck mytext.txt -files -locations -blocks
Thanks,
Rahul
On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
Hi,
>
>
>Could you tell me how to locate where store each replica of a file in HDFS?
>
>
>Correctly speaking, if I create a file in HDFS(replicate factor:3),how to find the DataNodes which store its each block and replicas?
>
>
>
>Best Wishes,
>Yifan
>____________________
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Mahmood Naderan <nt...@yahoo.com>.
>hadoop fsck mytext.txt -files -locations -blocks
I expect something like a tag which is attached to each block (say block X) that shows the position of the replicated block of X. The method you mentioned is a user level task. Am I right?
Regards,
Mahmood
________________________________
From: Rahul Bhattacharjee <ra...@gmail.com>
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>; 一凡 李 <zh...@yahoo.com.cn>
Sent: Tuesday, June 4, 2013 9:34 AM
Subject: Re: how to locate the replicas of a file in HDFS?
hadoop fsck mytext.txt -files -locations -blocks
Thanks,
Rahul
On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
Hi,
>
>
>Could you tell me how to locate where store each replica of a file in HDFS?
>
>
>Correctly speaking, if I create a file in HDFS(replicate factor:3),how to find the DataNodes which store its each block and replicas?
>
>
>
>Best Wishes,
>Yifan
>____________________
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Rahul Bhattacharjee <ra...@gmail.com>.
hadoop fsck mytext.txt -files -locations -blocks
Thanks,
Rahul
On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
> Hi,
>
> Could you tell me how to locate where store each replica of a file in HDFS?
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas?
>
>
> Best Wishes,
> Yifan
> ____________________
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Sandeep Nemuri <nh...@gmail.com>.
Try this command
hadoop fsck <file path> -files -blocks
On Tue, Jun 4, 2013 at 3:41 PM, zangxiangyu <za...@qiyi.com> wrote:
> hadoop fsck * <path>* -files -blocks -locations –racks****
>
> ** **
>
> replace <path> with real path J****
>
> ** **
>
> *From:* 一凡 李 [mailto:zhuazhua_box@yahoo.com.cn]
> *Sent:* Tuesday, June 04, 2013 12:49 PM
> *To:* user@hadoop.apache.org
> *Subject:* how to locate the replicas of a file in HDFS?****
>
> ** **
>
> Hi,****
>
> ** **
>
> Could you tell me how to locate where store each replica of a file in HDFS?
> ****
>
> ** **
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas? ****
>
> ** **
>
> ****
>
> Best Wishes,
> Yifan*
> ____________________*
>
> * *
>
--
--Regards
Sandeep Nemuri
Re: how to locate the replicas of a file in HDFS?
Posted by Sandeep Nemuri <nh...@gmail.com>.
Try this command
hadoop fsck <file path> -files -blocks
On Tue, Jun 4, 2013 at 3:41 PM, zangxiangyu <za...@qiyi.com> wrote:
> hadoop fsck * <path>* -files -blocks -locations –racks****
>
> ** **
>
> replace <path> with real path J****
>
> ** **
>
> *From:* 一凡 李 [mailto:zhuazhua_box@yahoo.com.cn]
> *Sent:* Tuesday, June 04, 2013 12:49 PM
> *To:* user@hadoop.apache.org
> *Subject:* how to locate the replicas of a file in HDFS?****
>
> ** **
>
> Hi,****
>
> ** **
>
> Could you tell me how to locate where store each replica of a file in HDFS?
> ****
>
> ** **
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas? ****
>
> ** **
>
> ****
>
> Best Wishes,
> Yifan*
> ____________________*
>
> * *
>
--
--Regards
Sandeep Nemuri
Re: how to locate the replicas of a file in HDFS?
Posted by Sandeep Nemuri <nh...@gmail.com>.
Try this command
hadoop fsck <file path> -files -blocks
On Tue, Jun 4, 2013 at 3:41 PM, zangxiangyu <za...@qiyi.com> wrote:
> hadoop fsck * <path>* -files -blocks -locations –racks****
>
> ** **
>
> replace <path> with real path J****
>
> ** **
>
> *From:* 一凡 李 [mailto:zhuazhua_box@yahoo.com.cn]
> *Sent:* Tuesday, June 04, 2013 12:49 PM
> *To:* user@hadoop.apache.org
> *Subject:* how to locate the replicas of a file in HDFS?****
>
> ** **
>
> Hi,****
>
> ** **
>
> Could you tell me how to locate where store each replica of a file in HDFS?
> ****
>
> ** **
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas? ****
>
> ** **
>
> ****
>
> Best Wishes,
> Yifan*
> ____________________*
>
> * *
>
--
--Regards
Sandeep Nemuri
Re: how to locate the replicas of a file in HDFS?
Posted by Sandeep Nemuri <nh...@gmail.com>.
Try this command
hadoop fsck <file path> -files -blocks
On Tue, Jun 4, 2013 at 3:41 PM, zangxiangyu <za...@qiyi.com> wrote:
> hadoop fsck * <path>* -files -blocks -locations –racks****
>
> ** **
>
> replace <path> with real path J****
>
> ** **
>
> *From:* 一凡 李 [mailto:zhuazhua_box@yahoo.com.cn]
> *Sent:* Tuesday, June 04, 2013 12:49 PM
> *To:* user@hadoop.apache.org
> *Subject:* how to locate the replicas of a file in HDFS?****
>
> ** **
>
> Hi,****
>
> ** **
>
> Could you tell me how to locate where store each replica of a file in HDFS?
> ****
>
> ** **
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas? ****
>
> ** **
>
> ****
>
> Best Wishes,
> Yifan*
> ____________________*
>
> * *
>
--
--Regards
Sandeep Nemuri
RE: how to locate the replicas of a file in HDFS?
Posted by zangxiangyu <za...@qiyi.com>.
hadoop fsck <path> -files -blocks -locations –racks
replace <path> with real path J
From: 一凡 李 [mailto:zhuazhua_box@yahoo.com.cn]
Sent: Tuesday, June 04, 2013 12:49 PM
To: user@hadoop.apache.org
Subject: how to locate the replicas of a file in HDFS?
Hi,
Could you tell me how to locate where store each replica of a file in HDFS?
Correctly speaking, if I create a file in HDFS(replicate factor:3),how to find the DataNodes which store its each block and replicas?
Best Wishes,
Yifan
____________________
RE: how to locate the replicas of a file in HDFS?
Posted by zangxiangyu <za...@qiyi.com>.
hadoop fsck <path> -files -blocks -locations –racks
replace <path> with real path J
From: 一凡 李 [mailto:zhuazhua_box@yahoo.com.cn]
Sent: Tuesday, June 04, 2013 12:49 PM
To: user@hadoop.apache.org
Subject: how to locate the replicas of a file in HDFS?
Hi,
Could you tell me how to locate where store each replica of a file in HDFS?
Correctly speaking, if I create a file in HDFS(replicate factor:3),how to find the DataNodes which store its each block and replicas?
Best Wishes,
Yifan
____________________
Re: how to locate the replicas of a file in HDFS?
Posted by Rahul Bhattacharjee <ra...@gmail.com>.
hadoop fsck mytext.txt -files -locations -blocks
Thanks,
Rahul
On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
> Hi,
>
> Could you tell me how to locate where store each replica of a file in HDFS?
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas?
>
>
> Best Wishes,
> Yifan
> ____________________
>
>
RE: how to locate the replicas of a file in HDFS?
Posted by zangxiangyu <za...@qiyi.com>.
hadoop fsck <path> -files -blocks -locations –racks
replace <path> with real path J
From: 一凡 李 [mailto:zhuazhua_box@yahoo.com.cn]
Sent: Tuesday, June 04, 2013 12:49 PM
To: user@hadoop.apache.org
Subject: how to locate the replicas of a file in HDFS?
Hi,
Could you tell me how to locate where store each replica of a file in HDFS?
Correctly speaking, if I create a file in HDFS(replicate factor:3),how to find the DataNodes which store its each block and replicas?
Best Wishes,
Yifan
____________________
RE: how to locate the replicas of a file in HDFS?
Posted by zangxiangyu <za...@qiyi.com>.
hadoop fsck <path> -files -blocks -locations –racks
replace <path> with real path J
From: 一凡 李 [mailto:zhuazhua_box@yahoo.com.cn]
Sent: Tuesday, June 04, 2013 12:49 PM
To: user@hadoop.apache.org
Subject: how to locate the replicas of a file in HDFS?
Hi,
Could you tell me how to locate where store each replica of a file in HDFS?
Correctly speaking, if I create a file in HDFS(replicate factor:3),how to find the DataNodes which store its each block and replicas?
Best Wishes,
Yifan
____________________
Re: how to locate the replicas of a file in HDFS?
Posted by Rahul Bhattacharjee <ra...@gmail.com>.
hadoop fsck mytext.txt -files -locations -blocks
Thanks,
Rahul
On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
> Hi,
>
> Could you tell me how to locate where store each replica of a file in HDFS?
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas?
>
>
> Best Wishes,
> Yifan
> ____________________
>
>
Re: how to locate the replicas of a file in HDFS?
Posted by Rahul Bhattacharjee <ra...@gmail.com>.
hadoop fsck mytext.txt -files -locations -blocks
Thanks,
Rahul
On Tue, Jun 4, 2013 at 10:19 AM, 一凡 李 <zh...@yahoo.com.cn> wrote:
> Hi,
>
> Could you tell me how to locate where store each replica of a file in HDFS?
>
> Correctly speaking, if I create a file in HDFS(replicate factor:3),how to
> find the DataNodes which store its each block and replicas?
>
>
> Best Wishes,
> Yifan
> ____________________
>
>