You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Mark Kerzner <ma...@shmsoft.com> on 2014/01/23 19:39:00 UTC
hdfs fsck -locations
Hi,
hdfs fsck -locations
is supposed to show every block with its location? Is location the ip of
the datanode?
Thank you,
Mark
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Got it, Harsh, now I see it all.
Thank you,
Mark
On Fri, Jan 24, 2014 at 7:49 PM, Harsh J <ha...@cloudera.com> wrote:
> The right syntax is to use "-files -blocks -locations", so it drills
> down all the way. You are not "missing" a feature - this has existed
> for as long as I've known HDFS.
>
> In Rodrigo's output, he's seeing a BlockPool ID, which is not
> equivalent to a location, but just carries an IP in it for
> identification purposes.
>
> On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Yes, Rodrigo,
> >
> > that's what I was looking for. So in my install I somehow don't have it
> at
> > all. Was asked by my students, so I got the answer.
> >
> > Mark
> >
> >
> > On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo
> > <Ro...@netapp.com> wrote:
> >>
> >> Mark,
> >>
> >> there we go ;-)
> >>
> >> Rodrigo Nascimento
> >> Systems Engineer @ Brazil
> >> Mobile +55 11 991.873.810
> >>
> >> Sent from my iPhone
> >>
> >> Begin forwarded message:
> >>
> >> From: "Nascimento, Rodrigo" <Ro...@netapp.com>
> >> Date: 24 de janeiro de 2014 15:59:33 BRST
> >> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> >> Subject: RE: hdfs fsck -locations
> >>
> >> Hi Mark,
> >>
> >> It is a sample from my sandbox. Your question is about the part that is
> in
> >> RED at the output below, right?
> >>
> >> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
> >> Connecting to namenode via http://sandbox.hortonworks.com:50070
> >> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
> >> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
> >> .
> >> /user/ambari-qa/passwd: Under replicated
> >> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target
> Replicas
> >> is 3 but found 1 replica(s).
> >> Status: HEALTHY
> >> Total size: 1708 B
> >> Total dirs: 0
> >> Total files: 1
> >> Total symlinks: 0
> >> Total blocks (validated): 1 (avg. block size 1708 B)
> >> Minimally replicated blocks: 1 (100.0 %)
> >> Over-replicated blocks: 0 (0.0 %)
> >> Under-replicated blocks: 1 (100.0 %)
> >> Mis-replicated blocks: 0 (0.0 %)
> >> Default replication factor: 3
> >> Average block replication: 1.0
> >> Corrupt blocks: 0
> >> Missing replicas: 2 (66.666664 %)
> >> Number of data-nodes: 1
> >> Number of racks: 1
> >> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
> >>
> >>
> >> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
> >> [root@sandbox ~]#
> >>
> >> Rod Nascimento
> >>
> >> ________________________________
> >> From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
> >> Sent: Friday, January 24, 2014 3:34 PM
> >> To: user@hadoop.apache.org
> >> Subject: RE: hdfs fsck -locations
> >>
> >> I’m not seeing locations flag yet.
> >>
> >>
> >>
> >> Rod Nascimento
> >>
> >> Systems Engineer @ Brazil
> >>
> >>
> >>
> >> People don’t buy WHAT you do. They buy WHY you do it.
> >>
> >>
> >>
> >> From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> >> Sent: Friday, January 24, 2014 3:16 PM
> >> To: Hadoop User
> >> Subject: Re: hdfs fsck -locations
> >>
> >>
> >>
> >> Sorry, did not copy the full command
> >>
> >>
> >>
> >> hdfs fsck /user/mark/data/word_count.csv -locations
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >> Total size: 7217 B
> >>
> >> Total dirs: 0
> >>
> >> Total files: 1
> >>
> >> Total blocks (validated): 1 (avg. block size 7217 B)
> >>
> >> Minimally replicated blocks: 1 (100.0 %)
> >>
> >> Over-replicated blocks: 0 (0.0 %)
> >>
> >> Under-replicated blocks: 0 (0.0 %)
> >>
> >> Mis-replicated blocks: 0 (0.0 %)
> >>
> >> Default replication factor: 1
> >>
> >> Average block replication: 1.0
> >>
> >> Corrupt blocks: 0
> >>
> >> Missing replicas: 0 (0.0 %)
> >>
> >> Number of data-nodes: 1
> >>
> >> Number of racks: 1
> >>
> >> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >> Sorry, but what was the question? I also do not see a locations option
> >> flag.
> >>
> >> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>
> wrote:
> >>
> >> Here is an example
> >>
> >>
> >>
> >> hdfs fsck /user/mark/data/word_count.csv
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >> Total size: 7217 B
> >>
> >> Total dirs: 0
> >>
> >> Total files: 1
> >>
> >> Total blocks (validated): 1 (avg. block size 7217 B)
> >>
> >> Minimally replicated blocks: 1 (100.0 %)
> >>
> >> Over-replicated blocks: 0 (0.0 %)
> >>
> >> Under-replicated blocks: 0 (0.0 %)
> >>
> >> Mis-replicated blocks: 0 (0.0 %)
> >>
> >> Default replication factor: 1
> >>
> >> Average block replication: 1.0
> >>
> >> Corrupt blocks: 0
> >>
> >> Missing replicas: 0 (0.0 %)
> >>
> >> Number of data-nodes: 1
> >>
> >> Number of racks: 1
> >>
> >> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >> Hi Mark,
> >>
> >> Yes, the locations are shown as IP.
> >>
> >>
> >> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <
> mark.kerzner@shmsoft.com>
> >> wrote:
> >> > Hi,
> >> >
> >> > hdfs fsck -locations
> >> >
> >> > is supposed to show every block with its location? Is location the ip
> of
> >> > the
> >> > datanode?
> >> >
> >> > Thank you,
> >> > Mark
> >>
> >>
> >> --
> >> Harsh J
> >>
> >>
> >>
> >>
> >
> >
>
>
>
> --
> Harsh J
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Got it, Harsh, now I see it all.
Thank you,
Mark
On Fri, Jan 24, 2014 at 7:49 PM, Harsh J <ha...@cloudera.com> wrote:
> The right syntax is to use "-files -blocks -locations", so it drills
> down all the way. You are not "missing" a feature - this has existed
> for as long as I've known HDFS.
>
> In Rodrigo's output, he's seeing a BlockPool ID, which is not
> equivalent to a location, but just carries an IP in it for
> identification purposes.
>
> On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Yes, Rodrigo,
> >
> > that's what I was looking for. So in my install I somehow don't have it
> at
> > all. Was asked by my students, so I got the answer.
> >
> > Mark
> >
> >
> > On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo
> > <Ro...@netapp.com> wrote:
> >>
> >> Mark,
> >>
> >> there we go ;-)
> >>
> >> Rodrigo Nascimento
> >> Systems Engineer @ Brazil
> >> Mobile +55 11 991.873.810
> >>
> >> Sent from my iPhone
> >>
> >> Begin forwarded message:
> >>
> >> From: "Nascimento, Rodrigo" <Ro...@netapp.com>
> >> Date: 24 de janeiro de 2014 15:59:33 BRST
> >> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> >> Subject: RE: hdfs fsck -locations
> >>
> >> Hi Mark,
> >>
> >> It is a sample from my sandbox. Your question is about the part that is
> in
> >> RED at the output below, right?
> >>
> >> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
> >> Connecting to namenode via http://sandbox.hortonworks.com:50070
> >> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
> >> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
> >> .
> >> /user/ambari-qa/passwd: Under replicated
> >> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target
> Replicas
> >> is 3 but found 1 replica(s).
> >> Status: HEALTHY
> >> Total size: 1708 B
> >> Total dirs: 0
> >> Total files: 1
> >> Total symlinks: 0
> >> Total blocks (validated): 1 (avg. block size 1708 B)
> >> Minimally replicated blocks: 1 (100.0 %)
> >> Over-replicated blocks: 0 (0.0 %)
> >> Under-replicated blocks: 1 (100.0 %)
> >> Mis-replicated blocks: 0 (0.0 %)
> >> Default replication factor: 3
> >> Average block replication: 1.0
> >> Corrupt blocks: 0
> >> Missing replicas: 2 (66.666664 %)
> >> Number of data-nodes: 1
> >> Number of racks: 1
> >> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
> >>
> >>
> >> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
> >> [root@sandbox ~]#
> >>
> >> Rod Nascimento
> >>
> >> ________________________________
> >> From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
> >> Sent: Friday, January 24, 2014 3:34 PM
> >> To: user@hadoop.apache.org
> >> Subject: RE: hdfs fsck -locations
> >>
> >> I’m not seeing locations flag yet.
> >>
> >>
> >>
> >> Rod Nascimento
> >>
> >> Systems Engineer @ Brazil
> >>
> >>
> >>
> >> People don’t buy WHAT you do. They buy WHY you do it.
> >>
> >>
> >>
> >> From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> >> Sent: Friday, January 24, 2014 3:16 PM
> >> To: Hadoop User
> >> Subject: Re: hdfs fsck -locations
> >>
> >>
> >>
> >> Sorry, did not copy the full command
> >>
> >>
> >>
> >> hdfs fsck /user/mark/data/word_count.csv -locations
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >> Total size: 7217 B
> >>
> >> Total dirs: 0
> >>
> >> Total files: 1
> >>
> >> Total blocks (validated): 1 (avg. block size 7217 B)
> >>
> >> Minimally replicated blocks: 1 (100.0 %)
> >>
> >> Over-replicated blocks: 0 (0.0 %)
> >>
> >> Under-replicated blocks: 0 (0.0 %)
> >>
> >> Mis-replicated blocks: 0 (0.0 %)
> >>
> >> Default replication factor: 1
> >>
> >> Average block replication: 1.0
> >>
> >> Corrupt blocks: 0
> >>
> >> Missing replicas: 0 (0.0 %)
> >>
> >> Number of data-nodes: 1
> >>
> >> Number of racks: 1
> >>
> >> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >> Sorry, but what was the question? I also do not see a locations option
> >> flag.
> >>
> >> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>
> wrote:
> >>
> >> Here is an example
> >>
> >>
> >>
> >> hdfs fsck /user/mark/data/word_count.csv
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >> Total size: 7217 B
> >>
> >> Total dirs: 0
> >>
> >> Total files: 1
> >>
> >> Total blocks (validated): 1 (avg. block size 7217 B)
> >>
> >> Minimally replicated blocks: 1 (100.0 %)
> >>
> >> Over-replicated blocks: 0 (0.0 %)
> >>
> >> Under-replicated blocks: 0 (0.0 %)
> >>
> >> Mis-replicated blocks: 0 (0.0 %)
> >>
> >> Default replication factor: 1
> >>
> >> Average block replication: 1.0
> >>
> >> Corrupt blocks: 0
> >>
> >> Missing replicas: 0 (0.0 %)
> >>
> >> Number of data-nodes: 1
> >>
> >> Number of racks: 1
> >>
> >> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >> Hi Mark,
> >>
> >> Yes, the locations are shown as IP.
> >>
> >>
> >> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <
> mark.kerzner@shmsoft.com>
> >> wrote:
> >> > Hi,
> >> >
> >> > hdfs fsck -locations
> >> >
> >> > is supposed to show every block with its location? Is location the ip
> of
> >> > the
> >> > datanode?
> >> >
> >> > Thank you,
> >> > Mark
> >>
> >>
> >> --
> >> Harsh J
> >>
> >>
> >>
> >>
> >
> >
>
>
>
> --
> Harsh J
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Got it, Harsh, now I see it all.
Thank you,
Mark
On Fri, Jan 24, 2014 at 7:49 PM, Harsh J <ha...@cloudera.com> wrote:
> The right syntax is to use "-files -blocks -locations", so it drills
> down all the way. You are not "missing" a feature - this has existed
> for as long as I've known HDFS.
>
> In Rodrigo's output, he's seeing a BlockPool ID, which is not
> equivalent to a location, but just carries an IP in it for
> identification purposes.
>
> On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Yes, Rodrigo,
> >
> > that's what I was looking for. So in my install I somehow don't have it
> at
> > all. Was asked by my students, so I got the answer.
> >
> > Mark
> >
> >
> > On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo
> > <Ro...@netapp.com> wrote:
> >>
> >> Mark,
> >>
> >> there we go ;-)
> >>
> >> Rodrigo Nascimento
> >> Systems Engineer @ Brazil
> >> Mobile +55 11 991.873.810
> >>
> >> Sent from my iPhone
> >>
> >> Begin forwarded message:
> >>
> >> From: "Nascimento, Rodrigo" <Ro...@netapp.com>
> >> Date: 24 de janeiro de 2014 15:59:33 BRST
> >> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> >> Subject: RE: hdfs fsck -locations
> >>
> >> Hi Mark,
> >>
> >> It is a sample from my sandbox. Your question is about the part that is
> in
> >> RED at the output below, right?
> >>
> >> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
> >> Connecting to namenode via http://sandbox.hortonworks.com:50070
> >> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
> >> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
> >> .
> >> /user/ambari-qa/passwd: Under replicated
> >> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target
> Replicas
> >> is 3 but found 1 replica(s).
> >> Status: HEALTHY
> >> Total size: 1708 B
> >> Total dirs: 0
> >> Total files: 1
> >> Total symlinks: 0
> >> Total blocks (validated): 1 (avg. block size 1708 B)
> >> Minimally replicated blocks: 1 (100.0 %)
> >> Over-replicated blocks: 0 (0.0 %)
> >> Under-replicated blocks: 1 (100.0 %)
> >> Mis-replicated blocks: 0 (0.0 %)
> >> Default replication factor: 3
> >> Average block replication: 1.0
> >> Corrupt blocks: 0
> >> Missing replicas: 2 (66.666664 %)
> >> Number of data-nodes: 1
> >> Number of racks: 1
> >> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
> >>
> >>
> >> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
> >> [root@sandbox ~]#
> >>
> >> Rod Nascimento
> >>
> >> ________________________________
> >> From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
> >> Sent: Friday, January 24, 2014 3:34 PM
> >> To: user@hadoop.apache.org
> >> Subject: RE: hdfs fsck -locations
> >>
> >> I’m not seeing locations flag yet.
> >>
> >>
> >>
> >> Rod Nascimento
> >>
> >> Systems Engineer @ Brazil
> >>
> >>
> >>
> >> People don’t buy WHAT you do. They buy WHY you do it.
> >>
> >>
> >>
> >> From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> >> Sent: Friday, January 24, 2014 3:16 PM
> >> To: Hadoop User
> >> Subject: Re: hdfs fsck -locations
> >>
> >>
> >>
> >> Sorry, did not copy the full command
> >>
> >>
> >>
> >> hdfs fsck /user/mark/data/word_count.csv -locations
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >> Total size: 7217 B
> >>
> >> Total dirs: 0
> >>
> >> Total files: 1
> >>
> >> Total blocks (validated): 1 (avg. block size 7217 B)
> >>
> >> Minimally replicated blocks: 1 (100.0 %)
> >>
> >> Over-replicated blocks: 0 (0.0 %)
> >>
> >> Under-replicated blocks: 0 (0.0 %)
> >>
> >> Mis-replicated blocks: 0 (0.0 %)
> >>
> >> Default replication factor: 1
> >>
> >> Average block replication: 1.0
> >>
> >> Corrupt blocks: 0
> >>
> >> Missing replicas: 0 (0.0 %)
> >>
> >> Number of data-nodes: 1
> >>
> >> Number of racks: 1
> >>
> >> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >> Sorry, but what was the question? I also do not see a locations option
> >> flag.
> >>
> >> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>
> wrote:
> >>
> >> Here is an example
> >>
> >>
> >>
> >> hdfs fsck /user/mark/data/word_count.csv
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >> Total size: 7217 B
> >>
> >> Total dirs: 0
> >>
> >> Total files: 1
> >>
> >> Total blocks (validated): 1 (avg. block size 7217 B)
> >>
> >> Minimally replicated blocks: 1 (100.0 %)
> >>
> >> Over-replicated blocks: 0 (0.0 %)
> >>
> >> Under-replicated blocks: 0 (0.0 %)
> >>
> >> Mis-replicated blocks: 0 (0.0 %)
> >>
> >> Default replication factor: 1
> >>
> >> Average block replication: 1.0
> >>
> >> Corrupt blocks: 0
> >>
> >> Missing replicas: 0 (0.0 %)
> >>
> >> Number of data-nodes: 1
> >>
> >> Number of racks: 1
> >>
> >> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >> Hi Mark,
> >>
> >> Yes, the locations are shown as IP.
> >>
> >>
> >> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <
> mark.kerzner@shmsoft.com>
> >> wrote:
> >> > Hi,
> >> >
> >> > hdfs fsck -locations
> >> >
> >> > is supposed to show every block with its location? Is location the ip
> of
> >> > the
> >> > datanode?
> >> >
> >> > Thank you,
> >> > Mark
> >>
> >>
> >> --
> >> Harsh J
> >>
> >>
> >>
> >>
> >
> >
>
>
>
> --
> Harsh J
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Got it, Harsh, now I see it all.
Thank you,
Mark
On Fri, Jan 24, 2014 at 7:49 PM, Harsh J <ha...@cloudera.com> wrote:
> The right syntax is to use "-files -blocks -locations", so it drills
> down all the way. You are not "missing" a feature - this has existed
> for as long as I've known HDFS.
>
> In Rodrigo's output, he's seeing a BlockPool ID, which is not
> equivalent to a location, but just carries an IP in it for
> identification purposes.
>
> On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Yes, Rodrigo,
> >
> > that's what I was looking for. So in my install I somehow don't have it
> at
> > all. Was asked by my students, so I got the answer.
> >
> > Mark
> >
> >
> > On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo
> > <Ro...@netapp.com> wrote:
> >>
> >> Mark,
> >>
> >> there we go ;-)
> >>
> >> Rodrigo Nascimento
> >> Systems Engineer @ Brazil
> >> Mobile +55 11 991.873.810
> >>
> >> Sent from my iPhone
> >>
> >> Begin forwarded message:
> >>
> >> From: "Nascimento, Rodrigo" <Ro...@netapp.com>
> >> Date: 24 de janeiro de 2014 15:59:33 BRST
> >> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> >> Subject: RE: hdfs fsck -locations
> >>
> >> Hi Mark,
> >>
> >> It is a sample from my sandbox. Your question is about the part that is
> in
> >> RED at the output below, right?
> >>
> >> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
> >> Connecting to namenode via http://sandbox.hortonworks.com:50070
> >> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
> >> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
> >> .
> >> /user/ambari-qa/passwd: Under replicated
> >> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target
> Replicas
> >> is 3 but found 1 replica(s).
> >> Status: HEALTHY
> >> Total size: 1708 B
> >> Total dirs: 0
> >> Total files: 1
> >> Total symlinks: 0
> >> Total blocks (validated): 1 (avg. block size 1708 B)
> >> Minimally replicated blocks: 1 (100.0 %)
> >> Over-replicated blocks: 0 (0.0 %)
> >> Under-replicated blocks: 1 (100.0 %)
> >> Mis-replicated blocks: 0 (0.0 %)
> >> Default replication factor: 3
> >> Average block replication: 1.0
> >> Corrupt blocks: 0
> >> Missing replicas: 2 (66.666664 %)
> >> Number of data-nodes: 1
> >> Number of racks: 1
> >> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
> >>
> >>
> >> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
> >> [root@sandbox ~]#
> >>
> >> Rod Nascimento
> >>
> >> ________________________________
> >> From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
> >> Sent: Friday, January 24, 2014 3:34 PM
> >> To: user@hadoop.apache.org
> >> Subject: RE: hdfs fsck -locations
> >>
> >> I’m not seeing locations flag yet.
> >>
> >>
> >>
> >> Rod Nascimento
> >>
> >> Systems Engineer @ Brazil
> >>
> >>
> >>
> >> People don’t buy WHAT you do. They buy WHY you do it.
> >>
> >>
> >>
> >> From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> >> Sent: Friday, January 24, 2014 3:16 PM
> >> To: Hadoop User
> >> Subject: Re: hdfs fsck -locations
> >>
> >>
> >>
> >> Sorry, did not copy the full command
> >>
> >>
> >>
> >> hdfs fsck /user/mark/data/word_count.csv -locations
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >> Total size: 7217 B
> >>
> >> Total dirs: 0
> >>
> >> Total files: 1
> >>
> >> Total blocks (validated): 1 (avg. block size 7217 B)
> >>
> >> Minimally replicated blocks: 1 (100.0 %)
> >>
> >> Over-replicated blocks: 0 (0.0 %)
> >>
> >> Under-replicated blocks: 0 (0.0 %)
> >>
> >> Mis-replicated blocks: 0 (0.0 %)
> >>
> >> Default replication factor: 1
> >>
> >> Average block replication: 1.0
> >>
> >> Corrupt blocks: 0
> >>
> >> Missing replicas: 0 (0.0 %)
> >>
> >> Number of data-nodes: 1
> >>
> >> Number of racks: 1
> >>
> >> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >> Sorry, but what was the question? I also do not see a locations option
> >> flag.
> >>
> >> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>
> wrote:
> >>
> >> Here is an example
> >>
> >>
> >>
> >> hdfs fsck /user/mark/data/word_count.csv
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >> Total size: 7217 B
> >>
> >> Total dirs: 0
> >>
> >> Total files: 1
> >>
> >> Total blocks (validated): 1 (avg. block size 7217 B)
> >>
> >> Minimally replicated blocks: 1 (100.0 %)
> >>
> >> Over-replicated blocks: 0 (0.0 %)
> >>
> >> Under-replicated blocks: 0 (0.0 %)
> >>
> >> Mis-replicated blocks: 0 (0.0 %)
> >>
> >> Default replication factor: 1
> >>
> >> Average block replication: 1.0
> >>
> >> Corrupt blocks: 0
> >>
> >> Missing replicas: 0 (0.0 %)
> >>
> >> Number of data-nodes: 1
> >>
> >> Number of racks: 1
> >>
> >> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >> Hi Mark,
> >>
> >> Yes, the locations are shown as IP.
> >>
> >>
> >> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <
> mark.kerzner@shmsoft.com>
> >> wrote:
> >> > Hi,
> >> >
> >> > hdfs fsck -locations
> >> >
> >> > is supposed to show every block with its location? Is location the ip
> of
> >> > the
> >> > datanode?
> >> >
> >> > Thank you,
> >> > Mark
> >>
> >>
> >> --
> >> Harsh J
> >>
> >>
> >>
> >>
> >
> >
>
>
>
> --
> Harsh J
>
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
The right syntax is to use "-files -blocks -locations", so it drills
down all the way. You are not "missing" a feature - this has existed
for as long as I've known HDFS.
In Rodrigo's output, he's seeing a BlockPool ID, which is not
equivalent to a location, but just carries an IP in it for
identification purposes.
On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <ma...@shmsoft.com> wrote:
> Yes, Rodrigo,
>
> that's what I was looking for. So in my install I somehow don't have it at
> all. Was asked by my students, so I got the answer.
>
> Mark
>
>
> On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo
> <Ro...@netapp.com> wrote:
>>
>> Mark,
>>
>> there we go ;-)
>>
>> Rodrigo Nascimento
>> Systems Engineer @ Brazil
>> Mobile +55 11 991.873.810
>>
>> Sent from my iPhone
>>
>> Begin forwarded message:
>>
>> From: "Nascimento, Rodrigo" <Ro...@netapp.com>
>> Date: 24 de janeiro de 2014 15:59:33 BRST
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: RE: hdfs fsck -locations
>>
>> Hi Mark,
>>
>> It is a sample from my sandbox. Your question is about the part that is in
>> RED at the output below, right?
>>
>> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
>> Connecting to namenode via http://sandbox.hortonworks.com:50070
>> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
>> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
>> .
>> /user/ambari-qa/passwd: Under replicated
>> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas
>> is 3 but found 1 replica(s).
>> Status: HEALTHY
>> Total size: 1708 B
>> Total dirs: 0
>> Total files: 1
>> Total symlinks: 0
>> Total blocks (validated): 1 (avg. block size 1708 B)
>> Minimally replicated blocks: 1 (100.0 %)
>> Over-replicated blocks: 0 (0.0 %)
>> Under-replicated blocks: 1 (100.0 %)
>> Mis-replicated blocks: 0 (0.0 %)
>> Default replication factor: 3
>> Average block replication: 1.0
>> Corrupt blocks: 0
>> Missing replicas: 2 (66.666664 %)
>> Number of data-nodes: 1
>> Number of racks: 1
>> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
>>
>>
>> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
>> [root@sandbox ~]#
>>
>> Rod Nascimento
>>
>> ________________________________
>> From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
>> Sent: Friday, January 24, 2014 3:34 PM
>> To: user@hadoop.apache.org
>> Subject: RE: hdfs fsck -locations
>>
>> I’m not seeing locations flag yet.
>>
>>
>>
>> Rod Nascimento
>>
>> Systems Engineer @ Brazil
>>
>>
>>
>> People don’t buy WHAT you do. They buy WHY you do it.
>>
>>
>>
>> From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>> Sent: Friday, January 24, 2014 3:16 PM
>> To: Hadoop User
>> Subject: Re: hdfs fsck -locations
>>
>>
>>
>> Sorry, did not copy the full command
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv -locations
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>
>>
>>
>>
>>
>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Sorry, but what was the question? I also do not see a locations option
>> flag.
>>
>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>> Here is an example
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip of
>> > the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>> --
>> Harsh J
>>
>>
>>
>>
>
>
--
Harsh J
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
The right syntax is to use "-files -blocks -locations", so it drills
down all the way. You are not "missing" a feature - this has existed
for as long as I've known HDFS.
In Rodrigo's output, he's seeing a BlockPool ID, which is not
equivalent to a location, but just carries an IP in it for
identification purposes.
On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <ma...@shmsoft.com> wrote:
> Yes, Rodrigo,
>
> that's what I was looking for. So in my install I somehow don't have it at
> all. Was asked by my students, so I got the answer.
>
> Mark
>
>
> On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo
> <Ro...@netapp.com> wrote:
>>
>> Mark,
>>
>> there we go ;-)
>>
>> Rodrigo Nascimento
>> Systems Engineer @ Brazil
>> Mobile +55 11 991.873.810
>>
>> Sent from my iPhone
>>
>> Begin forwarded message:
>>
>> From: "Nascimento, Rodrigo" <Ro...@netapp.com>
>> Date: 24 de janeiro de 2014 15:59:33 BRST
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: RE: hdfs fsck -locations
>>
>> Hi Mark,
>>
>> It is a sample from my sandbox. Your question is about the part that is in
>> RED at the output below, right?
>>
>> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
>> Connecting to namenode via http://sandbox.hortonworks.com:50070
>> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
>> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
>> .
>> /user/ambari-qa/passwd: Under replicated
>> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas
>> is 3 but found 1 replica(s).
>> Status: HEALTHY
>> Total size: 1708 B
>> Total dirs: 0
>> Total files: 1
>> Total symlinks: 0
>> Total blocks (validated): 1 (avg. block size 1708 B)
>> Minimally replicated blocks: 1 (100.0 %)
>> Over-replicated blocks: 0 (0.0 %)
>> Under-replicated blocks: 1 (100.0 %)
>> Mis-replicated blocks: 0 (0.0 %)
>> Default replication factor: 3
>> Average block replication: 1.0
>> Corrupt blocks: 0
>> Missing replicas: 2 (66.666664 %)
>> Number of data-nodes: 1
>> Number of racks: 1
>> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
>>
>>
>> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
>> [root@sandbox ~]#
>>
>> Rod Nascimento
>>
>> ________________________________
>> From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
>> Sent: Friday, January 24, 2014 3:34 PM
>> To: user@hadoop.apache.org
>> Subject: RE: hdfs fsck -locations
>>
>> I’m not seeing locations flag yet.
>>
>>
>>
>> Rod Nascimento
>>
>> Systems Engineer @ Brazil
>>
>>
>>
>> People don’t buy WHAT you do. They buy WHY you do it.
>>
>>
>>
>> From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>> Sent: Friday, January 24, 2014 3:16 PM
>> To: Hadoop User
>> Subject: Re: hdfs fsck -locations
>>
>>
>>
>> Sorry, did not copy the full command
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv -locations
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>
>>
>>
>>
>>
>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Sorry, but what was the question? I also do not see a locations option
>> flag.
>>
>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>> Here is an example
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip of
>> > the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>> --
>> Harsh J
>>
>>
>>
>>
>
>
--
Harsh J
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
The right syntax is to use "-files -blocks -locations", so it drills
down all the way. You are not "missing" a feature - this has existed
for as long as I've known HDFS.
In Rodrigo's output, he's seeing a BlockPool ID, which is not
equivalent to a location, but just carries an IP in it for
identification purposes.
On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <ma...@shmsoft.com> wrote:
> Yes, Rodrigo,
>
> that's what I was looking for. So in my install I somehow don't have it at
> all. Was asked by my students, so I got the answer.
>
> Mark
>
>
> On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo
> <Ro...@netapp.com> wrote:
>>
>> Mark,
>>
>> there we go ;-)
>>
>> Rodrigo Nascimento
>> Systems Engineer @ Brazil
>> Mobile +55 11 991.873.810
>>
>> Sent from my iPhone
>>
>> Begin forwarded message:
>>
>> From: "Nascimento, Rodrigo" <Ro...@netapp.com>
>> Date: 24 de janeiro de 2014 15:59:33 BRST
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: RE: hdfs fsck -locations
>>
>> Hi Mark,
>>
>> It is a sample from my sandbox. Your question is about the part that is in
>> RED at the output below, right?
>>
>> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
>> Connecting to namenode via http://sandbox.hortonworks.com:50070
>> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
>> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
>> .
>> /user/ambari-qa/passwd: Under replicated
>> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas
>> is 3 but found 1 replica(s).
>> Status: HEALTHY
>> Total size: 1708 B
>> Total dirs: 0
>> Total files: 1
>> Total symlinks: 0
>> Total blocks (validated): 1 (avg. block size 1708 B)
>> Minimally replicated blocks: 1 (100.0 %)
>> Over-replicated blocks: 0 (0.0 %)
>> Under-replicated blocks: 1 (100.0 %)
>> Mis-replicated blocks: 0 (0.0 %)
>> Default replication factor: 3
>> Average block replication: 1.0
>> Corrupt blocks: 0
>> Missing replicas: 2 (66.666664 %)
>> Number of data-nodes: 1
>> Number of racks: 1
>> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
>>
>>
>> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
>> [root@sandbox ~]#
>>
>> Rod Nascimento
>>
>> ________________________________
>> From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
>> Sent: Friday, January 24, 2014 3:34 PM
>> To: user@hadoop.apache.org
>> Subject: RE: hdfs fsck -locations
>>
>> I’m not seeing locations flag yet.
>>
>>
>>
>> Rod Nascimento
>>
>> Systems Engineer @ Brazil
>>
>>
>>
>> People don’t buy WHAT you do. They buy WHY you do it.
>>
>>
>>
>> From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>> Sent: Friday, January 24, 2014 3:16 PM
>> To: Hadoop User
>> Subject: Re: hdfs fsck -locations
>>
>>
>>
>> Sorry, did not copy the full command
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv -locations
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>
>>
>>
>>
>>
>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Sorry, but what was the question? I also do not see a locations option
>> flag.
>>
>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>> Here is an example
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip of
>> > the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>> --
>> Harsh J
>>
>>
>>
>>
>
>
--
Harsh J
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
The right syntax is to use "-files -blocks -locations", so it drills
down all the way. You are not "missing" a feature - this has existed
for as long as I've known HDFS.
In Rodrigo's output, he's seeing a BlockPool ID, which is not
equivalent to a location, but just carries an IP in it for
identification purposes.
On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <ma...@shmsoft.com> wrote:
> Yes, Rodrigo,
>
> that's what I was looking for. So in my install I somehow don't have it at
> all. Was asked by my students, so I got the answer.
>
> Mark
>
>
> On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo
> <Ro...@netapp.com> wrote:
>>
>> Mark,
>>
>> there we go ;-)
>>
>> Rodrigo Nascimento
>> Systems Engineer @ Brazil
>> Mobile +55 11 991.873.810
>>
>> Sent from my iPhone
>>
>> Begin forwarded message:
>>
>> From: "Nascimento, Rodrigo" <Ro...@netapp.com>
>> Date: 24 de janeiro de 2014 15:59:33 BRST
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: RE: hdfs fsck -locations
>>
>> Hi Mark,
>>
>> It is a sample from my sandbox. Your question is about the part that is in
>> RED at the output below, right?
>>
>> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
>> Connecting to namenode via http://sandbox.hortonworks.com:50070
>> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
>> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
>> .
>> /user/ambari-qa/passwd: Under replicated
>> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas
>> is 3 but found 1 replica(s).
>> Status: HEALTHY
>> Total size: 1708 B
>> Total dirs: 0
>> Total files: 1
>> Total symlinks: 0
>> Total blocks (validated): 1 (avg. block size 1708 B)
>> Minimally replicated blocks: 1 (100.0 %)
>> Over-replicated blocks: 0 (0.0 %)
>> Under-replicated blocks: 1 (100.0 %)
>> Mis-replicated blocks: 0 (0.0 %)
>> Default replication factor: 3
>> Average block replication: 1.0
>> Corrupt blocks: 0
>> Missing replicas: 2 (66.666664 %)
>> Number of data-nodes: 1
>> Number of racks: 1
>> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
>>
>>
>> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
>> [root@sandbox ~]#
>>
>> Rod Nascimento
>>
>> ________________________________
>> From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
>> Sent: Friday, January 24, 2014 3:34 PM
>> To: user@hadoop.apache.org
>> Subject: RE: hdfs fsck -locations
>>
>> I’m not seeing locations flag yet.
>>
>>
>>
>> Rod Nascimento
>>
>> Systems Engineer @ Brazil
>>
>>
>>
>> People don’t buy WHAT you do. They buy WHY you do it.
>>
>>
>>
>> From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>> Sent: Friday, January 24, 2014 3:16 PM
>> To: Hadoop User
>> Subject: Re: hdfs fsck -locations
>>
>>
>>
>> Sorry, did not copy the full command
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv -locations
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>
>>
>>
>>
>>
>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Sorry, but what was the question? I also do not see a locations option
>> flag.
>>
>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>> Here is an example
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip of
>> > the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>> --
>> Harsh J
>>
>>
>>
>>
>
>
--
Harsh J
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Yes, Rodrigo,
that's what I was looking for. So in my install I somehow don't have it at
all. Was asked by my students, so I got the answer.
Mark
On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Mark,
>
> there we go ;-)
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
> Mobile +55 11 991.873.810
>
> Sent from my iPhone
>
> Begin forwarded message:
>
> *From:* "Nascimento, Rodrigo" <Ro...@netapp.com>
> *Date:* 24 de janeiro de 2014 15:59:33 BRST
> *To:* "user@hadoop.apache.org" <us...@hadoop.apache.org>
> *Subject:* *RE: hdfs fsck -locations*
>
> Hi Mark,
>
> It is a sample from my sandbox. Your question is about the part that is in
> RED at the output below, right?
>
> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
> Connecting to namenode via http://sandbox.hortonworks.com:50070
> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
> .
> */user/ambari-qa/passwd: Under replicated
> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas
> is 3 but found 1 replica(s).*
> Status: HEALTHY
> Total size: 1708 B
> Total dirs: 0
> Total files: 1
> Total symlinks: 0
> Total blocks (validated): 1 (avg. block size 1708 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 1 (100.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 3
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 2 (66.666664 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
>
>
> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
> [root@sandbox ~]#
>
> Rod Nascimento
>
> ------------------------------
> *From:* Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
> *Sent:* Friday, January 24, 2014 3:34 PM
> *To:* user@hadoop.apache.org
> *Subject:* RE: hdfs fsck -locations
>
> I’m not seeing locations flag yet.
>
>
>
> *Rod Nascimento*
>
> *Systems Engineer @ Brazil*
>
>
>
> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>
>
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
>
> *Sent:* Friday, January 24, 2014 3:16 PM
> *To:* Hadoop User
> *Subject:* Re: hdfs fsck -locations
>
>
>
> Sorry, did not copy the full command
>
>
>
> hdfs fsck /user/mark/data/word_count.csv -locations
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>
>
>
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>
>
>
>
>
> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Sorry, but what was the question? I also do not see a locations option
> flag.
>
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> Here is an example
>
>
>
> hdfs fsck /user/mark/data/word_count.csv
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
> --
> Harsh J
>
>
>
>
>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Yes, Rodrigo,
that's what I was looking for. So in my install I somehow don't have it at
all. Was asked by my students, so I got the answer.
Mark
On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Mark,
>
> there we go ;-)
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
> Mobile +55 11 991.873.810
>
> Sent from my iPhone
>
> Begin forwarded message:
>
> *From:* "Nascimento, Rodrigo" <Ro...@netapp.com>
> *Date:* 24 de janeiro de 2014 15:59:33 BRST
> *To:* "user@hadoop.apache.org" <us...@hadoop.apache.org>
> *Subject:* *RE: hdfs fsck -locations*
>
> Hi Mark,
>
> It is a sample from my sandbox. Your question is about the part that is in
> RED at the output below, right?
>
> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
> Connecting to namenode via http://sandbox.hortonworks.com:50070
> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
> .
> */user/ambari-qa/passwd: Under replicated
> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas
> is 3 but found 1 replica(s).*
> Status: HEALTHY
> Total size: 1708 B
> Total dirs: 0
> Total files: 1
> Total symlinks: 0
> Total blocks (validated): 1 (avg. block size 1708 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 1 (100.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 3
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 2 (66.666664 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
>
>
> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
> [root@sandbox ~]#
>
> Rod Nascimento
>
> ------------------------------
> *From:* Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
> *Sent:* Friday, January 24, 2014 3:34 PM
> *To:* user@hadoop.apache.org
> *Subject:* RE: hdfs fsck -locations
>
> I’m not seeing locations flag yet.
>
>
>
> *Rod Nascimento*
>
> *Systems Engineer @ Brazil*
>
>
>
> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>
>
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
>
> *Sent:* Friday, January 24, 2014 3:16 PM
> *To:* Hadoop User
> *Subject:* Re: hdfs fsck -locations
>
>
>
> Sorry, did not copy the full command
>
>
>
> hdfs fsck /user/mark/data/word_count.csv -locations
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>
>
>
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>
>
>
>
>
> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Sorry, but what was the question? I also do not see a locations option
> flag.
>
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> Here is an example
>
>
>
> hdfs fsck /user/mark/data/word_count.csv
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
> --
> Harsh J
>
>
>
>
>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Yes, Rodrigo,
that's what I was looking for. So in my install I somehow don't have it at
all. Was asked by my students, so I got the answer.
Mark
On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Mark,
>
> there we go ;-)
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
> Mobile +55 11 991.873.810
>
> Sent from my iPhone
>
> Begin forwarded message:
>
> *From:* "Nascimento, Rodrigo" <Ro...@netapp.com>
> *Date:* 24 de janeiro de 2014 15:59:33 BRST
> *To:* "user@hadoop.apache.org" <us...@hadoop.apache.org>
> *Subject:* *RE: hdfs fsck -locations*
>
> Hi Mark,
>
> It is a sample from my sandbox. Your question is about the part that is in
> RED at the output below, right?
>
> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
> Connecting to namenode via http://sandbox.hortonworks.com:50070
> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
> .
> */user/ambari-qa/passwd: Under replicated
> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas
> is 3 but found 1 replica(s).*
> Status: HEALTHY
> Total size: 1708 B
> Total dirs: 0
> Total files: 1
> Total symlinks: 0
> Total blocks (validated): 1 (avg. block size 1708 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 1 (100.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 3
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 2 (66.666664 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
>
>
> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
> [root@sandbox ~]#
>
> Rod Nascimento
>
> ------------------------------
> *From:* Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
> *Sent:* Friday, January 24, 2014 3:34 PM
> *To:* user@hadoop.apache.org
> *Subject:* RE: hdfs fsck -locations
>
> I’m not seeing locations flag yet.
>
>
>
> *Rod Nascimento*
>
> *Systems Engineer @ Brazil*
>
>
>
> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>
>
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
>
> *Sent:* Friday, January 24, 2014 3:16 PM
> *To:* Hadoop User
> *Subject:* Re: hdfs fsck -locations
>
>
>
> Sorry, did not copy the full command
>
>
>
> hdfs fsck /user/mark/data/word_count.csv -locations
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>
>
>
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>
>
>
>
>
> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Sorry, but what was the question? I also do not see a locations option
> flag.
>
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> Here is an example
>
>
>
> hdfs fsck /user/mark/data/word_count.csv
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
> --
> Harsh J
>
>
>
>
>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Yes, Rodrigo,
that's what I was looking for. So in my install I somehow don't have it at
all. Was asked by my students, so I got the answer.
Mark
On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Mark,
>
> there we go ;-)
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
> Mobile +55 11 991.873.810
>
> Sent from my iPhone
>
> Begin forwarded message:
>
> *From:* "Nascimento, Rodrigo" <Ro...@netapp.com>
> *Date:* 24 de janeiro de 2014 15:59:33 BRST
> *To:* "user@hadoop.apache.org" <us...@hadoop.apache.org>
> *Subject:* *RE: hdfs fsck -locations*
>
> Hi Mark,
>
> It is a sample from my sandbox. Your question is about the part that is in
> RED at the output below, right?
>
> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
> Connecting to namenode via http://sandbox.hortonworks.com:50070
> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
> .
> */user/ambari-qa/passwd: Under replicated
> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas
> is 3 but found 1 replica(s).*
> Status: HEALTHY
> Total size: 1708 B
> Total dirs: 0
> Total files: 1
> Total symlinks: 0
> Total blocks (validated): 1 (avg. block size 1708 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 1 (100.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 3
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 2 (66.666664 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
>
>
> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
> [root@sandbox ~]#
>
> Rod Nascimento
>
> ------------------------------
> *From:* Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
> *Sent:* Friday, January 24, 2014 3:34 PM
> *To:* user@hadoop.apache.org
> *Subject:* RE: hdfs fsck -locations
>
> I’m not seeing locations flag yet.
>
>
>
> *Rod Nascimento*
>
> *Systems Engineer @ Brazil*
>
>
>
> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>
>
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
>
> *Sent:* Friday, January 24, 2014 3:16 PM
> *To:* Hadoop User
> *Subject:* Re: hdfs fsck -locations
>
>
>
> Sorry, did not copy the full command
>
>
>
> hdfs fsck /user/mark/data/word_count.csv -locations
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>
>
>
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>
>
>
>
>
> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Sorry, but what was the question? I also do not see a locations option
> flag.
>
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> Here is an example
>
>
>
> hdfs fsck /user/mark/data/word_count.csv
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
> --
> Harsh J
>
>
>
>
>
>
Fwd: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Mark,
there we go ;-)
Rodrigo Nascimento
Systems Engineer @ Brazil
Mobile +55 11 991.873.810
Sent from my iPhone
Begin forwarded message:
From: "Nascimento, Rodrigo" <Ro...@netapp.com>>
Date: 24 de janeiro de 2014 15:59:33 BRST
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: RE: hdfs fsck -locations
Hi Mark,
It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right?
[root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
Connecting to namenode via http://sandbox.hortonworks.com:50070
FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
.
/user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s).
Status: HEALTHY
Total size: 1708 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 1708 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 2 (66.666664 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
[root@sandbox ~]#
Rod Nascimento
________________________________
From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com<ma...@netapp.com>]
Sent: Friday, January 24, 2014 3:34 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: hdfs fsck -locations
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Fwd: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Mark,
there we go ;-)
Rodrigo Nascimento
Systems Engineer @ Brazil
Mobile +55 11 991.873.810
Sent from my iPhone
Begin forwarded message:
From: "Nascimento, Rodrigo" <Ro...@netapp.com>>
Date: 24 de janeiro de 2014 15:59:33 BRST
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: RE: hdfs fsck -locations
Hi Mark,
It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right?
[root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
Connecting to namenode via http://sandbox.hortonworks.com:50070
FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
.
/user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s).
Status: HEALTHY
Total size: 1708 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 1708 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 2 (66.666664 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
[root@sandbox ~]#
Rod Nascimento
________________________________
From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com<ma...@netapp.com>]
Sent: Friday, January 24, 2014 3:34 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: hdfs fsck -locations
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Fwd: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Mark,
there we go ;-)
Rodrigo Nascimento
Systems Engineer @ Brazil
Mobile +55 11 991.873.810
Sent from my iPhone
Begin forwarded message:
From: "Nascimento, Rodrigo" <Ro...@netapp.com>>
Date: 24 de janeiro de 2014 15:59:33 BRST
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: RE: hdfs fsck -locations
Hi Mark,
It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right?
[root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
Connecting to namenode via http://sandbox.hortonworks.com:50070
FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
.
/user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s).
Status: HEALTHY
Total size: 1708 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 1708 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 2 (66.666664 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
[root@sandbox ~]#
Rod Nascimento
________________________________
From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com<ma...@netapp.com>]
Sent: Friday, January 24, 2014 3:34 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: hdfs fsck -locations
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Fwd: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Mark,
there we go ;-)
Rodrigo Nascimento
Systems Engineer @ Brazil
Mobile +55 11 991.873.810
Sent from my iPhone
Begin forwarded message:
From: "Nascimento, Rodrigo" <Ro...@netapp.com>>
Date: 24 de janeiro de 2014 15:59:33 BRST
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: RE: hdfs fsck -locations
Hi Mark,
It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right?
[root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
Connecting to namenode via http://sandbox.hortonworks.com:50070
FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
.
/user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s).
Status: HEALTHY
Total size: 1708 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 1708 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 2 (66.666664 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
[root@sandbox ~]#
Rod Nascimento
________________________________
From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com<ma...@netapp.com>]
Sent: Friday, January 24, 2014 3:34 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: hdfs fsck -locations
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
RE: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Hi Mark,
It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right?
[root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
Connecting to namenode via http://sandbox.hortonworks.com:50070
FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
.
/user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s).
Status: HEALTHY
Total size: 1708 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 1708 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 2 (66.666664 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
[root@sandbox ~]#
Rod Nascimento
________________________________
From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
Sent: Friday, January 24, 2014 3:34 PM
To: user@hadoop.apache.org
Subject: RE: hdfs fsck -locations
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Can you send me your output?
hadoop version
Hadoop 2.0.0-cdh4.5.0
Subversion
git://ubuntu64-12-04-mk1/var/lib/jenkins/workspace/generic-package-ubuntu64-12-04/CDH4.5.0-Packaging-Hadoop-2013-11-20_14-31-53/hadoop-2.0.0+1518-1.cdh4.5.0.p0.24~precise/src/hadoop-common-project/hadoop-common
-r 8e266e052e423af592871e2dfe09d54c03f6a0e8
Compiled by jenkins on Wed Nov 20 15:10:35 PST 2013
>From source with checksum 9848b0f85b461913ed63fa19c2b79ccc
This command was run using /usr/lib/hadoop/hadoop-common-2.0.0-cdh4.5.0.jar
On Fri, Jan 24, 2014 at 3:14 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Mark,
>
> Did you see that your output is different from mine?
>
> which Is your hadoop versioks
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
> Mobile +55 11 991.873.810
>
> Sent from my iPhone
>
> On 24/01/2014, at 18:31, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> HI, Rodrigo, I am fine thank you :)
>
> Here is the _complete_ output
>
> mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
> Connecting to namenode via http://mark-7:50070
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
> .Status: HEALTHY
> Total size: 7217 B
> Total dirs: 0
> Total files: 1
> Total blocks (validated): 1 (avg. block size 7217 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 0 (0.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 1
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 0 (0.0 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
> mark@mark-7:~$
>
>
> On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <
> Rodrigo.Nascimento@netapp.com> wrote:
>
>> Hi Mark,
>>
>> How are you?
>>
>> at your output from word_count.csv, the portion related to the nlock
>> locations is missing. This is the reason why I told you in my previous
>> message that "I'm not seeing locations yet."
>>
>> please, take a look at my last e-mail.
>>
>> All the best,
>>
>> Rodrigo Nascimento
>> Systems Engineer @ Brazil
>>
>> Sent from my iPhone
>>
>> On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv *-locations*
>>
>>
>> On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
>> Rodrigo.Nascimento@netapp.com> wrote:
>>
>>> I’m not seeing locations flag yet.
>>>
>>>
>>>
>>> *Rod Nascimento*
>>>
>>> *Systems Engineer @ Brazil*
>>>
>>>
>>>
>>> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>>>
>>>
>>>
>>> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>>> *Sent:* Friday, January 24, 2014 3:16 PM
>>> *To:* Hadoop User
>>> *Subject:* Re: hdfs fsck -locations
>>>
>>>
>>>
>>> Sorry, did not copy the full command
>>>
>>>
>>>
>>> hdfs fsck /user/mark/data/word_count.csv -locations
>>>
>>> Connecting to namenode via http://mark-7:50070
>>>
>>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>>
>>> .Status: HEALTHY
>>>
>>> Total size: 7217 B
>>>
>>> Total dirs: 0
>>>
>>> Total files: 1
>>>
>>> Total blocks (validated): 1 (avg. block size 7217 B)
>>>
>>> Minimally replicated blocks: 1 (100.0 %)
>>>
>>> Over-replicated blocks: 0 (0.0 %)
>>>
>>> Under-replicated blocks: 0 (0.0 %)
>>>
>>> Mis-replicated blocks: 0 (0.0 %)
>>>
>>> Default replication factor: 1
>>>
>>> Average block replication: 1.0
>>>
>>> Corrupt blocks: 0
>>>
>>> Missing replicas: 0 (0.0 %)
>>>
>>> Number of data-nodes: 1
>>>
>>> Number of racks: 1
>>>
>>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>>
>>>
>>>
>>>
>>>
>>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Sorry, but what was the question? I also do not see a locations option
>>> flag.
>>>
>>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>
>>> wrote:
>>>
>>> Here is an example
>>>
>>>
>>>
>>> hdfs fsck /user/mark/data/word_count.csv
>>>
>>> Connecting to namenode via http://mark-7:50070
>>>
>>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>>
>>> .Status: HEALTHY
>>>
>>> Total size: 7217 B
>>>
>>> Total dirs: 0
>>>
>>> Total files: 1
>>>
>>> Total blocks (validated): 1 (avg. block size 7217 B)
>>>
>>> Minimally replicated blocks: 1 (100.0 %)
>>>
>>> Over-replicated blocks: 0 (0.0 %)
>>>
>>> Under-replicated blocks: 0 (0.0 %)
>>>
>>> Mis-replicated blocks: 0 (0.0 %)
>>>
>>> Default replication factor: 1
>>>
>>> Average block replication: 1.0
>>>
>>> Corrupt blocks: 0
>>>
>>> Missing replicas: 0 (0.0 %)
>>>
>>> Number of data-nodes: 1
>>>
>>> Number of racks: 1
>>>
>>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Hi Mark,
>>>
>>> Yes, the locations are shown as IP.
>>>
>>>
>>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > hdfs fsck -locations
>>> >
>>> > is supposed to show every block with its location? Is location the ip
>>> of the
>>> > datanode?
>>> >
>>> > Thank you,
>>> > Mark
>>>
>>>
>>> --
>>> Harsh J
>>>
>>>
>>>
>>>
>>>
>>
>>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Can you send me your output?
hadoop version
Hadoop 2.0.0-cdh4.5.0
Subversion
git://ubuntu64-12-04-mk1/var/lib/jenkins/workspace/generic-package-ubuntu64-12-04/CDH4.5.0-Packaging-Hadoop-2013-11-20_14-31-53/hadoop-2.0.0+1518-1.cdh4.5.0.p0.24~precise/src/hadoop-common-project/hadoop-common
-r 8e266e052e423af592871e2dfe09d54c03f6a0e8
Compiled by jenkins on Wed Nov 20 15:10:35 PST 2013
>From source with checksum 9848b0f85b461913ed63fa19c2b79ccc
This command was run using /usr/lib/hadoop/hadoop-common-2.0.0-cdh4.5.0.jar
On Fri, Jan 24, 2014 at 3:14 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Mark,
>
> Did you see that your output is different from mine?
>
> which Is your hadoop versioks
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
> Mobile +55 11 991.873.810
>
> Sent from my iPhone
>
> On 24/01/2014, at 18:31, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> HI, Rodrigo, I am fine thank you :)
>
> Here is the _complete_ output
>
> mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
> Connecting to namenode via http://mark-7:50070
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
> .Status: HEALTHY
> Total size: 7217 B
> Total dirs: 0
> Total files: 1
> Total blocks (validated): 1 (avg. block size 7217 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 0 (0.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 1
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 0 (0.0 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
> mark@mark-7:~$
>
>
> On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <
> Rodrigo.Nascimento@netapp.com> wrote:
>
>> Hi Mark,
>>
>> How are you?
>>
>> at your output from word_count.csv, the portion related to the nlock
>> locations is missing. This is the reason why I told you in my previous
>> message that "I'm not seeing locations yet."
>>
>> please, take a look at my last e-mail.
>>
>> All the best,
>>
>> Rodrigo Nascimento
>> Systems Engineer @ Brazil
>>
>> Sent from my iPhone
>>
>> On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv *-locations*
>>
>>
>> On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
>> Rodrigo.Nascimento@netapp.com> wrote:
>>
>>> I’m not seeing locations flag yet.
>>>
>>>
>>>
>>> *Rod Nascimento*
>>>
>>> *Systems Engineer @ Brazil*
>>>
>>>
>>>
>>> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>>>
>>>
>>>
>>> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>>> *Sent:* Friday, January 24, 2014 3:16 PM
>>> *To:* Hadoop User
>>> *Subject:* Re: hdfs fsck -locations
>>>
>>>
>>>
>>> Sorry, did not copy the full command
>>>
>>>
>>>
>>> hdfs fsck /user/mark/data/word_count.csv -locations
>>>
>>> Connecting to namenode via http://mark-7:50070
>>>
>>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>>
>>> .Status: HEALTHY
>>>
>>> Total size: 7217 B
>>>
>>> Total dirs: 0
>>>
>>> Total files: 1
>>>
>>> Total blocks (validated): 1 (avg. block size 7217 B)
>>>
>>> Minimally replicated blocks: 1 (100.0 %)
>>>
>>> Over-replicated blocks: 0 (0.0 %)
>>>
>>> Under-replicated blocks: 0 (0.0 %)
>>>
>>> Mis-replicated blocks: 0 (0.0 %)
>>>
>>> Default replication factor: 1
>>>
>>> Average block replication: 1.0
>>>
>>> Corrupt blocks: 0
>>>
>>> Missing replicas: 0 (0.0 %)
>>>
>>> Number of data-nodes: 1
>>>
>>> Number of racks: 1
>>>
>>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>>
>>>
>>>
>>>
>>>
>>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Sorry, but what was the question? I also do not see a locations option
>>> flag.
>>>
>>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>
>>> wrote:
>>>
>>> Here is an example
>>>
>>>
>>>
>>> hdfs fsck /user/mark/data/word_count.csv
>>>
>>> Connecting to namenode via http://mark-7:50070
>>>
>>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>>
>>> .Status: HEALTHY
>>>
>>> Total size: 7217 B
>>>
>>> Total dirs: 0
>>>
>>> Total files: 1
>>>
>>> Total blocks (validated): 1 (avg. block size 7217 B)
>>>
>>> Minimally replicated blocks: 1 (100.0 %)
>>>
>>> Over-replicated blocks: 0 (0.0 %)
>>>
>>> Under-replicated blocks: 0 (0.0 %)
>>>
>>> Mis-replicated blocks: 0 (0.0 %)
>>>
>>> Default replication factor: 1
>>>
>>> Average block replication: 1.0
>>>
>>> Corrupt blocks: 0
>>>
>>> Missing replicas: 0 (0.0 %)
>>>
>>> Number of data-nodes: 1
>>>
>>> Number of racks: 1
>>>
>>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Hi Mark,
>>>
>>> Yes, the locations are shown as IP.
>>>
>>>
>>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > hdfs fsck -locations
>>> >
>>> > is supposed to show every block with its location? Is location the ip
>>> of the
>>> > datanode?
>>> >
>>> > Thank you,
>>> > Mark
>>>
>>>
>>> --
>>> Harsh J
>>>
>>>
>>>
>>>
>>>
>>
>>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Can you send me your output?
hadoop version
Hadoop 2.0.0-cdh4.5.0
Subversion
git://ubuntu64-12-04-mk1/var/lib/jenkins/workspace/generic-package-ubuntu64-12-04/CDH4.5.0-Packaging-Hadoop-2013-11-20_14-31-53/hadoop-2.0.0+1518-1.cdh4.5.0.p0.24~precise/src/hadoop-common-project/hadoop-common
-r 8e266e052e423af592871e2dfe09d54c03f6a0e8
Compiled by jenkins on Wed Nov 20 15:10:35 PST 2013
>From source with checksum 9848b0f85b461913ed63fa19c2b79ccc
This command was run using /usr/lib/hadoop/hadoop-common-2.0.0-cdh4.5.0.jar
On Fri, Jan 24, 2014 at 3:14 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Mark,
>
> Did you see that your output is different from mine?
>
> which Is your hadoop versioks
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
> Mobile +55 11 991.873.810
>
> Sent from my iPhone
>
> On 24/01/2014, at 18:31, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> HI, Rodrigo, I am fine thank you :)
>
> Here is the _complete_ output
>
> mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
> Connecting to namenode via http://mark-7:50070
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
> .Status: HEALTHY
> Total size: 7217 B
> Total dirs: 0
> Total files: 1
> Total blocks (validated): 1 (avg. block size 7217 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 0 (0.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 1
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 0 (0.0 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
> mark@mark-7:~$
>
>
> On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <
> Rodrigo.Nascimento@netapp.com> wrote:
>
>> Hi Mark,
>>
>> How are you?
>>
>> at your output from word_count.csv, the portion related to the nlock
>> locations is missing. This is the reason why I told you in my previous
>> message that "I'm not seeing locations yet."
>>
>> please, take a look at my last e-mail.
>>
>> All the best,
>>
>> Rodrigo Nascimento
>> Systems Engineer @ Brazil
>>
>> Sent from my iPhone
>>
>> On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv *-locations*
>>
>>
>> On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
>> Rodrigo.Nascimento@netapp.com> wrote:
>>
>>> I’m not seeing locations flag yet.
>>>
>>>
>>>
>>> *Rod Nascimento*
>>>
>>> *Systems Engineer @ Brazil*
>>>
>>>
>>>
>>> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>>>
>>>
>>>
>>> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>>> *Sent:* Friday, January 24, 2014 3:16 PM
>>> *To:* Hadoop User
>>> *Subject:* Re: hdfs fsck -locations
>>>
>>>
>>>
>>> Sorry, did not copy the full command
>>>
>>>
>>>
>>> hdfs fsck /user/mark/data/word_count.csv -locations
>>>
>>> Connecting to namenode via http://mark-7:50070
>>>
>>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>>
>>> .Status: HEALTHY
>>>
>>> Total size: 7217 B
>>>
>>> Total dirs: 0
>>>
>>> Total files: 1
>>>
>>> Total blocks (validated): 1 (avg. block size 7217 B)
>>>
>>> Minimally replicated blocks: 1 (100.0 %)
>>>
>>> Over-replicated blocks: 0 (0.0 %)
>>>
>>> Under-replicated blocks: 0 (0.0 %)
>>>
>>> Mis-replicated blocks: 0 (0.0 %)
>>>
>>> Default replication factor: 1
>>>
>>> Average block replication: 1.0
>>>
>>> Corrupt blocks: 0
>>>
>>> Missing replicas: 0 (0.0 %)
>>>
>>> Number of data-nodes: 1
>>>
>>> Number of racks: 1
>>>
>>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>>
>>>
>>>
>>>
>>>
>>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Sorry, but what was the question? I also do not see a locations option
>>> flag.
>>>
>>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>
>>> wrote:
>>>
>>> Here is an example
>>>
>>>
>>>
>>> hdfs fsck /user/mark/data/word_count.csv
>>>
>>> Connecting to namenode via http://mark-7:50070
>>>
>>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>>
>>> .Status: HEALTHY
>>>
>>> Total size: 7217 B
>>>
>>> Total dirs: 0
>>>
>>> Total files: 1
>>>
>>> Total blocks (validated): 1 (avg. block size 7217 B)
>>>
>>> Minimally replicated blocks: 1 (100.0 %)
>>>
>>> Over-replicated blocks: 0 (0.0 %)
>>>
>>> Under-replicated blocks: 0 (0.0 %)
>>>
>>> Mis-replicated blocks: 0 (0.0 %)
>>>
>>> Default replication factor: 1
>>>
>>> Average block replication: 1.0
>>>
>>> Corrupt blocks: 0
>>>
>>> Missing replicas: 0 (0.0 %)
>>>
>>> Number of data-nodes: 1
>>>
>>> Number of racks: 1
>>>
>>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Hi Mark,
>>>
>>> Yes, the locations are shown as IP.
>>>
>>>
>>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > hdfs fsck -locations
>>> >
>>> > is supposed to show every block with its location? Is location the ip
>>> of the
>>> > datanode?
>>> >
>>> > Thank you,
>>> > Mark
>>>
>>>
>>> --
>>> Harsh J
>>>
>>>
>>>
>>>
>>>
>>
>>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Can you send me your output?
hadoop version
Hadoop 2.0.0-cdh4.5.0
Subversion
git://ubuntu64-12-04-mk1/var/lib/jenkins/workspace/generic-package-ubuntu64-12-04/CDH4.5.0-Packaging-Hadoop-2013-11-20_14-31-53/hadoop-2.0.0+1518-1.cdh4.5.0.p0.24~precise/src/hadoop-common-project/hadoop-common
-r 8e266e052e423af592871e2dfe09d54c03f6a0e8
Compiled by jenkins on Wed Nov 20 15:10:35 PST 2013
>From source with checksum 9848b0f85b461913ed63fa19c2b79ccc
This command was run using /usr/lib/hadoop/hadoop-common-2.0.0-cdh4.5.0.jar
On Fri, Jan 24, 2014 at 3:14 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Mark,
>
> Did you see that your output is different from mine?
>
> which Is your hadoop versioks
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
> Mobile +55 11 991.873.810
>
> Sent from my iPhone
>
> On 24/01/2014, at 18:31, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> HI, Rodrigo, I am fine thank you :)
>
> Here is the _complete_ output
>
> mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
> Connecting to namenode via http://mark-7:50070
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
> .Status: HEALTHY
> Total size: 7217 B
> Total dirs: 0
> Total files: 1
> Total blocks (validated): 1 (avg. block size 7217 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 0 (0.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 1
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 0 (0.0 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
> mark@mark-7:~$
>
>
> On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <
> Rodrigo.Nascimento@netapp.com> wrote:
>
>> Hi Mark,
>>
>> How are you?
>>
>> at your output from word_count.csv, the portion related to the nlock
>> locations is missing. This is the reason why I told you in my previous
>> message that "I'm not seeing locations yet."
>>
>> please, take a look at my last e-mail.
>>
>> All the best,
>>
>> Rodrigo Nascimento
>> Systems Engineer @ Brazil
>>
>> Sent from my iPhone
>>
>> On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv *-locations*
>>
>>
>> On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
>> Rodrigo.Nascimento@netapp.com> wrote:
>>
>>> I’m not seeing locations flag yet.
>>>
>>>
>>>
>>> *Rod Nascimento*
>>>
>>> *Systems Engineer @ Brazil*
>>>
>>>
>>>
>>> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>>>
>>>
>>>
>>> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>>> *Sent:* Friday, January 24, 2014 3:16 PM
>>> *To:* Hadoop User
>>> *Subject:* Re: hdfs fsck -locations
>>>
>>>
>>>
>>> Sorry, did not copy the full command
>>>
>>>
>>>
>>> hdfs fsck /user/mark/data/word_count.csv -locations
>>>
>>> Connecting to namenode via http://mark-7:50070
>>>
>>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>>
>>> .Status: HEALTHY
>>>
>>> Total size: 7217 B
>>>
>>> Total dirs: 0
>>>
>>> Total files: 1
>>>
>>> Total blocks (validated): 1 (avg. block size 7217 B)
>>>
>>> Minimally replicated blocks: 1 (100.0 %)
>>>
>>> Over-replicated blocks: 0 (0.0 %)
>>>
>>> Under-replicated blocks: 0 (0.0 %)
>>>
>>> Mis-replicated blocks: 0 (0.0 %)
>>>
>>> Default replication factor: 1
>>>
>>> Average block replication: 1.0
>>>
>>> Corrupt blocks: 0
>>>
>>> Missing replicas: 0 (0.0 %)
>>>
>>> Number of data-nodes: 1
>>>
>>> Number of racks: 1
>>>
>>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>>
>>>
>>>
>>>
>>>
>>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Sorry, but what was the question? I also do not see a locations option
>>> flag.
>>>
>>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>
>>> wrote:
>>>
>>> Here is an example
>>>
>>>
>>>
>>> hdfs fsck /user/mark/data/word_count.csv
>>>
>>> Connecting to namenode via http://mark-7:50070
>>>
>>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>>
>>> .Status: HEALTHY
>>>
>>> Total size: 7217 B
>>>
>>> Total dirs: 0
>>>
>>> Total files: 1
>>>
>>> Total blocks (validated): 1 (avg. block size 7217 B)
>>>
>>> Minimally replicated blocks: 1 (100.0 %)
>>>
>>> Over-replicated blocks: 0 (0.0 %)
>>>
>>> Under-replicated blocks: 0 (0.0 %)
>>>
>>> Mis-replicated blocks: 0 (0.0 %)
>>>
>>> Default replication factor: 1
>>>
>>> Average block replication: 1.0
>>>
>>> Corrupt blocks: 0
>>>
>>> Missing replicas: 0 (0.0 %)
>>>
>>> Number of data-nodes: 1
>>>
>>> Number of racks: 1
>>>
>>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Hi Mark,
>>>
>>> Yes, the locations are shown as IP.
>>>
>>>
>>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > hdfs fsck -locations
>>> >
>>> > is supposed to show every block with its location? Is location the ip
>>> of the
>>> > datanode?
>>> >
>>> > Thank you,
>>> > Mark
>>>
>>>
>>> --
>>> Harsh J
>>>
>>>
>>>
>>>
>>>
>>
>>
>
Re: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Mark,
Did you see that your output is different from mine?
which Is your hadoop versioks
Rodrigo Nascimento
Systems Engineer @ Brazil
Mobile +55 11 991.873.810
Sent from my iPhone
On 24/01/2014, at 18:31, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
HI, Rodrigo, I am fine thank you :)
Here is the _complete_ output
mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
mark@mark-7:~$
On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
Hi Mark,
How are you?
at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet."
please, take a look at my last e-mail.
All the best,
Rodrigo Nascimento
Systems Engineer @ Brazil
Sent from my iPhone
On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
hdfs fsck /user/mark/data/word_count.csv -locations
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Mark,
Did you see that your output is different from mine?
which Is your hadoop versioks
Rodrigo Nascimento
Systems Engineer @ Brazil
Mobile +55 11 991.873.810
Sent from my iPhone
On 24/01/2014, at 18:31, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
HI, Rodrigo, I am fine thank you :)
Here is the _complete_ output
mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
mark@mark-7:~$
On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
Hi Mark,
How are you?
at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet."
please, take a look at my last e-mail.
All the best,
Rodrigo Nascimento
Systems Engineer @ Brazil
Sent from my iPhone
On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
hdfs fsck /user/mark/data/word_count.csv -locations
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Mark,
Did you see that your output is different from mine?
which Is your hadoop versioks
Rodrigo Nascimento
Systems Engineer @ Brazil
Mobile +55 11 991.873.810
Sent from my iPhone
On 24/01/2014, at 18:31, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
HI, Rodrigo, I am fine thank you :)
Here is the _complete_ output
mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
mark@mark-7:~$
On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
Hi Mark,
How are you?
at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet."
please, take a look at my last e-mail.
All the best,
Rodrigo Nascimento
Systems Engineer @ Brazil
Sent from my iPhone
On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
hdfs fsck /user/mark/data/word_count.csv -locations
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Mark,
Did you see that your output is different from mine?
which Is your hadoop versioks
Rodrigo Nascimento
Systems Engineer @ Brazil
Mobile +55 11 991.873.810
Sent from my iPhone
On 24/01/2014, at 18:31, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
HI, Rodrigo, I am fine thank you :)
Here is the _complete_ output
mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
mark@mark-7:~$
On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
Hi Mark,
How are you?
at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet."
please, take a look at my last e-mail.
All the best,
Rodrigo Nascimento
Systems Engineer @ Brazil
Sent from my iPhone
On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
hdfs fsck /user/mark/data/word_count.csv -locations
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
HI, Rodrigo, I am fine thank you :)
Here is the _complete_ output
mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
mark@mark-7:~$
On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Hi Mark,
>
> How are you?
>
> at your output from word_count.csv, the portion related to the nlock
> locations is missing. This is the reason why I told you in my previous
> message that "I'm not seeing locations yet."
>
> please, take a look at my last e-mail.
>
> All the best,
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
>
> Sent from my iPhone
>
> On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
>
> hdfs fsck /user/mark/data/word_count.csv *-locations*
>
>
> On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
> Rodrigo.Nascimento@netapp.com> wrote:
>
>> I’m not seeing locations flag yet.
>>
>>
>>
>> *Rod Nascimento*
>>
>> *Systems Engineer @ Brazil*
>>
>>
>>
>> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>>
>>
>>
>> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>> *Sent:* Friday, January 24, 2014 3:16 PM
>> *To:* Hadoop User
>> *Subject:* Re: hdfs fsck -locations
>>
>>
>>
>> Sorry, did not copy the full command
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv -locations
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>
>>
>>
>>
>>
>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Sorry, but what was the question? I also do not see a locations option
>> flag.
>>
>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>> Here is an example
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip
>> of the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>> --
>> Harsh J
>>
>>
>>
>>
>>
>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
HI, Rodrigo, I am fine thank you :)
Here is the _complete_ output
mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
mark@mark-7:~$
On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Hi Mark,
>
> How are you?
>
> at your output from word_count.csv, the portion related to the nlock
> locations is missing. This is the reason why I told you in my previous
> message that "I'm not seeing locations yet."
>
> please, take a look at my last e-mail.
>
> All the best,
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
>
> Sent from my iPhone
>
> On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
>
> hdfs fsck /user/mark/data/word_count.csv *-locations*
>
>
> On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
> Rodrigo.Nascimento@netapp.com> wrote:
>
>> I’m not seeing locations flag yet.
>>
>>
>>
>> *Rod Nascimento*
>>
>> *Systems Engineer @ Brazil*
>>
>>
>>
>> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>>
>>
>>
>> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>> *Sent:* Friday, January 24, 2014 3:16 PM
>> *To:* Hadoop User
>> *Subject:* Re: hdfs fsck -locations
>>
>>
>>
>> Sorry, did not copy the full command
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv -locations
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>
>>
>>
>>
>>
>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Sorry, but what was the question? I also do not see a locations option
>> flag.
>>
>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>> Here is an example
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip
>> of the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>> --
>> Harsh J
>>
>>
>>
>>
>>
>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
HI, Rodrigo, I am fine thank you :)
Here is the _complete_ output
mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
mark@mark-7:~$
On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Hi Mark,
>
> How are you?
>
> at your output from word_count.csv, the portion related to the nlock
> locations is missing. This is the reason why I told you in my previous
> message that "I'm not seeing locations yet."
>
> please, take a look at my last e-mail.
>
> All the best,
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
>
> Sent from my iPhone
>
> On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
>
> hdfs fsck /user/mark/data/word_count.csv *-locations*
>
>
> On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
> Rodrigo.Nascimento@netapp.com> wrote:
>
>> I’m not seeing locations flag yet.
>>
>>
>>
>> *Rod Nascimento*
>>
>> *Systems Engineer @ Brazil*
>>
>>
>>
>> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>>
>>
>>
>> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>> *Sent:* Friday, January 24, 2014 3:16 PM
>> *To:* Hadoop User
>> *Subject:* Re: hdfs fsck -locations
>>
>>
>>
>> Sorry, did not copy the full command
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv -locations
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>
>>
>>
>>
>>
>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Sorry, but what was the question? I also do not see a locations option
>> flag.
>>
>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>> Here is an example
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip
>> of the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>> --
>> Harsh J
>>
>>
>>
>>
>>
>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
HI, Rodrigo, I am fine thank you :)
Here is the _complete_ output
mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
mark@mark-7:~$
On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> Hi Mark,
>
> How are you?
>
> at your output from word_count.csv, the portion related to the nlock
> locations is missing. This is the reason why I told you in my previous
> message that "I'm not seeing locations yet."
>
> please, take a look at my last e-mail.
>
> All the best,
>
> Rodrigo Nascimento
> Systems Engineer @ Brazil
>
> Sent from my iPhone
>
> On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
>
> hdfs fsck /user/mark/data/word_count.csv *-locations*
>
>
> On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
> Rodrigo.Nascimento@netapp.com> wrote:
>
>> I’m not seeing locations flag yet.
>>
>>
>>
>> *Rod Nascimento*
>>
>> *Systems Engineer @ Brazil*
>>
>>
>>
>> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>>
>>
>>
>> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
>> *Sent:* Friday, January 24, 2014 3:16 PM
>> *To:* Hadoop User
>> *Subject:* Re: hdfs fsck -locations
>>
>>
>>
>> Sorry, did not copy the full command
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv -locations
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>>
>>
>>
>>
>>
>> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Sorry, but what was the question? I also do not see a locations option
>> flag.
>>
>> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>>
>> Here is an example
>>
>>
>>
>> hdfs fsck /user/mark/data/word_count.csv
>>
>> Connecting to namenode via http://mark-7:50070
>>
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>>
>> .Status: HEALTHY
>>
>> Total size: 7217 B
>>
>> Total dirs: 0
>>
>> Total files: 1
>>
>> Total blocks (validated): 1 (avg. block size 7217 B)
>>
>> Minimally replicated blocks: 1 (100.0 %)
>>
>> Over-replicated blocks: 0 (0.0 %)
>>
>> Under-replicated blocks: 0 (0.0 %)
>>
>> Mis-replicated blocks: 0 (0.0 %)
>>
>> Default replication factor: 1
>>
>> Average block replication: 1.0
>>
>> Corrupt blocks: 0
>>
>> Missing replicas: 0 (0.0 %)
>>
>> Number of data-nodes: 1
>>
>> Number of racks: 1
>>
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip
>> of the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>> --
>> Harsh J
>>
>>
>>
>>
>>
>
>
Re: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Hi Mark,
How are you?
at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet."
please, take a look at my last e-mail.
All the best,
Rodrigo Nascimento
Systems Engineer @ Brazil
Sent from my iPhone
On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
hdfs fsck /user/mark/data/word_count.csv -locations
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Hi Mark,
How are you?
at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet."
please, take a look at my last e-mail.
All the best,
Rodrigo Nascimento
Systems Engineer @ Brazil
Sent from my iPhone
On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
hdfs fsck /user/mark/data/word_count.csv -locations
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Hi Mark,
How are you?
at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet."
please, take a look at my last e-mail.
All the best,
Rodrigo Nascimento
Systems Engineer @ Brazil
Sent from my iPhone
On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
hdfs fsck /user/mark/data/word_count.csv -locations
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Hi Mark,
How are you?
at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet."
please, take a look at my last e-mail.
All the best,
Rodrigo Nascimento
Systems Engineer @ Brazil
Sent from my iPhone
On 24/01/2014, at 16:40, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
hdfs fsck /user/mark/data/word_count.csv -locations
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <Ro...@netapp.com>> wrote:
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com<ma...@shmsoft.com>]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
hdfs fsck /user/mark/data/word_count.csv *-locations*
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> I’m not seeing locations flag yet.
>
>
>
> *Rod Nascimento*
>
> *Systems Engineer @ Brazil*
>
>
>
> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>
>
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> *Sent:* Friday, January 24, 2014 3:16 PM
> *To:* Hadoop User
> *Subject:* Re: hdfs fsck -locations
>
>
>
> Sorry, did not copy the full command
>
>
>
> hdfs fsck /user/mark/data/word_count.csv -locations
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>
>
>
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>
>
>
>
>
> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Sorry, but what was the question? I also do not see a locations option
> flag.
>
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> Here is an example
>
>
>
> hdfs fsck /user/mark/data/word_count.csv
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
> --
> Harsh J
>
>
>
>
>
RE: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Hi Mark,
It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right?
[root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
Connecting to namenode via http://sandbox.hortonworks.com:50070
FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
.
/user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s).
Status: HEALTHY
Total size: 1708 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 1708 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 2 (66.666664 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
[root@sandbox ~]#
Rod Nascimento
________________________________
From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
Sent: Friday, January 24, 2014 3:34 PM
To: user@hadoop.apache.org
Subject: RE: hdfs fsck -locations
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
hdfs fsck /user/mark/data/word_count.csv *-locations*
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> I’m not seeing locations flag yet.
>
>
>
> *Rod Nascimento*
>
> *Systems Engineer @ Brazil*
>
>
>
> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>
>
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> *Sent:* Friday, January 24, 2014 3:16 PM
> *To:* Hadoop User
> *Subject:* Re: hdfs fsck -locations
>
>
>
> Sorry, did not copy the full command
>
>
>
> hdfs fsck /user/mark/data/word_count.csv -locations
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>
>
>
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>
>
>
>
>
> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Sorry, but what was the question? I also do not see a locations option
> flag.
>
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> Here is an example
>
>
>
> hdfs fsck /user/mark/data/word_count.csv
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
> --
> Harsh J
>
>
>
>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
hdfs fsck /user/mark/data/word_count.csv *-locations*
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> I’m not seeing locations flag yet.
>
>
>
> *Rod Nascimento*
>
> *Systems Engineer @ Brazil*
>
>
>
> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>
>
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> *Sent:* Friday, January 24, 2014 3:16 PM
> *To:* Hadoop User
> *Subject:* Re: hdfs fsck -locations
>
>
>
> Sorry, did not copy the full command
>
>
>
> hdfs fsck /user/mark/data/word_count.csv -locations
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>
>
>
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>
>
>
>
>
> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Sorry, but what was the question? I also do not see a locations option
> flag.
>
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> Here is an example
>
>
>
> hdfs fsck /user/mark/data/word_count.csv
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
> --
> Harsh J
>
>
>
>
>
RE: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Hi Mark,
It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right?
[root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
Connecting to namenode via http://sandbox.hortonworks.com:50070
FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
.
/user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s).
Status: HEALTHY
Total size: 1708 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 1708 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 2 (66.666664 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
[root@sandbox ~]#
Rod Nascimento
________________________________
From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
Sent: Friday, January 24, 2014 3:34 PM
To: user@hadoop.apache.org
Subject: RE: hdfs fsck -locations
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
RE: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
Hi Mark,
It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right?
[root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations
Connecting to namenode via http://sandbox.hortonworks.com:50070
FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
.
/user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s).
Status: HEALTHY
Total size: 1708 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 1708 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 2 (66.666664 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
[root@sandbox ~]#
Rod Nascimento
________________________________
From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
Sent: Friday, January 24, 2014 3:34 PM
To: user@hadoop.apache.org
Subject: RE: hdfs fsck -locations
I’m not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don’t buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
hdfs fsck /user/mark/data/word_count.csv *-locations*
On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo <
Rodrigo.Nascimento@netapp.com> wrote:
> I’m not seeing locations flag yet.
>
>
>
> *Rod Nascimento*
>
> *Systems Engineer @ Brazil*
>
>
>
> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.*
>
>
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> *Sent:* Friday, January 24, 2014 3:16 PM
> *To:* Hadoop User
> *Subject:* Re: hdfs fsck -locations
>
>
>
> Sorry, did not copy the full command
>
>
>
> hdfs fsck /user/mark/data/word_count.csv -locations
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
>
>
>
>
>
> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
>
>
>
>
>
> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Sorry, but what was the question? I also do not see a locations option
> flag.
>
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
> Here is an example
>
>
>
> hdfs fsck /user/mark/data/word_count.csv
>
> Connecting to namenode via http://mark-7:50070
>
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>
> .Status: HEALTHY
>
> Total size: 7217 B
>
> Total dirs: 0
>
> Total files: 1
>
> Total blocks (validated): 1 (avg. block size 7217 B)
>
> Minimally replicated blocks: 1 (100.0 %)
>
> Over-replicated blocks: 0 (0.0 %)
>
> Under-replicated blocks: 0 (0.0 %)
>
> Mis-replicated blocks: 0 (0.0 %)
>
> Default replication factor: 1
>
> Average block replication: 1.0
>
> Corrupt blocks: 0
>
> Missing replicas: 0 (0.0 %)
>
> Number of data-nodes: 1
>
> Number of racks: 1
>
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
> --
> Harsh J
>
>
>
>
>
RE: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
I'm not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don't buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
RE: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
I'm not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don't buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
RE: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
I'm not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don't buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
RE: hdfs fsck -locations
Posted by "Nascimento, Rodrigo" <Ro...@netapp.com>.
I'm not seeing locations flag yet.
Rod Nascimento
Systems Engineer @ Brazil
People don't buy WHAT you do. They buy WHY you do it.
From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com>> wrote:
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com>> wrote:
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com>> wrote:
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
> Sorry, but what was the question? I also do not see a locations option
> flag.
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
>> Here is an example
>>
>> hdfs fsck /user/mark/data/word_count.csv
>> Connecting to namenode via http://mark-7:50070
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>> .Status: HEALTHY
>> Total size: 7217 B
>> Total dirs: 0
>> Total files: 1
>> Total blocks (validated): 1 (avg. block size 7217 B)
>> Minimally replicated blocks: 1 (100.0 %)
>> Over-replicated blocks: 0 (0.0 %)
>> Under-replicated blocks: 0 (0.0 %)
>> Mis-replicated blocks: 0 (0.0 %)
>> Default replication factor: 1
>> Average block replication: 1.0
>> Corrupt blocks: 0
>> Missing replicas: 0 (0.0 %)
>> Number of data-nodes: 1
>> Number of racks: 1
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>>> Hi Mark,
>>>
>>> Yes, the locations are shown as IP.
>>>
>>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > hdfs fsck -locations
>>> >
>>> > is supposed to show every block with its location? Is location the ip
>>> of the
>>> > datanode?
>>> >
>>> > Thank you,
>>> > Mark
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
> Sorry, but what was the question? I also do not see a locations option
> flag.
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
>> Here is an example
>>
>> hdfs fsck /user/mark/data/word_count.csv
>> Connecting to namenode via http://mark-7:50070
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>> .Status: HEALTHY
>> Total size: 7217 B
>> Total dirs: 0
>> Total files: 1
>> Total blocks (validated): 1 (avg. block size 7217 B)
>> Minimally replicated blocks: 1 (100.0 %)
>> Over-replicated blocks: 0 (0.0 %)
>> Under-replicated blocks: 0 (0.0 %)
>> Mis-replicated blocks: 0 (0.0 %)
>> Default replication factor: 1
>> Average block replication: 1.0
>> Corrupt blocks: 0
>> Missing replicas: 0 (0.0 %)
>> Number of data-nodes: 1
>> Number of racks: 1
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>>> Hi Mark,
>>>
>>> Yes, the locations are shown as IP.
>>>
>>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > hdfs fsck -locations
>>> >
>>> > is supposed to show every block with its location? Is location the ip
>>> of the
>>> > datanode?
>>> >
>>> > Thank you,
>>> > Mark
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
> Sorry, but what was the question? I also do not see a locations option
> flag.
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
>> Here is an example
>>
>> hdfs fsck /user/mark/data/word_count.csv
>> Connecting to namenode via http://mark-7:50070
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>> .Status: HEALTHY
>> Total size: 7217 B
>> Total dirs: 0
>> Total files: 1
>> Total blocks (validated): 1 (avg. block size 7217 B)
>> Minimally replicated blocks: 1 (100.0 %)
>> Over-replicated blocks: 0 (0.0 %)
>> Under-replicated blocks: 0 (0.0 %)
>> Mis-replicated blocks: 0 (0.0 %)
>> Default replication factor: 1
>> Average block replication: 1.0
>> Corrupt blocks: 0
>> Missing replicas: 0 (0.0 %)
>> Number of data-nodes: 1
>> Number of racks: 1
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>>> Hi Mark,
>>>
>>> Yes, the locations are shown as IP.
>>>
>>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > hdfs fsck -locations
>>> >
>>> > is supposed to show every block with its location? Is location the ip
>>> of the
>>> > datanode?
>>> >
>>> > Thank you,
>>> > Mark
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Sorry, did not copy the full command
hdfs fsck /user/mark/data/word_count.csv -locations
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <ha...@cloudera.com> wrote:
> Sorry, but what was the question? I also do not see a locations option
> flag.
> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
>
>> Here is an example
>>
>> hdfs fsck /user/mark/data/word_count.csv
>> Connecting to namenode via http://mark-7:50070
>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
>> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
>> .Status: HEALTHY
>> Total size: 7217 B
>> Total dirs: 0
>> Total files: 1
>> Total blocks (validated): 1 (avg. block size 7217 B)
>> Minimally replicated blocks: 1 (100.0 %)
>> Over-replicated blocks: 0 (0.0 %)
>> Under-replicated blocks: 0 (0.0 %)
>> Mis-replicated blocks: 0 (0.0 %)
>> Default replication factor: 1
>> Average block replication: 1.0
>> Corrupt blocks: 0
>> Missing replicas: 0 (0.0 %)
>> Number of data-nodes: 1
>> Number of racks: 1
>> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>>
>>
>>
>> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>>> Hi Mark,
>>>
>>> Yes, the locations are shown as IP.
>>>
>>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > hdfs fsck -locations
>>> >
>>> > is supposed to show every block with its location? Is location the ip
>>> of the
>>> > datanode?
>>> >
>>> > Thank you,
>>> > Mark
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>>
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
> Here is an example
>
> hdfs fsck /user/mark/data/word_count.csv
> Connecting to namenode via http://mark-7:50070
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
> .Status: HEALTHY
> Total size: 7217 B
> Total dirs: 0
> Total files: 1
> Total blocks (validated): 1 (avg. block size 7217 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 0 (0.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 1
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 0 (0.0 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip
>> of the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
> Here is an example
>
> hdfs fsck /user/mark/data/word_count.csv
> Connecting to namenode via http://mark-7:50070
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
> .Status: HEALTHY
> Total size: 7217 B
> Total dirs: 0
> Total files: 1
> Total blocks (validated): 1 (avg. block size 7217 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 0 (0.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 1
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 0 (0.0 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip
>> of the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
> Here is an example
>
> hdfs fsck /user/mark/data/word_count.csv
> Connecting to namenode via http://mark-7:50070
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
> .Status: HEALTHY
> Total size: 7217 B
> Total dirs: 0
> Total files: 1
> Total blocks (validated): 1 (avg. block size 7217 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 0 (0.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 1
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 0 (0.0 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip
>> of the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
Sorry, but what was the question? I also do not see a locations option flag.
On Jan 24, 2014 7:17 PM, "Mark Kerzner" <ma...@shmsoft.com> wrote:
> Here is an example
>
> hdfs fsck /user/mark/data/word_count.csv
> Connecting to namenode via http://mark-7:50070
> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
> .Status: HEALTHY
> Total size: 7217 B
> Total dirs: 0
> Total files: 1
> Total blocks (validated): 1 (avg. block size 7217 B)
> Minimally replicated blocks: 1 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 0 (0.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 1
> Average block replication: 1.0
> Corrupt blocks: 0
> Missing replicas: 0 (0.0 %)
> Number of data-nodes: 1
> Number of racks: 1
> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
>
>
>
> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi Mark,
>>
>> Yes, the locations are shown as IP.
>>
>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
>> wrote:
>> > Hi,
>> >
>> > hdfs fsck -locations
>> >
>> > is supposed to show every block with its location? Is location the ip
>> of the
>> > datanode?
>> >
>> > Thank you,
>> > Mark
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
>
> --
> Harsh J
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
>
> --
> Harsh J
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
>
> --
> Harsh J
>
Re: hdfs fsck -locations
Posted by Mark Kerzner <ma...@shmsoft.com>.
Here is an example
hdfs fsck /user/mark/data/word_count.csv
Connecting to namenode via http://mark-7:50070
FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
/user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
.Status: HEALTHY
Total size: 7217 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 7217 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 1
Average block replication: 1.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <ha...@cloudera.com> wrote:
> Hi Mark,
>
> Yes, the locations are shown as IP.
>
> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com>
> wrote:
> > Hi,
> >
> > hdfs fsck -locations
> >
> > is supposed to show every block with its location? Is location the ip of
> the
> > datanode?
> >
> > Thank you,
> > Mark
>
>
>
> --
> Harsh J
>
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J
Re: hdfs fsck -locations
Posted by Harsh J <ha...@cloudera.com>.
Hi Mark,
Yes, the locations are shown as IP.
On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <ma...@shmsoft.com> wrote:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip of the
> datanode?
>
> Thank you,
> Mark
--
Harsh J