You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by hadoop hive <ha...@gmail.com> on 2014/07/02 07:33:57 UTC
Re: How to make hdfs data rack aware
Try running fsck, it will also validate the block placement as well as
replication.
On Jun 27, 2014 6:49 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
wrote:
> My topology script is working fine for data I am writing to hdfs. My
> question is how to make the
> Existing data topoloy compaliant?
>
> Thanks,
> sam
>
> From: karthikeyan S <ka...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Thursday, June 26, 2014 at 8:23 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Re: How to make hdfs data rack aware
>
> Hi
>
> Did you try supplying the script name in hdfs-site.xml as the value for
> the parameter topology.script.file.name?
>
> Thanks
>
> Karthik
> On Jun 26, 2014 6:02 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
> wrote:
>
>> Hi All,
>>
>> We setup a cluster with out rackawareness, now that we got right
>> rackawreness script, we wanted to
>> Make sure data is rack aware.
>> How we make sure existing data is rack aware and NN should move copies if
>> all of them are in the
>> Same rack?
>>
>> Thanks,
>> Sam
>>
>
Re: How to make hdfs data rack aware
Posted by Adam Kawa <ka...@gmail.com>.
You can run
$ sudo -u hdfs hdfs dfsadmin -report | grep "Hostname" -A 1
2014-07-02 7:33 GMT+02:00 hadoop hive <ha...@gmail.com>:
> Try running fsck, it will also validate the block placement as well as
> replication.
> On Jun 27, 2014 6:49 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
> wrote:
>
>> My topology script is working fine for data I am writing to hdfs. My
>> question is how to make the
>> Existing data topoloy compaliant?
>>
>> Thanks,
>> sam
>>
>> From: karthikeyan S <ka...@gmail.com>
>> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Date: Thursday, June 26, 2014 at 8:23 PM
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: Re: How to make hdfs data rack aware
>>
>> Hi
>>
>> Did you try supplying the script name in hdfs-site.xml as the value for
>> the parameter topology.script.file.name?
>>
>> Thanks
>>
>> Karthik
>> On Jun 26, 2014 6:02 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> We setup a cluster with out rackawareness, now that we got right
>>> rackawreness script, we wanted to
>>> Make sure data is rack aware.
>>> How we make sure existing data is rack aware and NN should move copies
>>> if all of them are in the
>>> Same rack?
>>>
>>> Thanks,
>>> Sam
>>>
>>
Re: How to make hdfs data rack aware
Posted by Adam Kawa <ka...@gmail.com>.
You can run
$ sudo -u hdfs hdfs dfsadmin -report | grep "Hostname" -A 1
2014-07-02 7:33 GMT+02:00 hadoop hive <ha...@gmail.com>:
> Try running fsck, it will also validate the block placement as well as
> replication.
> On Jun 27, 2014 6:49 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
> wrote:
>
>> My topology script is working fine for data I am writing to hdfs. My
>> question is how to make the
>> Existing data topoloy compaliant?
>>
>> Thanks,
>> sam
>>
>> From: karthikeyan S <ka...@gmail.com>
>> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Date: Thursday, June 26, 2014 at 8:23 PM
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: Re: How to make hdfs data rack aware
>>
>> Hi
>>
>> Did you try supplying the script name in hdfs-site.xml as the value for
>> the parameter topology.script.file.name?
>>
>> Thanks
>>
>> Karthik
>> On Jun 26, 2014 6:02 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> We setup a cluster with out rackawareness, now that we got right
>>> rackawreness script, we wanted to
>>> Make sure data is rack aware.
>>> How we make sure existing data is rack aware and NN should move copies
>>> if all of them are in the
>>> Same rack?
>>>
>>> Thanks,
>>> Sam
>>>
>>
Re: How to make hdfs data rack aware
Posted by Adam Kawa <ka...@gmail.com>.
You can run
$ sudo -u hdfs hdfs dfsadmin -report | grep "Hostname" -A 1
2014-07-02 7:33 GMT+02:00 hadoop hive <ha...@gmail.com>:
> Try running fsck, it will also validate the block placement as well as
> replication.
> On Jun 27, 2014 6:49 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
> wrote:
>
>> My topology script is working fine for data I am writing to hdfs. My
>> question is how to make the
>> Existing data topoloy compaliant?
>>
>> Thanks,
>> sam
>>
>> From: karthikeyan S <ka...@gmail.com>
>> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Date: Thursday, June 26, 2014 at 8:23 PM
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: Re: How to make hdfs data rack aware
>>
>> Hi
>>
>> Did you try supplying the script name in hdfs-site.xml as the value for
>> the parameter topology.script.file.name?
>>
>> Thanks
>>
>> Karthik
>> On Jun 26, 2014 6:02 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> We setup a cluster with out rackawareness, now that we got right
>>> rackawreness script, we wanted to
>>> Make sure data is rack aware.
>>> How we make sure existing data is rack aware and NN should move copies
>>> if all of them are in the
>>> Same rack?
>>>
>>> Thanks,
>>> Sam
>>>
>>
Re: How to make hdfs data rack aware
Posted by Adam Kawa <ka...@gmail.com>.
You can run
$ sudo -u hdfs hdfs dfsadmin -report | grep "Hostname" -A 1
2014-07-02 7:33 GMT+02:00 hadoop hive <ha...@gmail.com>:
> Try running fsck, it will also validate the block placement as well as
> replication.
> On Jun 27, 2014 6:49 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
> wrote:
>
>> My topology script is working fine for data I am writing to hdfs. My
>> question is how to make the
>> Existing data topoloy compaliant?
>>
>> Thanks,
>> sam
>>
>> From: karthikeyan S <ka...@gmail.com>
>> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Date: Thursday, June 26, 2014 at 8:23 PM
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: Re: How to make hdfs data rack aware
>>
>> Hi
>>
>> Did you try supplying the script name in hdfs-site.xml as the value for
>> the parameter topology.script.file.name?
>>
>> Thanks
>>
>> Karthik
>> On Jun 26, 2014 6:02 AM, "Kilaru, Sambaiah" <Sa...@intuit.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> We setup a cluster with out rackawareness, now that we got right
>>> rackawreness script, we wanted to
>>> Make sure data is rack aware.
>>> How we make sure existing data is rack aware and NN should move copies
>>> if all of them are in the
>>> Same rack?
>>>
>>> Thanks,
>>> Sam
>>>
>>