You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by modemide <mo...@gmail.com> on 2011/08/19 18:45:28 UTC
Help - Rack Topology Script - Hadoop 0.20 (CDH3u1)
Hi all,
I've tried to make a rack topology script. I've written it in python
and it works if I call it with the following arguments:
10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26 10.2.0.31
10.2.0.33
The output is:
/rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
Should the output be on newlines or is any whitespace sufficient?
Additionally, my cluster's datanodes have DNS names such as:
r1dn02
r2dn05
etc...
I restarted the namenode in my running cluster (after configuring the
topology script setting in core-site.xml).
I ran a job and checked what the job tracker thinks the rack id's are
and it showed default-rack.
Can anyone tell me what I'm doing wrong?
Thanks,
tim
Re: Help - Rack Topology Script - Hadoop 0.20 (CDH3u1)
Posted by Edward Capriolo <ed...@gmail.com>.
On Sun, Aug 21, 2011 at 10:22 AM, Joey Echeverria <jo...@cloudera.com> wrote:
> Not that I know of.
>
> -Joey
>
> On Fri, Aug 19, 2011 at 1:16 PM, modemide <mo...@gmail.com> wrote:
> > Ha, what a silly mistake.
> >
> > Thank you Joey.
> >
> > Do you also happen to know of an easier way to tell which racks the
> > jobtracker/namenode think each node is in?
> >
> >
> >
> > On 8/19/11, Joey Echeverria <jo...@cloudera.com> wrote:
> >> Did you restart the JobTracker?
> >>
> >> -Joey
> >>
> >> On Fri, Aug 19, 2011 at 12:45 PM, modemide <mo...@gmail.com> wrote:
> >>> Hi all,
> >>> I've tried to make a rack topology script. I've written it in python
> >>> and it works if I call it with the following arguments:
> >>> 10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26 10.2.0.31
> >>> 10.2.0.33
> >>>
> >>> The output is:
> >>> /rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
> >>> Should the output be on newlines or is any whitespace sufficient?
> >>>
> >>> Additionally, my cluster's datanodes have DNS names such as:
> >>> r1dn02
> >>> r2dn05
> >>> etc...
> >>>
> >>> I restarted the namenode in my running cluster (after configuring the
> >>> topology script setting in core-site.xml).
> >>> I ran a job and checked what the job tracker thinks the rack id's are
> >>> and it showed default-rack.
> >>> Can anyone tell me what I'm doing wrong?
> >>>
> >>> Thanks,
> >>> tim
> >>>
> >>
> >>
> >>
> >> --
> >> Joseph Echeverria
> >> Cloudera, Inc.
> >> 443.305.9434
> >>
> >
>
>
>
> --
> Joseph Echeverria
> Cloudera, Inc.
> 443.305.9434
>
If you run the hdfs balancer application the application displays the
topology it learns from from the topology script. Assuming your jobtracker
started with the same configuration you have your answer.
Re: Help - Rack Topology Script - Hadoop 0.20 (CDH3u1)
Posted by Joey Echeverria <jo...@cloudera.com>.
Not that I know of.
-Joey
On Fri, Aug 19, 2011 at 1:16 PM, modemide <mo...@gmail.com> wrote:
> Ha, what a silly mistake.
>
> Thank you Joey.
>
> Do you also happen to know of an easier way to tell which racks the
> jobtracker/namenode think each node is in?
>
>
>
> On 8/19/11, Joey Echeverria <jo...@cloudera.com> wrote:
>> Did you restart the JobTracker?
>>
>> -Joey
>>
>> On Fri, Aug 19, 2011 at 12:45 PM, modemide <mo...@gmail.com> wrote:
>>> Hi all,
>>> I've tried to make a rack topology script. I've written it in python
>>> and it works if I call it with the following arguments:
>>> 10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26 10.2.0.31
>>> 10.2.0.33
>>>
>>> The output is:
>>> /rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
>>> Should the output be on newlines or is any whitespace sufficient?
>>>
>>> Additionally, my cluster's datanodes have DNS names such as:
>>> r1dn02
>>> r2dn05
>>> etc...
>>>
>>> I restarted the namenode in my running cluster (after configuring the
>>> topology script setting in core-site.xml).
>>> I ran a job and checked what the job tracker thinks the rack id's are
>>> and it showed default-rack.
>>> Can anyone tell me what I'm doing wrong?
>>>
>>> Thanks,
>>> tim
>>>
>>
>>
>>
>> --
>> Joseph Echeverria
>> Cloudera, Inc.
>> 443.305.9434
>>
>
--
Joseph Echeverria
Cloudera, Inc.
443.305.9434
Re: Help - Rack Topology Script - Hadoop 0.20 (CDH3u1)
Posted by modemide <mo...@gmail.com>.
Ha, what a silly mistake.
Thank you Joey.
Do you also happen to know of an easier way to tell which racks the
jobtracker/namenode think each node is in?
On 8/19/11, Joey Echeverria <jo...@cloudera.com> wrote:
> Did you restart the JobTracker?
>
> -Joey
>
> On Fri, Aug 19, 2011 at 12:45 PM, modemide <mo...@gmail.com> wrote:
>> Hi all,
>> I've tried to make a rack topology script. I've written it in python
>> and it works if I call it with the following arguments:
>> 10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26 10.2.0.31
>> 10.2.0.33
>>
>> The output is:
>> /rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
>> Should the output be on newlines or is any whitespace sufficient?
>>
>> Additionally, my cluster's datanodes have DNS names such as:
>> r1dn02
>> r2dn05
>> etc...
>>
>> I restarted the namenode in my running cluster (after configuring the
>> topology script setting in core-site.xml).
>> I ran a job and checked what the job tracker thinks the rack id's are
>> and it showed default-rack.
>> Can anyone tell me what I'm doing wrong?
>>
>> Thanks,
>> tim
>>
>
>
>
> --
> Joseph Echeverria
> Cloudera, Inc.
> 443.305.9434
>
Re: Help - Rack Topology Script - Hadoop 0.20 (CDH3u1)
Posted by Joey Echeverria <jo...@cloudera.com>.
Did you restart the JobTracker?
-Joey
On Fri, Aug 19, 2011 at 12:45 PM, modemide <mo...@gmail.com> wrote:
> Hi all,
> I've tried to make a rack topology script. I've written it in python
> and it works if I call it with the following arguments:
> 10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26 10.2.0.31
> 10.2.0.33
>
> The output is:
> /rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
> Should the output be on newlines or is any whitespace sufficient?
>
> Additionally, my cluster's datanodes have DNS names such as:
> r1dn02
> r2dn05
> etc...
>
> I restarted the namenode in my running cluster (after configuring the
> topology script setting in core-site.xml).
> I ran a job and checked what the job tracker thinks the rack id's are
> and it showed default-rack.
> Can anyone tell me what I'm doing wrong?
>
> Thanks,
> tim
>
--
Joseph Echeverria
Cloudera, Inc.
443.305.9434