You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Patai Sangbutsarakum <si...@gmail.com> on 2012/09/26 22:46:42 UTC
splitting jobtracker and namenode
Hi Hadoopers,
My production Hadoop 0.20.2 cluster has been running Pri. namenode and
Jobtracker together in the same box for long time. it's time to
separate them. I am trying to write up a procedure on this to make
sure that it won't cause any issues.
- Decrease TTL of DNS entry for value of mapred.job.tracker
- Prepare new name in DNS entry for value of mapred.job.tracker
- Change value of mapred.job.tracker (in mapred-site.xml) to new
server and propagate the new config to whole cluster and clients (
where Hadoop jobs are submitted from )
- Create directory defined in mapred.local.dir on new server.
- Make sure that new jobtracker server is not in slave or master file.
- run stop-mapred.sh on current jobtracker server
- run start-mapred.sh on new jobtracker server
Do I miss anything ?
Please suggest.
Patai
Re: splitting jobtracker and namenode
Posted by Patai Sangbutsarakum <si...@gmail.com>.
Thanks Ted, that's true.
On Wed, Sep 26, 2012 at 3:07 PM, Ted Dunning <td...@maprtech.com> wrote:
> Why are you changing the TTL on DNS if you aren't moving the name? If you
> are just changing the config to a new name, then caching won't matter.
>
>
> On Wed, Sep 26, 2012 at 1:46 PM, Patai Sangbutsarakum
> <si...@gmail.com> wrote:
>>
>> Hi Hadoopers,
>>
>> My production Hadoop 0.20.2 cluster has been running Pri. namenode and
>> Jobtracker together in the same box for long time. it's time to
>> separate them. I am trying to write up a procedure on this to make
>> sure that it won't cause any issues.
>>
>> - Decrease TTL of DNS entry for value of mapred.job.tracker
>> - Prepare new name in DNS entry for value of mapred.job.tracker
>> - Change value of mapred.job.tracker (in mapred-site.xml) to new
>> server and propagate the new config to whole cluster and clients (
>> where Hadoop jobs are submitted from )
>> - Create directory defined in mapred.local.dir on new server.
>> - Make sure that new jobtracker server is not in slave or master file.
>> - run stop-mapred.sh on current jobtracker server
>> - run start-mapred.sh on new jobtracker server
>>
>> Do I miss anything ?
>>
>> Please suggest.
>>
>> Patai
>
>
Re: splitting jobtracker and namenode
Posted by Patai Sangbutsarakum <si...@gmail.com>.
Thanks Ted, that's true.
On Wed, Sep 26, 2012 at 3:07 PM, Ted Dunning <td...@maprtech.com> wrote:
> Why are you changing the TTL on DNS if you aren't moving the name? If you
> are just changing the config to a new name, then caching won't matter.
>
>
> On Wed, Sep 26, 2012 at 1:46 PM, Patai Sangbutsarakum
> <si...@gmail.com> wrote:
>>
>> Hi Hadoopers,
>>
>> My production Hadoop 0.20.2 cluster has been running Pri. namenode and
>> Jobtracker together in the same box for long time. it's time to
>> separate them. I am trying to write up a procedure on this to make
>> sure that it won't cause any issues.
>>
>> - Decrease TTL of DNS entry for value of mapred.job.tracker
>> - Prepare new name in DNS entry for value of mapred.job.tracker
>> - Change value of mapred.job.tracker (in mapred-site.xml) to new
>> server and propagate the new config to whole cluster and clients (
>> where Hadoop jobs are submitted from )
>> - Create directory defined in mapred.local.dir on new server.
>> - Make sure that new jobtracker server is not in slave or master file.
>> - run stop-mapred.sh on current jobtracker server
>> - run start-mapred.sh on new jobtracker server
>>
>> Do I miss anything ?
>>
>> Please suggest.
>>
>> Patai
>
>
Re: splitting jobtracker and namenode
Posted by Patai Sangbutsarakum <si...@gmail.com>.
Thanks Ted, that's true.
On Wed, Sep 26, 2012 at 3:07 PM, Ted Dunning <td...@maprtech.com> wrote:
> Why are you changing the TTL on DNS if you aren't moving the name? If you
> are just changing the config to a new name, then caching won't matter.
>
>
> On Wed, Sep 26, 2012 at 1:46 PM, Patai Sangbutsarakum
> <si...@gmail.com> wrote:
>>
>> Hi Hadoopers,
>>
>> My production Hadoop 0.20.2 cluster has been running Pri. namenode and
>> Jobtracker together in the same box for long time. it's time to
>> separate them. I am trying to write up a procedure on this to make
>> sure that it won't cause any issues.
>>
>> - Decrease TTL of DNS entry for value of mapred.job.tracker
>> - Prepare new name in DNS entry for value of mapred.job.tracker
>> - Change value of mapred.job.tracker (in mapred-site.xml) to new
>> server and propagate the new config to whole cluster and clients (
>> where Hadoop jobs are submitted from )
>> - Create directory defined in mapred.local.dir on new server.
>> - Make sure that new jobtracker server is not in slave or master file.
>> - run stop-mapred.sh on current jobtracker server
>> - run start-mapred.sh on new jobtracker server
>>
>> Do I miss anything ?
>>
>> Please suggest.
>>
>> Patai
>
>
Re: splitting jobtracker and namenode
Posted by Patai Sangbutsarakum <si...@gmail.com>.
Thanks Ted, that's true.
On Wed, Sep 26, 2012 at 3:07 PM, Ted Dunning <td...@maprtech.com> wrote:
> Why are you changing the TTL on DNS if you aren't moving the name? If you
> are just changing the config to a new name, then caching won't matter.
>
>
> On Wed, Sep 26, 2012 at 1:46 PM, Patai Sangbutsarakum
> <si...@gmail.com> wrote:
>>
>> Hi Hadoopers,
>>
>> My production Hadoop 0.20.2 cluster has been running Pri. namenode and
>> Jobtracker together in the same box for long time. it's time to
>> separate them. I am trying to write up a procedure on this to make
>> sure that it won't cause any issues.
>>
>> - Decrease TTL of DNS entry for value of mapred.job.tracker
>> - Prepare new name in DNS entry for value of mapred.job.tracker
>> - Change value of mapred.job.tracker (in mapred-site.xml) to new
>> server and propagate the new config to whole cluster and clients (
>> where Hadoop jobs are submitted from )
>> - Create directory defined in mapred.local.dir on new server.
>> - Make sure that new jobtracker server is not in slave or master file.
>> - run stop-mapred.sh on current jobtracker server
>> - run start-mapred.sh on new jobtracker server
>>
>> Do I miss anything ?
>>
>> Please suggest.
>>
>> Patai
>
>
Re: splitting jobtracker and namenode
Posted by Ted Dunning <td...@maprtech.com>.
Why are you changing the TTL on DNS if you aren't moving the name? If you
are just changing the config to a new name, then caching won't matter.
On Wed, Sep 26, 2012 at 1:46 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Hi Hadoopers,
>
> My production Hadoop 0.20.2 cluster has been running Pri. namenode and
> Jobtracker together in the same box for long time. it's time to
> separate them. I am trying to write up a procedure on this to make
> sure that it won't cause any issues.
>
> - Decrease TTL of DNS entry for value of mapred.job.tracker
> - Prepare new name in DNS entry for value of mapred.job.tracker
> - Change value of mapred.job.tracker (in mapred-site.xml) to new
> server and propagate the new config to whole cluster and clients (
> where Hadoop jobs are submitted from )
> - Create directory defined in mapred.local.dir on new server.
> - Make sure that new jobtracker server is not in slave or master file.
> - run stop-mapred.sh on current jobtracker server
> - run start-mapred.sh on new jobtracker server
>
> Do I miss anything ?
>
> Please suggest.
>
> Patai
>
Re: splitting jobtracker and namenode
Posted by Ted Dunning <td...@maprtech.com>.
Why are you changing the TTL on DNS if you aren't moving the name? If you
are just changing the config to a new name, then caching won't matter.
On Wed, Sep 26, 2012 at 1:46 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Hi Hadoopers,
>
> My production Hadoop 0.20.2 cluster has been running Pri. namenode and
> Jobtracker together in the same box for long time. it's time to
> separate them. I am trying to write up a procedure on this to make
> sure that it won't cause any issues.
>
> - Decrease TTL of DNS entry for value of mapred.job.tracker
> - Prepare new name in DNS entry for value of mapred.job.tracker
> - Change value of mapred.job.tracker (in mapred-site.xml) to new
> server and propagate the new config to whole cluster and clients (
> where Hadoop jobs are submitted from )
> - Create directory defined in mapred.local.dir on new server.
> - Make sure that new jobtracker server is not in slave or master file.
> - run stop-mapred.sh on current jobtracker server
> - run start-mapred.sh on new jobtracker server
>
> Do I miss anything ?
>
> Please suggest.
>
> Patai
>
Re: splitting jobtracker and namenode
Posted by Ted Dunning <td...@maprtech.com>.
Why are you changing the TTL on DNS if you aren't moving the name? If you
are just changing the config to a new name, then caching won't matter.
On Wed, Sep 26, 2012 at 1:46 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Hi Hadoopers,
>
> My production Hadoop 0.20.2 cluster has been running Pri. namenode and
> Jobtracker together in the same box for long time. it's time to
> separate them. I am trying to write up a procedure on this to make
> sure that it won't cause any issues.
>
> - Decrease TTL of DNS entry for value of mapred.job.tracker
> - Prepare new name in DNS entry for value of mapred.job.tracker
> - Change value of mapred.job.tracker (in mapred-site.xml) to new
> server and propagate the new config to whole cluster and clients (
> where Hadoop jobs are submitted from )
> - Create directory defined in mapred.local.dir on new server.
> - Make sure that new jobtracker server is not in slave or master file.
> - run stop-mapred.sh on current jobtracker server
> - run start-mapred.sh on new jobtracker server
>
> Do I miss anything ?
>
> Please suggest.
>
> Patai
>
Re: splitting jobtracker and namenode
Posted by Ted Dunning <td...@maprtech.com>.
Why are you changing the TTL on DNS if you aren't moving the name? If you
are just changing the config to a new name, then caching won't matter.
On Wed, Sep 26, 2012 at 1:46 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Hi Hadoopers,
>
> My production Hadoop 0.20.2 cluster has been running Pri. namenode and
> Jobtracker together in the same box for long time. it's time to
> separate them. I am trying to write up a procedure on this to make
> sure that it won't cause any issues.
>
> - Decrease TTL of DNS entry for value of mapred.job.tracker
> - Prepare new name in DNS entry for value of mapred.job.tracker
> - Change value of mapred.job.tracker (in mapred-site.xml) to new
> server and propagate the new config to whole cluster and clients (
> where Hadoop jobs are submitted from )
> - Create directory defined in mapred.local.dir on new server.
> - Make sure that new jobtracker server is not in slave or master file.
> - run stop-mapred.sh on current jobtracker server
> - run start-mapred.sh on new jobtracker server
>
> Do I miss anything ?
>
> Please suggest.
>
> Patai
>