Question on Secondary Namenode

[praveenesh kumar <pr...@gmail.com>]

Hi,
Masters file in $HADOOP_HOME/conf tells you about where exactly the
SecondaryNamenode deamon should run. Correct me please if I am wrong.

My doubt is Do all Datanodes should know where secondary namenode is
running or Only namenode should be knowing where secondary namenode is
running ?
The reason I am asking this thing is, to know Do I need to copy the
contents $HADOOP_HOME/conf/masters file in all Datanodes or only Namenode
and Secondary Namenode should have this entry in their master file.

Also, if I am right, the following property, tells  the location of
Namenode to Secondary Namenode, so that it can communicate with namenode to
create checkpoints.

<property>

<name>dfs.http.address</name>

<value>namenode:50070</value>

<description>

The address and the base port where the dfs namenode web ui will listen on.

If the port is 0 then the server will start on a free port.

</description>

</property>


so this property should be define only in the secondary namenode, Does this
property also needs to be define in all Datanodes ?


Thanks,

Praveenesh

Re: Question on Secondary Namenode

[Harsh J <ha...@cloudera.com>]

Praveenesh,

On Mon, Jan 9, 2012 at 11:38 AM, praveenesh kumar <pr...@gmail.com> wrote:
> Hi,
> Masters file in $HADOOP_HOME/conf tells you about where exactly the
> SecondaryNamenode deamon should run. Correct me please if I am wrong.

Correct. But do not run > 1 SNN. Depending on the version of HDFS you
use, you may run into a corruption issue which has only recently been
fixed.

> My doubt is Do all Datanodes should know where secondary namenode is
> running or Only namenode should be knowing where secondary namenode is
> running ?

The SNN works on a pull-mechanism, so its the converse: The SNN should
know where the NN runs and both should be able to communicate with one
another over the network.

> The reason I am asking this thing is, to know Do I need to copy the
> contents $HADOOP_HOME/conf/masters file in all Datanodes or only Namenode
> and Secondary Namenode should have this entry in their master file.

That file is only used by your scripts, not by HDFS itself. Feel free
to keep it only in the place where you usually execute
start-all/stop-all scripts from. Or even sync it across, it does not
matter.

> Also, if I am right, the following property, tells  the location of
> Namenode to Secondary Namenode, so that it can communicate with namenode to
> create checkpoints.
>
> <property>
>
> <name>dfs.http.address</name>

Yes that is the right property.

> so this property should be define only in the secondary namenode, Does this
> property also needs to be define in all Datanodes ?

The SNN *needs* it. DataNodes do not *need* it. Should be fine if you
want just the SNN to carry this.

-- 
Harsh J
(Frankly been a long time since I had to worry about where what config
goes, given tools that are available today.)