You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Manish Bhoge <ma...@rocketmail.com> on 2013/07/25 08:58:30 UTC

Multiple data node and namenode ?

I am configuring cluster and starting with first machine. So i have configured the core-site, hdfs-site and mapred-site to run hadoop only on 1 machine. But somehow i am getting 2 copies of data node, namenode and secondary namenode. Not able to figure out why ?



SecurityAuth-hdfs.audit 0 bytes Jul 17, 2013 4:34:56 AM
hadoop-hdfs-datanode-INBEDU011997A.log 243358 bytes Jul 25, 2013 2:29:15 AM
hadoop-hdfs-datanode-myhost-1.log 1893692 bytes Jul 25, 2013 2:50:42 AM
hadoop-hdfs-namenode-INBEDU011997A.log 19545397 bytes Jul 25, 2013 2:50:40 AM
hadoop-hdfs-namenode-myhost-1.log 2438271 bytes Jul 25, 2013 2:48:57 AM
hadoop-hdfs-secondarynamenode-INBEDU011997A.log 4061047 bytes Jul 25, 2013 2:30:10 AM
hadoop-hdfs-secondarynamenode-myhost-1.log 9707957 bytes Jul 25, 2013 2:46:18 AM

RE: Multiple data node and namenode ?

Posted by Devaraj k <de...@huawei.com>.
I could see two data node's log files. But the modified times for both files are different with the difference as 21 mins. I doubt whether they are really running. By any chance have you updated the host name in the machine and then restarted the data node?

Could you check with 'ps -ef | grep datanode' shell command to know how many datanode processes are running at this moment.

Thanks
Devaraj k


-----Original Message-----
From: Manish Bhoge [mailto:manishbhoge@rocketmail.com] 
Sent: 25 July 2013 12:56
To: common-user@hadoop.apache.org
Subject: Re: Multiple data node and namenode ?

Unfortunately, I don't have JPS installed. But as you can see it shows 2 data nodes.
 hadoop-hdfs-datanode-INBEDU011997A.log 243358 bytes Jul 25, 2013 2:29:15 AM hadoop-hdfs-datanode-myhost-1.log 1893692 bytes Jul 25, 2013 2:50:42 AM


is it because i have given 3 directory under dfs-data.dir in hdfs-site.xml?


________________________________
 From: Devaraj k <de...@huawei.com>
To: "common-user@hadoop.apache.org" <co...@hadoop.apache.org>
Sent: Thursday, 25 July 2013 12:41 PM
Subject: RE: Multiple data node and namenode ?
 

Hi Manish,

  Can you check how many data node processes are running really in the machine using the command 'jps' or 'ps'. 

Thanks
Devaraj k


-----Original Message-----
From: Manish Bhoge [mailto:manishbhoge@rocketmail.com] 
Sent: 25 July 2013 12:29
To: common-user@hadoop.apache.org
Subject: Multiple data node and namenode ?

I am configuring cluster and starting with first machine. So i have configured the core-site, hdfs-site and mapred-site to run hadoop only on 1 machine. But somehow i am getting 2 copies of data node, namenode and secondary namenode. Not able to figure out why ?



SecurityAuth-hdfs.audit 0 bytes Jul 17, 2013 4:34:56 AM hadoop-hdfs-datanode-INBEDU011997A.log 243358 bytes Jul 25, 2013 2:29:15 AM hadoop-hdfs-datanode-myhost-1.log 1893692 bytes Jul 25, 2013 2:50:42 AM hadoop-hdfs-namenode-INBEDU011997A.log 19545397 bytes Jul 25, 2013 2:50:40 AM hadoop-hdfs-namenode-myhost-1.log 2438271 bytes Jul 25, 2013 2:48:57 AM hadoop-hdfs-secondarynamenode-INBEDU011997A.log 4061047 bytes Jul 25, 2013 2:30:10 AM hadoop-hdfs-secondarynamenode-myhost-1.log 9707957 bytes Jul 25, 2013 2:46:18 AM

Re: Multiple data node and namenode ?

Posted by Manish Bhoge <ma...@rocketmail.com>.
Unfortunately, I don't have JPS installed. But as you can see it shows 2 data nodes. 
 hadoop-hdfs-datanode-INBEDU011997A.log 243358 bytes Jul 25, 2013 2:29:15 AM 
hadoop-hdfs-datanode-myhost-1.log 1893692 bytes Jul 25, 2013 2:50:42 AM


is it because i have given 3 directory under dfs-data.dir in hdfs-site.xml?


________________________________
 From: Devaraj k <de...@huawei.com>
To: "common-user@hadoop.apache.org" <co...@hadoop.apache.org> 
Sent: Thursday, 25 July 2013 12:41 PM
Subject: RE: Multiple data node and namenode ?
 

Hi Manish,

  Can you check how many data node processes are running really in the machine using the command 'jps' or 'ps'. 

Thanks
Devaraj k


-----Original Message-----
From: Manish Bhoge [mailto:manishbhoge@rocketmail.com] 
Sent: 25 July 2013 12:29
To: common-user@hadoop.apache.org
Subject: Multiple data node and namenode ?

I am configuring cluster and starting with first machine. So i have configured the core-site, hdfs-site and mapred-site to run hadoop only on 1 machine. But somehow i am getting 2 copies of data node, namenode and secondary namenode. Not able to figure out why ?



SecurityAuth-hdfs.audit 0 bytes Jul 17, 2013 4:34:56 AM hadoop-hdfs-datanode-INBEDU011997A.log 243358 bytes Jul 25, 2013 2:29:15 AM hadoop-hdfs-datanode-myhost-1.log 1893692 bytes Jul 25, 2013 2:50:42 AM hadoop-hdfs-namenode-INBEDU011997A.log 19545397 bytes Jul 25, 2013 2:50:40 AM hadoop-hdfs-namenode-myhost-1.log 2438271 bytes Jul 25, 2013 2:48:57 AM hadoop-hdfs-secondarynamenode-INBEDU011997A.log 4061047 bytes Jul 25, 2013 2:30:10 AM hadoop-hdfs-secondarynamenode-myhost-1.log 9707957 bytes Jul 25, 2013 2:46:18 AM

RE: Multiple data node and namenode ?

Posted by Devaraj k <de...@huawei.com>.
Hi Manish,

  Can you check how many data node processes are running really in the machine using the command 'jps' or 'ps'. 

Thanks
Devaraj k


-----Original Message-----
From: Manish Bhoge [mailto:manishbhoge@rocketmail.com] 
Sent: 25 July 2013 12:29
To: common-user@hadoop.apache.org
Subject: Multiple data node and namenode ?

I am configuring cluster and starting with first machine. So i have configured the core-site, hdfs-site and mapred-site to run hadoop only on 1 machine. But somehow i am getting 2 copies of data node, namenode and secondary namenode. Not able to figure out why ?



SecurityAuth-hdfs.audit 0 bytes Jul 17, 2013 4:34:56 AM hadoop-hdfs-datanode-INBEDU011997A.log 243358 bytes Jul 25, 2013 2:29:15 AM hadoop-hdfs-datanode-myhost-1.log 1893692 bytes Jul 25, 2013 2:50:42 AM hadoop-hdfs-namenode-INBEDU011997A.log 19545397 bytes Jul 25, 2013 2:50:40 AM hadoop-hdfs-namenode-myhost-1.log 2438271 bytes Jul 25, 2013 2:48:57 AM hadoop-hdfs-secondarynamenode-INBEDU011997A.log 4061047 bytes Jul 25, 2013 2:30:10 AM hadoop-hdfs-secondarynamenode-myhost-1.log 9707957 bytes Jul 25, 2013 2:46:18 AM