You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Arbi Akhina <ar...@gmail.com> on 2015/06/16 14:53:05 UTC
Two hadoop nodes on same machine while a second machine not joining
the cluster
Hi,
I have a test cluster of two machines, on both of them hadoop is installed.
I've configured the hadoop cluster but on admin UI (as in the below
picture) I see that two nodes are running on the same master machine, and
that the other machine has no Hadoop node.
[image: enter image description here]
On master machine following services are running:
~$ jps
26310 ResourceManager
27593 Jps
26216 DataNode
26135 NameNode
26557 NodeManager
26701 JobHistoryServer
On the slave machine:
~$ jps
2614 DataNode
2920 Jps
2707 NodeManager
I don't why the slave is not joining the cluster (It was before). I tried
to shutdown all servers on both machines and format HDFS then restarting
everything but that didn't help. Any help to figure what's causing that
behavior is appreciated.
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by Arbi Akhina <ar...@gmail.com>.
Fixed, the two machines had same hostname :(
On Tue, Jun 16, 2015 at 5:55 PM, Arbi Akhina <ar...@gmail.com> wrote:
> Here is the content of configuration files on both machines:
>
> Here is the content of *core-site.xml*
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://hadoop-server</value>
> </property>
> </configuration>
>
> Here is the content of *mapred-site.xml*
> <configuration>
> <property>
> <name>mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
> </configuration>
>
>
> On Tue, Jun 16, 2015 at 4:37 PM, vedavyasa reddy <
> vedavyas.veda92@gmail.com> wrote:
>
>> hi,
>> check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host
>> name of the master is given correctly or not. i.e.,
>> "hdfs://master-hostname:9000"
>>
>> and if these is correct then check the "mapred-site.xml" whether "job
>> Tracker address" is given correct or not.
>>
>> On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I have a test cluster of two machines, on both of them hadoop is
>>> installed. I've configured the hadoop cluster but on admin UI (as in the
>>> below picture) I see that two nodes are running on the same master machine,
>>> and that the other machine has no Hadoop node.
>>>
>>> [image: enter image description here]
>>>
>>> On master machine following services are running:
>>>
>>> ~$ jps
>>> 26310 ResourceManager
>>> 27593 Jps
>>> 26216 DataNode
>>> 26135 NameNode
>>> 26557 NodeManager
>>> 26701 JobHistoryServer
>>>
>>> On the slave machine:
>>>
>>> ~$ jps
>>> 2614 DataNode
>>> 2920 Jps
>>> 2707 NodeManager
>>>
>>> I don't why the slave is not joining the cluster (It was before). I
>>> tried to shutdown all servers on both machines and format HDFS then
>>> restarting everything but that didn't help. Any help to figure what's
>>> causing that behavior is appreciated.
>>>
>>
>>
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by Arbi Akhina <ar...@gmail.com>.
Fixed, the two machines had same hostname :(
On Tue, Jun 16, 2015 at 5:55 PM, Arbi Akhina <ar...@gmail.com> wrote:
> Here is the content of configuration files on both machines:
>
> Here is the content of *core-site.xml*
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://hadoop-server</value>
> </property>
> </configuration>
>
> Here is the content of *mapred-site.xml*
> <configuration>
> <property>
> <name>mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
> </configuration>
>
>
> On Tue, Jun 16, 2015 at 4:37 PM, vedavyasa reddy <
> vedavyas.veda92@gmail.com> wrote:
>
>> hi,
>> check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host
>> name of the master is given correctly or not. i.e.,
>> "hdfs://master-hostname:9000"
>>
>> and if these is correct then check the "mapred-site.xml" whether "job
>> Tracker address" is given correct or not.
>>
>> On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I have a test cluster of two machines, on both of them hadoop is
>>> installed. I've configured the hadoop cluster but on admin UI (as in the
>>> below picture) I see that two nodes are running on the same master machine,
>>> and that the other machine has no Hadoop node.
>>>
>>> [image: enter image description here]
>>>
>>> On master machine following services are running:
>>>
>>> ~$ jps
>>> 26310 ResourceManager
>>> 27593 Jps
>>> 26216 DataNode
>>> 26135 NameNode
>>> 26557 NodeManager
>>> 26701 JobHistoryServer
>>>
>>> On the slave machine:
>>>
>>> ~$ jps
>>> 2614 DataNode
>>> 2920 Jps
>>> 2707 NodeManager
>>>
>>> I don't why the slave is not joining the cluster (It was before). I
>>> tried to shutdown all servers on both machines and format HDFS then
>>> restarting everything but that didn't help. Any help to figure what's
>>> causing that behavior is appreciated.
>>>
>>
>>
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by Arbi Akhina <ar...@gmail.com>.
Fixed, the two machines had same hostname :(
On Tue, Jun 16, 2015 at 5:55 PM, Arbi Akhina <ar...@gmail.com> wrote:
> Here is the content of configuration files on both machines:
>
> Here is the content of *core-site.xml*
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://hadoop-server</value>
> </property>
> </configuration>
>
> Here is the content of *mapred-site.xml*
> <configuration>
> <property>
> <name>mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
> </configuration>
>
>
> On Tue, Jun 16, 2015 at 4:37 PM, vedavyasa reddy <
> vedavyas.veda92@gmail.com> wrote:
>
>> hi,
>> check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host
>> name of the master is given correctly or not. i.e.,
>> "hdfs://master-hostname:9000"
>>
>> and if these is correct then check the "mapred-site.xml" whether "job
>> Tracker address" is given correct or not.
>>
>> On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I have a test cluster of two machines, on both of them hadoop is
>>> installed. I've configured the hadoop cluster but on admin UI (as in the
>>> below picture) I see that two nodes are running on the same master machine,
>>> and that the other machine has no Hadoop node.
>>>
>>> [image: enter image description here]
>>>
>>> On master machine following services are running:
>>>
>>> ~$ jps
>>> 26310 ResourceManager
>>> 27593 Jps
>>> 26216 DataNode
>>> 26135 NameNode
>>> 26557 NodeManager
>>> 26701 JobHistoryServer
>>>
>>> On the slave machine:
>>>
>>> ~$ jps
>>> 2614 DataNode
>>> 2920 Jps
>>> 2707 NodeManager
>>>
>>> I don't why the slave is not joining the cluster (It was before). I
>>> tried to shutdown all servers on both machines and format HDFS then
>>> restarting everything but that didn't help. Any help to figure what's
>>> causing that behavior is appreciated.
>>>
>>
>>
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by Arbi Akhina <ar...@gmail.com>.
Fixed, the two machines had same hostname :(
On Tue, Jun 16, 2015 at 5:55 PM, Arbi Akhina <ar...@gmail.com> wrote:
> Here is the content of configuration files on both machines:
>
> Here is the content of *core-site.xml*
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://hadoop-server</value>
> </property>
> </configuration>
>
> Here is the content of *mapred-site.xml*
> <configuration>
> <property>
> <name>mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
> </configuration>
>
>
> On Tue, Jun 16, 2015 at 4:37 PM, vedavyasa reddy <
> vedavyas.veda92@gmail.com> wrote:
>
>> hi,
>> check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host
>> name of the master is given correctly or not. i.e.,
>> "hdfs://master-hostname:9000"
>>
>> and if these is correct then check the "mapred-site.xml" whether "job
>> Tracker address" is given correct or not.
>>
>> On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I have a test cluster of two machines, on both of them hadoop is
>>> installed. I've configured the hadoop cluster but on admin UI (as in the
>>> below picture) I see that two nodes are running on the same master machine,
>>> and that the other machine has no Hadoop node.
>>>
>>> [image: enter image description here]
>>>
>>> On master machine following services are running:
>>>
>>> ~$ jps
>>> 26310 ResourceManager
>>> 27593 Jps
>>> 26216 DataNode
>>> 26135 NameNode
>>> 26557 NodeManager
>>> 26701 JobHistoryServer
>>>
>>> On the slave machine:
>>>
>>> ~$ jps
>>> 2614 DataNode
>>> 2920 Jps
>>> 2707 NodeManager
>>>
>>> I don't why the slave is not joining the cluster (It was before). I
>>> tried to shutdown all servers on both machines and format HDFS then
>>> restarting everything but that didn't help. Any help to figure what's
>>> causing that behavior is appreciated.
>>>
>>
>>
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by Arbi Akhina <ar...@gmail.com>.
Here is the content of configuration files on both machines:
Here is the content of *core-site.xml*
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://hadoop-server</value>
</property>
</configuration>
Here is the content of *mapred-site.xml*
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
On Tue, Jun 16, 2015 at 4:37 PM, vedavyasa reddy <ve...@gmail.com>
wrote:
> hi,
> check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host
> name of the master is given correctly or not. i.e.,
> "hdfs://master-hostname:9000"
>
> and if these is correct then check the "mapred-site.xml" whether "job
> Tracker address" is given correct or not.
>
> On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I have a test cluster of two machines, on both of them hadoop is
>> installed. I've configured the hadoop cluster but on admin UI (as in the
>> below picture) I see that two nodes are running on the same master machine,
>> and that the other machine has no Hadoop node.
>>
>> [image: enter image description here]
>>
>> On master machine following services are running:
>>
>> ~$ jps
>> 26310 ResourceManager
>> 27593 Jps
>> 26216 DataNode
>> 26135 NameNode
>> 26557 NodeManager
>> 26701 JobHistoryServer
>>
>> On the slave machine:
>>
>> ~$ jps
>> 2614 DataNode
>> 2920 Jps
>> 2707 NodeManager
>>
>> I don't why the slave is not joining the cluster (It was before). I tried
>> to shutdown all servers on both machines and format HDFS then restarting
>> everything but that didn't help. Any help to figure what's causing that
>> behavior is appreciated.
>>
>
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by Arbi Akhina <ar...@gmail.com>.
Here is the content of configuration files on both machines:
Here is the content of *core-site.xml*
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://hadoop-server</value>
</property>
</configuration>
Here is the content of *mapred-site.xml*
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
On Tue, Jun 16, 2015 at 4:37 PM, vedavyasa reddy <ve...@gmail.com>
wrote:
> hi,
> check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host
> name of the master is given correctly or not. i.e.,
> "hdfs://master-hostname:9000"
>
> and if these is correct then check the "mapred-site.xml" whether "job
> Tracker address" is given correct or not.
>
> On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I have a test cluster of two machines, on both of them hadoop is
>> installed. I've configured the hadoop cluster but on admin UI (as in the
>> below picture) I see that two nodes are running on the same master machine,
>> and that the other machine has no Hadoop node.
>>
>> [image: enter image description here]
>>
>> On master machine following services are running:
>>
>> ~$ jps
>> 26310 ResourceManager
>> 27593 Jps
>> 26216 DataNode
>> 26135 NameNode
>> 26557 NodeManager
>> 26701 JobHistoryServer
>>
>> On the slave machine:
>>
>> ~$ jps
>> 2614 DataNode
>> 2920 Jps
>> 2707 NodeManager
>>
>> I don't why the slave is not joining the cluster (It was before). I tried
>> to shutdown all servers on both machines and format HDFS then restarting
>> everything but that didn't help. Any help to figure what's causing that
>> behavior is appreciated.
>>
>
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by Arbi Akhina <ar...@gmail.com>.
Here is the content of configuration files on both machines:
Here is the content of *core-site.xml*
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://hadoop-server</value>
</property>
</configuration>
Here is the content of *mapred-site.xml*
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
On Tue, Jun 16, 2015 at 4:37 PM, vedavyasa reddy <ve...@gmail.com>
wrote:
> hi,
> check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host
> name of the master is given correctly or not. i.e.,
> "hdfs://master-hostname:9000"
>
> and if these is correct then check the "mapred-site.xml" whether "job
> Tracker address" is given correct or not.
>
> On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I have a test cluster of two machines, on both of them hadoop is
>> installed. I've configured the hadoop cluster but on admin UI (as in the
>> below picture) I see that two nodes are running on the same master machine,
>> and that the other machine has no Hadoop node.
>>
>> [image: enter image description here]
>>
>> On master machine following services are running:
>>
>> ~$ jps
>> 26310 ResourceManager
>> 27593 Jps
>> 26216 DataNode
>> 26135 NameNode
>> 26557 NodeManager
>> 26701 JobHistoryServer
>>
>> On the slave machine:
>>
>> ~$ jps
>> 2614 DataNode
>> 2920 Jps
>> 2707 NodeManager
>>
>> I don't why the slave is not joining the cluster (It was before). I tried
>> to shutdown all servers on both machines and format HDFS then restarting
>> everything but that didn't help. Any help to figure what's causing that
>> behavior is appreciated.
>>
>
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by Arbi Akhina <ar...@gmail.com>.
Here is the content of configuration files on both machines:
Here is the content of *core-site.xml*
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://hadoop-server</value>
</property>
</configuration>
Here is the content of *mapred-site.xml*
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
On Tue, Jun 16, 2015 at 4:37 PM, vedavyasa reddy <ve...@gmail.com>
wrote:
> hi,
> check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host
> name of the master is given correctly or not. i.e.,
> "hdfs://master-hostname:9000"
>
> and if these is correct then check the "mapred-site.xml" whether "job
> Tracker address" is given correct or not.
>
> On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I have a test cluster of two machines, on both of them hadoop is
>> installed. I've configured the hadoop cluster but on admin UI (as in the
>> below picture) I see that two nodes are running on the same master machine,
>> and that the other machine has no Hadoop node.
>>
>> [image: enter image description here]
>>
>> On master machine following services are running:
>>
>> ~$ jps
>> 26310 ResourceManager
>> 27593 Jps
>> 26216 DataNode
>> 26135 NameNode
>> 26557 NodeManager
>> 26701 JobHistoryServer
>>
>> On the slave machine:
>>
>> ~$ jps
>> 2614 DataNode
>> 2920 Jps
>> 2707 NodeManager
>>
>> I don't why the slave is not joining the cluster (It was before). I tried
>> to shutdown all servers on both machines and format HDFS then restarting
>> everything but that didn't help. Any help to figure what's causing that
>> behavior is appreciated.
>>
>
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by vedavyasa reddy <ve...@gmail.com>.
hi,
check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host name
of the master is given correctly or not. i.e., "hdfs://master-hostname:9000"
and if these is correct then check the "mapred-site.xml" whether "job
Tracker address" is given correct or not.
On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com> wrote:
> Hi,
>
> I have a test cluster of two machines, on both of them hadoop is
> installed. I've configured the hadoop cluster but on admin UI (as in the
> below picture) I see that two nodes are running on the same master machine,
> and that the other machine has no Hadoop node.
>
> [image: enter image description here]
>
> On master machine following services are running:
>
> ~$ jps
> 26310 ResourceManager
> 27593 Jps
> 26216 DataNode
> 26135 NameNode
> 26557 NodeManager
> 26701 JobHistoryServer
>
> On the slave machine:
>
> ~$ jps
> 2614 DataNode
> 2920 Jps
> 2707 NodeManager
>
> I don't why the slave is not joining the cluster (It was before). I tried
> to shutdown all servers on both machines and format HDFS then restarting
> everything but that didn't help. Any help to figure what's causing that
> behavior is appreciated.
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by vedavyasa reddy <ve...@gmail.com>.
hi,
check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host name
of the master is given correctly or not. i.e., "hdfs://master-hostname:9000"
and if these is correct then check the "mapred-site.xml" whether "job
Tracker address" is given correct or not.
On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com> wrote:
> Hi,
>
> I have a test cluster of two machines, on both of them hadoop is
> installed. I've configured the hadoop cluster but on admin UI (as in the
> below picture) I see that two nodes are running on the same master machine,
> and that the other machine has no Hadoop node.
>
> [image: enter image description here]
>
> On master machine following services are running:
>
> ~$ jps
> 26310 ResourceManager
> 27593 Jps
> 26216 DataNode
> 26135 NameNode
> 26557 NodeManager
> 26701 JobHistoryServer
>
> On the slave machine:
>
> ~$ jps
> 2614 DataNode
> 2920 Jps
> 2707 NodeManager
>
> I don't why the slave is not joining the cluster (It was before). I tried
> to shutdown all servers on both machines and format HDFS then restarting
> everything but that didn't help. Any help to figure what's causing that
> behavior is appreciated.
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by vedavyasa reddy <ve...@gmail.com>.
hi,
check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host name
of the master is given correctly or not. i.e., "hdfs://master-hostname:9000"
and if these is correct then check the "mapred-site.xml" whether "job
Tracker address" is given correct or not.
On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com> wrote:
> Hi,
>
> I have a test cluster of two machines, on both of them hadoop is
> installed. I've configured the hadoop cluster but on admin UI (as in the
> below picture) I see that two nodes are running on the same master machine,
> and that the other machine has no Hadoop node.
>
> [image: enter image description here]
>
> On master machine following services are running:
>
> ~$ jps
> 26310 ResourceManager
> 27593 Jps
> 26216 DataNode
> 26135 NameNode
> 26557 NodeManager
> 26701 JobHistoryServer
>
> On the slave machine:
>
> ~$ jps
> 2614 DataNode
> 2920 Jps
> 2707 NodeManager
>
> I don't why the slave is not joining the cluster (It was before). I tried
> to shutdown all servers on both machines and format HDFS then restarting
> everything but that didn't help. Any help to figure what's causing that
> behavior is appreciated.
>
Re: Two hadoop nodes on same machine while a second machine not
joining the cluster
Posted by vedavyasa reddy <ve...@gmail.com>.
hi,
check in slave node Hadoop_home/etc/hadoop/"core-site.xml"whether host name
of the master is given correctly or not. i.e., "hdfs://master-hostname:9000"
and if these is correct then check the "mapred-site.xml" whether "job
Tracker address" is given correct or not.
On Tue, Jun 16, 2015 at 6:23 PM, Arbi Akhina <ar...@gmail.com> wrote:
> Hi,
>
> I have a test cluster of two machines, on both of them hadoop is
> installed. I've configured the hadoop cluster but on admin UI (as in the
> below picture) I see that two nodes are running on the same master machine,
> and that the other machine has no Hadoop node.
>
> [image: enter image description here]
>
> On master machine following services are running:
>
> ~$ jps
> 26310 ResourceManager
> 27593 Jps
> 26216 DataNode
> 26135 NameNode
> 26557 NodeManager
> 26701 JobHistoryServer
>
> On the slave machine:
>
> ~$ jps
> 2614 DataNode
> 2920 Jps
> 2707 NodeManager
>
> I don't why the slave is not joining the cluster (It was before). I tried
> to shutdown all servers on both machines and format HDFS then restarting
> everything but that didn't help. Any help to figure what's causing that
> behavior is appreciated.
>