You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Charles Robertson <ch...@gmail.com> on 2014/09/12 00:06:08 UTC
Change master node with Ambari?
I'm playing around with a 2-node cluster running Hortonworks Data Platform
2.1, running on AWS EC2 instances. Today I discovered that for some reason
my master node had decided to no longer recognise my private key, so I
could no longer SSH in to the host to work on the project.
In this case it's not really a show-stopper as I have all the important
stuff saved elsewhere and I am rebuilding the ambari server node.
My question is, is it possible to change the master node and so preserve
the data in HDFS in this situation?
I could probably prevent this from happening again by having a secondary
authorised key for the host, but I would expect there should be some
mechanism to change the master node. Is there one? I couldn't find a way of
doing it through Ambari, so perhaps it's a more manual process?
Regards,
Charles
RE: Change master node with Ambari?
Posted by "Liu, Yi A" <yi...@intel.com>.
1. If your cluster is still alive, then you can build a new cluster and using distcp to migrate all data to the new cluster.
2. Suppose the master node you mean is the NameNode, the second approach needs you to copy fsimage/edit logs from master node, I think it doesn’t work for you since you can’t access it.
Regards,
Yi Liu
From: Charles Robertson [mailto:charles.robertson@gmail.com]
Sent: Friday, September 12, 2014 6:06 AM
To: user@hadoop.apache.org
Subject: Change master node with Ambari?
I'm playing around with a 2-node cluster running Hortonworks Data Platform 2.1, running on AWS EC2 instances. Today I discovered that for some reason my master node had decided to no longer recognise my private key, so I could no longer SSH in to the host to work on the project.
In this case it's not really a show-stopper as I have all the important stuff saved elsewhere and I am rebuilding the ambari server node.
My question is, is it possible to change the master node and so preserve the data in HDFS in this situation?
I could probably prevent this from happening again by having a secondary authorised key for the host, but I would expect there should be some mechanism to change the master node. Is there one? I couldn't find a way of doing it through Ambari, so perhaps it's a more manual process?
Regards,
Charles
RE: Change master node with Ambari?
Posted by "Liu, Yi A" <yi...@intel.com>.
1. If your cluster is still alive, then you can build a new cluster and using distcp to migrate all data to the new cluster.
2. Suppose the master node you mean is the NameNode, the second approach needs you to copy fsimage/edit logs from master node, I think it doesn’t work for you since you can’t access it.
Regards,
Yi Liu
From: Charles Robertson [mailto:charles.robertson@gmail.com]
Sent: Friday, September 12, 2014 6:06 AM
To: user@hadoop.apache.org
Subject: Change master node with Ambari?
I'm playing around with a 2-node cluster running Hortonworks Data Platform 2.1, running on AWS EC2 instances. Today I discovered that for some reason my master node had decided to no longer recognise my private key, so I could no longer SSH in to the host to work on the project.
In this case it's not really a show-stopper as I have all the important stuff saved elsewhere and I am rebuilding the ambari server node.
My question is, is it possible to change the master node and so preserve the data in HDFS in this situation?
I could probably prevent this from happening again by having a secondary authorised key for the host, but I would expect there should be some mechanism to change the master node. Is there one? I couldn't find a way of doing it through Ambari, so perhaps it's a more manual process?
Regards,
Charles
RE: Change master node with Ambari?
Posted by "Liu, Yi A" <yi...@intel.com>.
1. If your cluster is still alive, then you can build a new cluster and using distcp to migrate all data to the new cluster.
2. Suppose the master node you mean is the NameNode, the second approach needs you to copy fsimage/edit logs from master node, I think it doesn’t work for you since you can’t access it.
Regards,
Yi Liu
From: Charles Robertson [mailto:charles.robertson@gmail.com]
Sent: Friday, September 12, 2014 6:06 AM
To: user@hadoop.apache.org
Subject: Change master node with Ambari?
I'm playing around with a 2-node cluster running Hortonworks Data Platform 2.1, running on AWS EC2 instances. Today I discovered that for some reason my master node had decided to no longer recognise my private key, so I could no longer SSH in to the host to work on the project.
In this case it's not really a show-stopper as I have all the important stuff saved elsewhere and I am rebuilding the ambari server node.
My question is, is it possible to change the master node and so preserve the data in HDFS in this situation?
I could probably prevent this from happening again by having a secondary authorised key for the host, but I would expect there should be some mechanism to change the master node. Is there one? I couldn't find a way of doing it through Ambari, so perhaps it's a more manual process?
Regards,
Charles
RE: Change master node with Ambari?
Posted by "Liu, Yi A" <yi...@intel.com>.
1. If your cluster is still alive, then you can build a new cluster and using distcp to migrate all data to the new cluster.
2. Suppose the master node you mean is the NameNode, the second approach needs you to copy fsimage/edit logs from master node, I think it doesn’t work for you since you can’t access it.
Regards,
Yi Liu
From: Charles Robertson [mailto:charles.robertson@gmail.com]
Sent: Friday, September 12, 2014 6:06 AM
To: user@hadoop.apache.org
Subject: Change master node with Ambari?
I'm playing around with a 2-node cluster running Hortonworks Data Platform 2.1, running on AWS EC2 instances. Today I discovered that for some reason my master node had decided to no longer recognise my private key, so I could no longer SSH in to the host to work on the project.
In this case it's not really a show-stopper as I have all the important stuff saved elsewhere and I am rebuilding the ambari server node.
My question is, is it possible to change the master node and so preserve the data in HDFS in this situation?
I could probably prevent this from happening again by having a secondary authorised key for the host, but I would expect there should be some mechanism to change the master node. Is there one? I couldn't find a way of doing it through Ambari, so perhaps it's a more manual process?
Regards,
Charles