You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Shashank Agarwal <sh...@gmail.com> on 2013/03/14 05:38:04 UTC
Regarding: Merging two hadoop clusters
Hey Guys,
I have two different hadoop clusters in production. One cluster is used as
backing for HBase and the other for other things. Both hadoop clusters are
using the same version 1.0 and I want to merge them and make them one. I
know, one possible solution is to copy the data across, but the data is
really huge on these clusters and it will hard for me to compromise with
huge downtime.
Is there any optimal way to merge two hadoop clusters.
~Shashank
Re: Regarding: Merging two hadoop clusters
Posted by Suresh Srinivas <su...@hortonworks.com>.
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
This is not a supported feature. Hence this activity would require
understanding low level Hadoop details, quite a bit of hacking and is not
straightforward. Copying data from the clusters is the simplest solution.
Re: Regarding: Merging two hadoop clusters
Posted by al...@aim.com.
Are those clusters located in separate LANs?
-----Original Message-----
From: Azuryy Yu <az...@gmail.com>
To: user <us...@hadoop.apache.org>
Sent: Thu, Mar 14, 2013 5:06 am
Subject: Re: Regarding: Merging two hadoop clusters
there is no good idea for your question. but for hadoop-2.x, its easy with HDFS federation.
On Mar 14, 2013 12:38 PM, "Shashank Agarwal" <sh...@gmail.com> wrote:
Hey Guys,
I have two different hadoop clusters in production. One cluster is used as backing for HBase and the other for other things. Both hadoop clusters are using the same version 1.0 and I want to merge them and make them one. I know, one possible solution is to copy the data across, but the data is really huge on these clusters and it will hard for me to compromise with huge downtime.
Is there any optimal way to merge two hadoop clusters.
~Shashank
Re: Regarding: Merging two hadoop clusters
Posted by al...@aim.com.
Are those clusters located in separate LANs?
-----Original Message-----
From: Azuryy Yu <az...@gmail.com>
To: user <us...@hadoop.apache.org>
Sent: Thu, Mar 14, 2013 5:06 am
Subject: Re: Regarding: Merging two hadoop clusters
there is no good idea for your question. but for hadoop-2.x, its easy with HDFS federation.
On Mar 14, 2013 12:38 PM, "Shashank Agarwal" <sh...@gmail.com> wrote:
Hey Guys,
I have two different hadoop clusters in production. One cluster is used as backing for HBase and the other for other things. Both hadoop clusters are using the same version 1.0 and I want to merge them and make them one. I know, one possible solution is to copy the data across, but the data is really huge on these clusters and it will hard for me to compromise with huge downtime.
Is there any optimal way to merge two hadoop clusters.
~Shashank
Re: Regarding: Merging two hadoop clusters
Posted by al...@aim.com.
Are those clusters located in separate LANs?
-----Original Message-----
From: Azuryy Yu <az...@gmail.com>
To: user <us...@hadoop.apache.org>
Sent: Thu, Mar 14, 2013 5:06 am
Subject: Re: Regarding: Merging two hadoop clusters
there is no good idea for your question. but for hadoop-2.x, its easy with HDFS federation.
On Mar 14, 2013 12:38 PM, "Shashank Agarwal" <sh...@gmail.com> wrote:
Hey Guys,
I have two different hadoop clusters in production. One cluster is used as backing for HBase and the other for other things. Both hadoop clusters are using the same version 1.0 and I want to merge them and make them one. I know, one possible solution is to copy the data across, but the data is really huge on these clusters and it will hard for me to compromise with huge downtime.
Is there any optimal way to merge two hadoop clusters.
~Shashank
Re: Regarding: Merging two hadoop clusters
Posted by al...@aim.com.
Are those clusters located in separate LANs?
-----Original Message-----
From: Azuryy Yu <az...@gmail.com>
To: user <us...@hadoop.apache.org>
Sent: Thu, Mar 14, 2013 5:06 am
Subject: Re: Regarding: Merging two hadoop clusters
there is no good idea for your question. but for hadoop-2.x, its easy with HDFS federation.
On Mar 14, 2013 12:38 PM, "Shashank Agarwal" <sh...@gmail.com> wrote:
Hey Guys,
I have two different hadoop clusters in production. One cluster is used as backing for HBase and the other for other things. Both hadoop clusters are using the same version 1.0 and I want to merge them and make them one. I know, one possible solution is to copy the data across, but the data is really huge on these clusters and it will hard for me to compromise with huge downtime.
Is there any optimal way to merge two hadoop clusters.
~Shashank
Re: Regarding: Merging two hadoop clusters
Posted by Azuryy Yu <az...@gmail.com>.
there is no good idea for your question. but for hadoop-2.x, its easy with
HDFS federation.
On Mar 14, 2013 12:38 PM, "Shashank Agarwal" <sh...@gmail.com>
wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
Re: Regarding: Merging two hadoop clusters
Posted by Suresh Srinivas <su...@hortonworks.com>.
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
This is not a supported feature. Hence this activity would require
understanding low level Hadoop details, quite a bit of hacking and is not
straightforward. Copying data from the clusters is the simplest solution.
Re: Regarding: Merging two hadoop clusters
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Like Vivek and Vinod are saying, using distcp might be the solution.
But you need to make sure you have enought free space in one cluster
to received the data from the other one. If you don't have, you might
need to re-assign some on the nodes from one cluster to the other one
to "concentrate" the free space on a singlue cluster, then to the
migration? That was you will not have any down-time. But you need to
make sure nother is written on the cluster you are migrating to not
miss any data.
JM
2013/3/14 vivek <vi...@gmail.com>:
> Distcp is the simplest approach you can use (it will copy data parallely
> using mappers and reducers).
>
>
>
> On Thu, Mar 14, 2013 at 12:16 PM, Vinod Kumar Vavilapalli
> <vi...@hortonworks.com> wrote:
>>
>>
>> Copy data into one of the clusters using distcp *without* downtime
>> (assuming you have enough capacity) and then merge the clusters?
>>
>> Thanks,
>> +Vinod Kumar Vavilapalli
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
>>
>> Hey Guys,
>>
>> I have two different hadoop clusters in production. One cluster is used as
>> backing for HBase and the other for other things. Both hadoop clusters are
>> using the same version 1.0 and I want to merge them and make them one. I
>> know, one possible solution is to copy the data across, but the data is
>> really huge on these clusters and it will hard for me to compromise with
>> huge downtime.
>> Is there any optimal way to merge two hadoop clusters.
>>
>> ~Shashank
>>
>>
>
>
>
> --
>
>
>
>
>
>
>
> Thanks and Regards,
>
> VIVEK KOUL
Re: Regarding: Merging two hadoop clusters
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Like Vivek and Vinod are saying, using distcp might be the solution.
But you need to make sure you have enought free space in one cluster
to received the data from the other one. If you don't have, you might
need to re-assign some on the nodes from one cluster to the other one
to "concentrate" the free space on a singlue cluster, then to the
migration? That was you will not have any down-time. But you need to
make sure nother is written on the cluster you are migrating to not
miss any data.
JM
2013/3/14 vivek <vi...@gmail.com>:
> Distcp is the simplest approach you can use (it will copy data parallely
> using mappers and reducers).
>
>
>
> On Thu, Mar 14, 2013 at 12:16 PM, Vinod Kumar Vavilapalli
> <vi...@hortonworks.com> wrote:
>>
>>
>> Copy data into one of the clusters using distcp *without* downtime
>> (assuming you have enough capacity) and then merge the clusters?
>>
>> Thanks,
>> +Vinod Kumar Vavilapalli
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
>>
>> Hey Guys,
>>
>> I have two different hadoop clusters in production. One cluster is used as
>> backing for HBase and the other for other things. Both hadoop clusters are
>> using the same version 1.0 and I want to merge them and make them one. I
>> know, one possible solution is to copy the data across, but the data is
>> really huge on these clusters and it will hard for me to compromise with
>> huge downtime.
>> Is there any optimal way to merge two hadoop clusters.
>>
>> ~Shashank
>>
>>
>
>
>
> --
>
>
>
>
>
>
>
> Thanks and Regards,
>
> VIVEK KOUL
Re: Regarding: Merging two hadoop clusters
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Like Vivek and Vinod are saying, using distcp might be the solution.
But you need to make sure you have enought free space in one cluster
to received the data from the other one. If you don't have, you might
need to re-assign some on the nodes from one cluster to the other one
to "concentrate" the free space on a singlue cluster, then to the
migration? That was you will not have any down-time. But you need to
make sure nother is written on the cluster you are migrating to not
miss any data.
JM
2013/3/14 vivek <vi...@gmail.com>:
> Distcp is the simplest approach you can use (it will copy data parallely
> using mappers and reducers).
>
>
>
> On Thu, Mar 14, 2013 at 12:16 PM, Vinod Kumar Vavilapalli
> <vi...@hortonworks.com> wrote:
>>
>>
>> Copy data into one of the clusters using distcp *without* downtime
>> (assuming you have enough capacity) and then merge the clusters?
>>
>> Thanks,
>> +Vinod Kumar Vavilapalli
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
>>
>> Hey Guys,
>>
>> I have two different hadoop clusters in production. One cluster is used as
>> backing for HBase and the other for other things. Both hadoop clusters are
>> using the same version 1.0 and I want to merge them and make them one. I
>> know, one possible solution is to copy the data across, but the data is
>> really huge on these clusters and it will hard for me to compromise with
>> huge downtime.
>> Is there any optimal way to merge two hadoop clusters.
>>
>> ~Shashank
>>
>>
>
>
>
> --
>
>
>
>
>
>
>
> Thanks and Regards,
>
> VIVEK KOUL
Re: Regarding: Merging two hadoop clusters
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Like Vivek and Vinod are saying, using distcp might be the solution.
But you need to make sure you have enought free space in one cluster
to received the data from the other one. If you don't have, you might
need to re-assign some on the nodes from one cluster to the other one
to "concentrate" the free space on a singlue cluster, then to the
migration? That was you will not have any down-time. But you need to
make sure nother is written on the cluster you are migrating to not
miss any data.
JM
2013/3/14 vivek <vi...@gmail.com>:
> Distcp is the simplest approach you can use (it will copy data parallely
> using mappers and reducers).
>
>
>
> On Thu, Mar 14, 2013 at 12:16 PM, Vinod Kumar Vavilapalli
> <vi...@hortonworks.com> wrote:
>>
>>
>> Copy data into one of the clusters using distcp *without* downtime
>> (assuming you have enough capacity) and then merge the clusters?
>>
>> Thanks,
>> +Vinod Kumar Vavilapalli
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
>>
>> Hey Guys,
>>
>> I have two different hadoop clusters in production. One cluster is used as
>> backing for HBase and the other for other things. Both hadoop clusters are
>> using the same version 1.0 and I want to merge them and make them one. I
>> know, one possible solution is to copy the data across, but the data is
>> really huge on these clusters and it will hard for me to compromise with
>> huge downtime.
>> Is there any optimal way to merge two hadoop clusters.
>>
>> ~Shashank
>>
>>
>
>
>
> --
>
>
>
>
>
>
>
> Thanks and Regards,
>
> VIVEK KOUL
Re: Regarding: Merging two hadoop clusters
Posted by vivek <vi...@gmail.com>.
Distcp is the simplest approach you can use (it will copy data parallely
using mappers and reducers).
On Thu, Mar 14, 2013 at 12:16 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> Copy data into one of the clusters using distcp *without* downtime
> (assuming you have enough capacity) and then merge the clusters?
>
> Thanks,
> +Vinod Kumar Vavilapalli
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
>
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
>
>
--
Thanks and Regards,
VIVEK KOUL
Re: Regarding: Merging two hadoop clusters
Posted by vivek <vi...@gmail.com>.
Distcp is the simplest approach you can use (it will copy data parallely
using mappers and reducers).
On Thu, Mar 14, 2013 at 12:16 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> Copy data into one of the clusters using distcp *without* downtime
> (assuming you have enough capacity) and then merge the clusters?
>
> Thanks,
> +Vinod Kumar Vavilapalli
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
>
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
>
>
--
Thanks and Regards,
VIVEK KOUL
Re: Regarding: Merging two hadoop clusters
Posted by vivek <vi...@gmail.com>.
Distcp is the simplest approach you can use (it will copy data parallely
using mappers and reducers).
On Thu, Mar 14, 2013 at 12:16 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> Copy data into one of the clusters using distcp *without* downtime
> (assuming you have enough capacity) and then merge the clusters?
>
> Thanks,
> +Vinod Kumar Vavilapalli
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
>
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
>
>
--
Thanks and Regards,
VIVEK KOUL
Re: Regarding: Merging two hadoop clusters
Posted by vivek <vi...@gmail.com>.
Distcp is the simplest approach you can use (it will copy data parallely
using mappers and reducers).
On Thu, Mar 14, 2013 at 12:16 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> Copy data into one of the clusters using distcp *without* downtime
> (assuming you have enough capacity) and then merge the clusters?
>
> Thanks,
> +Vinod Kumar Vavilapalli
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
>
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
>
>
--
Thanks and Regards,
VIVEK KOUL
Re: Regarding: Merging two hadoop clusters
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Copy data into one of the clusters using distcp *without* downtime (assuming you have enough capacity) and then merge the clusters?
Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/
On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as backing for HBase and the other for other things. Both hadoop clusters are using the same version 1.0 and I want to merge them and make them one. I know, one possible solution is to copy the data across, but the data is really huge on these clusters and it will hard for me to compromise with huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
Re: Regarding: Merging two hadoop clusters
Posted by Azuryy Yu <az...@gmail.com>.
there is no good idea for your question. but for hadoop-2.x, its easy with
HDFS federation.
On Mar 14, 2013 12:38 PM, "Shashank Agarwal" <sh...@gmail.com>
wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
Re: Regarding: Merging two hadoop clusters
Posted by Azuryy Yu <az...@gmail.com>.
there is no good idea for your question. but for hadoop-2.x, its easy with
HDFS federation.
On Mar 14, 2013 12:38 PM, "Shashank Agarwal" <sh...@gmail.com>
wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
Re: Regarding: Merging two hadoop clusters
Posted by Azuryy Yu <az...@gmail.com>.
there is no good idea for your question. but for hadoop-2.x, its easy with
HDFS federation.
On Mar 14, 2013 12:38 PM, "Shashank Agarwal" <sh...@gmail.com>
wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
Re: Regarding: Merging two hadoop clusters
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Copy data into one of the clusters using distcp *without* downtime (assuming you have enough capacity) and then merge the clusters?
Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/
On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as backing for HBase and the other for other things. Both hadoop clusters are using the same version 1.0 and I want to merge them and make them one. I know, one possible solution is to copy the data across, but the data is really huge on these clusters and it will hard for me to compromise with huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
Re: Regarding: Merging two hadoop clusters
Posted by Lake Chang <la...@gmail.com>.
Actually I have the same question. Can anyone please kindly give a solution?
Thanks,
- Aaron
On Thu, Mar 14, 2013 at 12:38 PM, Shashank Agarwal <
shashankagarwal1706@gmail.com> wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
Re: Regarding: Merging two hadoop clusters
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Copy data into one of the clusters using distcp *without* downtime (assuming you have enough capacity) and then merge the clusters?
Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/
On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as backing for HBase and the other for other things. Both hadoop clusters are using the same version 1.0 and I want to merge them and make them one. I know, one possible solution is to copy the data across, but the data is really huge on these clusters and it will hard for me to compromise with huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
Re: Regarding: Merging two hadoop clusters
Posted by Suresh Srinivas <su...@hortonworks.com>.
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
This is not a supported feature. Hence this activity would require
understanding low level Hadoop details, quite a bit of hacking and is not
straightforward. Copying data from the clusters is the simplest solution.
Re: Regarding: Merging two hadoop clusters
Posted by Lake Chang <la...@gmail.com>.
Actually I have the same question. Can anyone please kindly give a solution?
Thanks,
- Aaron
On Thu, Mar 14, 2013 at 12:38 PM, Shashank Agarwal <
shashankagarwal1706@gmail.com> wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
Re: Regarding: Merging two hadoop clusters
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Copy data into one of the clusters using distcp *without* downtime (assuming you have enough capacity) and then merge the clusters?
Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/
On Mar 13, 2013, at 9:38 PM, Shashank Agarwal wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as backing for HBase and the other for other things. Both hadoop clusters are using the same version 1.0 and I want to merge them and make them one. I know, one possible solution is to copy the data across, but the data is really huge on these clusters and it will hard for me to compromise with huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
Re: Regarding: Merging two hadoop clusters
Posted by Lake Chang <la...@gmail.com>.
Actually I have the same question. Can anyone please kindly give a solution?
Thanks,
- Aaron
On Thu, Mar 14, 2013 at 12:38 PM, Shashank Agarwal <
shashankagarwal1706@gmail.com> wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
Re: Regarding: Merging two hadoop clusters
Posted by Lake Chang <la...@gmail.com>.
Actually I have the same question. Can anyone please kindly give a solution?
Thanks,
- Aaron
On Thu, Mar 14, 2013 at 12:38 PM, Shashank Agarwal <
shashankagarwal1706@gmail.com> wrote:
> Hey Guys,
>
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
> ~Shashank
>
Re: Regarding: Merging two hadoop clusters
Posted by Suresh Srinivas <su...@hortonworks.com>.
> I have two different hadoop clusters in production. One cluster is used as
> backing for HBase and the other for other things. Both hadoop clusters are
> using the same version 1.0 and I want to merge them and make them one. I
> know, one possible solution is to copy the data across, but the data is
> really huge on these clusters and it will hard for me to compromise with
> huge downtime.
> Is there any optimal way to merge two hadoop clusters.
>
This is not a supported feature. Hence this activity would require
understanding low level Hadoop details, quite a bit of hacking and is not
straightforward. Copying data from the clusters is the simplest solution.