You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by samir das mohapatra <sa...@gmail.com> on 2013/02/22 07:37:21 UTC
How To Load Data Between two cluster
Hi All,
I have senario in my project dev,
The senario is, I have a bussiness data in HBASE which is different
from Development cluster but the requirement is , how to bring the data
from Hbase which is on one cluster to HDFS another cluster
CASE1:
Let cluster-1 and Cluster-2
cluster-1(Hbase) ---------------> cluster2(hdfs)
cluster-1(Hbase) <--------------- cluster2(hdfs)
Case2:
cluster-1(Hbase) ---------------> cluster2(hive)
cluster-1(Hbase) <--------------- cluster2(hive)
Note:- Before going forward I am looking good solution , which will be
best fit.
Regards,
samir.
Re: How To Load Data Between two cluster
Posted by Mahesh Balija <ba...@gmail.com>.
Hi Samir,
One solution for your Case 1 can be,
1) First you export your data in Hbase (Cluster1) to the
HDFS (Cluster1)
2) And then using distcp command you can copy the data
from HDFS (cluster1) to HDFS (cluster2)
and vice versa
I am NOT sure whether this is the optimized solution, prolly
you can check for other approaches.
Case 2:- After case 1 you can build Hive tables on the HDFS
(Cluster2)
Best,
Mahesh Balija,
CalsoftLabs.
On Fri, Feb 22, 2013 at 12:07 PM, samir das mohapatra <
samir.helpdoc@gmail.com> wrote:
> Hi All,
> I have senario in my project dev,
> The senario is, I have a bussiness data in HBASE which is different
> from Development cluster but the requirement is , how to bring the data
> from Hbase which is on one cluster to HDFS another cluster
>
> CASE1:
> Let cluster-1 and Cluster-2
>
> cluster-1(Hbase) ---------------> cluster2(hdfs)
> cluster-1(Hbase) <--------------- cluster2(hdfs)
>
> Case2:
> cluster-1(Hbase) ---------------> cluster2(hive)
> cluster-1(Hbase) <--------------- cluster2(hive)
>
>
> Note:- Before going forward I am looking good solution , which will be
> best fit.
>
>
> Regards,
> samir.
>
Re: How To Load Data Between two cluster
Posted by Mahesh Balija <ba...@gmail.com>.
Hi Samir,
One solution for your Case 1 can be,
1) First you export your data in Hbase (Cluster1) to the
HDFS (Cluster1)
2) And then using distcp command you can copy the data
from HDFS (cluster1) to HDFS (cluster2)
and vice versa
I am NOT sure whether this is the optimized solution, prolly
you can check for other approaches.
Case 2:- After case 1 you can build Hive tables on the HDFS
(Cluster2)
Best,
Mahesh Balija,
CalsoftLabs.
On Fri, Feb 22, 2013 at 12:07 PM, samir das mohapatra <
samir.helpdoc@gmail.com> wrote:
> Hi All,
> I have senario in my project dev,
> The senario is, I have a bussiness data in HBASE which is different
> from Development cluster but the requirement is , how to bring the data
> from Hbase which is on one cluster to HDFS another cluster
>
> CASE1:
> Let cluster-1 and Cluster-2
>
> cluster-1(Hbase) ---------------> cluster2(hdfs)
> cluster-1(Hbase) <--------------- cluster2(hdfs)
>
> Case2:
> cluster-1(Hbase) ---------------> cluster2(hive)
> cluster-1(Hbase) <--------------- cluster2(hive)
>
>
> Note:- Before going forward I am looking good solution , which will be
> best fit.
>
>
> Regards,
> samir.
>
Re: How To Load Data Between two cluster
Posted by Mahesh Balija <ba...@gmail.com>.
Hi Samir,
One solution for your Case 1 can be,
1) First you export your data in Hbase (Cluster1) to the
HDFS (Cluster1)
2) And then using distcp command you can copy the data
from HDFS (cluster1) to HDFS (cluster2)
and vice versa
I am NOT sure whether this is the optimized solution, prolly
you can check for other approaches.
Case 2:- After case 1 you can build Hive tables on the HDFS
(Cluster2)
Best,
Mahesh Balija,
CalsoftLabs.
On Fri, Feb 22, 2013 at 12:07 PM, samir das mohapatra <
samir.helpdoc@gmail.com> wrote:
> Hi All,
> I have senario in my project dev,
> The senario is, I have a bussiness data in HBASE which is different
> from Development cluster but the requirement is , how to bring the data
> from Hbase which is on one cluster to HDFS another cluster
>
> CASE1:
> Let cluster-1 and Cluster-2
>
> cluster-1(Hbase) ---------------> cluster2(hdfs)
> cluster-1(Hbase) <--------------- cluster2(hdfs)
>
> Case2:
> cluster-1(Hbase) ---------------> cluster2(hive)
> cluster-1(Hbase) <--------------- cluster2(hive)
>
>
> Note:- Before going forward I am looking good solution , which will be
> best fit.
>
>
> Regards,
> samir.
>
Re: How To Load Data Between two cluster
Posted by Mahesh Balija <ba...@gmail.com>.
Hi Samir,
One solution for your Case 1 can be,
1) First you export your data in Hbase (Cluster1) to the
HDFS (Cluster1)
2) And then using distcp command you can copy the data
from HDFS (cluster1) to HDFS (cluster2)
and vice versa
I am NOT sure whether this is the optimized solution, prolly
you can check for other approaches.
Case 2:- After case 1 you can build Hive tables on the HDFS
(Cluster2)
Best,
Mahesh Balija,
CalsoftLabs.
On Fri, Feb 22, 2013 at 12:07 PM, samir das mohapatra <
samir.helpdoc@gmail.com> wrote:
> Hi All,
> I have senario in my project dev,
> The senario is, I have a bussiness data in HBASE which is different
> from Development cluster but the requirement is , how to bring the data
> from Hbase which is on one cluster to HDFS another cluster
>
> CASE1:
> Let cluster-1 and Cluster-2
>
> cluster-1(Hbase) ---------------> cluster2(hdfs)
> cluster-1(Hbase) <--------------- cluster2(hdfs)
>
> Case2:
> cluster-1(Hbase) ---------------> cluster2(hive)
> cluster-1(Hbase) <--------------- cluster2(hive)
>
>
> Note:- Before going forward I am looking good solution , which will be
> best fit.
>
>
> Regards,
> samir.
>