You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by Vimal Jain <vk...@gmail.com> on 2014/07/22 07:51:20 UTC

Replication in Hbase

Hi,
I have 2 Hbase clusters setup in different data center.
Both are configured in pseudo-distributed mode.
I followed the steps in Hbase Replication
<https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements>
.
But i am getting following logs in master cluster's region server log.

2014-07-22 11:19:19,186 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication ip-10-14-24-19%2C60020%2C1405945008796.1406006236991 at
134
2014-07-22 11:19:19,193 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
currentNbOperations:0 and seenEntries:0 and size: 0
2014-07-22 11:19:19,193 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Nothing
to replicate, sleeping 1000 times 10


What i am missing here ?
Also in one of the requirements mentioned in the above post , it says
zookeeper should not be managed by Hbase.But i have Hbase managing
zookeeper in both clusters.


Please help here.
Basically , i want to copy all data from one cluster to another which are
geographically distant.


-- 
Thanks and Regards,
Vimal Jain

Re: Replication in Hbase

Posted by Ted Yu <yu...@gmail.com>.

Please also see 17.8.8 under http://hbase.apache.org/book.html#ops.snapshots

Cheers

On Jul 22, 2014, at 1:57 AM, Wellington Chevreuil <we...@gmail.com> wrote:

> I think you need to run a ZK instance apart from HBase. But if your main goal is to copy data from one cluster to another, you may use other options, such as CopyTable, Bulkload, or Export/Import tools. Replication will not copy data already inserted previously on your source Hbase, it only replicates transactions as it happens, from source to destination, only once replication has been enabled.
> 
> http://hbase.apache.org/book/ops_mgt.html#copytable
> 
> http://hbase.apache.org/book/ops_mgt.html#export
> 
> http://hbase.apache.org/book/ops_mgt.html#completebulkload
> 
> 
> 
> 
> 
> On 22 Jul 2014, at 07:09, Vimal Jain <vk...@gmail.com> wrote:
> 
>> One more info ,
>> After putting some data in master cluster , i am getting below in its
>> regionserver log.
>> 
>> 2014-07-22 11:38:14,032 INFO
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting
>> 1 rs from peer cluster # 1
>> 2014-07-22 11:38:14,032 INFO
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
>> Choosing peer XYZ,60020,1406009110004
>> 2014-07-22 11:38:15,032 DEBUG
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Since
>> we are unable to replicate, sleeping 1000 times 10
>> 
>> where XYZ is my slave cluster's machine.
>> 
>> 
>> 
>> On Tue, Jul 22, 2014 at 11:21 AM, Vimal Jain <vk...@gmail.com> wrote:
>> 
>>> Hi,
>>> I have 2 Hbase clusters setup in different data center.
>>> Both are configured in pseudo-distributed mode.
>>> I followed the steps in Hbase Replication
>>> <https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements>
>>> .
>>> But i am getting following logs in master cluster's region server log.
>>> 
>>> 2014-07-22 11:19:19,186 DEBUG
>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
>>> log for replication ip-10-14-24-19%2C60020%2C1405945008796.1406006236991 at
>>> 134
>>> 2014-07-22 11:19:19,193 DEBUG
>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
>>> currentNbOperations:0 and seenEntries:0 and size: 0
>>> 2014-07-22 11:19:19,193 DEBUG
>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Nothing
>>> to replicate, sleeping 1000 times 10
>>> 
>>> 
>>> What i am missing here ?
>>> Also in one of the requirements mentioned in the above post , it says
>>> zookeeper should not be managed by Hbase.But i have Hbase managing
>>> zookeeper in both clusters.
>>> 
>>> 
>>> Please help here.
>>> Basically , i want to copy all data from one cluster to another which are
>>> geographically distant.
>>> 
>>> 
>>> --
>>> Thanks and Regards,
>>> Vimal Jain
>> 
>> 
>> 
>> -- 
>> Thanks and Regards,
>> Vimal Jain
>

Re: Replication in Hbase

Posted by Wellington Chevreuil <we...@gmail.com>.

I think you need to run a ZK instance apart from HBase. But if your main goal is to copy data from one cluster to another, you may use other options, such as CopyTable, Bulkload, or Export/Import tools. Replication will not copy data already inserted previously on your source Hbase, it only replicates transactions as it happens, from source to destination, only once replication has been enabled.

http://hbase.apache.org/book/ops_mgt.html#copytable

http://hbase.apache.org/book/ops_mgt.html#export

http://hbase.apache.org/book/ops_mgt.html#completebulkload





On 22 Jul 2014, at 07:09, Vimal Jain <vk...@gmail.com> wrote:

> One more info ,
> After putting some data in master cluster , i am getting below in its
> regionserver log.
> 
> 2014-07-22 11:38:14,032 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting
> 1 rs from peer cluster # 1
> 2014-07-22 11:38:14,032 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Choosing peer XYZ,60020,1406009110004
> 2014-07-22 11:38:15,032 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Since
> we are unable to replicate, sleeping 1000 times 10
> 
> where XYZ is my slave cluster's machine.
> 
> 
> 
> On Tue, Jul 22, 2014 at 11:21 AM, Vimal Jain <vk...@gmail.com> wrote:
> 
>> Hi,
>> I have 2 Hbase clusters setup in different data center.
>> Both are configured in pseudo-distributed mode.
>> I followed the steps in Hbase Replication
>> <https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements>
>> .
>> But i am getting following logs in master cluster's region server log.
>> 
>> 2014-07-22 11:19:19,186 DEBUG
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
>> log for replication ip-10-14-24-19%2C60020%2C1405945008796.1406006236991 at
>> 134
>> 2014-07-22 11:19:19,193 DEBUG
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
>> currentNbOperations:0 and seenEntries:0 and size: 0
>> 2014-07-22 11:19:19,193 DEBUG
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Nothing
>> to replicate, sleeping 1000 times 10
>> 
>> 
>> What i am missing here ?
>> Also in one of the requirements mentioned in the above post , it says
>> zookeeper should not be managed by Hbase.But i have Hbase managing
>> zookeeper in both clusters.
>> 
>> 
>> Please help here.
>> Basically , i want to copy all data from one cluster to another which are
>> geographically distant.
>> 
>> 
>> --
>> Thanks and Regards,
>> Vimal Jain
>> 
> 
> 
> 
> -- 
> Thanks and Regards,
> Vimal Jain

Re: Replication in Hbase

Posted by Vimal Jain <vk...@gmail.com>.

One more info ,
After putting some data in master cluster , i am getting below in its
regionserver log.

2014-07-22 11:38:14,032 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting
1 rs from peer cluster # 1
2014-07-22 11:38:14,032 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Choosing peer XYZ,60020,1406009110004
2014-07-22 11:38:15,032 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Since
we are unable to replicate, sleeping 1000 times 10

where XYZ is my slave cluster's machine.



On Tue, Jul 22, 2014 at 11:21 AM, Vimal Jain <vk...@gmail.com> wrote:

> Hi,
> I have 2 Hbase clusters setup in different data center.
> Both are configured in pseudo-distributed mode.
> I followed the steps in Hbase Replication
> <https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements>
> .
> But i am getting following logs in master cluster's region server log.
>
> 2014-07-22 11:19:19,186 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
> log for replication ip-10-14-24-19%2C60020%2C1405945008796.1406006236991 at
> 134
> 2014-07-22 11:19:19,193 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> currentNbOperations:0 and seenEntries:0 and size: 0
> 2014-07-22 11:19:19,193 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Nothing
> to replicate, sleeping 1000 times 10
>
>
> What i am missing here ?
> Also in one of the requirements mentioned in the above post , it says
> zookeeper should not be managed by Hbase.But i have Hbase managing
> zookeeper in both clusters.
>
>
> Please help here.
> Basically , i want to copy all data from one cluster to another which are
> geographically distant.
>
>
> --
> Thanks and Regards,
> Vimal Jain
>



-- 
Thanks and Regards,
Vimal Jain