You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Juan P." <go...@gmail.com> on 2012/12/07 19:14:38 UTC
PROD/DR - Replication
I was reading up on HBase Replication and wanted to make sure I'm not
missing something.
Given that replication happens asynchronously the replication strategy has
an "eventually consistent" policy.
I was considering using this feature for Production / Disaster Recovery
setup.
Is there a way to enforce Consistency so that if my PROD environment should
ever go down, I can 100% sure that DR will be completely up to date?
Thank you,
Juan
Re: PROD/DR - Replication
Posted by Amandeep Khurana <am...@gmail.com>.
What failure condition are you trying to safeguard against? A full
data center failure? That's when you would lose your entire cluster
and need the DR to kick in. Otherwise, you could deploy such that an
entire rack failure or even a row failure won't take you down. Just
span across multiple racks and rows. For a full data center failure,
the current replication scheme only gives you asynchronous replication
for reasons Ian mentioned.
On Dec 7, 2012, at 10:15 AM, "Juan P." <go...@gmail.com> wrote:
> I was reading up on HBase Replication and wanted to make sure I'm not
> missing something.
>
> Given that replication happens asynchronously the replication strategy has
> an "eventually consistent" policy.
>
> I was considering using this feature for Production / Disaster Recovery
> setup.
>
> Is there a way to enforce Consistency so that if my PROD environment should
> ever go down, I can 100% sure that DR will be completely up to date?
>
> Thank you,
> Juan
PROD/DR - Replication
Posted by Andrew Purtell <ap...@apache.org>.
Agree with what Ian says below except I'd say with work it's possible to do
cross DC quorum writes in a layer on top of HBase that persists into
HBase, like
when Google built Megastore on BigTable. At this year's VLDB if I recall
correctly there was a megastore-like system with an interesting variation built
on HBase presented in http://arxiv.org/abs/1208.0270 .
On Saturday, December 8, 2012, Ian Varley wrote:
> Juan,
>
> No; that would mean every single write to HBase has to wait for an ACK
> from a remote data center, which would decrease your cluster throughput
> dramatically. If you need that, consider other database solutions.
>
> Ian
>
> On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
>
> I was reading up on HBase Replication and wanted to make sure I'm not
> missing something.
>
> Given that replication happens asynchronously the replication strategy has
> an "eventually consistent" policy.
>
> I was considering using this feature for Production / Disaster Recovery
> setup.
>
> Is there a way to enforce Consistency so that if my PROD environment should
> ever go down, I can 100% sure that DR will be completely up to date?
>
> Thank you,
> Juan
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: PROD/DR - Replication
Posted by Ian Varley <iv...@salesforce.com>.
Ha - that's what I get for trying to answer list emails from my phone. :)
Ian
On Dec 7, 2012, at 1:58 PM, sriraam h wrote:
Thanks Ian. I DID miss the point. The person who started the chain is a different person :)
- Sri
________________________________
From: Ian Varley <iv...@salesforce.com>>
To: "user@hbase.apache.org<ma...@hbase.apache.org>" <us...@hbase.apache.org>>
Sent: Saturday, 8 December 2012, 1:21
Subject: Re: PROD/DR - Replication
Yes, I think so. A single HBase cluster can't (or, at least, really shouldn't) span multiple data centers; the strong consistency you refer to is only available within a cluster.
But the replication you were referring to in your initial email is cross-data center, between two or more clusters. That's where you can't get strong consistency.
Ian
On Dec 7, 2012, at 1:38 PM, "sriraam h" <el...@yahoo.com>> wrote:
"Strongly consistent reads/writes: HBase is not an "eventually consistent" DataStore. This makes it very suitable for tasks such as high-speed counter aggregation"
http://hbase.apache.org/book/architecture.html
Am I missing something ?
- Sri
________________________________
From: Ian Varley <iv...@salesforce.com>>
To: "user@hbase.apache.org<ma...@hbase.apache.org>" <us...@hbase.apache.org>>
Sent: Friday, 7 December 2012, 23:49
Subject: Re: PROD/DR - Replication
Juan,
No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.
Ian
On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
I was reading up on HBase Replication and wanted to make sure I'm not
missing something.
Given that replication happens asynchronously the replication strategy has
an "eventually consistent" policy.
I was considering using this feature for Production / Disaster Recovery
setup.
Is there a way to enforce Consistency so that if my PROD environment should
ever go down, I can 100% sure that DR will be completely up to date?
Thank you,
Juan
Re: PROD/DR - Replication
Posted by sriraam h <el...@yahoo.com>.
Thanks Ian. I DID miss the point. The person who started the chain is a different person :)
- Sri
>________________________________
> From: Ian Varley <iv...@salesforce.com>
>To: "user@hbase.apache.org" <us...@hbase.apache.org>
>Sent: Saturday, 8 December 2012, 1:21
>Subject: Re: PROD/DR - Replication
>
>Yes, I think so. A single HBase cluster can't (or, at least, really shouldn't) span multiple data centers; the strong consistency you refer to is only available within a cluster.
>
>But the replication you were referring to in your initial email is cross-data center, between two or more clusters. That's where you can't get strong consistency.
>
>Ian
>
>
>
>On Dec 7, 2012, at 1:38 PM, "sriraam h" <el...@yahoo.com> wrote:
>
>> "Strongly consistent reads/writes: HBase is not an "eventually consistent" DataStore. This makes it very suitable for tasks such as high-speed counter aggregation"
>>
>> http://hbase.apache.org/book/architecture.html
>>
>>
>> Am I missing something ?
>>
>> - Sri
>>
>>
>>
>>> ________________________________
>>> From: Ian Varley <iv...@salesforce.com>
>>> To: "user@hbase.apache.org" <us...@hbase.apache.org>
>>> Sent: Friday, 7 December 2012, 23:49
>>> Subject: Re: PROD/DR - Replication
>>>
>>> Juan,
>>>
>>> No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.
>>>
>>> Ian
>>>
>>> On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
>>>
>>> I was reading up on HBase Replication and wanted to make sure I'm not
>>> missing something.
>>>
>>> Given that replication happens asynchronously the replication strategy has
>>> an "eventually consistent" policy.
>>>
>>> I was considering using this feature for Production / Disaster Recovery
>>> setup.
>>>
>>> Is there a way to enforce Consistency so that if my PROD environment should
>>> ever go down, I can 100% sure that DR will be completely up to date?
>>>
>>> Thank you,
>>> Juan
>>>
>>>
>>>
>
>
>
Re: PROD/DR - Replication
Posted by Ian Varley <iv...@salesforce.com>.
Yes, I think so. A single HBase cluster can't (or, at least, really shouldn't) span multiple data centers; the strong consistency you refer to is only available within a cluster.
But the replication you were referring to in your initial email is cross-data center, between two or more clusters. That's where you can't get strong consistency.
Ian
On Dec 7, 2012, at 1:38 PM, "sriraam h" <el...@yahoo.com> wrote:
> "Strongly consistent reads/writes: HBase is not an "eventually consistent" DataStore. This makes it very suitable for tasks such as high-speed counter aggregation"
>
> http://hbase.apache.org/book/architecture.html
>
>
> Am I missing something ?
>
> - Sri
>
>
>
>> ________________________________
>> From: Ian Varley <iv...@salesforce.com>
>> To: "user@hbase.apache.org" <us...@hbase.apache.org>
>> Sent: Friday, 7 December 2012, 23:49
>> Subject: Re: PROD/DR - Replication
>>
>> Juan,
>>
>> No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.
>>
>> Ian
>>
>> On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
>>
>> I was reading up on HBase Replication and wanted to make sure I'm not
>> missing something.
>>
>> Given that replication happens asynchronously the replication strategy has
>> an "eventually consistent" policy.
>>
>> I was considering using this feature for Production / Disaster Recovery
>> setup.
>>
>> Is there a way to enforce Consistency so that if my PROD environment should
>> ever go down, I can 100% sure that DR will be completely up to date?
>>
>> Thank you,
>> Juan
>>
>>
>>
Re: PROD/DR - Replication
Posted by sriraam h <el...@yahoo.com>.
"Strongly consistent reads/writes: HBase is not an "eventually consistent" DataStore. This makes it very suitable for tasks such as high-speed counter aggregation"
http://hbase.apache.org/book/architecture.html
Am I missing something ?
- Sri
>________________________________
> From: Ian Varley <iv...@salesforce.com>
>To: "user@hbase.apache.org" <us...@hbase.apache.org>
>Sent: Friday, 7 December 2012, 23:49
>Subject: Re: PROD/DR - Replication
>
>Juan,
>
>No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.
>
>Ian
>
>On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
>
>I was reading up on HBase Replication and wanted to make sure I'm not
>missing something.
>
>Given that replication happens asynchronously the replication strategy has
>an "eventually consistent" policy.
>
>I was considering using this feature for Production / Disaster Recovery
>setup.
>
>Is there a way to enforce Consistency so that if my PROD environment should
>ever go down, I can 100% sure that DR will be completely up to date?
>
>Thank you,
>Juan
>
>
>
>
Re: PROD/DR - Replication
Posted by Ian Varley <iv...@salesforce.com>.
Juan,
No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.
Ian
On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
I was reading up on HBase Replication and wanted to make sure I'm not
missing something.
Given that replication happens asynchronously the replication strategy has
an "eventually consistent" policy.
I was considering using this feature for Production / Disaster Recovery
setup.
Is there a way to enforce Consistency so that if my PROD environment should
ever go down, I can 100% sure that DR will be completely up to date?
Thank you,
Juan