You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Konstantin Boudnik <co...@apache.org> on 2014/04/08 01:15:57 UTC
Bringing consensus based strong consistency into HBase
Guys,
As some of you might have noticed there is a number of new JIRAs opened
recently that are aiming on abstracting and separating ZK out of HBase guts
and making it an implementation detail, rather than a center of attention
for some parts of the HBase.
I would like to send around a short document written by Mikhail Antonov that
is trying to clarify a couple of points about this whole effort.
People are asking me about this off-line, so I decided to send this around so
we can have a lively and wider discussion about this initiative and the
development.
Here's the JIRA and the link to the PDF document.
https://issues.apache.org/jira/browse/HBASE-10866
https://issues.apache.org/jira/secure/attachment/12637957/HBaseConsensus.pdf
The umbrella JIRA for this effort is here
https://issues.apache.org/jira/browse/HBASE-10909
--
Regardsk
Cos
Re: Bringing consensus based strong consistency into HBase
Posted by Mikhail Antonov <ol...@gmail.com>.
Ted, thanks for the comments on Jira.
I will put this on shared google doc and send out link as you guys advised
so it'd be easier to collaborate on.
Mikhail
2014-04-18 10:49 GMT-07:00 Stack <st...@duboce.net>:
> The doc helps Mikhail. Its a fun read. Thanks.
> St.Ack
>
>
> On Thu, Apr 17, 2014 at 9:29 PM, Mikhail Antonov <olorinbant@gmail.com
> >wrote:
>
> > Guys,
> >
> > attached is a reworked version of this document which includes:
> >
> >
> > - "starting fresh" approach to consensus operations in HBase
> > - benefits it could bring
> > - code pointers and list of modifications required to abstract ZK
> >
> > I'd appreciate review and feedbacks! The document is also attached to
> > HBASE-10909 for reference and comments.
> >
> > Thanks,
> >
> > Mikhail
> >
> >
> > 2014-04-09 16:33 GMT-07:00 Mikhail Antonov <ol...@gmail.com>:
> >
> > Guys,
> >>
> >> as was advised, I moved the PDF document to umbrella jira, here -
> >> https://issues.apache.org/jira/browse/HBASE-10909.
> >>
> >> Thanks,
> >> Mikhail
> >>
> >>
> >> 2014-04-08 8:34 GMT-07:00 Andrew Purtell <ap...@apache.org>:
> >>
> >> Hi Cos,
> >>>
> >>> Thanks for providing this overview, and more importantly, considering
> >>> these
> >>> contributions. Reading through the relevant JIRAs, looks to me like
> HBase
> >>> will be in a better position than before to have this abstraction in
> >>> place,
> >>> provided the changes are fully completed.
> >>>
> >>> Thanks again for the kind consideration.
> >>>
> >>>
> >>> On Mon, Apr 7, 2014 at 4:15 PM, Konstantin Boudnik <co...@apache.org>
> >>> wrote:
> >>>
> >>> > Guys,
> >>> >
> >>> > As some of you might have noticed there is a number of new JIRAs
> opened
> >>> > recently that are aiming on abstracting and separating ZK out of
> HBase
> >>> guts
> >>> > and making it an implementation detail, rather than a center of
> >>> attention
> >>> > for some parts of the HBase.
> >>> >
> >>> > I would like to send around a short document written by Mikhail
> Antonov
> >>> > that
> >>> > is trying to clarify a couple of points about this whole effort.
> >>> > People are asking me about this off-line, so I decided to send this
> >>> around
> >>> > so
> >>> > we can have a lively and wider discussion about this initiative and
> the
> >>> > development.
> >>> >
> >>> > Here's the JIRA and the link to the PDF document.
> >>> > https://issues.apache.org/jira/browse/HBASE-10866
> >>> >
> >>> >
> >>>
> https://issues.apache.org/jira/secure/attachment/12637957/HBaseConsensus.pdf
> >>> >
> >>> > The umbrella JIRA for this effort is here
> >>> > https://issues.apache.org/jira/browse/HBASE-10909
> >>> >
> >>> > --
> >>> > Regardsk
> >>> > Cos
> >>> >
> >>> >
> >>>
> >>>
> >>> --
> >>> Best regards,
> >>>
> >>> - Andy
> >>>
> >>> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >>> (via Tom White)
> >>>
> >>
> >>
> >>
> >> --
> >> Thanks,
> >> Michael Antonov
> >>
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
> >
>
--
Thanks,
Michael Antonov
Re: Bringing consensus based strong consistency into HBase
Posted by Mikhail Antonov <ol...@gmail.com>.
The google doc has been looked at by quite a few guys, I think.
I'm about to create some more jiras around abstracting ZooKeeper, and would
really appreciate if someone has specific comments on the section of
document outlining the steps to abstract ZK - to check some important
pieces aren't forgotten.
Thanks!
Mikhail
2014-04-18 10:49 GMT-07:00 Stack <st...@duboce.net>:
> The doc helps Mikhail. Its a fun read. Thanks.
> St.Ack
>
>
> On Thu, Apr 17, 2014 at 9:29 PM, Mikhail Antonov <olorinbant@gmail.com
> >wrote:
>
> > Guys,
> >
> > attached is a reworked version of this document which includes:
> >
> >
> > - "starting fresh" approach to consensus operations in HBase
> > - benefits it could bring
> > - code pointers and list of modifications required to abstract ZK
> >
> > I'd appreciate review and feedbacks! The document is also attached to
> > HBASE-10909 for reference and comments.
> >
> > Thanks,
> >
> > Mikhail
> >
> >
> > 2014-04-09 16:33 GMT-07:00 Mikhail Antonov <ol...@gmail.com>:
> >
> > Guys,
> >>
> >> as was advised, I moved the PDF document to umbrella jira, here -
> >> https://issues.apache.org/jira/browse/HBASE-10909.
> >>
> >> Thanks,
> >> Mikhail
> >>
> >>
> >> 2014-04-08 8:34 GMT-07:00 Andrew Purtell <ap...@apache.org>:
> >>
> >> Hi Cos,
> >>>
> >>> Thanks for providing this overview, and more importantly, considering
> >>> these
> >>> contributions. Reading through the relevant JIRAs, looks to me like
> HBase
> >>> will be in a better position than before to have this abstraction in
> >>> place,
> >>> provided the changes are fully completed.
> >>>
> >>> Thanks again for the kind consideration.
> >>>
> >>>
> >>> On Mon, Apr 7, 2014 at 4:15 PM, Konstantin Boudnik <co...@apache.org>
> >>> wrote:
> >>>
> >>> > Guys,
> >>> >
> >>> > As some of you might have noticed there is a number of new JIRAs
> opened
> >>> > recently that are aiming on abstracting and separating ZK out of
> HBase
> >>> guts
> >>> > and making it an implementation detail, rather than a center of
> >>> attention
> >>> > for some parts of the HBase.
> >>> >
> >>> > I would like to send around a short document written by Mikhail
> Antonov
> >>> > that
> >>> > is trying to clarify a couple of points about this whole effort.
> >>> > People are asking me about this off-line, so I decided to send this
> >>> around
> >>> > so
> >>> > we can have a lively and wider discussion about this initiative and
> the
> >>> > development.
> >>> >
> >>> > Here's the JIRA and the link to the PDF document.
> >>> > https://issues.apache.org/jira/browse/HBASE-10866
> >>> >
> >>> >
> >>>
> https://issues.apache.org/jira/secure/attachment/12637957/HBaseConsensus.pdf
> >>> >
> >>> > The umbrella JIRA for this effort is here
> >>> > https://issues.apache.org/jira/browse/HBASE-10909
> >>> >
> >>> > --
> >>> > Regardsk
> >>> > Cos
> >>> >
> >>> >
> >>>
> >>>
> >>> --
> >>> Best regards,
> >>>
> >>> - Andy
> >>>
> >>> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >>> (via Tom White)
> >>>
> >>
> >>
> >>
> >> --
> >> Thanks,
> >> Michael Antonov
> >>
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
> >
>
--
Thanks,
Michael Antonov
Re: Bringing consensus based strong consistency into HBase
Posted by Stack <st...@duboce.net>.
The doc helps Mikhail. Its a fun read. Thanks.
St.Ack
On Thu, Apr 17, 2014 at 9:29 PM, Mikhail Antonov <ol...@gmail.com>wrote:
> Guys,
>
> attached is a reworked version of this document which includes:
>
>
> - "starting fresh" approach to consensus operations in HBase
> - benefits it could bring
> - code pointers and list of modifications required to abstract ZK
>
> I'd appreciate review and feedbacks! The document is also attached to
> HBASE-10909 for reference and comments.
>
> Thanks,
>
> Mikhail
>
>
> 2014-04-09 16:33 GMT-07:00 Mikhail Antonov <ol...@gmail.com>:
>
> Guys,
>>
>> as was advised, I moved the PDF document to umbrella jira, here -
>> https://issues.apache.org/jira/browse/HBASE-10909.
>>
>> Thanks,
>> Mikhail
>>
>>
>> 2014-04-08 8:34 GMT-07:00 Andrew Purtell <ap...@apache.org>:
>>
>> Hi Cos,
>>>
>>> Thanks for providing this overview, and more importantly, considering
>>> these
>>> contributions. Reading through the relevant JIRAs, looks to me like HBase
>>> will be in a better position than before to have this abstraction in
>>> place,
>>> provided the changes are fully completed.
>>>
>>> Thanks again for the kind consideration.
>>>
>>>
>>> On Mon, Apr 7, 2014 at 4:15 PM, Konstantin Boudnik <co...@apache.org>
>>> wrote:
>>>
>>> > Guys,
>>> >
>>> > As some of you might have noticed there is a number of new JIRAs opened
>>> > recently that are aiming on abstracting and separating ZK out of HBase
>>> guts
>>> > and making it an implementation detail, rather than a center of
>>> attention
>>> > for some parts of the HBase.
>>> >
>>> > I would like to send around a short document written by Mikhail Antonov
>>> > that
>>> > is trying to clarify a couple of points about this whole effort.
>>> > People are asking me about this off-line, so I decided to send this
>>> around
>>> > so
>>> > we can have a lively and wider discussion about this initiative and the
>>> > development.
>>> >
>>> > Here's the JIRA and the link to the PDF document.
>>> > https://issues.apache.org/jira/browse/HBASE-10866
>>> >
>>> >
>>> https://issues.apache.org/jira/secure/attachment/12637957/HBaseConsensus.pdf
>>> >
>>> > The umbrella JIRA for this effort is here
>>> > https://issues.apache.org/jira/browse/HBASE-10909
>>> >
>>> > --
>>> > Regardsk
>>> > Cos
>>> >
>>> >
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> - Andy
>>>
>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>> (via Tom White)
>>>
>>
>>
>>
>> --
>> Thanks,
>> Michael Antonov
>>
>
>
>
> --
> Thanks,
> Michael Antonov
>
Re: Bringing consensus based strong consistency into HBase
Posted by Ted Yu <yu...@gmail.com>.
I put some comments on HBASE-10909 after reading the updated proposal.
Using Google Docs would make review process smooth.
Cheers
On Fri, Apr 18, 2014 at 10:28 AM, Nick Dimiduk <nd...@gmail.com> wrote:
> For future reference, attachments are filtered out by the list serve. For
> working drafts, we've had success using a shared editing service (Google
> Docs, specifically). Best to attach later-stage docs to JIRAs.
>
>
> On Thu, Apr 17, 2014 at 9:29 PM, Mikhail Antonov <olorinbant@gmail.com
> >wrote:
>
> > Guys,
> >
> > attached is a reworked version of this document which includes:
> >
> >
> > - "starting fresh" approach to consensus operations in HBase
> > - benefits it could bring
> > - code pointers and list of modifications required to abstract ZK
> >
> > I'd appreciate review and feedbacks! The document is also attached to
> > HBASE-10909 for reference and comments.
> >
> > Thanks,
> >
> > Mikhail
> >
> >
> > 2014-04-09 16:33 GMT-07:00 Mikhail Antonov <ol...@gmail.com>:
> >
> > Guys,
> >>
> >> as was advised, I moved the PDF document to umbrella jira, here -
> >> https://issues.apache.org/jira/browse/HBASE-10909.
> >>
> >> Thanks,
> >> Mikhail
> >>
> >>
> >> 2014-04-08 8:34 GMT-07:00 Andrew Purtell <ap...@apache.org>:
> >>
> >> Hi Cos,
> >>>
> >>> Thanks for providing this overview, and more importantly, considering
> >>> these
> >>> contributions. Reading through the relevant JIRAs, looks to me like
> HBase
> >>> will be in a better position than before to have this abstraction in
> >>> place,
> >>> provided the changes are fully completed.
> >>>
> >>> Thanks again for the kind consideration.
> >>>
> >>>
> >>> On Mon, Apr 7, 2014 at 4:15 PM, Konstantin Boudnik <co...@apache.org>
> >>> wrote:
> >>>
> >>> > Guys,
> >>> >
> >>> > As some of you might have noticed there is a number of new JIRAs
> opened
> >>> > recently that are aiming on abstracting and separating ZK out of
> HBase
> >>> guts
> >>> > and making it an implementation detail, rather than a center of
> >>> attention
> >>> > for some parts of the HBase.
> >>> >
> >>> > I would like to send around a short document written by Mikhail
> Antonov
> >>> > that
> >>> > is trying to clarify a couple of points about this whole effort.
> >>> > People are asking me about this off-line, so I decided to send this
> >>> around
> >>> > so
> >>> > we can have a lively and wider discussion about this initiative and
> the
> >>> > development.
> >>> >
> >>> > Here's the JIRA and the link to the PDF document.
> >>> > https://issues.apache.org/jira/browse/HBASE-10866
> >>> >
> >>> >
> >>>
> https://issues.apache.org/jira/secure/attachment/12637957/HBaseConsensus.pdf
> >>> >
> >>> > The umbrella JIRA for this effort is here
> >>> > https://issues.apache.org/jira/browse/HBASE-10909
> >>> >
> >>> > --
> >>> > Regardsk
> >>> > Cos
> >>> >
> >>> >
> >>>
> >>>
> >>> --
> >>> Best regards,
> >>>
> >>> - Andy
> >>>
> >>> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >>> (via Tom White)
> >>>
> >>
> >>
> >>
> >> --
> >> Thanks,
> >> Michael Antonov
> >>
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
> >
>
Re: Bringing consensus based strong consistency into HBase
Posted by Nick Dimiduk <nd...@gmail.com>.
For future reference, attachments are filtered out by the list serve. For
working drafts, we've had success using a shared editing service (Google
Docs, specifically). Best to attach later-stage docs to JIRAs.
On Thu, Apr 17, 2014 at 9:29 PM, Mikhail Antonov <ol...@gmail.com>wrote:
> Guys,
>
> attached is a reworked version of this document which includes:
>
>
> - "starting fresh" approach to consensus operations in HBase
> - benefits it could bring
> - code pointers and list of modifications required to abstract ZK
>
> I'd appreciate review and feedbacks! The document is also attached to
> HBASE-10909 for reference and comments.
>
> Thanks,
>
> Mikhail
>
>
> 2014-04-09 16:33 GMT-07:00 Mikhail Antonov <ol...@gmail.com>:
>
> Guys,
>>
>> as was advised, I moved the PDF document to umbrella jira, here -
>> https://issues.apache.org/jira/browse/HBASE-10909.
>>
>> Thanks,
>> Mikhail
>>
>>
>> 2014-04-08 8:34 GMT-07:00 Andrew Purtell <ap...@apache.org>:
>>
>> Hi Cos,
>>>
>>> Thanks for providing this overview, and more importantly, considering
>>> these
>>> contributions. Reading through the relevant JIRAs, looks to me like HBase
>>> will be in a better position than before to have this abstraction in
>>> place,
>>> provided the changes are fully completed.
>>>
>>> Thanks again for the kind consideration.
>>>
>>>
>>> On Mon, Apr 7, 2014 at 4:15 PM, Konstantin Boudnik <co...@apache.org>
>>> wrote:
>>>
>>> > Guys,
>>> >
>>> > As some of you might have noticed there is a number of new JIRAs opened
>>> > recently that are aiming on abstracting and separating ZK out of HBase
>>> guts
>>> > and making it an implementation detail, rather than a center of
>>> attention
>>> > for some parts of the HBase.
>>> >
>>> > I would like to send around a short document written by Mikhail Antonov
>>> > that
>>> > is trying to clarify a couple of points about this whole effort.
>>> > People are asking me about this off-line, so I decided to send this
>>> around
>>> > so
>>> > we can have a lively and wider discussion about this initiative and the
>>> > development.
>>> >
>>> > Here's the JIRA and the link to the PDF document.
>>> > https://issues.apache.org/jira/browse/HBASE-10866
>>> >
>>> >
>>> https://issues.apache.org/jira/secure/attachment/12637957/HBaseConsensus.pdf
>>> >
>>> > The umbrella JIRA for this effort is here
>>> > https://issues.apache.org/jira/browse/HBASE-10909
>>> >
>>> > --
>>> > Regardsk
>>> > Cos
>>> >
>>> >
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> - Andy
>>>
>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>> (via Tom White)
>>>
>>
>>
>>
>> --
>> Thanks,
>> Michael Antonov
>>
>
>
>
> --
> Thanks,
> Michael Antonov
>
Re: Bringing consensus based strong consistency into HBase
Posted by Mikhail Antonov <ol...@gmail.com>.
Guys,
attached is a reworked version of this document which includes:
- "starting fresh" approach to consensus operations in HBase
- benefits it could bring
- code pointers and list of modifications required to abstract ZK
I'd appreciate review and feedbacks! The document is also attached to
HBASE-10909 for reference and comments.
Thanks,
Mikhail
2014-04-09 16:33 GMT-07:00 Mikhail Antonov <ol...@gmail.com>:
> Guys,
>
> as was advised, I moved the PDF document to umbrella jira, here -
> https://issues.apache.org/jira/browse/HBASE-10909.
>
> Thanks,
> Mikhail
>
>
> 2014-04-08 8:34 GMT-07:00 Andrew Purtell <ap...@apache.org>:
>
> Hi Cos,
>>
>> Thanks for providing this overview, and more importantly, considering
>> these
>> contributions. Reading through the relevant JIRAs, looks to me like HBase
>> will be in a better position than before to have this abstraction in
>> place,
>> provided the changes are fully completed.
>>
>> Thanks again for the kind consideration.
>>
>>
>> On Mon, Apr 7, 2014 at 4:15 PM, Konstantin Boudnik <co...@apache.org>
>> wrote:
>>
>> > Guys,
>> >
>> > As some of you might have noticed there is a number of new JIRAs opened
>> > recently that are aiming on abstracting and separating ZK out of HBase
>> guts
>> > and making it an implementation detail, rather than a center of
>> attention
>> > for some parts of the HBase.
>> >
>> > I would like to send around a short document written by Mikhail Antonov
>> > that
>> > is trying to clarify a couple of points about this whole effort.
>> > People are asking me about this off-line, so I decided to send this
>> around
>> > so
>> > we can have a lively and wider discussion about this initiative and the
>> > development.
>> >
>> > Here's the JIRA and the link to the PDF document.
>> > https://issues.apache.org/jira/browse/HBASE-10866
>> >
>> >
>> https://issues.apache.org/jira/secure/attachment/12637957/HBaseConsensus.pdf
>> >
>> > The umbrella JIRA for this effort is here
>> > https://issues.apache.org/jira/browse/HBASE-10909
>> >
>> > --
>> > Regardsk
>> > Cos
>> >
>> >
>>
>>
>> --
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>
>
>
> --
> Thanks,
> Michael Antonov
>
--
Thanks,
Michael Antonov
Re: Bringing consensus based strong consistency into HBase
Posted by Mikhail Antonov <ol...@gmail.com>.
Guys,
as was advised, I moved the PDF document to umbrella jira, here -
https://issues.apache.org/jira/browse/HBASE-10909.
Thanks,
Mikhail
2014-04-08 8:34 GMT-07:00 Andrew Purtell <ap...@apache.org>:
> Hi Cos,
>
> Thanks for providing this overview, and more importantly, considering these
> contributions. Reading through the relevant JIRAs, looks to me like HBase
> will be in a better position than before to have this abstraction in place,
> provided the changes are fully completed.
>
> Thanks again for the kind consideration.
>
>
> On Mon, Apr 7, 2014 at 4:15 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> > Guys,
> >
> > As some of you might have noticed there is a number of new JIRAs opened
> > recently that are aiming on abstracting and separating ZK out of HBase
> guts
> > and making it an implementation detail, rather than a center of attention
> > for some parts of the HBase.
> >
> > I would like to send around a short document written by Mikhail Antonov
> > that
> > is trying to clarify a couple of points about this whole effort.
> > People are asking me about this off-line, so I decided to send this
> around
> > so
> > we can have a lively and wider discussion about this initiative and the
> > development.
> >
> > Here's the JIRA and the link to the PDF document.
> > https://issues.apache.org/jira/browse/HBASE-10866
> >
> >
> https://issues.apache.org/jira/secure/attachment/12637957/HBaseConsensus.pdf
> >
> > The umbrella JIRA for this effort is here
> > https://issues.apache.org/jira/browse/HBASE-10909
> >
> > --
> > Regardsk
> > Cos
> >
> >
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
--
Thanks,
Michael Antonov
Re: Bringing consensus based strong consistency into HBase
Posted by Andrew Purtell <ap...@apache.org>.
Hi Cos,
Thanks for providing this overview, and more importantly, considering these
contributions. Reading through the relevant JIRAs, looks to me like HBase
will be in a better position than before to have this abstraction in place,
provided the changes are fully completed.
Thanks again for the kind consideration.
On Mon, Apr 7, 2014 at 4:15 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Guys,
>
> As some of you might have noticed there is a number of new JIRAs opened
> recently that are aiming on abstracting and separating ZK out of HBase guts
> and making it an implementation detail, rather than a center of attention
> for some parts of the HBase.
>
> I would like to send around a short document written by Mikhail Antonov
> that
> is trying to clarify a couple of points about this whole effort.
> People are asking me about this off-line, so I decided to send this around
> so
> we can have a lively and wider discussion about this initiative and the
> development.
>
> Here's the JIRA and the link to the PDF document.
> https://issues.apache.org/jira/browse/HBASE-10866
>
> https://issues.apache.org/jira/secure/attachment/12637957/HBaseConsensus.pdf
>
> The umbrella JIRA for this effort is here
> https://issues.apache.org/jira/browse/HBASE-10909
>
> --
> Regardsk
> Cos
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)