You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Jaromir Hamala <ja...@gmail.com> on 2013/11/08 14:54:34 UTC
Simpler Replicated LevelDB store?
Hi guys,
I was looking at the implementation of replicated LevelDB store. It
currently requires Apache ZooKeeper. I was thinking how to simplify it and
it seems me it should be possible to implement both electing master and
member discovery mechanisms by Hazelcast.
What do you think about it? Hazelcast implements distributed locking, it's
very easy to use and has no external dependencies. I'm hoping to simplify
deployment and easy to use of ActiveMQ HA capabilities.
Cheers,
Jaromir
--
“Perfection is achieved, not when there is nothing more to add, but when
there is nothing left to take away.”
Antoine de Saint Exupéry
Re: Simpler Replicated LevelDB store?
Posted by Hiram Chirino <hi...@hiramchirino.com>.
Split brain is the hardest thing to deal with when working with
distributed master elections. For example: you also need to control
how fast you can detect the network split scenario and make sure that
we don't elect a new master faster than the old master can be shutdown
when it's determined he's in the e minority part of the split.
On Fri, Nov 8, 2013 at 9:47 AM, Jaromir Hamala <ja...@gmail.com> wrote:
> Cool! I was looking at it originally myself, but I'm not sure if I
> understand all consequences. Is there anything else to be aware of besides
> the split-brain scenario?
>
> Cheers,
> Jaromir
>
>
> On Fri, Nov 8, 2013 at 2:35 PM, Hiram Chirino <hi...@hiramchirino.com>wrote:
>
>> Ok sounds like a plan. :)
>>
>> On Fri, Nov 8, 2013 at 9:20 AM, Jaromir Hamala <ja...@gmail.com>
>> wrote:
>> > There is nothing really fancy:
>> >
>> http://www.hazelcast.com/docs/3.0/manual/multi_html/ch12s02.html#NetworkPartitioning
>> > I believe there is no way how to specify a minimum quorum in order to
>> > obtain a distributed lock. This could lead to a situation where 2 servers
>> > will act as masters during the split-brain scenario, however I believe it
>> > should be possible to implement this in ActiveMQ code - something like
>> >
>> > if (currentClusterSize < minimumSize) { dontEvenTryBeObtainAMasterLock..
>> }
>> >
>> >
>> > On Fri, Nov 8, 2013 at 2:11 PM, Hiram Chirino <hiram@hiramchirino.com
>> >wrote:
>> >
>> >> Would love to have multiple options. My biggest question is how does
>> >> Hazelcast deal with network splits?
>> >>
>> >> On Fri, Nov 8, 2013 at 8:54 AM, Jaromir Hamala <
>> jaromir.hamala@gmail.com>
>> >> wrote:
>> >> > Hi guys,
>> >> >
>> >> > I was looking at the implementation of replicated LevelDB store. It
>> >> > currently requires Apache ZooKeeper. I was thinking how to simplify it
>> >> and
>> >> > it seems me it should be possible to implement both electing master
>> and
>> >> > member discovery mechanisms by Hazelcast.
>> >> >
>> >> > What do you think about it? Hazelcast implements distributed locking,
>> >> it's
>> >> > very easy to use and has no external dependencies. I'm hoping to
>> simplify
>> >> > deployment and easy to use of ActiveMQ HA capabilities.
>> >> >
>> >> > Cheers,
>> >> > Jaromir
>> >> >
>> >> >
>> >> > --
>> >> > “Perfection is achieved, not when there is nothing more to add, but
>> when
>> >> > there is nothing left to take away.”
>> >> > Antoine de Saint Exupéry
>> >>
>> >>
>> >>
>> >> --
>> >> Hiram Chirino
>> >>
>> >> Engineering | Red Hat, Inc.
>> >>
>> >> hchirino@redhat.com | fusesource.com | redhat.com
>> >>
>> >> skype: hiramchirino | twitter: @hiramchirino
>> >>
>> >> blog: Hiram Chirino's Bit Mojo
>> >>
>> >
>> >
>> >
>> > --
>> > “Perfection is achieved, not when there is nothing more to add, but when
>> > there is nothing left to take away.”
>> > Antoine de Saint Exupéry
>>
>>
>>
>> --
>> Hiram Chirino
>>
>> Engineering | Red Hat, Inc.
>>
>> hchirino@redhat.com | fusesource.com | redhat.com
>>
>> skype: hiramchirino | twitter: @hiramchirino
>>
>> blog: Hiram Chirino's Bit Mojo
>>
>
>
>
> --
> “Perfection is achieved, not when there is nothing more to add, but when
> there is nothing left to take away.”
> Antoine de Saint Exupéry
--
Hiram Chirino
Engineering | Red Hat, Inc.
hchirino@redhat.com | fusesource.com | redhat.com
skype: hiramchirino | twitter: @hiramchirino
blog: Hiram Chirino's Bit Mojo
Re: Simpler Replicated LevelDB store?
Posted by Jaromir Hamala <ja...@gmail.com>.
Cool! I was looking at it originally myself, but I'm not sure if I
understand all consequences. Is there anything else to be aware of besides
the split-brain scenario?
Cheers,
Jaromir
On Fri, Nov 8, 2013 at 2:35 PM, Hiram Chirino <hi...@hiramchirino.com>wrote:
> Ok sounds like a plan. :)
>
> On Fri, Nov 8, 2013 at 9:20 AM, Jaromir Hamala <ja...@gmail.com>
> wrote:
> > There is nothing really fancy:
> >
> http://www.hazelcast.com/docs/3.0/manual/multi_html/ch12s02.html#NetworkPartitioning
> > I believe there is no way how to specify a minimum quorum in order to
> > obtain a distributed lock. This could lead to a situation where 2 servers
> > will act as masters during the split-brain scenario, however I believe it
> > should be possible to implement this in ActiveMQ code - something like
> >
> > if (currentClusterSize < minimumSize) { dontEvenTryBeObtainAMasterLock..
> }
> >
> >
> > On Fri, Nov 8, 2013 at 2:11 PM, Hiram Chirino <hiram@hiramchirino.com
> >wrote:
> >
> >> Would love to have multiple options. My biggest question is how does
> >> Hazelcast deal with network splits?
> >>
> >> On Fri, Nov 8, 2013 at 8:54 AM, Jaromir Hamala <
> jaromir.hamala@gmail.com>
> >> wrote:
> >> > Hi guys,
> >> >
> >> > I was looking at the implementation of replicated LevelDB store. It
> >> > currently requires Apache ZooKeeper. I was thinking how to simplify it
> >> and
> >> > it seems me it should be possible to implement both electing master
> and
> >> > member discovery mechanisms by Hazelcast.
> >> >
> >> > What do you think about it? Hazelcast implements distributed locking,
> >> it's
> >> > very easy to use and has no external dependencies. I'm hoping to
> simplify
> >> > deployment and easy to use of ActiveMQ HA capabilities.
> >> >
> >> > Cheers,
> >> > Jaromir
> >> >
> >> >
> >> > --
> >> > “Perfection is achieved, not when there is nothing more to add, but
> when
> >> > there is nothing left to take away.”
> >> > Antoine de Saint Exupéry
> >>
> >>
> >>
> >> --
> >> Hiram Chirino
> >>
> >> Engineering | Red Hat, Inc.
> >>
> >> hchirino@redhat.com | fusesource.com | redhat.com
> >>
> >> skype: hiramchirino | twitter: @hiramchirino
> >>
> >> blog: Hiram Chirino's Bit Mojo
> >>
> >
> >
> >
> > --
> > “Perfection is achieved, not when there is nothing more to add, but when
> > there is nothing left to take away.”
> > Antoine de Saint Exupéry
>
>
>
> --
> Hiram Chirino
>
> Engineering | Red Hat, Inc.
>
> hchirino@redhat.com | fusesource.com | redhat.com
>
> skype: hiramchirino | twitter: @hiramchirino
>
> blog: Hiram Chirino's Bit Mojo
>
--
“Perfection is achieved, not when there is nothing more to add, but when
there is nothing left to take away.”
Antoine de Saint Exupéry
Re: Simpler Replicated LevelDB store?
Posted by Hiram Chirino <hi...@hiramchirino.com>.
Ok sounds like a plan. :)
On Fri, Nov 8, 2013 at 9:20 AM, Jaromir Hamala <ja...@gmail.com> wrote:
> There is nothing really fancy:
> http://www.hazelcast.com/docs/3.0/manual/multi_html/ch12s02.html#NetworkPartitioning
> I believe there is no way how to specify a minimum quorum in order to
> obtain a distributed lock. This could lead to a situation where 2 servers
> will act as masters during the split-brain scenario, however I believe it
> should be possible to implement this in ActiveMQ code - something like
>
> if (currentClusterSize < minimumSize) { dontEvenTryBeObtainAMasterLock.. }
>
>
> On Fri, Nov 8, 2013 at 2:11 PM, Hiram Chirino <hi...@hiramchirino.com>wrote:
>
>> Would love to have multiple options. My biggest question is how does
>> Hazelcast deal with network splits?
>>
>> On Fri, Nov 8, 2013 at 8:54 AM, Jaromir Hamala <ja...@gmail.com>
>> wrote:
>> > Hi guys,
>> >
>> > I was looking at the implementation of replicated LevelDB store. It
>> > currently requires Apache ZooKeeper. I was thinking how to simplify it
>> and
>> > it seems me it should be possible to implement both electing master and
>> > member discovery mechanisms by Hazelcast.
>> >
>> > What do you think about it? Hazelcast implements distributed locking,
>> it's
>> > very easy to use and has no external dependencies. I'm hoping to simplify
>> > deployment and easy to use of ActiveMQ HA capabilities.
>> >
>> > Cheers,
>> > Jaromir
>> >
>> >
>> > --
>> > “Perfection is achieved, not when there is nothing more to add, but when
>> > there is nothing left to take away.”
>> > Antoine de Saint Exupéry
>>
>>
>>
>> --
>> Hiram Chirino
>>
>> Engineering | Red Hat, Inc.
>>
>> hchirino@redhat.com | fusesource.com | redhat.com
>>
>> skype: hiramchirino | twitter: @hiramchirino
>>
>> blog: Hiram Chirino's Bit Mojo
>>
>
>
>
> --
> “Perfection is achieved, not when there is nothing more to add, but when
> there is nothing left to take away.”
> Antoine de Saint Exupéry
--
Hiram Chirino
Engineering | Red Hat, Inc.
hchirino@redhat.com | fusesource.com | redhat.com
skype: hiramchirino | twitter: @hiramchirino
blog: Hiram Chirino's Bit Mojo
Re: Simpler Replicated LevelDB store?
Posted by Jaromir Hamala <ja...@gmail.com>.
There is nothing really fancy:
http://www.hazelcast.com/docs/3.0/manual/multi_html/ch12s02.html#NetworkPartitioning
I believe there is no way how to specify a minimum quorum in order to
obtain a distributed lock. This could lead to a situation where 2 servers
will act as masters during the split-brain scenario, however I believe it
should be possible to implement this in ActiveMQ code - something like
if (currentClusterSize < minimumSize) { dontEvenTryBeObtainAMasterLock.. }
On Fri, Nov 8, 2013 at 2:11 PM, Hiram Chirino <hi...@hiramchirino.com>wrote:
> Would love to have multiple options. My biggest question is how does
> Hazelcast deal with network splits?
>
> On Fri, Nov 8, 2013 at 8:54 AM, Jaromir Hamala <ja...@gmail.com>
> wrote:
> > Hi guys,
> >
> > I was looking at the implementation of replicated LevelDB store. It
> > currently requires Apache ZooKeeper. I was thinking how to simplify it
> and
> > it seems me it should be possible to implement both electing master and
> > member discovery mechanisms by Hazelcast.
> >
> > What do you think about it? Hazelcast implements distributed locking,
> it's
> > very easy to use and has no external dependencies. I'm hoping to simplify
> > deployment and easy to use of ActiveMQ HA capabilities.
> >
> > Cheers,
> > Jaromir
> >
> >
> > --
> > “Perfection is achieved, not when there is nothing more to add, but when
> > there is nothing left to take away.”
> > Antoine de Saint Exupéry
>
>
>
> --
> Hiram Chirino
>
> Engineering | Red Hat, Inc.
>
> hchirino@redhat.com | fusesource.com | redhat.com
>
> skype: hiramchirino | twitter: @hiramchirino
>
> blog: Hiram Chirino's Bit Mojo
>
--
“Perfection is achieved, not when there is nothing more to add, but when
there is nothing left to take away.”
Antoine de Saint Exupéry
Re: Simpler Replicated LevelDB store?
Posted by Hiram Chirino <hi...@hiramchirino.com>.
Would love to have multiple options. My biggest question is how does
Hazelcast deal with network splits?
On Fri, Nov 8, 2013 at 8:54 AM, Jaromir Hamala <ja...@gmail.com> wrote:
> Hi guys,
>
> I was looking at the implementation of replicated LevelDB store. It
> currently requires Apache ZooKeeper. I was thinking how to simplify it and
> it seems me it should be possible to implement both electing master and
> member discovery mechanisms by Hazelcast.
>
> What do you think about it? Hazelcast implements distributed locking, it's
> very easy to use and has no external dependencies. I'm hoping to simplify
> deployment and easy to use of ActiveMQ HA capabilities.
>
> Cheers,
> Jaromir
>
>
> --
> “Perfection is achieved, not when there is nothing more to add, but when
> there is nothing left to take away.”
> Antoine de Saint Exupéry
--
Hiram Chirino
Engineering | Red Hat, Inc.
hchirino@redhat.com | fusesource.com | redhat.com
skype: hiramchirino | twitter: @hiramchirino
blog: Hiram Chirino's Bit Mojo