You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by Bruce Schuchardt <bs...@pivotal.io> on 2016/07/25 22:17:56 UTC
Re: Geode as Blocking Queue
Geode has its own mechanisms for server auto-discovery. Geode
applications contact a Locator service which assigns them to a server.
Once connected to the server the application receives metadata telling
it about all of the cache servers so it can directly go to the correct
server for an operation. Metadata is refreshed if there are server-side
changes. The Locator also includes load-balancing so it can make
decisions about what servers can take on additional load for
subscription processing (interest registration and continuous-queries).
Le 7/25/2016 1:43 PM, Chaudhary, Sushil (CONT) a crit :
> Any support for Geode AWS auto discovery in multi clustered
> environments? Any key/id configuration which help to discover all
> cluster node and auto connect?
Re: Geode as Blocking Queue
Posted by Gregory Chase <gc...@pivotal.io>.
Hi Sushil,
Thursday's Geode Clubhouse will be discussing event queue's and continuous
query. If you'd like, please feel free to bring some questions to the live
discussion.
From the email announcement:
> Dear Apache Geode (incubating) community,
> Don't miss the next Geode Clubhouse community meeting, next Thursday, July
> 28 at 9AM Pacific.
> Join the meeting at this link
> <https://pivotalcommunity.adobeconnect.com/clubhouse/>. You can add a
> reminder to your calendar with this link
> <
> https://www.google.com/calendar/event?eid=cTlwZGdzOTcyZm9oZ2Q0NTRuZGJia2xnb2cgcGl2b3RhbC5pb191OGtndnVhaGprYm9oMWduZmh2NXRzMnY5Y0Bn&ctz=America/Los_Angeles
> >
> .
> Luke Shannon and Mike Stolz will discuss the event-handling and queuing
> capabilities of Apache Geode, as well as the recently added Continuous
> Query feature.
> We'll also take questions, and show a demo of these capabilities in the
> context of a Geode-powered application.
> See you next Thursday!
-Greg
On Tue, Jul 26, 2016 at 9:40 AM, Dan Smith <ds...@pivotal.io> wrote:
> I think an Async Event queue is what you want to use. Geode doesn't have
> an implementation of BlockingQueue, but if you are concerned about latency
> I think a push model like the aync event queues should be just as good as a
> polling model like a BlockingQueue. Async event queues do provide
> redundancy and balancing data across the cluster.
>
> -Dan
>
> On Tue, Jul 26, 2016 at 8:41 AM, Chaudhary, Sushil (CONT) <
> Sushil.Chaudhary@capitalone.com> wrote:
>
>> Hi Anthony,
>> In our case, we want to avoid even a milliseconds of latency ( that could
>> happen because of Async way of Messaging communication). We need to use
>> IMDB as Queue, not necessary a blocking Queue. Unbounded/bounded Queue
>> should be have persistence in the same way, Geode does for regions/map.
>>
>> - In case of node failure, it should be backed up by redundant copy on
>> other node. ( same as region)
>> - Application process will poll object from Queue, instead of being pushed
>> by Async Event. The only reason is to avoid latency. Will Async call not
>> add any latency in the complete operation.
>> - it should balance the cluster in the same way, as geode does for
>> regions.
>>
>>
>> Hope this give clarity on the use case. Let me know your thoughts.
>>
>> Sushil Chaudhary
>> Email: sushil.chaudhary@capitalone.com
>>
>>
>>
>>
>>
>>
>> On 7/26/16, 11:13 AM, "Anthony Baker" <ab...@pivotal.io> wrote:
>>
>> >Geode has been used in a lot of applications that involve event delivery.
>> > What is your use case?
>> >
>> >j.u.c.BlockingQueue can be useful but in the context of a distributed
>> >system there are important semantics to consider:
>> >
>> >- What happens when a node fails? Are the unacknowledged entries
>> >redelivered? What about missed entries if a client disconnects?
>> >- Are there any delivery guarantees provided such as ³At least once²,
>> >³Once and only once², or ³Best effort but you will lose data under
>> >certain conditions²?
>> >- Can it load balance across a cluster and handle 1000¹s of connected
>> >clients?
>> >
>> >I wonder if the BlockingQueue is the best abstraction to capture all
>> >those ideas.
>> >
>> >Anthony
>> >
>> >> On Jul 25, 2016, at 8:31 PM, Chaudhary, Sushil (CONT)
>> >><Su...@capitalone.com> wrote:
>> >>
>> >> Thanks. Any update on Queuing?
>> >>
>> >> Sushil Chaudhary
>> >> Email: sushil.chaudhary@capitalone.com
>> >>
>> >
>>
>> ________________________________________________________
>>
>> The information contained in this e-mail is confidential and/or
>> proprietary to Capital One and/or its affiliates and may only be used
>> solely in performance of work or services for Capital One. The information
>> transmitted herewith is intended only for use by the individual or entity
>> to which it is addressed. If the reader of this message is not the intended
>> recipient, you are hereby notified that any review, retransmission,
>> dissemination, distribution, copying or other use of, or taking of any
>> action in reliance upon this information is strictly prohibited. If you
>> have received this communication in error, please contact the sender and
>> delete the material from your computer.
>>
>>
>
--
Greg Chase
Global Head, Big Data Communities
http://www.pivotal.io/big-data
Pivotal Software
http://www.pivotal.io/
650-215-0477
@GregChase
Blog: http://geekmarketing.biz/
Re: Geode as Blocking Queue
Posted by Dan Smith <ds...@pivotal.io>.
I think an Async Event queue is what you want to use. Geode doesn't have an
implementation of BlockingQueue, but if you are concerned about latency I
think a push model like the aync event queues should be just as good as a
polling model like a BlockingQueue. Async event queues do provide
redundancy and balancing data across the cluster.
-Dan
On Tue, Jul 26, 2016 at 8:41 AM, Chaudhary, Sushil (CONT) <
Sushil.Chaudhary@capitalone.com> wrote:
> Hi Anthony,
> In our case, we want to avoid even a milliseconds of latency ( that could
> happen because of Async way of Messaging communication). We need to use
> IMDB as Queue, not necessary a blocking Queue. Unbounded/bounded Queue
> should be have persistence in the same way, Geode does for regions/map.
>
> - In case of node failure, it should be backed up by redundant copy on
> other node. ( same as region)
> - Application process will poll object from Queue, instead of being pushed
> by Async Event. The only reason is to avoid latency. Will Async call not
> add any latency in the complete operation.
> - it should balance the cluster in the same way, as geode does for regions.
>
>
> Hope this give clarity on the use case. Let me know your thoughts.
>
> Sushil Chaudhary
> Email: sushil.chaudhary@capitalone.com
>
>
>
>
>
>
> On 7/26/16, 11:13 AM, "Anthony Baker" <ab...@pivotal.io> wrote:
>
> >Geode has been used in a lot of applications that involve event delivery.
> > What is your use case?
> >
> >j.u.c.BlockingQueue can be useful but in the context of a distributed
> >system there are important semantics to consider:
> >
> >- What happens when a node fails? Are the unacknowledged entries
> >redelivered? What about missed entries if a client disconnects?
> >- Are there any delivery guarantees provided such as ³At least once²,
> >³Once and only once², or ³Best effort but you will lose data under
> >certain conditions²?
> >- Can it load balance across a cluster and handle 1000¹s of connected
> >clients?
> >
> >I wonder if the BlockingQueue is the best abstraction to capture all
> >those ideas.
> >
> >Anthony
> >
> >> On Jul 25, 2016, at 8:31 PM, Chaudhary, Sushil (CONT)
> >><Su...@capitalone.com> wrote:
> >>
> >> Thanks. Any update on Queuing?
> >>
> >> Sushil Chaudhary
> >> Email: sushil.chaudhary@capitalone.com
> >>
> >
>
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>
>
Re: Geode as Blocking Queue
Posted by "Chaudhary, Sushil (CONT)" <Su...@capitalone.com>.
Hi Anthony,
In our case, we want to avoid even a milliseconds of latency ( that could
happen because of Async way of Messaging communication). We need to use
IMDB as Queue, not necessary a blocking Queue. Unbounded/bounded Queue
should be have persistence in the same way, Geode does for regions/map.
- In case of node failure, it should be backed up by redundant copy on
other node. ( same as region)
- Application process will poll object from Queue, instead of being pushed
by Async Event. The only reason is to avoid latency. Will Async call not
add any latency in the complete operation.
- it should balance the cluster in the same way, as geode does for regions.
Hope this give clarity on the use case. Let me know your thoughts.
Sushil Chaudhary
Email: sushil.chaudhary@capitalone.com
On 7/26/16, 11:13 AM, "Anthony Baker" <ab...@pivotal.io> wrote:
>Geode has been used in a lot of applications that involve event delivery.
> What is your use case?
>
>j.u.c.BlockingQueue can be useful but in the context of a distributed
>system there are important semantics to consider:
>
>- What happens when a node fails? Are the unacknowledged entries
>redelivered? What about missed entries if a client disconnects?
>- Are there any delivery guarantees provided such as ³At least once²,
>³Once and only once², or ³Best effort but you will lose data under
>certain conditions²?
>- Can it load balance across a cluster and handle 1000¹s of connected
>clients?
>
>I wonder if the BlockingQueue is the best abstraction to capture all
>those ideas.
>
>Anthony
>
>> On Jul 25, 2016, at 8:31 PM, Chaudhary, Sushil (CONT)
>><Su...@capitalone.com> wrote:
>>
>> Thanks. Any update on Queuing?
>>
>> Sushil Chaudhary
>> Email: sushil.chaudhary@capitalone.com
>>
>
________________________________________________________
The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
Re: Geode as Blocking Queue
Posted by Anthony Baker <ab...@pivotal.io>.
Geode has been used in a lot of applications that involve event delivery. What is your use case?
j.u.c.BlockingQueue can be useful but in the context of a distributed system there are important semantics to consider:
- What happens when a node fails? Are the unacknowledged entries redelivered? What about missed entries if a client disconnects?
- Are there any delivery guarantees provided such as “At least once”, “Once and only once”, or “Best effort but you will lose data under certain conditions”?
- Can it load balance across a cluster and handle 1000’s of connected clients?
I wonder if the BlockingQueue is the best abstraction to capture all those ideas.
Anthony
> On Jul 25, 2016, at 8:31 PM, Chaudhary, Sushil (CONT) <Su...@capitalone.com> wrote:
>
> Thanks. Any update on Queuing?
>
> Sushil Chaudhary
> Email: sushil.chaudhary@capitalone.com
>
Re: Geode as Blocking Queue
Posted by "Chaudhary, Sushil (CONT)" <Su...@capitalone.com>.
Thanks. Any update on Queuing?
Sushil Chaudhary
Email: sushil.chaudhary@capitalone.com
On 7/25/16, 6:17 PM, "Bruce Schuchardt" <bs...@pivotal.io> wrote:
>Geode has its own mechanisms for server auto-discovery. Geode
>applications contact a Locator service which assigns them to a server.
>Once connected to the server the application receives metadata telling
>it about all of the cache servers so it can directly go to the correct
>server for an operation. Metadata is refreshed if there are server-side
>changes. The Locator also includes load-balancing so it can make
>decisions about what servers can take on additional load for
>subscription processing (interest registration and continuous-queries).
>
>Le 7/25/2016 à 1:43 PM, Chaudhary, Sushil (CONT) a écrit :
>> Any support for Geode AWS auto discovery in multi clustered
>> environments? Any key/id configuration which help to discover all
>> cluster node and auto connect?
>
>
________________________________________________________
The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.