AsyncEventListener - How to detect duplicate events

[Nilkanth Patel <ni...@gmail.com>]

Hello,

*i am writing a **AsyncEventListener implementation as following where
i am getting duplicate events (same event multiple times). How can i
detect whether a particular event is duplicate or not?*


public boolean processEvents(List<AsyncEvent> events) {  for
(AsyncEvent asyncEvent : events) {
    GatewaySenderEventImpl event = (GatewaySenderEventImpl) (asyncEvent);
    final Operation op = event.getOperation();    if
(!event.getPossibleDuplicate()) {
      if (op == Operation.CREATE) {
        //create event
      } else if (op == Operation.UPDATE) {
        //update event
      } else if (op == Operation.DESTROY) {
        //destroy event
      } else {
        /*other event*/ }
    } else {
    //duplicate event
    }  }}


Thanks in advance.

Nilkanth.

Re: AsyncEventListener - How to detect duplicate events

[Michael Stolz <ms...@pivotal.io>]

To expand on this, the only way to be sure which of the possible duplicates
are actual duplicates would be to look in the backing store you are writing
behind into to see if the event was already written.

Since most of the events in GemFire are atomic in nature, it usually
doesn't matter if you write the same event to the backing store a second
time. However, if you are doing some sort of processing in the
AsyncEventListener you may need to know not to re-process and then you
would have to rely on the backing store to tell you if this is a duplicate
event that has already been processed.

--
Mike Stolz
GemFire Principal Product Manager
Mobile: 631-835-4771

On Wed, Sep 23, 2015 at 1:21 PM, Barry Oglesby <bo...@pivotal.io> wrote:

> You shouldn't be seeing duplicate events in the normal case. Can you share
> your entire configuration?
>
> In the normal case where all members are stable, each event is sent to the
> primary member for that event. From there it is replicated to the secondary
> member(s). Each member delivers the event to its local AsyncEventListener
> which will be primary in the primary member and secondary in the secondary
> member. The only member that should be processing the event in the
> AsyncEventListener is the primary one. This is the normal case.
>
> In the event of primary member failure, the secondary member will take over
> as primary. At this time, the AsyncEventListener might see duplicate events
> which would be ones that were being processed by the primary
> AsyncEventListener right before the primary member failed. These events may
> have been completed in the former primary member, but the notification to
> the former secondary (now primary) member to remove them from its queue
> hadn't been made, so they are still in the secondary member's queue. In
> this case, as soon as the secondary member becomes primary, these events
> will be delivered to its AsyncEventListener as possible duplicates. The
> getPossibleDuplicate method is the way to detect whether these events are
> possible duplicates.
>
>
> Barry Oglesby
> GemFire Advanced Customer Engineering (ACE)
> For immediate support please contact Pivotal Support at
> http://support.pivotal.io/
>
>
> On Wed, Sep 23, 2015 at 1:09 AM, Nilkanth Patel <nilkanth.hpatel@gmail.com
> >
> wrote:
>
> > Hello,
> >
> > *i am writing a **AsyncEventListener implementation as following where
> > i am getting duplicate events (same event multiple times). How can i
> > detect whether a particular event is duplicate or not?*
> >
> >
> > public boolean processEvents(List<AsyncEvent> events) {  for
> > (AsyncEvent asyncEvent : events) {
> >     GatewaySenderEventImpl event = (GatewaySenderEventImpl) (asyncEvent);
> >     final Operation op = event.getOperation();    if
> > (!event.getPossibleDuplicate()) {
> >       if (op == Operation.CREATE) {
> >         //create event
> >       } else if (op == Operation.UPDATE) {
> >         //update event
> >       } else if (op == Operation.DESTROY) {
> >         //destroy event
> >       } else {
> >         /*other event*/ }
> >     } else {
> >     //duplicate event
> >     }  }}
> >
> >
> > Thanks in advance.
> >
> > Nilkanth.
> >
>

Re: AsyncEventListener - How to detect duplicate events

[Barry Oglesby <bo...@pivotal.io>]

You shouldn't be seeing duplicate events in the normal case. Can you share
your entire configuration?

In the normal case where all members are stable, each event is sent to the
primary member for that event. From there it is replicated to the secondary
member(s). Each member delivers the event to its local AsyncEventListener
which will be primary in the primary member and secondary in the secondary
member. The only member that should be processing the event in the
AsyncEventListener is the primary one. This is the normal case.

In the event of primary member failure, the secondary member will take over
as primary. At this time, the AsyncEventListener might see duplicate events
which would be ones that were being processed by the primary
AsyncEventListener right before the primary member failed. These events may
have been completed in the former primary member, but the notification to
the former secondary (now primary) member to remove them from its queue
hadn't been made, so they are still in the secondary member's queue. In
this case, as soon as the secondary member becomes primary, these events
will be delivered to its AsyncEventListener as possible duplicates. The
getPossibleDuplicate method is the way to detect whether these events are
possible duplicates.


Barry Oglesby
GemFire Advanced Customer Engineering (ACE)
For immediate support please contact Pivotal Support at
http://support.pivotal.io/


On Wed, Sep 23, 2015 at 1:09 AM, Nilkanth Patel <ni...@gmail.com>
wrote:

> Hello,
>
> *i am writing a **AsyncEventListener implementation as following where
> i am getting duplicate events (same event multiple times). How can i
> detect whether a particular event is duplicate or not?*
>
>
> public boolean processEvents(List<AsyncEvent> events) {  for
> (AsyncEvent asyncEvent : events) {
>     GatewaySenderEventImpl event = (GatewaySenderEventImpl) (asyncEvent);
>     final Operation op = event.getOperation();    if
> (!event.getPossibleDuplicate()) {
>       if (op == Operation.CREATE) {
>         //create event
>       } else if (op == Operation.UPDATE) {
>         //update event
>       } else if (op == Operation.DESTROY) {
>         //destroy event
>       } else {
>         /*other event*/ }
>     } else {
>     //duplicate event
>     }  }}
>
>
> Thanks in advance.
>
> Nilkanth.
>