[jira] [Created] (GEODE-2675) AsyncEventListener may see a retried event come out of order

["Dan Smith (JIRA)" <ji...@apache.org>]

Dan Smith created GEODE-2675:
--------------------------------

             Summary: AsyncEventListener may see a retried event come out of order
                 Key: GEODE-2675
                 URL: https://issues.apache.org/jira/browse/GEODE-2675
             Project: Geode
          Issue Type: Bug
          Components: wan
            Reporter: Dan Smith


We discovered this as part of the geode lucene implementation - see GEODE-2674.

The basic issue is that if you are trying to write an AysncEventListener that writes to another data store, and you want that data store to be consistent with what is stored in gemfire, you may see inconsistencies because replayed events may arrive out of order. Here's the problem, from GEODE-2674

# An accessor starts doing a put
# The put goes to the current primary
# Primary distributes the put to the secondary
# Primary closes it's cache
# New primary does a destroy on the same entry
# The accessor retries the put because the old primary closed the cache
# The retried put is added to the queue, after the destroy. But it is not added to the region, because the region detects that it is a retry.
# The AsyncEventListener sees the put as the last event.

There are a couple of workarounds
# Your AsyncEventListener can fetch the correct value from the region, rather than rely on the value in the AsyncEvent. This is what we did for [lucene|https://github.com/apache/geode/blob/1602a2683408ef608554d4c22a8e7c8c61f3e946/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventListener.java#L92]
# You can use AsyncEvent.getEventSequenceID and ignore duplicate events. This requires keeping track of the previously seen sequence ids.

The real fix is to make sure the AsyncEventQueue ignores retried events. That probably means the queue needs to keep track of previously seen sequence ids and not enqueue events if we can see that they have previously been enqueued.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)