[jira] [Commented] (GEODE-5631) failedBatchRemovalMessageKeys never cleared

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/GEODE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593917#comment-16593917 ] 

ASF subversion and git services commented on GEODE-5631:
--------------------------------------------------------

Commit 85953f085b13815405cb9bf3c5b1bca77c3c9a5e in geode's branch refs/heads/develop from [~nabarunnag]
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=85953f0 ]

GEODE-5631: failedBatchRemovalMessageKeys not used after GII (#2375)

        * After GII a flag is set to indicate that failedBatchRemovalMessageKeys has been processed
	* If this flag is set, no more entries will be put into failedBatchRemovalMessageKeys.

> failedBatchRemovalMessageKeys never cleared
> -------------------------------------------
>
>                 Key: GEODE-5631
>                 URL: https://issues.apache.org/jira/browse/GEODE-5631
>             Project: Geode
>          Issue Type: Bug
>          Components: wan
>            Reporter: nabarun
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> *+Experiment setup+*:
> * Region A created with async event listener attached to it
> * For every event processed by the async listener, a new entry is put into another region, Region B.
> * There is a client which does 1 million operations on 1500 keys on Region A. [to trigger conflation.] 
> * 3 servers, 1 locator and 1 client.
> +*Issue:*+
> It was confirmed that after upgrading to 1.6.0 , we saw an increase in the memory footprint after all operations are completed.
> +*Cause:*+
> * We had a data structure to store all the queue removal messages that comes in when the secondary is in process of GII, called failedBatchRemovalMessageKeys.
> * Two removal messages were sent to the secondary for a single event, one from the processor which was processing the event and another from the conflation thread which conflated the event and hence wants the secondary to remove it.
> * Of the two messages whichever comes first, it removes the event from the queue.
> * When the second message comes in, and we try to remove it from the queue, it hits an EntryNotFoundException. This makes the message think that secondary is in GII and hence stores it in the failedBatchRemovalMessageKeys, and expects that when GII is complete, this message will be processed.
> * But GII was already done long before, and this data structure, failedBatchRemovalMessageKeys keeps storing messages which are never removed and hence a large memory footprint.
> +*Fix:*+
> The data structure failedBatchRemovalMessageKeys is not used anymore if it was already processed once, as GII happens once in a server’s lifecycle.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)