[GitHub] [geode] boglesby commented on pull request #7422: GEODE-10105: when stopping gw sender, for pending events set possible…

[GitBox <gi...@apache.org>]

boglesby commented on pull request #7422:
URL: https://github.com/apache/geode/pull/7422#issuecomment-1071901510


   I think this is a good change, but I'm still reviewing it. So far, I've been looking at it in the context of killing the remote receiver (not stopping the local sender).
   
   One thing I see that is definitely positive is that it is setting possibleDuplicate on events that were not being set before. And the new ones being set are almost certainly going to be duplicates on the remote server.
   
   Here is an example with some logging.
   
   Several batches are sent and acked successfully (0-6):
   ```
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue about to dispatch batchId=0
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue done dispatch batchId=0; success=true
   AckReaderThread for : Event Processor for GatewaySender_ny_1: AckReaderThread.run acked successful batchId=0
   
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue about to dispatch batchId=1
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue done dispatch batchId=1; success=true
   AckReaderThread for : Event Processor for GatewaySender_ny_1: AckReaderThread.run acked successful batchId=1
   
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue about to dispatch batchId=2
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue done dispatch batchId=2; success=true
   AckReaderThread for : Event Processor for GatewaySender_ny_1: AckReaderThread.run acked successful batchId=2
   
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue about to dispatch batchId=3
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue done dispatch batchId=3; success=true
   AckReaderThread for : Event Processor for GatewaySender_ny_1: AckReaderThread.run acked successful batchId=3
   
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue about to dispatch batchId=4
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue done dispatch batchId=4; success=true
   AckReaderThread for : Event Processor for GatewaySender_ny_1: AckReaderThread.run acked successful batchId=4
   
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue about to dispatch batchId=5
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue done dispatch batchId=5; success=true
   AckReaderThread for : Event Processor for GatewaySender_ny_1: AckReaderThread.run acked successful batchId=5
   
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue about to dispatch batchId=6
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue done dispatch batchId=6; success=true
   AckReaderThread for : Event Processor for GatewaySender_ny_1: AckReaderThread.run acked successful batchId=6
   ```
   Batch 7 is sent successfully, but the AckReaderThread receives a null ack because the receiver is gone:
   ```
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue about to dispatch batchId=7
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue done dispatch batchId=7; success=true
   AckReaderThread for : Event Processor for GatewaySender_ny_1: AckReaderThread.run null ack
   AckReaderThread for : Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.handleException resetLastPeekedEvents=true
   ```
   These events are not set to possibleDuplicates, but they almost certainly are.
   
   Also, handleException sets resetLastPeekedEvents to true (which is what causes pendingEventsInBatchesMarkAsPossibleDuplicate to be called below).
   
   Batch 8 is unsuccessfully sent:
   ```
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue about to dispatch batchId=8
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.resetBatchId about to reset batchId=8
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.processQueue done dispatch batchId=8; success=false
   ```
   This causes handleUnSuccessfulBatchDispatch to be called which sets possibleDuplicate to true on all the events in the batch:
   ```
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.handleUnSuccessfulBatchDispatch about to setPossibleDuplicate eventsSize=100
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5304; possibleDuplicate=true
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5417; possibleDuplicate=true
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5530; possibleDuplicate=true
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5086; possibleDuplicate=true
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5332; possibleDuplicate=true
   ...
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=8802; possibleDuplicate=true
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=6434; possibleDuplicate=true
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=7087; possibleDuplicate=true
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=7775; possibleDuplicate=true
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=8915; possibleDuplicate=true
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.handleUnSuccessfulBatchDispatch done setPossibleDuplicate eventsSize=100
   ```
   The processQueue loop iteration is done and since resetLastPeekedEvents=true, pendingEventsInBatchesMarkAsPossibleDuplicate is called. The batchIdToEventsMap contains 2 entries, namely batches 7 and 8.
   
   This method sets possibleDuplicate for the events in both of these batches. The events in batch 7 were the ones that didn't receive an ack and have not been set possibleDuplicate previously. The events in batch 8 have already been set, so the setPossibleDuplicate call is short-circuited. I'm not sure if its not just easier to call setPossibleDuplicate every time.
   
   In any event, at the end of this call, all the events in batches 7 and 8 are possibleDuplicate.
   ```
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.pendingEventsInBatchesMarkAsPossibleDuplicate batchIdToEventsMapSize=2; batchIdToEventsMapKeys=[7, 8]
   
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.pendingEventsInBatchesMarkAsPossibleDuplicate about to setPossibleDuplicate batchId=7; eventsSize=100
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5573; possibleDuplicate=true; this.possibleDuplicate=false
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=8305; possibleDuplicate=true; this.possibleDuplicate=false
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5053; possibleDuplicate=true; this.possibleDuplicate=false
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5638; possibleDuplicate=true; this.possibleDuplicate=false
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=3835; possibleDuplicate=true; this.possibleDuplicate=false
   
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=6758; possibleDuplicate=true; this.possibleDuplicate=false
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5407; possibleDuplicate=true; this.possibleDuplicate=false
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5191; possibleDuplicate=true; this.possibleDuplicate=false
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=5219; possibleDuplicate=true; this.possibleDuplicate=false
   Event Processor for GatewaySender_ny_1: GatewaySenderEventImpl.setPossibleDuplicate queueKey=9661; possibleDuplicate=true; this.possibleDuplicate=false
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.pendingEventsInBatchesMarkAsPossibleDuplicate done setPossibleDuplicate batchId=7; eventsSize=100
   
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.pendingEventsInBatchesMarkAsPossibleDuplicate about to setPossibleDuplicate batchId=8; eventsSize=100
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.pendingEventsInBatchesMarkAsPossibleDuplicate done setPossibleDuplicate batchId=8; eventsSize=100
   
   Event Processor for GatewaySender_ny_1: AbstractGatewaySenderEventProcessor.pendingEventsInBatchesMarkAsPossibleDuplicate done batchIdToEventsMapSize=2; batchIdToEventsMapKeys=[7, 8]
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org