[jira] [Commented] (IGNITE-426) Make sure continuous queries notifications are not missed in case primary node fails

["Valentin Kulichenko (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681118#comment-14681118 ] 

Valentin Kulichenko commented on IGNITE-426:
--------------------------------------------

Made some progress on the ticket. The following is done for the moment:
* Implement backup queue flush in case of topology changing. This should be done from {{GridDhtPartitionsExchangeFuture.init()}} method. Need to make sure that this happens anytime when primary node changes (if it leaves topology OR another node joins becomes primary). Otherwise updates can be lost.
* Add a per-listener map on client that will track latest update counter value for each partition. Use it to discard duplicates.
* If partition is evicted, we need to remove all entries for this partition from the backup queue, otherwise there is a chance of memory leak. Should be done from {{GridDhtLocalPartition.tryEvict()}} method.
* Add good tests for continuous queries with node restarts.

> Make sure continuous queries notifications are not missed in case primary node fails
> ------------------------------------------------------------------------------------
>
>                 Key: IGNITE-426
>                 URL: https://issues.apache.org/jira/browse/IGNITE-426
>             Project: Ignite
>          Issue Type: Sub-task
>          Components: cache
>            Reporter: Yakov Zhdanov
>            Assignee: Valentin Kulichenko
>            Priority: Critical
>
> * Maintain updates queue on backup node(s) in addition to primary node.
> * If primary node crushes, this queue is flushed to listening clients.
> * To avoid notification duplicates we will have a per-partition update counter. Once an entry in some partition is updated, counter for this partition is incremented on both primary and backups. The value of this counter is also sent along with the update to the client, which also maintains the copy of this mapping. If at some moment it receives an update with the counter less than in its local map, this update is a duplicate and can be discarded.
> * To cleanup the backup queue we will use communication acks. When batch receival is acked by the client, we will send special ack message to backup nodes that will remove items that are not longer needed. This message has to contain partition to latest update counter map.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)