[jira] [Updated] (IGNITE-21588) CMG commands idempotency is broken

["Kirill Tkalenko (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kirill Tkalenko updated IGNITE-21588:
-------------------------------------
    Fix Version/s: 3.0.0-beta2

> CMG commands idempotency is broken
> ----------------------------------
>
>                 Key: IGNITE-21588
>                 URL: https://issues.apache.org/jira/browse/IGNITE-21588
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Ivan Bessonov
>            Assignee: Kirill Tkalenko
>            Priority: Major
>              Labels: ignite-3
>             Fix For: 3.0.0-beta2
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> When handling commands like {{JoinReadyCommand}} and {{NodesLeaveCommand}} we do the following:
>  * Read local state with {{{}readLogicalTopology(){}}}.
>  * Modify state according to the command.
>  * {*}Increase version{*}.
>  * Write new state with {{{}saveSnapshotToStorage(snapshot){}}}.
> The problem lies in reading and writing of the state - it' local, and version value is not replicated.
> What happens when we restart the node:
>  * It starts without local storage snapshot, with appliedIndex == 0, which is a {*}state in the past{*}.
>  * We apply commands that were already applied before restart.
>  * We apply these commands to locally saved topology snapshot.
>  * This logical topology snapshot has a *state in the future* when compared to appliedIndex == 0.
>  * As a result, when we re-apply some commands, we *increase the version* one more time, thus breaking data consistency between nodes.
> This would have been fine if we only used this version locally. But distribution zones rely on the consistency of the version between all nodes in cluster. This might break DZ data nodes handling if any of the cluster nodes restarts.
> How to fix:
>  * Either drop the storage if there's no storage snapshot, this will restore consistency
>  * or never start CMG group from a snapshot, but rather start it from the latest storage data.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)