[jira] [Updated] (IGNITE-18060) Prepare design and tickets breakdown for rebalance over replicas

["Kirill Gusakov (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-18060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kirill Gusakov updated IGNITE-18060:
------------------------------------
    Attachment: rebalance_redesign.md

> Prepare design and tickets breakdown for rebalance over replicas
> ----------------------------------------------------------------
>
>                 Key: IGNITE-18060
>                 URL: https://issues.apache.org/jira/browse/IGNITE-18060
>             Project: Ignite
>          Issue Type: New Feature
>            Reporter: Alexander Lapin
>            Assignee: Kirill Gusakov
>            Priority: Major
>              Labels: ignite-3
>         Attachments: rebalance_redesign.md
>
>
> h3. Motivation
> Current rebalance design specified in 
> {code:java}
> ignite-3/modules/table/tech-notes/rebalance.md {code}
> is a bit complicated. The good news is that it can be simplified using the following mechanics:
>  * Meta storage over learners, meaning that every cluster node will have meta storage locally.
>  * Rebalance over replicas, meaning that there will be useful abstraction of (at most) single actor in replication group, better known as primary replica.
>  * Meta storage over version values, meaning that it'll be possible to eliminate the requirement of single-threaded meta storage watch, that, among other things, will also eliminate watch-redeployment procedure.
> All in all, given ticket is only about over replicas part. So, it's required to prototype and design new rebalance algorithm on top of at most single actor invariant. Among the key potential difficulties, I would highlight the following set:
>  * Stale triggers. Primary replica that manages rebalance could still be notified about stale updates, that will required ms invokes over trigger revision. 
>  * It's required to take into consideration in-memory replicas switch logic.
> h3. Definition of Done
>  * Either new design along with tickets breakdown for rebalance over replicas (at most single actor) is ready or list of substantial arguments that the use of replicas does not simplify the rebalancing algorithm is provided.
>  * Proposed replica based logic should be simple, it's not worth to waste time for micro enhancements. In that case we should consider system table as an alternative.
>  * Thus high level design of rebalance over system table is also expected. All in all we should compare whether it's easier to have (ms based, replica based) approach or implement system tables and build rebalance logic on top of it.  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)