You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ratis.apache.org by "Ivan Andika (Jira)" <ji...@apache.org> on 2026/07/10 06:38:00 UTC

[jira] [Updated] (RATIS-2597) Local StateMachine Sharding within a Ratis group

     [ https://issues.apache.org/jira/browse/RATIS-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Andika updated RATIS-2597:
-------------------------------
    Summary: Local StateMachine Sharding within a Ratis group  (was: Local Sharding for StateMachine in a Ratis group)

> Local StateMachine Sharding within a Ratis group
> ------------------------------------------------
>
>                 Key: RATIS-2597
>                 URL: https://issues.apache.org/jira/browse/RATIS-2597
>             Project: Ratis
>          Issue Type: New Feature
>            Reporter: Ivan Andika
>            Assignee: Ivan Andika
>            Priority: Major
>
> Simply an idea. 
> Currently, each Ratis group contains a single Raft log and StateMachine (+ updater). This makes every Ratis transactions to be serialized. However, in the case of key value store there are possible cases where there are some independent shard (e.g. prefix) where operations is guaranteed to be within that shard). In this case, the single StateMachine can cause operations across independent shards to content with each other. In the context of Apache Ozone, currently metadata operations are independent for different buckets, but since Ozone uses a single Ratis group for the namespace, one bucket operations can block other bucket operations.
> We can technically create multiple Ratis groups where each Ratis group is a shard. This is usually done by either creating a static sharding mechanism (e.g. divide the namespace to 256 shards) or using a separate shard manager that will be able to do split, merge a single namespace to multiple shards (e.g. TiKV, HBase regions, CockroachDB, etc). However, usually this causes more complexity (especially the later), since we have to manage N Raft groups, each with their own leader and follower combination. Additionally, in the case of Ozone, we want OM to still share a single RocksDB state so that we can halt the Raft group if there is a InstallSnapshot. This is very difficult if we have multiple independent Raft groups.
> What if we can make a single Raft group be transparently contain multiple Raft logs, each with a single StateMachineUpdater. The Raft group contains only a single leader and followers combination, but underneath there are sharded logs that can be used to improve the paralellism. This way, the Raft group is transparent to client and client does not need to keep track of different Raft groups with different leaders. One requirement is that each request now needs to specify a shard key (e.g. in Ozone case the shard key is the bucket name or ID) so that it can be routed to the correct StateMachine shard. In the future, we can have some kind of 2PC protocol if we want to do cross-shard transactions.
> Drawbacks
>  * The leader for sharded Ratis is shared among the shards which can be the new bottleneck
>  * Raft implementation can become more complicated since we need to keep track of the states of each shard.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)