You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ratis.apache.org by Asad Awadia <as...@gmail.com> on 2022/03/06 05:11:02 UTC
Change group id
Can the group id be changed for a state machine that has data/transactions applied?
Will it create a new blank state machine to replicate? Or it will be able to still work on existing data?
Re: Change group id
Posted by Asad Awadia <as...@gmail.com>.
Yes, I looked into ozone. Couldn't tell how what the operational overhead is.
Thanks as always!
________________________________
From: Tsz Wo Sze <sz...@gmail.com>
Sent: Monday, March 7, 2022 5:51:09 AM
To: user@ratis.apache.org <us...@ratis.apache.org>
Subject: Re: Change group id
Hi Asad,
Your application may consider managing a different ID, says dbID. The dbID normally maps to a Ratis groupID. Whenever a rename of the groupID is needed, we may map that groupID to a different dbID.
BTW, Apache Ozone may suit your use case well. It is a distributed object store, which maintains multiple containers. Each container uses a Ratis group when it is being built. It supports creating multiple containers on the same set of nodes.
Tsz-Wo
On Mon, Mar 7, 2022 at 1:05 AM Asad Awadia <as...@gmail.com>> wrote:
I was thinking about our earlier thread of a single threaded applying and sending keys to task queue for multithread processing etc.
Each thread gets a group id but they write to the same state machine DB and to scale up more threads with more group ids get added. (As part of that group ids may move around in unexpected ways) Then more servers to scale out with their own group ids.
The ingress protion then maps client requests key ranges to group ids and forwards it to the right place?
Bigger picture:
What I am trying to figure out is can I build a datastore that can be "easily" dynamically resized as the volume of data grows.
________________________________
From: Tsz Wo Sze <sz...@gmail.com>>
Sent: Sunday, March 6, 2022 3:28:29 AM
To: user@ratis.apache.org<ma...@ratis.apache.org> <us...@ratis.apache.org>>
Subject: Re: Change group id
Hi Asad,
Currently, we do not support changing group id. What is your use case?
Tsz-Wo
On Sun, Mar 6, 2022 at 1:11 PM Asad Awadia <as...@gmail.com>> wrote:
Can the group id be changed for a state machine that has data/transactions applied?
Will it create a new blank state machine to replicate? Or it will be able to still work on existing data?
Re: Change group id
Posted by Tsz Wo Sze <sz...@gmail.com>.
Hi Asad,
Your application may consider managing a different ID, says dbID. The
dbID normally maps to a Ratis groupID. Whenever a rename of the groupID is
needed, we may map that groupID to a different dbID.
BTW, Apache Ozone may suit your use case well. It is a distributed object
store, which maintains multiple containers. Each container uses a Ratis
group when it is being built. It supports creating multiple containers on
the same set of nodes.
Tsz-Wo
On Mon, Mar 7, 2022 at 1:05 AM Asad Awadia <as...@gmail.com> wrote:
> I was thinking about our earlier thread of a single threaded applying and
> sending keys to task queue for multithread processing etc.
>
> Each thread gets a group id but they write to the same state machine DB
> and to scale up more threads with more group ids get added. (As part of
> that group ids may move around in unexpected ways) Then more servers to
> scale out with their own group ids.
>
> The ingress protion then maps client requests key ranges to group ids and
> forwards it to the right place?
>
> Bigger picture:
> What I am trying to figure out is can I build a datastore that can be
> "easily" dynamically resized as the volume of data grows.
>
>
>
> ------------------------------
> *From:* Tsz Wo Sze <sz...@gmail.com>
> *Sent:* Sunday, March 6, 2022 3:28:29 AM
> *To:* user@ratis.apache.org <us...@ratis.apache.org>
> *Subject:* Re: Change group id
>
> Hi Asad,
>
> Currently, we do not support changing group id. What is your use case?
>
> Tsz-Wo
>
>
> On Sun, Mar 6, 2022 at 1:11 PM Asad Awadia <as...@gmail.com> wrote:
>
> Can the group id be changed for a state machine that has data/transactions
> applied?
>
> Will it create a new blank state machine to replicate? Or it will be able
> to still work on existing data?
>
>
Re: Change group id
Posted by Asad Awadia <as...@gmail.com>.
I was thinking about our earlier thread of a single threaded applying and sending keys to task queue for multithread processing etc.
Each thread gets a group id but they write to the same state machine DB and to scale up more threads with more group ids get added. (As part of that group ids may move around in unexpected ways) Then more servers to scale out with their own group ids.
The ingress protion then maps client requests key ranges to group ids and forwards it to the right place?
Bigger picture:
What I am trying to figure out is can I build a datastore that can be "easily" dynamically resized as the volume of data grows.
________________________________
From: Tsz Wo Sze <sz...@gmail.com>
Sent: Sunday, March 6, 2022 3:28:29 AM
To: user@ratis.apache.org <us...@ratis.apache.org>
Subject: Re: Change group id
Hi Asad,
Currently, we do not support changing group id. What is your use case?
Tsz-Wo
On Sun, Mar 6, 2022 at 1:11 PM Asad Awadia <as...@gmail.com>> wrote:
Can the group id be changed for a state machine that has data/transactions applied?
Will it create a new blank state machine to replicate? Or it will be able to still work on existing data?
Re: Change group id
Posted by Tsz Wo Sze <sz...@gmail.com>.
Hi Asad,
Currently, we do not support changing group id. What is your use case?
Tsz-Wo
On Sun, Mar 6, 2022 at 1:11 PM Asad Awadia <as...@gmail.com> wrote:
> Can the group id be changed for a state machine that has data/transactions
> applied?
>
> Will it create a new blank state machine to replicate? Or it will be able
> to still work on existing data?
>