You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ratis.apache.org by William Song <sz...@163.com> on 2023/09/01 01:35:31 UTC
Re: [DISCUSS] Ratis 3.0.0 release
+1 looking forward to 3.0.0, especially pluggable metrics. I’ll start testing 3.0.0 in IoTDB.
There are some WIP patches related to lease read and snapshot, could we wait until these patches are merged?
William
> 2023年8月31日 02:49，Tsz Wo Sze <sz...@gmail.com> 写道：
>
> Hi dev,
>
> The master branch was set to 3.0.0-SNAPSHOT for more than a year. How
> about we roll a Ratis 3.0.0 release?
>
> I recently tested Ratis 3.0.0 with Apache Ozone [1, 2]. It needs a few
> changes as shown below:
> - Set RaftStorage.StartupOption.RECOVER when building a RaftServer.
> - Ozone uses dropwizard3 so it needs the ratis-metrics-dropwizard3
> dependency.
> - Ozone uses some internal ratis-metrics API. We have to update the code
> to use the public API in ratis-metrics-api.
>
> [1] https://issues.apache.org/jira/browse/HDDS-9192
> [2] https://github.com/szetszwo/ozone/pull/6
>
> Tsz-Wo
Re: [DISCUSS] Ratis 3.0.0 release
Posted by William Song <sz...@163.com>.
Thanks for the explanation! Then it’s ok to see src tarball being different to git repo. The only differences are from ratis-experiments.
William
> 2023年11月9日 01:35，Tsz Wo Sze <sz...@gmail.com> 写道：
>
> Yes, it is intentional not to build ratis-experiments by default. It needs
> the "-Pexperiments-build" profile. Also, we should not include it in
> releases.
>
> Tsz-Wo
>
> On Wed, Nov 8, 2023 at 12:06 AM William Song <sz...@163.com> wrote:
>
>> Hi,
>>
>> It appears that the source files of the "ratis-experiments" module are not
>> included in the build. Is this omission intentional?
>>
>> William
>>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Tsz Wo Sze <sz...@gmail.com>.
Yes, it is intentional not to build ratis-experiments by default. It needs
the "-Pexperiments-build" profile. Also, we should not include it in
releases.
Tsz-Wo
On Wed, Nov 8, 2023 at 12:06 AM William Song <sz...@163.com> wrote:
> Hi,
>
> It appears that the source files of the "ratis-experiments" module are not
> included in the build. Is this omission intentional?
>
> William
>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by William Song <sz...@163.com>.
Hi,
It appears that the source files of the "ratis-experiments" module are not included in the build. Is this omission intentional?
William
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Tsz Wo Sze <sz...@gmail.com>.
That's great! Thanks, William!
Tsz-Wo
On Tue, Nov 7, 2023 at 9:28 PM William Song <sz...@163.com> wrote:
> The four JIRAs are now all merged. The long-anticipated ratis 3.0.0 is
> just around the corner!
> I’ll start 3.0 releasing process if there are no other tickets to catch
> on board.
>
> Regards,
> William
>
>
> >
> > The ratis-thirdparty vote has passed [1]. Now, we have the following
> four
> > JIRAs. Then, we could roll the 3.0.0 release.
> >
> > - RATIS-1927. Use double check to eliminate data race in ReadRequests
> #958
> > - RATIS-1928. Join the LogAppenders when closing the server. #959
> > - RATIS-1929. Bump ratis-thirdparty version to 1.0.5 #960
> > - RATIS-1930. Add a conf for enable/disable majority-add. #961
> >
> > Tsz-Wo
> > [1] https://lists.apache.org/thread/pwof4rov5gdhnpsys8cvdt2vdgs0qzk3
> >
> >
> > On Wed, Oct 25, 2023 at 5:34 PM Tsz Wo Sze <sz...@gmail.com> wrote:
> >
> >> It is a great idea to start rolling a ratis-thirdparty release!
> >>
> >> BTW, both RATIS-1920 and RATIS-1916 are now merged. Thanks.
> >>
> >> Tsz-Wo
> >>
> >>
> >> On Tue, Oct 24, 2023 at 11:23 PM William Song <sz...@163.com>
> wrote:
> >>
> >>> It’s thrilling to see the 3.0.0 on the corner. Meanwhile, let me
> prepare
> >>> for the ratis-thirdparty release first.
> >>>
> >>> William
> >>>
> >>>> 2023年10月25日 10:42，Tsz Wo Sze <sz...@gmail.com> 写道：
> >>>>
> >>>>> ... we can include RATIS-1911 "Add MembershipManager example” ...
> >>>>
> >>>> Sure, RATIS-1911 is now merged.
> >>>>
> >>>>> ... added a new parameter in Ratis-1918 to control the node shutdown
> >>>> threshold. ...
> >>>>
> >>>> Yes, RATIS-1918 will be included in Ratis 3.0.0.
> >>>>
> >>>> I was running tests for RATIS-1901 "Update Counter example to
> benchmark
> >>>> performance" and then found RATIS-1920. So, we have two blockers now:
> >>>> - RATIS-1920 "NPE in AppendLogResponseHandler"
> >>>> - RATIS-1916 "OrderAsync does not call handReply".
> >>>>
> >>>> Tsz-Wo
> >>>
> >>>
>
>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by William Song <sz...@163.com>.
The four JIRAs are now all merged. The long-anticipated ratis 3.0.0 is just around the corner!
I’ll start 3.0 releasing process if there are no other tickets to catch on board.
Regards,
William
>
> The ratis-thirdparty vote has passed [1]. Now, we have the following four
> JIRAs. Then, we could roll the 3.0.0 release.
>
> - RATIS-1927. Use double check to eliminate data race in ReadRequests #958
> - RATIS-1928. Join the LogAppenders when closing the server. #959
> - RATIS-1929. Bump ratis-thirdparty version to 1.0.5 #960
> - RATIS-1930. Add a conf for enable/disable majority-add. #961
>
> Tsz-Wo
> [1] https://lists.apache.org/thread/pwof4rov5gdhnpsys8cvdt2vdgs0qzk3
>
>
> On Wed, Oct 25, 2023 at 5:34 PM Tsz Wo Sze <sz...@gmail.com> wrote:
>
>> It is a great idea to start rolling a ratis-thirdparty release!
>>
>> BTW, both RATIS-1920 and RATIS-1916 are now merged. Thanks.
>>
>> Tsz-Wo
>>
>>
>> On Tue, Oct 24, 2023 at 11:23 PM William Song <sz...@163.com> wrote:
>>
>>> It’s thrilling to see the 3.0.0 on the corner. Meanwhile, let me prepare
>>> for the ratis-thirdparty release first.
>>>
>>> William
>>>
>>>> 2023年10月25日 10:42，Tsz Wo Sze <sz...@gmail.com> 写道：
>>>>
>>>>> ... we can include RATIS-1911 "Add MembershipManager example” ...
>>>>
>>>> Sure, RATIS-1911 is now merged.
>>>>
>>>>> ... added a new parameter in Ratis-1918 to control the node shutdown
>>>> threshold. ...
>>>>
>>>> Yes, RATIS-1918 will be included in Ratis 3.0.0.
>>>>
>>>> I was running tests for RATIS-1901 "Update Counter example to benchmark
>>>> performance" and then found RATIS-1920. So, we have two blockers now:
>>>> - RATIS-1920 "NPE in AppendLogResponseHandler"
>>>> - RATIS-1916 "OrderAsync does not call handReply".
>>>>
>>>> Tsz-Wo
>>>
>>>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Tsz Wo Sze <sz...@gmail.com>.
The ratis-thirdparty vote has passed [1]. Now, we have the following four
JIRAs. Then, we could roll the 3.0.0 release.
- RATIS-1927. Use double check to eliminate data race in ReadRequests #958
- RATIS-1928. Join the LogAppenders when closing the server. #959
- RATIS-1929. Bump ratis-thirdparty version to 1.0.5 #960
- RATIS-1930. Add a conf for enable/disable majority-add. #961
Tsz-Wo
[1] https://lists.apache.org/thread/pwof4rov5gdhnpsys8cvdt2vdgs0qzk3
On Wed, Oct 25, 2023 at 5:34 PM Tsz Wo Sze <sz...@gmail.com> wrote:
> It is a great idea to start rolling a ratis-thirdparty release!
>
> BTW, both RATIS-1920 and RATIS-1916 are now merged. Thanks.
>
> Tsz-Wo
>
>
> On Tue, Oct 24, 2023 at 11:23 PM William Song <sz...@163.com> wrote:
>
>> It’s thrilling to see the 3.0.0 on the corner. Meanwhile, let me prepare
>> for the ratis-thirdparty release first.
>>
>> William
>>
>> > 2023年10月25日 10:42，Tsz Wo Sze <sz...@gmail.com> 写道：
>> >
>> >> ... we can include RATIS-1911 "Add MembershipManager example” ...
>> >
>> > Sure, RATIS-1911 is now merged.
>> >
>> >> ... added a new parameter in Ratis-1918 to control the node shutdown
>> > threshold. ...
>> >
>> > Yes, RATIS-1918 will be included in Ratis 3.0.0.
>> >
>> > I was running tests for RATIS-1901 "Update Counter example to benchmark
>> > performance" and then found RATIS-1920. So, we have two blockers now:
>> > - RATIS-1920 "NPE in AppendLogResponseHandler"
>> > - RATIS-1916 "OrderAsync does not call handReply".
>> >
>> > Tsz-Wo
>>
>>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Tsz Wo Sze <sz...@gmail.com>.
It is a great idea to start rolling a ratis-thirdparty release!
BTW, both RATIS-1920 and RATIS-1916 are now merged. Thanks.
Tsz-Wo
On Tue, Oct 24, 2023 at 11:23 PM William Song <sz...@163.com> wrote:
> It’s thrilling to see the 3.0.0 on the corner. Meanwhile, let me prepare
> for the ratis-thirdparty release first.
>
> William
>
> > 2023年10月25日 10:42，Tsz Wo Sze <sz...@gmail.com> 写道：
> >
> >> ... we can include RATIS-1911 "Add MembershipManager example” ...
> >
> > Sure, RATIS-1911 is now merged.
> >
> >> ... added a new parameter in Ratis-1918 to control the node shutdown
> > threshold. ...
> >
> > Yes, RATIS-1918 will be included in Ratis 3.0.0.
> >
> > I was running tests for RATIS-1901 "Update Counter example to benchmark
> > performance" and then found RATIS-1920. So, we have two blockers now:
> > - RATIS-1920 "NPE in AppendLogResponseHandler"
> > - RATIS-1916 "OrderAsync does not call handReply".
> >
> > Tsz-Wo
>
>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by William Song <sz...@163.com>.
It’s thrilling to see the 3.0.0 on the corner. Meanwhile, let me prepare for the ratis-thirdparty release first.
William
> 2023年10月25日 10:42，Tsz Wo Sze <sz...@gmail.com> 写道：
>
>> ... we can include RATIS-1911 "Add MembershipManager example” ...
>
> Sure, RATIS-1911 is now merged.
>
>> ... added a new parameter in Ratis-1918 to control the node shutdown
> threshold. ...
>
> Yes, RATIS-1918 will be included in Ratis 3.0.0.
>
> I was running tests for RATIS-1901 "Update Counter example to benchmark
> performance" and then found RATIS-1920. So, we have two blockers now:
> - RATIS-1920 "NPE in AppendLogResponseHandler"
> - RATIS-1916 "OrderAsync does not call handReply".
>
> Tsz-Wo
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Tsz Wo Sze <sz...@gmail.com>.
> ... we can include RATIS-1911 "Add MembershipManager example” ...
Sure, RATIS-1911 is now merged.
> ... added a new parameter in Ratis-1918 to control the node shutdown
threshold. ...
Yes, RATIS-1918 will be included in Ratis 3.0.0.
I was running tests for RATIS-1901 "Update Counter example to benchmark
performance" and then found RATIS-1920. So, we have two blockers now:
- RATIS-1920 "NPE in AppendLogResponseHandler"
- RATIS-1916 "OrderAsync does not call handReply".
Tsz-Wo
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Xinyu Tan <ta...@apache.org>.
Hi, Ratis Community
I just added a new parameter in Ratis-1918 to control the node shutdown threshold. Since PR has already been merged, I'd like to include that commit in 3.0.0 if it's convenient.
Thanks
----------------------------
Xinyu Tan
On 2023/10/23 21:09:52 Tsz Wo Sze wrote:
> Hi William and others,
>
> It turns out that both RATIS-1900 and RATIS-872 are quite big and
> non-trivial. Since they are not incompatible changes, we may include them
> in a post Ratis 3.0.0 release (say, 3.0.1).
>
> Just checked the pull request queue [1]. It seems that only RATIS-1916
> "OrderAsync does not call handReply" needs to get in. The others are
> optional. What do you think?
>
> Tsz-Wo
> [1] https://github.com/apache/ratis/pulls
>
>
> On Thu, Oct 12, 2023 at 9:32 AM Tsz Wo Sze <sz...@gmail.com> wrote:
>
> > William,
> >
> > Some tests in Apache Ozone are failing with recent Ratis 3.0.0 snapshots.
> > The reason is RATIS-1902.
> >
> > I would also like to get RATIS-1900 and maybe RATIS-872. RATIS-1900 is a
> > simple performance improvement. RATIS-872 is a good idea to reduce the
> > retry cache memory usage.
> >
> > Good to hear that your tests went well!
> >
> > Tsz-Wo
> >
> >
> > On Thu, Oct 12, 2023 at 3:37 AM Yaolong Liu <ly...@163.com> wrote:
> >
> >>
> >>
> >> +1 for the new Ratis 3.0.0 release.
> >>
> >>
> >> Thanks Tsz-Wo for starting the discussion on 3.0.0 release and William
> >> for testing.
> >> Looking forward to the new release.
> >>
> >>
> >> Regards,
> >> - Yaolong
> >>
> >>
> >>
> >>
> >>
> >> At 2023-10-12 18:13:44, "William Song" <sz...@163.com> wrote:
> >> >The performance regression testing was successful. The 3.0.0 version we
> >> packaged after the commit 68b394a470e4e1e4a36b44bbaeffc6d7d0601b5a
> >> > showed no significant degradation in performance.
> >> >
> >> >Resource management, including CPU, memory, JVM GC, disk, and network,
> >> is also performing well.
> >> >
> >> > > You may also consider doing some compatibility tests (old/new clients
> >> talking to new/old servers)
> >> >
> >> >So far, compatibility testing has yielded positive results as well.
> >> >
> >> >
> >> >Are there any other patches expected to be included in 3.0.0? If not,
> >> I’ll start rolling out the release candidates ;)
> >> >
> >> >William
> >> >
> >> >
> >> >
> >> >> 2023年10月10日 03:19，Tsz Wo Sze <sz...@gmail.com> 写道：
> >> >>
> >> >> Hi William,
> >> >>
> >> >> It is great to hear that the IoTDB is working well with Ratis 3.0!
> >> You may
> >> >> also consider doing some compatibility tests (old/new clients talking
> >> to
> >> >> new/old servers), which is expected to work except for Streaming
> >> >> (RATIS-1569).
> >> >>
> >> >>> Also, I’d be happy to take on the release manager role. Please let me
> >> >> know if there are anything I can help with.
> >> >>
> >> >> Sure. Thanks a lot for taking the release manager role!
> >> >>
> >> >> Tsz-Wo
> >>
> >
>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by William Song <sz...@163.com>.
Hi Tsz-Wo,
+1 for including RATIS-1900 and RATIS-872 in 3.0.1. We also plan to bring some other patches after 3.0.0 release.
Besides, I think we can include RATIS-1911 "Add MembershipManager example” , since it’s a user-experience related patch (examples). I can add a corresponding doc of membership management if we decide to include RATIS-1911.
William
> 2023年10月24日 05:09，Tsz Wo Sze <sz...@gmail.com> 写道：
>
> Hi William and others,
>
> It turns out that both RATIS-1900 and RATIS-872 are quite big and
> non-trivial. Since they are not incompatible changes, we may include them
> in a post Ratis 3.0.0 release (say, 3.0.1).
>
> Just checked the pull request queue [1]. It seems that only RATIS-1916
> "OrderAsync does not call handReply" needs to get in. The others are
> optional. What do you think?
>
> Tsz-Wo
> [1] https://github.com/apache/ratis/pulls
>
>
> On Thu, Oct 12, 2023 at 9:32 AM Tsz Wo Sze <sz...@gmail.com> wrote:
>
>> William,
>>
>> Some tests in Apache Ozone are failing with recent Ratis 3.0.0 snapshots.
>> The reason is RATIS-1902.
>>
>> I would also like to get RATIS-1900 and maybe RATIS-872. RATIS-1900 is a
>> simple performance improvement. RATIS-872 is a good idea to reduce the
>> retry cache memory usage.
>>
>> Good to hear that your tests went well!
>>
>> Tsz-Wo
>>
>>
>> On Thu, Oct 12, 2023 at 3:37 AM Yaolong Liu <ly...@163.com> wrote:
>>
>>>
>>>
>>> +1 for the new Ratis 3.0.0 release.
>>>
>>>
>>> Thanks Tsz-Wo for starting the discussion on 3.0.0 release and William
>>> for testing.
>>> Looking forward to the new release.
>>>
>>>
>>> Regards,
>>> - Yaolong
>>>
>>>
>>>
>>>
>>>
>>> At 2023-10-12 18:13:44, "William Song" <sz...@163.com> wrote:
>>>> The performance regression testing was successful. The 3.0.0 version we
>>> packaged after the commit 68b394a470e4e1e4a36b44bbaeffc6d7d0601b5a
>>>> showed no significant degradation in performance.
>>>>
>>>> Resource management, including CPU, memory, JVM GC, disk, and network,
>>> is also performing well.
>>>>
>>>>> You may also consider doing some compatibility tests (old/new clients
>>> talking to new/old servers)
>>>>
>>>> So far, compatibility testing has yielded positive results as well.
>>>>
>>>>
>>>> Are there any other patches expected to be included in 3.0.0? If not,
>>> I’ll start rolling out the release candidates ;)
>>>>
>>>> William
>>>>
>>>>
>>>>
>>>>> 2023年10月10日 03:19，Tsz Wo Sze <sz...@gmail.com> 写道：
>>>>>
>>>>> Hi William,
>>>>>
>>>>> It is great to hear that the IoTDB is working well with Ratis 3.0!
>>> You may
>>>>> also consider doing some compatibility tests (old/new clients talking
>>> to
>>>>> new/old servers), which is expected to work except for Streaming
>>>>> (RATIS-1569).
>>>>>
>>>>>> Also, I’d be happy to take on the release manager role. Please let me
>>>>> know if there are anything I can help with.
>>>>>
>>>>> Sure. Thanks a lot for taking the release manager role!
>>>>>
>>>>> Tsz-Wo
>>>
>>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Xinyu Tan <tx...@gmail.com>.
Hi,
+1 for including big changes in 3.0.1.
Looking forward to Ratis 3.0.0! It already has a lot of new features and
bug fixes!
Thanks
----------------------------
Xinyu Tan
Tsz Wo Sze <sz...@gmail.com> 于2023年10月24日周二 05:10写道：
> Hi William and others,
>
> It turns out that both RATIS-1900 and RATIS-872 are quite big and
> non-trivial. Since they are not incompatible changes, we may include them
> in a post Ratis 3.0.0 release (say, 3.0.1).
>
> Just checked the pull request queue [1]. It seems that only RATIS-1916
> "OrderAsync does not call handReply" needs to get in. The others are
> optional. What do you think?
>
> Tsz-Wo
> [1] https://github.com/apache/ratis/pulls
>
>
> On Thu, Oct 12, 2023 at 9:32 AM Tsz Wo Sze <sz...@gmail.com> wrote:
>
> > William,
> >
> > Some tests in Apache Ozone are failing with recent Ratis 3.0.0 snapshots.
> > The reason is RATIS-1902.
> >
> > I would also like to get RATIS-1900 and maybe RATIS-872. RATIS-1900 is a
> > simple performance improvement. RATIS-872 is a good idea to reduce the
> > retry cache memory usage.
> >
> > Good to hear that your tests went well!
> >
> > Tsz-Wo
> >
> >
> > On Thu, Oct 12, 2023 at 3:37 AM Yaolong Liu <ly...@163.com> wrote:
> >
> >>
> >>
> >> +1 for the new Ratis 3.0.0 release.
> >>
> >>
> >> Thanks Tsz-Wo for starting the discussion on 3.0.0 release and William
> >> for testing.
> >> Looking forward to the new release.
> >>
> >>
> >> Regards,
> >> - Yaolong
> >>
> >>
> >>
> >>
> >>
> >> At 2023-10-12 18:13:44, "William Song" <sz...@163.com> wrote:
> >> >The performance regression testing was successful. The 3.0.0 version we
> >> packaged after the commit 68b394a470e4e1e4a36b44bbaeffc6d7d0601b5a
> >> > showed no significant degradation in performance.
> >> >
> >> >Resource management, including CPU, memory, JVM GC, disk, and network,
> >> is also performing well.
> >> >
> >> > > You may also consider doing some compatibility tests (old/new
> clients
> >> talking to new/old servers)
> >> >
> >> >So far, compatibility testing has yielded positive results as well.
> >> >
> >> >
> >> >Are there any other patches expected to be included in 3.0.0? If not,
> >> I’ll start rolling out the release candidates ;)
> >> >
> >> >William
> >> >
> >> >
> >> >
> >> >> 2023年10月10日 03:19，Tsz Wo Sze <sz...@gmail.com> 写道：
> >> >>
> >> >> Hi William,
> >> >>
> >> >> It is great to hear that the IoTDB is working well with Ratis 3.0!
> >> You may
> >> >> also consider doing some compatibility tests (old/new clients talking
> >> to
> >> >> new/old servers), which is expected to work except for Streaming
> >> >> (RATIS-1569).
> >> >>
> >> >>> Also, I’d be happy to take on the release manager role. Please let
> me
> >> >> know if there are anything I can help with.
> >> >>
> >> >> Sure. Thanks a lot for taking the release manager role!
> >> >>
> >> >> Tsz-Wo
> >>
> >
>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Tsz Wo Sze <sz...@gmail.com>.
Hi William and others,
It turns out that both RATIS-1900 and RATIS-872 are quite big and
non-trivial. Since they are not incompatible changes, we may include them
in a post Ratis 3.0.0 release (say, 3.0.1).
Just checked the pull request queue [1]. It seems that only RATIS-1916
"OrderAsync does not call handReply" needs to get in. The others are
optional. What do you think?
Tsz-Wo
[1] https://github.com/apache/ratis/pulls
On Thu, Oct 12, 2023 at 9:32 AM Tsz Wo Sze <sz...@gmail.com> wrote:
> William,
>
> Some tests in Apache Ozone are failing with recent Ratis 3.0.0 snapshots.
> The reason is RATIS-1902.
>
> I would also like to get RATIS-1900 and maybe RATIS-872. RATIS-1900 is a
> simple performance improvement. RATIS-872 is a good idea to reduce the
> retry cache memory usage.
>
> Good to hear that your tests went well!
>
> Tsz-Wo
>
>
> On Thu, Oct 12, 2023 at 3:37 AM Yaolong Liu <ly...@163.com> wrote:
>
>>
>>
>> +1 for the new Ratis 3.0.0 release.
>>
>>
>> Thanks Tsz-Wo for starting the discussion on 3.0.0 release and William
>> for testing.
>> Looking forward to the new release.
>>
>>
>> Regards,
>> - Yaolong
>>
>>
>>
>>
>>
>> At 2023-10-12 18:13:44, "William Song" <sz...@163.com> wrote:
>> >The performance regression testing was successful. The 3.0.0 version we
>> packaged after the commit 68b394a470e4e1e4a36b44bbaeffc6d7d0601b5a
>> > showed no significant degradation in performance.
>> >
>> >Resource management, including CPU, memory, JVM GC, disk, and network,
>> is also performing well.
>> >
>> > > You may also consider doing some compatibility tests (old/new clients
>> talking to new/old servers)
>> >
>> >So far, compatibility testing has yielded positive results as well.
>> >
>> >
>> >Are there any other patches expected to be included in 3.0.0? If not,
>> I’ll start rolling out the release candidates ;)
>> >
>> >William
>> >
>> >
>> >
>> >> 2023年10月10日 03:19，Tsz Wo Sze <sz...@gmail.com> 写道：
>> >>
>> >> Hi William,
>> >>
>> >> It is great to hear that the IoTDB is working well with Ratis 3.0!
>> You may
>> >> also consider doing some compatibility tests (old/new clients talking
>> to
>> >> new/old servers), which is expected to work except for Streaming
>> >> (RATIS-1569).
>> >>
>> >>> Also, I’d be happy to take on the release manager role. Please let me
>> >> know if there are anything I can help with.
>> >>
>> >> Sure. Thanks a lot for taking the release manager role!
>> >>
>> >> Tsz-Wo
>>
>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Tsz Wo Sze <sz...@gmail.com>.
William,
Some tests in Apache Ozone are failing with recent Ratis 3.0.0 snapshots.
The reason is RATIS-1902.
I would also like to get RATIS-1900 and maybe RATIS-872. RATIS-1900 is a
simple performance improvement. RATIS-872 is a good idea to reduce the
retry cache memory usage.
Good to hear that your tests went well!
Tsz-Wo
On Thu, Oct 12, 2023 at 3:37 AM Yaolong Liu <ly...@163.com> wrote:
>
>
> +1 for the new Ratis 3.0.0 release.
>
>
> Thanks Tsz-Wo for starting the discussion on 3.0.0 release and William for
> testing.
> Looking forward to the new release.
>
>
> Regards,
> - Yaolong
>
>
>
>
>
> At 2023-10-12 18:13:44, "William Song" <sz...@163.com> wrote:
> >The performance regression testing was successful. The 3.0.0 version we
> packaged after the commit 68b394a470e4e1e4a36b44bbaeffc6d7d0601b5a
> > showed no significant degradation in performance.
> >
> >Resource management, including CPU, memory, JVM GC, disk, and network, is
> also performing well.
> >
> > > You may also consider doing some compatibility tests (old/new clients
> talking to new/old servers)
> >
> >So far, compatibility testing has yielded positive results as well.
> >
> >
> >Are there any other patches expected to be included in 3.0.0? If not,
> I’ll start rolling out the release candidates ;)
> >
> >William
> >
> >
> >
> >> 2023年10月10日 03:19，Tsz Wo Sze <sz...@gmail.com> 写道：
> >>
> >> Hi William,
> >>
> >> It is great to hear that the IoTDB is working well with Ratis 3.0! You
> may
> >> also consider doing some compatibility tests (old/new clients talking to
> >> new/old servers), which is expected to work except for Streaming
> >> (RATIS-1569).
> >>
> >>> Also, I’d be happy to take on the release manager role. Please let me
> >> know if there are anything I can help with.
> >>
> >> Sure. Thanks a lot for taking the release manager role!
> >>
> >> Tsz-Wo
>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Yaolong Liu <ly...@163.com>.
+1 for the new Ratis 3.0.0 release.
Thanks Tsz-Wo for starting the discussion on 3.0.0 release and William for testing.
Looking forward to the new release.
Regards,
- Yaolong
At 2023-10-12 18:13:44, "William Song" <sz...@163.com> wrote:
>The performance regression testing was successful. The 3.0.0 version we packaged after the commit 68b394a470e4e1e4a36b44bbaeffc6d7d0601b5a
> showed no significant degradation in performance.
>
>Resource management, including CPU, memory, JVM GC, disk, and network, is also performing well.
>
> > You may also consider doing some compatibility tests (old/new clients talking to new/old servers)
>
>So far, compatibility testing has yielded positive results as well.
>
>
>Are there any other patches expected to be included in 3.0.0? If not, I’ll start rolling out the release candidates ;)
>
>William
>
>
>
>> 2023年10月10日 03:19，Tsz Wo Sze <sz...@gmail.com> 写道：
>>
>> Hi William,
>>
>> It is great to hear that the IoTDB is working well with Ratis 3.0! You may
>> also consider doing some compatibility tests (old/new clients talking to
>> new/old servers), which is expected to work except for Streaming
>> (RATIS-1569).
>>
>>> Also, I’d be happy to take on the release manager role. Please let me
>> know if there are anything I can help with.
>>
>> Sure. Thanks a lot for taking the release manager role!
>>
>> Tsz-Wo
Re: [DISCUSS] Ratis 3.0.0 release
Posted by William Song <sz...@163.com>.
The performance regression testing was successful. The 3.0.0 version we packaged after the commit 68b394a470e4e1e4a36b44bbaeffc6d7d0601b5a
showed no significant degradation in performance.
Resource management, including CPU, memory, JVM GC, disk, and network, is also performing well.
> You may also consider doing some compatibility tests (old/new clients talking to new/old servers)
So far, compatibility testing has yielded positive results as well.
Are there any other patches expected to be included in 3.0.0? If not, I’ll start rolling out the release candidates ;)
William
> 2023年10月10日 03:19，Tsz Wo Sze <sz...@gmail.com> 写道：
>
> Hi William,
>
> It is great to hear that the IoTDB is working well with Ratis 3.0! You may
> also consider doing some compatibility tests (old/new clients talking to
> new/old servers), which is expected to work except for Streaming
> (RATIS-1569).
>
>> Also, I’d be happy to take on the release manager role. Please let me
> know if there are anything I can help with.
>
> Sure. Thanks a lot for taking the release manager role!
>
> Tsz-Wo
>
>
> On Sat, Oct 7, 2023 at 8:58 AM William Song <sz...@163.com> wrote:
>
>> IoTDB now has no issues at all and we are fully on board! Currently we are
>> doing performance backtests and we will do a more thorough test once the
>> 3.0 official candidate releases.
>>
>> Also, I’d be happy to take on the release manager role. Please let me know
>> if there are anything I can help with.
>>
>> Regards,
>> William
>>
>>> 2023年9月16日 01:00，Tsz Wo Sze <sz...@gmail.com> 写道：
>>>
>>> William,
>>>
>>> Thanks a lot for testing 3.0.0 with IoTDB! Sure, we can wait for the
>>> patches. Please feel free to let me know if you need help on updating
>>> IoTDB to Ratis 3.0.0.
>>>
>>> Tsz-Wo
>>>
>>>
>>> On Thu, Aug 31, 2023 at 6:36 PM William Song <sz...@163.com> wrote:
>>>
>>>> +1 looking forward to 3.0.0, especially pluggable metrics. I’ll start
>>>> testing 3.0.0 in IoTDB.
>>>>
>>>> There are some WIP patches related to lease read and snapshot, could we
>>>> wait until these patches are merged?
>>>>
>>>> William
>>>>
>>>>> 2023年8月31日 02:49，Tsz Wo Sze <sz...@gmail.com> 写道：
>>>>>
>>>>> Hi dev,
>>>>>
>>>>> The master branch was set to 3.0.0-SNAPSHOT for more than a year. How
>>>>> about we roll a Ratis 3.0.0 release?
>>>>>
>>>>> I recently tested Ratis 3.0.0 with Apache Ozone [1, 2]. It needs a few
>>>>> changes as shown below:
>>>>> - Set RaftStorage.StartupOption.RECOVER when building a RaftServer.
>>>>> - Ozone uses dropwizard3 so it needs the ratis-metrics-dropwizard3
>>>>> dependency.
>>>>> - Ozone uses some internal ratis-metrics API. We have to update the
>> code
>>>>> to use the public API in ratis-metrics-api.
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/HDDS-9192
>>>>> [2] https://github.com/szetszwo/ozone/pull/6
>>>>>
>>>>> Tsz-Wo
>>>>
>>>>
>>
>>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Tsz Wo Sze <sz...@gmail.com>.
Hi William,
It is great to hear that the IoTDB is working well with Ratis 3.0! You may
also consider doing some compatibility tests (old/new clients talking to
new/old servers), which is expected to work except for Streaming
(RATIS-1569).
> Also, I’d be happy to take on the release manager role. Please let me
know if there are anything I can help with.
Sure. Thanks a lot for taking the release manager role!
Tsz-Wo
On Sat, Oct 7, 2023 at 8:58 AM William Song <sz...@163.com> wrote:
> IoTDB now has no issues at all and we are fully on board! Currently we are
> doing performance backtests and we will do a more thorough test once the
> 3.0 official candidate releases.
>
> Also, I’d be happy to take on the release manager role. Please let me know
> if there are anything I can help with.
>
> Regards,
> William
>
> > 2023年9月16日 01:00，Tsz Wo Sze <sz...@gmail.com> 写道：
> >
> > William,
> >
> > Thanks a lot for testing 3.0.0 with IoTDB! Sure, we can wait for the
> > patches. Please feel free to let me know if you need help on updating
> > IoTDB to Ratis 3.0.0.
> >
> > Tsz-Wo
> >
> >
> > On Thu, Aug 31, 2023 at 6:36 PM William Song <sz...@163.com> wrote:
> >
> >> +1 looking forward to 3.0.0, especially pluggable metrics. I’ll start
> >> testing 3.0.0 in IoTDB.
> >>
> >> There are some WIP patches related to lease read and snapshot, could we
> >> wait until these patches are merged?
> >>
> >> William
> >>
> >>> 2023年8月31日 02:49，Tsz Wo Sze <sz...@gmail.com> 写道：
> >>>
> >>> Hi dev,
> >>>
> >>> The master branch was set to 3.0.0-SNAPSHOT for more than a year. How
> >>> about we roll a Ratis 3.0.0 release?
> >>>
> >>> I recently tested Ratis 3.0.0 with Apache Ozone [1, 2]. It needs a few
> >>> changes as shown below:
> >>> - Set RaftStorage.StartupOption.RECOVER when building a RaftServer.
> >>> - Ozone uses dropwizard3 so it needs the ratis-metrics-dropwizard3
> >>> dependency.
> >>> - Ozone uses some internal ratis-metrics API. We have to update the
> code
> >>> to use the public API in ratis-metrics-api.
> >>>
> >>> [1] https://issues.apache.org/jira/browse/HDDS-9192
> >>> [2] https://github.com/szetszwo/ozone/pull/6
> >>>
> >>> Tsz-Wo
> >>
> >>
>
>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by William Song <sz...@163.com>.
IoTDB now has no issues at all and we are fully on board! Currently we are doing performance backtests and we will do a more thorough test once the 3.0 official candidate releases.
Also, I’d be happy to take on the release manager role. Please let me know if there are anything I can help with.
Regards,
William
> 2023年9月16日 01:00，Tsz Wo Sze <sz...@gmail.com> 写道：
>
> William,
>
> Thanks a lot for testing 3.0.0 with IoTDB! Sure, we can wait for the
> patches. Please feel free to let me know if you need help on updating
> IoTDB to Ratis 3.0.0.
>
> Tsz-Wo
>
>
> On Thu, Aug 31, 2023 at 6:36 PM William Song <sz...@163.com> wrote:
>
>> +1 looking forward to 3.0.0, especially pluggable metrics. I’ll start
>> testing 3.0.0 in IoTDB.
>>
>> There are some WIP patches related to lease read and snapshot, could we
>> wait until these patches are merged?
>>
>> William
>>
>>> 2023年8月31日 02:49，Tsz Wo Sze <sz...@gmail.com> 写道：
>>>
>>> Hi dev,
>>>
>>> The master branch was set to 3.0.0-SNAPSHOT for more than a year. How
>>> about we roll a Ratis 3.0.0 release?
>>>
>>> I recently tested Ratis 3.0.0 with Apache Ozone [1, 2]. It needs a few
>>> changes as shown below:
>>> - Set RaftStorage.StartupOption.RECOVER when building a RaftServer.
>>> - Ozone uses dropwizard3 so it needs the ratis-metrics-dropwizard3
>>> dependency.
>>> - Ozone uses some internal ratis-metrics API. We have to update the code
>>> to use the public API in ratis-metrics-api.
>>>
>>> [1] https://issues.apache.org/jira/browse/HDDS-9192
>>> [2] https://github.com/szetszwo/ozone/pull/6
>>>
>>> Tsz-Wo
>>
>>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by William Song <sz...@163.com>.
Hi tison,
The leader lease ticket: https://issues.apache.org/jira/browse/RATIS-1864.
Current progress: maintain lease through heartbeats https://issues.apache.org/jira/browse/RATIS-1866 and corresponding PR: https://github.com/apache/ratis/pull/898.
William
> 2023年9月16日 14:49，tison <wa...@gmail.com> 写道：
>
> May you share the link to lease read? That sounds interesting :)
>
> Tsz Wo Sze <sz...@gmail.com>于2023年9月16日 周六01:03写道：
>
>> William,
>>
>> Thanks a lot for testing 3.0.0 with IoTDB! Sure, we can wait for the
>> patches. Please feel free to let me know if you need help on updating
>> IoTDB to Ratis 3.0.0.
>>
>> Tsz-Wo
>>
>>
>> On Thu, Aug 31, 2023 at 6:36 PM William Song <sz...@163.com> wrote:
>>
>>> +1 looking forward to 3.0.0, especially pluggable metrics. I’ll start
>>> testing 3.0.0 in IoTDB.
>>>
>>> There are some WIP patches related to lease read and snapshot, could we
>>> wait until these patches are merged?
>>>
>>> William
>>>
>>>> 2023年8月31日 02:49，Tsz Wo Sze <sz...@gmail.com> 写道：
>>>>
>>>> Hi dev,
>>>>
>>>> The master branch was set to 3.0.0-SNAPSHOT for more than a year. How
>>>> about we roll a Ratis 3.0.0 release?
>>>>
>>>> I recently tested Ratis 3.0.0 with Apache Ozone [1, 2]. It needs a few
>>>> changes as shown below:
>>>> - Set RaftStorage.StartupOption.RECOVER when building a RaftServer.
>>>> - Ozone uses dropwizard3 so it needs the ratis-metrics-dropwizard3
>>>> dependency.
>>>> - Ozone uses some internal ratis-metrics API. We have to update the
>> code
>>>> to use the public API in ratis-metrics-api.
>>>>
>>>> [1] https://issues.apache.org/jira/browse/HDDS-9192
>>>> [2] https://github.com/szetszwo/ozone/pull/6
>>>>
>>>> Tsz-Wo
>>>
>>>
>>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by tison <wa...@gmail.com>.
May you share the link to lease read? That sounds interesting :)
Tsz Wo Sze <sz...@gmail.com>于2023年9月16日 周六01:03写道：
> William,
>
> Thanks a lot for testing 3.0.0 with IoTDB! Sure, we can wait for the
> patches. Please feel free to let me know if you need help on updating
> IoTDB to Ratis 3.0.0.
>
> Tsz-Wo
>
>
> On Thu, Aug 31, 2023 at 6:36 PM William Song <sz...@163.com> wrote:
>
> > +1 looking forward to 3.0.0, especially pluggable metrics. I’ll start
> > testing 3.0.0 in IoTDB.
> >
> > There are some WIP patches related to lease read and snapshot, could we
> > wait until these patches are merged?
> >
> > William
> >
> > > 2023年8月31日 02:49，Tsz Wo Sze <sz...@gmail.com> 写道：
> > >
> > > Hi dev,
> > >
> > > The master branch was set to 3.0.0-SNAPSHOT for more than a year. How
> > > about we roll a Ratis 3.0.0 release?
> > >
> > > I recently tested Ratis 3.0.0 with Apache Ozone [1, 2]. It needs a few
> > > changes as shown below:
> > > - Set RaftStorage.StartupOption.RECOVER when building a RaftServer.
> > > - Ozone uses dropwizard3 so it needs the ratis-metrics-dropwizard3
> > > dependency.
> > > - Ozone uses some internal ratis-metrics API. We have to update the
> code
> > > to use the public API in ratis-metrics-api.
> > >
> > > [1] https://issues.apache.org/jira/browse/HDDS-9192
> > > [2] https://github.com/szetszwo/ozone/pull/6
> > >
> > > Tsz-Wo
> >
> >
>
Re: [DISCUSS] Ratis 3.0.0 release
Posted by Tsz Wo Sze <sz...@gmail.com>.
William,
Thanks a lot for testing 3.0.0 with IoTDB! Sure, we can wait for the
patches. Please feel free to let me know if you need help on updating
IoTDB to Ratis 3.0.0.
Tsz-Wo
On Thu, Aug 31, 2023 at 6:36 PM William Song <sz...@163.com> wrote:
> +1 looking forward to 3.0.0, especially pluggable metrics. I’ll start
> testing 3.0.0 in IoTDB.
>
> There are some WIP patches related to lease read and snapshot, could we
> wait until these patches are merged?
>
> William
>
> > 2023年8月31日 02:49，Tsz Wo Sze <sz...@gmail.com> 写道：
> >
> > Hi dev,
> >
> > The master branch was set to 3.0.0-SNAPSHOT for more than a year. How
> > about we roll a Ratis 3.0.0 release?
> >
> > I recently tested Ratis 3.0.0 with Apache Ozone [1, 2]. It needs a few
> > changes as shown below:
> > - Set RaftStorage.StartupOption.RECOVER when building a RaftServer.
> > - Ozone uses dropwizard3 so it needs the ratis-metrics-dropwizard3
> > dependency.
> > - Ozone uses some internal ratis-metrics API. We have to update the code
> > to use the public API in ratis-metrics-api.
> >
> > [1] https://issues.apache.org/jira/browse/HDDS-9192
> > [2] https://github.com/szetszwo/ozone/pull/6
> >
> > Tsz-Wo
>
>