You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ozone.apache.org by Hanisha Koneru <hk...@cloudera.com.INVALID> on 2020/11/23 18:35:18 UTC
Enabling Ratis on OM by default
Hi All,
I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498 <https://issues.apache.org/jira/browse/HDDS-4498>.
Ratis on OM has been tested extensively and is stable now.
HDDS-4330 is an on-going effort to support bootstrapping a new OM node. The pre-requirement for bootstrapping a new OM node is that the cluster must have ratis enabled OM. This also the pre-requirement for enabling OM HA. Hence, by enabling ratis by default, we get rid of one extra step in enabling HA.
It is simple to upgrade an existing non-ratis cluster to ratis enabled cluster - set ozone.om.ratis.enabled to true and restart the OM.
Thanks
Hanisha
Re: Enabling Ratis on OM by default
Posted by Hanisha Koneru <hk...@cloudera.com.INVALID>.
Hi All,
Here are the results from the OM write performance test on single node ratis enabled OM vs non-ratis OM.
# Keys Per Sec # Transactions per Sec
Single Node Ratis Enabled OM 4584 9168
Non Ratis OM 8078 16156
Thanks
Hanisha
> On Dec 1, 2020, at 12:30 PM, Hanisha Koneru <hk...@cloudera.com> wrote:
>
> Thank you all the positive feedback.
>
> I did do a comparison testing some time back and at that time, the performance of OM with ratis enabled was around 60% of that of non-ratis. There were multiple improvements made after that so the numbers could have changed.
>
> I will run the tests again with the current code and post my findings.
>
> Thanks
> Hanisha
>
>> On Nov 25, 2020, at 10:32 PM, Sammi Chen <sa...@apache.org> wrote:
>>
>> +1
>>
>> We are considering enabling OM HA in our production cluster in the near
>> feature.
>>
>> It will be very helpful if there is some performance data for reference.
>>
>>
>> Regards,
>> Sammi
>>
>> On Tue, Nov 24, 2020 at 2:35 AM Hanisha Koneru <hk...@cloudera.com.invalid>
>> wrote:
>>
>>> Hi All,
>>>
>>> I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498 <
>>> https://issues.apache.org/jira/browse/HDDS-4498>.
>>>
>>> Ratis on OM has been tested extensively and is stable now.
>>>
>>> HDDS-4330 is an on-going effort to support bootstrapping a new OM node.
>>> The pre-requirement for bootstrapping a new OM node is that the cluster
>>> must have ratis enabled OM. This also the pre-requirement for enabling OM
>>> HA. Hence, by enabling ratis by default, we get rid of one extra step in
>>> enabling HA.
>>>
>>> It is simple to upgrade an existing non-ratis cluster to ratis enabled
>>> cluster - set ozone.om.ratis.enabled to true and restart the OM.
>>>
>>> Thanks
>>> Hanisha
>
Re: Enabling Ratis on OM by default
Posted by Hanisha Koneru <hk...@cloudera.com.INVALID>.
Thank you all the positive feedback.
I did do a comparison testing some time back and at that time, the performance of OM with ratis enabled was around 60% of that of non-ratis. There were multiple improvements made after that so the numbers could have changed.
I will run the tests again with the current code and post my findings.
Thanks
Hanisha
> On Nov 25, 2020, at 10:32 PM, Sammi Chen <sa...@apache.org> wrote:
>
> +1
>
> We are considering enabling OM HA in our production cluster in the near
> feature.
>
> It will be very helpful if there is some performance data for reference.
>
>
> Regards,
> Sammi
>
> On Tue, Nov 24, 2020 at 2:35 AM Hanisha Koneru <hk...@cloudera.com.invalid>
> wrote:
>
>> Hi All,
>>
>> I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498 <
>> https://issues.apache.org/jira/browse/HDDS-4498>.
>>
>> Ratis on OM has been tested extensively and is stable now.
>>
>> HDDS-4330 is an on-going effort to support bootstrapping a new OM node.
>> The pre-requirement for bootstrapping a new OM node is that the cluster
>> must have ratis enabled OM. This also the pre-requirement for enabling OM
>> HA. Hence, by enabling ratis by default, we get rid of one extra step in
>> enabling HA.
>>
>> It is simple to upgrade an existing non-ratis cluster to ratis enabled
>> cluster - set ozone.om.ratis.enabled to true and restart the OM.
>>
>> Thanks
>> Hanisha
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Enabling Ratis on OM by default
Posted by Sammi Chen <sa...@apache.org>.
+1
We are considering enabling OM HA in our production cluster in the near
feature.
It will be very helpful if there is some performance data for reference.
Regards,
Sammi
On Tue, Nov 24, 2020 at 2:35 AM Hanisha Koneru <hk...@cloudera.com.invalid>
wrote:
> Hi All,
>
> I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498 <
> https://issues.apache.org/jira/browse/HDDS-4498>.
>
> Ratis on OM has been tested extensively and is stable now.
>
> HDDS-4330 is an on-going effort to support bootstrapping a new OM node.
> The pre-requirement for bootstrapping a new OM node is that the cluster
> must have ratis enabled OM. This also the pre-requirement for enabling OM
> HA. Hence, by enabling ratis by default, we get rid of one extra step in
> enabling HA.
>
> It is simple to upgrade an existing non-ratis cluster to ratis enabled
> cluster - set ozone.om.ratis.enabled to true and restart the OM.
>
> Thanks
> Hanisha
Re: Enabling Ratis on OM by default
Posted by Mukul Kumar Singh <mk...@gmail.com>.
+1
Thanks,
Mukul
On 24/11/20 12:30 pm, Lokesh Jain wrote:
> +1
>
> Regards
> Lokesh
>
>> On 24-Nov-2020, at 11:05 AM, Shashikant Banerjee <sb...@cloudera.com.INVALID> wrote:
>>
>> +1
>>
>> Thanks
>> Shashi
>>
>> On Tue, Nov 24, 2020 at 8:39 AM Bharat Viswanadham <bh...@apache.org>
>> wrote:
>>
>>> +1
>>>
>>> Thanks,
>>> Bharat
>>>
>>> On Mon, Nov 23, 2020 at 4:21 PM Vivek Ratnavel <vi...@apache.org>
>>> wrote:
>>>
>>>> +1
>>>>
>>>> On Mon, Nov 23, 2020 at 1:54 PM Dinesh Chitlangia <di...@apache.org>
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Nov 23, 2020 at 1:35 PM Hanisha Koneru
>>>>> <hk...@cloudera.com.invalid>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I am proposing to enable Ratis by default on Ozone Manager -
>>> HDDS-4498
>>>> <
>>>>>> https://issues.apache.org/jira/browse/HDDS-4498>.
>>>>>>
>>>>>> Ratis on OM has been tested extensively and is stable now.
>>>>>>
>>>>>> HDDS-4330 is an on-going effort to support bootstrapping a new OM
>>> node.
>>>>>> The pre-requirement for bootstrapping a new OM node is that the
>>> cluster
>>>>>> must have ratis enabled OM. This also the pre-requirement for
>>> enabling
>>>> OM
>>>>>> HA. Hence, by enabling ratis by default, we get rid of one extra step
>>>> in
>>>>>> enabling HA.
>>>>>>
>>>>>> It is simple to upgrade an existing non-ratis cluster to ratis
>>> enabled
>>>>>> cluster - set ozone.om.ratis.enabled to true and restart the OM.
>>>>>>
>>>>>> Thanks
>>>>>> Hanisha
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Enabling Ratis on OM by default
Posted by Lokesh Jain <lj...@apache.org>.
+1
Regards
Lokesh
> On 24-Nov-2020, at 11:05 AM, Shashikant Banerjee <sb...@cloudera.com.INVALID> wrote:
>
> +1
>
> Thanks
> Shashi
>
> On Tue, Nov 24, 2020 at 8:39 AM Bharat Viswanadham <bh...@apache.org>
> wrote:
>
>> +1
>>
>> Thanks,
>> Bharat
>>
>> On Mon, Nov 23, 2020 at 4:21 PM Vivek Ratnavel <vi...@apache.org>
>> wrote:
>>
>>> +1
>>>
>>> On Mon, Nov 23, 2020 at 1:54 PM Dinesh Chitlangia <di...@apache.org>
>>> wrote:
>>>
>>>> +1
>>>>
>>>>
>>>>
>>>> On Mon, Nov 23, 2020 at 1:35 PM Hanisha Koneru
>>>> <hk...@cloudera.com.invalid>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> I am proposing to enable Ratis by default on Ozone Manager -
>> HDDS-4498
>>> <
>>>>> https://issues.apache.org/jira/browse/HDDS-4498>.
>>>>>
>>>>> Ratis on OM has been tested extensively and is stable now.
>>>>>
>>>>> HDDS-4330 is an on-going effort to support bootstrapping a new OM
>> node.
>>>>> The pre-requirement for bootstrapping a new OM node is that the
>> cluster
>>>>> must have ratis enabled OM. This also the pre-requirement for
>> enabling
>>> OM
>>>>> HA. Hence, by enabling ratis by default, we get rid of one extra step
>>> in
>>>>> enabling HA.
>>>>>
>>>>> It is simple to upgrade an existing non-ratis cluster to ratis
>> enabled
>>>>> cluster - set ozone.om.ratis.enabled to true and restart the OM.
>>>>>
>>>>> Thanks
>>>>> Hanisha
>>>>
>>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Enabling Ratis on OM by default
Posted by Shashikant Banerjee <sb...@cloudera.com.INVALID>.
+1
Thanks
Shashi
On Tue, Nov 24, 2020 at 8:39 AM Bharat Viswanadham <bh...@apache.org>
wrote:
> +1
>
> Thanks,
> Bharat
>
> On Mon, Nov 23, 2020 at 4:21 PM Vivek Ratnavel <vi...@apache.org>
> wrote:
>
> > +1
> >
> > On Mon, Nov 23, 2020 at 1:54 PM Dinesh Chitlangia <di...@apache.org>
> > wrote:
> >
> > > +1
> > >
> > >
> > >
> > > On Mon, Nov 23, 2020 at 1:35 PM Hanisha Koneru
> > > <hk...@cloudera.com.invalid>
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > I am proposing to enable Ratis by default on Ozone Manager -
> HDDS-4498
> > <
> > > > https://issues.apache.org/jira/browse/HDDS-4498>.
> > > >
> > > > Ratis on OM has been tested extensively and is stable now.
> > > >
> > > > HDDS-4330 is an on-going effort to support bootstrapping a new OM
> node.
> > > > The pre-requirement for bootstrapping a new OM node is that the
> cluster
> > > > must have ratis enabled OM. This also the pre-requirement for
> enabling
> > OM
> > > > HA. Hence, by enabling ratis by default, we get rid of one extra step
> > in
> > > > enabling HA.
> > > >
> > > > It is simple to upgrade an existing non-ratis cluster to ratis
> enabled
> > > > cluster - set ozone.om.ratis.enabled to true and restart the OM.
> > > >
> > > > Thanks
> > > > Hanisha
> > >
> >
>
Re: Enabling Ratis on OM by default
Posted by Bharat Viswanadham <bh...@apache.org>.
+1
Thanks,
Bharat
On Mon, Nov 23, 2020 at 4:21 PM Vivek Ratnavel <vi...@apache.org>
wrote:
> +1
>
> On Mon, Nov 23, 2020 at 1:54 PM Dinesh Chitlangia <di...@apache.org>
> wrote:
>
> > +1
> >
> >
> >
> > On Mon, Nov 23, 2020 at 1:35 PM Hanisha Koneru
> > <hk...@cloudera.com.invalid>
> > wrote:
> >
> > > Hi All,
> > >
> > > I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498
> <
> > > https://issues.apache.org/jira/browse/HDDS-4498>.
> > >
> > > Ratis on OM has been tested extensively and is stable now.
> > >
> > > HDDS-4330 is an on-going effort to support bootstrapping a new OM node.
> > > The pre-requirement for bootstrapping a new OM node is that the cluster
> > > must have ratis enabled OM. This also the pre-requirement for enabling
> OM
> > > HA. Hence, by enabling ratis by default, we get rid of one extra step
> in
> > > enabling HA.
> > >
> > > It is simple to upgrade an existing non-ratis cluster to ratis enabled
> > > cluster - set ozone.om.ratis.enabled to true and restart the OM.
> > >
> > > Thanks
> > > Hanisha
> >
>
Re: Enabling Ratis on OM by default
Posted by Vivek Ratnavel <vi...@apache.org>.
+1
On Mon, Nov 23, 2020 at 1:54 PM Dinesh Chitlangia <di...@apache.org>
wrote:
> +1
>
>
>
> On Mon, Nov 23, 2020 at 1:35 PM Hanisha Koneru
> <hk...@cloudera.com.invalid>
> wrote:
>
> > Hi All,
> >
> > I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498 <
> > https://issues.apache.org/jira/browse/HDDS-4498>.
> >
> > Ratis on OM has been tested extensively and is stable now.
> >
> > HDDS-4330 is an on-going effort to support bootstrapping a new OM node.
> > The pre-requirement for bootstrapping a new OM node is that the cluster
> > must have ratis enabled OM. This also the pre-requirement for enabling OM
> > HA. Hence, by enabling ratis by default, we get rid of one extra step in
> > enabling HA.
> >
> > It is simple to upgrade an existing non-ratis cluster to ratis enabled
> > cluster - set ozone.om.ratis.enabled to true and restart the OM.
> >
> > Thanks
> > Hanisha
>
Re: Enabling Ratis on OM by default
Posted by Dinesh Chitlangia <di...@apache.org>.
+1
On Mon, Nov 23, 2020 at 1:35 PM Hanisha Koneru <hk...@cloudera.com.invalid>
wrote:
> Hi All,
>
> I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498 <
> https://issues.apache.org/jira/browse/HDDS-4498>.
>
> Ratis on OM has been tested extensively and is stable now.
>
> HDDS-4330 is an on-going effort to support bootstrapping a new OM node.
> The pre-requirement for bootstrapping a new OM node is that the cluster
> must have ratis enabled OM. This also the pre-requirement for enabling OM
> HA. Hence, by enabling ratis by default, we get rid of one extra step in
> enabling HA.
>
> It is simple to upgrade an existing non-ratis cluster to ratis enabled
> cluster - set ozone.om.ratis.enabled to true and restart the OM.
>
> Thanks
> Hanisha
Re: Enabling Ratis on OM by default
Posted by "Elek, Marton" <el...@apache.org>.
Sounds like a reasonable step forward.
Just one question: do we have any data about the performance difference?
Will it be slower for non-HA use cases?
Thanks,
Marton
On 11/23/20 7:35 PM, Hanisha Koneru wrote:
> Hi All,
>
> I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498 <https://issues.apache.org/jira/browse/HDDS-4498>.
>
> Ratis on OM has been tested extensively and is stable now.
>
> HDDS-4330 is an on-going effort to support bootstrapping a new OM node. The pre-requirement for bootstrapping a new OM node is that the cluster must have ratis enabled OM. This also the pre-requirement for enabling OM HA. Hence, by enabling ratis by default, we get rid of one extra step in enabling HA.
>
> It is simple to upgrade an existing non-ratis cluster to ratis enabled cluster - set ozone.om.ratis.enabled to true and restart the OM.
>
> Thanks
> Hanisha
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Enabling Ratis on OM by default
Posted by Siyao Meng <si...@apache.org>.
+1
-Siyao
On Nov 23, 2020 at 12:45:20 PM, Jitendra Pandey <jn...@gmail.com> wrote:
> +1
>
> On Mon, Nov 23, 2020 at 12:35 PM Siddharth Wagle
> <sw...@cloudera.com.invalid> wrote:
>
> +1 for this change.
>
>
> Best,
>
> Sid
>
>
> On Mon, Nov 23, 2020 at 11:19 AM Arpit Agarwal
>
> <aa...@cloudera.com.invalid> wrote:
>
>
> > +1
>
> >
>
> >
>
> > > On Nov 23, 2020, at 10:35 AM, Hanisha Koneru
>
> > <hk...@cloudera.com.INVALID> wrote:
>
> > >
>
> > > Hi All,
>
> > >
>
> > > I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498
>
> <
>
> > https://issues.apache.org/jira/browse/HDDS-4498>.
>
> > >
>
> > > Ratis on OM has been tested extensively and is stable now.
>
> > >
>
> > > HDDS-4330 is an on-going effort to support bootstrapping a new OM node.
>
> > The pre-requirement for bootstrapping a new OM node is that the cluster
>
> > must have ratis enabled OM. This also the pre-requirement for enabling OM
>
> > HA. Hence, by enabling ratis by default, we get rid of one extra step in
>
> > enabling HA.
>
> > >
>
> > > It is simple to upgrade an existing non-ratis cluster to ratis enabled
>
> > cluster - set ozone.om.ratis.enabled to true and restart the OM.
>
> > >
>
> > > Thanks
>
> > > Hanisha
>
> >
>
> >
>
> > ---------------------------------------------------------------------
>
> > To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
>
> > For additional commands, e-mail: dev-help@ozone.apache.org
>
> >
>
> >
>
>
>
Re: Enabling Ratis on OM by default
Posted by Jitendra Pandey <jn...@gmail.com>.
+1
On Mon, Nov 23, 2020 at 12:35 PM Siddharth Wagle
<sw...@cloudera.com.invalid> wrote:
> +1 for this change.
>
> Best,
> Sid
>
> On Mon, Nov 23, 2020 at 11:19 AM Arpit Agarwal
> <aa...@cloudera.com.invalid> wrote:
>
> > +1
> >
> >
> > > On Nov 23, 2020, at 10:35 AM, Hanisha Koneru
> > <hk...@cloudera.com.INVALID> wrote:
> > >
> > > Hi All,
> > >
> > > I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498
> <
> > https://issues.apache.org/jira/browse/HDDS-4498>.
> > >
> > > Ratis on OM has been tested extensively and is stable now.
> > >
> > > HDDS-4330 is an on-going effort to support bootstrapping a new OM node.
> > The pre-requirement for bootstrapping a new OM node is that the cluster
> > must have ratis enabled OM. This also the pre-requirement for enabling OM
> > HA. Hence, by enabling ratis by default, we get rid of one extra step in
> > enabling HA.
> > >
> > > It is simple to upgrade an existing non-ratis cluster to ratis enabled
> > cluster - set ozone.om.ratis.enabled to true and restart the OM.
> > >
> > > Thanks
> > > Hanisha
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> > For additional commands, e-mail: dev-help@ozone.apache.org
> >
> >
>
Re: Enabling Ratis on OM by default
Posted by Siddharth Wagle <sw...@cloudera.com.INVALID>.
+1 for this change.
Best,
Sid
On Mon, Nov 23, 2020 at 11:19 AM Arpit Agarwal
<aa...@cloudera.com.invalid> wrote:
> +1
>
>
> > On Nov 23, 2020, at 10:35 AM, Hanisha Koneru
> <hk...@cloudera.com.INVALID> wrote:
> >
> > Hi All,
> >
> > I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498 <
> https://issues.apache.org/jira/browse/HDDS-4498>.
> >
> > Ratis on OM has been tested extensively and is stable now.
> >
> > HDDS-4330 is an on-going effort to support bootstrapping a new OM node.
> The pre-requirement for bootstrapping a new OM node is that the cluster
> must have ratis enabled OM. This also the pre-requirement for enabling OM
> HA. Hence, by enabling ratis by default, we get rid of one extra step in
> enabling HA.
> >
> > It is simple to upgrade an existing non-ratis cluster to ratis enabled
> cluster - set ozone.om.ratis.enabled to true and restart the OM.
> >
> > Thanks
> > Hanisha
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
>
Re: Enabling Ratis on OM by default
Posted by Arpit Agarwal <aa...@cloudera.com.INVALID>.
+1
> On Nov 23, 2020, at 10:35 AM, Hanisha Koneru <hk...@cloudera.com.INVALID> wrote:
>
> Hi All,
>
> I am proposing to enable Ratis by default on Ozone Manager - HDDS-4498 <https://issues.apache.org/jira/browse/HDDS-4498>.
>
> Ratis on OM has been tested extensively and is stable now.
>
> HDDS-4330 is an on-going effort to support bootstrapping a new OM node. The pre-requirement for bootstrapping a new OM node is that the cluster must have ratis enabled OM. This also the pre-requirement for enabling OM HA. Hence, by enabling ratis by default, we get rid of one extra step in enabling HA.
>
> It is simple to upgrade an existing non-ratis cluster to ratis enabled cluster - set ozone.om.ratis.enabled to true and restart the OM.
>
> Thanks
> Hanisha
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org