You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Lari Hotari <lh...@apache.org> on 2025/01/02 07:24:21 UTC
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
On 2024/12/30 02:15:52 Yubiao Feng wrote:
> Hi Lari
>
> > I disagree on making a change to the Pulsar 4.0 default value since
> Pulsar 4.0 has already been released.
>
> Since PR #9292 introduced a regression, we'd better make it the default
> carefully. We can change it as default-enabled at the following feature
> releases
Isn't the typical way to handle regressions and bugs to fix them?
I don't see a sensible argument for changing configuration defaults for 4.0.2 .
Instead, let's document configuration recommendations in a separate Pulsar 4.0 upgrade guide for users that are concerned about losing individual ack state if there's a need to downgrade from 4.0 to 3.0.
-Lari
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Lari Hotari <lh...@apache.org>.
Hi Yubiao,
There is no consensus to disable PR 9292 behavior ("Support large
number of unack message store for cursor recovery") in branch-4.0.
That's why I oppose what you are suggesting for branch-4.0. I also
suggest reverting the change in the master branch regarding the
default setting change. PR 23759 was merged without first reaching
consensus in this discussion thread.
According to Apache's decision making process, consensus means the
project as a whole has arrived at a decision that everyone can live
with.
In ASF projects, we don't want to vote. Instead, we want to reach
consensus through discussion.
In this thread, there are clearly opposing viewpoints that haven't
been reconciled:
- Yubiao and Baodi advocate for disabling PR 9292 to maintain
downgrade compatibility
- Enrico suggests keeping the current behavior in Pulsar 4.0.x and
documenting it
- I've argued for keeping PR 9292's behavior enabled due to its
performance benefits and due to the fact that it's already available
for Pulsar 4.0.0, 4.0.1 and 4.0.2 users
The proposal to merge changes to branch-4.0 and disable PR 9292 by
default in 4.0.3 faces several concerns:
- It would break expectations for existing Pulsar 4.0.0, 4.0.1 and 4.0.2 users
- Point releases (4.0.x) shouldn't introduce surprising behavioral changes
- The performance benefits of PR 9292 are significant for high-scale deployments
Alternative approaches have been proposed but not fully discussed:
- Adding comprehensive upgrade documentation for upgrading to Pulsar 4.0
- Providing clear configuration guidance for users who need downgrade
capability without losing individual acknowledgement state of
subscriptions
Given these unresolved matters, there isn't sufficient consensus to
proceed with making the changes that you are suggesting.
The Apache Way requires us to continue discussion when there is disagreement.
For reference, the Apache decision making process is documented here:
https://community.apache.org/committers/decisionMaking.html
Before making my suggestions for resolving this deadlock, I hope other
Pulsar community members could chime in.
-Lari
On Tue, 21 Jan 2025 at 04:32, Yubiao Feng
<yu...@streamnative.io.invalid> wrote:
>
> Hi all
>
> - To maintain compatibility, I will cherry-pick #9292 for branch-3 and
> branch-3.3 and cherry-pick and #23759 into branch-3., branch-3.3 and
> branch-4.0 after 48 hours. Please share your views if you have concerns.
>
>
> On Mon, Jan 20, 2025 at 9:47 AM Yubiao Feng <yu...@streamnative.io>
> wrote:
>
> > Hi all
> >
> > Since no more arguments, I merged the PR
> >
> > On Tue, Jan 14, 2025 at 5:05 PM Yubiao Feng <yu...@streamnative.io>
> > wrote:
> >
> >> Hi Lari
> >>
> >> > Let's start making progress:
> >> > - keep PR 9292 as default
> >> > - implement PR 23759 without changing the PR 9292 default
> >> > - write a proper upgrade guide for Pulsar 4.0 where rollback
> >> > considerations are explained
> >> > Makes sense?
> >>
> >> I do not think that it makes sense to me, there is no discussion to
> >> announce the feature before, and users never realized it in any doc so far.
> >>
> >> Thanks
> >> Yubiao Feng
> >>
> >> On Tue, Jan 14, 2025 at 3:56 PM Lari Hotari <lh...@apache.org> wrote:
> >>
> >>> On Tue, 14 Jan 2025 at 09:26, Baodi Shi <ba...@apache.org> wrote:
> >>> >
> >>> > hi, @lari and @enrico thanks for discuss.
> >>> >
> >>> > Yes, I agree that PR 9292 is a useful feature.
> >>> >
> >>> >
> >>> https://pulsar.apache.org/contribute/release-policy/#compatibility-between-releases
> >>> >
> >>> > When I look at our compatibility strategy, what we promised is
> >>> > compatibility from 3.0 -> 4.0 -> 3.0.
> >>> >
> >>> > Note that when the documentation mentions 3.0 and 4.0, I understand
> >>> > that they are the latest versions of 3.0 and 4.0, not the earlier
> >>> > ones.
> >>> >
> >>> > We need to ensure this, right?
> >>>
> >>> When we say there's support for downgrading (rollback), it doesn't
> >>> mean users don't need to take any action.
> >>>
> >>> We ensure downgrade compatibility by providing an upgrade guide that
> >>> explains how to configure the system to allow rollbacks without losing
> >>> state information. If we didn't do this, we'd always be stuck in the
> >>> same situation whenever a new LTS version comes out.
> >>>
> >>> Let's start making progress:
> >>> - keep PR 9292 as default
> >>> - implement PR 23759 without changing the PR 9292 default
> >>> - write a proper upgrade guide for Pulsar 4.0 where rollback
> >>> considerations are explained
> >>>
> >>> Makes sense?
> >>>
> >>>
> >>> -Lari
> >>>
> >>
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Yubiao Feng <yu...@streamnative.io.INVALID>.
Hi all
- To maintain compatibility, I will cherry-pick #9292 for branch-3 and
branch-3.3 and cherry-pick and #23759 into branch-3., branch-3.3 and
branch-4.0 after 48 hours. Please share your views if you have concerns.
On Mon, Jan 20, 2025 at 9:47 AM Yubiao Feng <yu...@streamnative.io>
wrote:
> Hi all
>
> Since no more arguments, I merged the PR
>
> On Tue, Jan 14, 2025 at 5:05 PM Yubiao Feng <yu...@streamnative.io>
> wrote:
>
>> Hi Lari
>>
>> > Let's start making progress:
>> > - keep PR 9292 as default
>> > - implement PR 23759 without changing the PR 9292 default
>> > - write a proper upgrade guide for Pulsar 4.0 where rollback
>> > considerations are explained
>> > Makes sense?
>>
>> I do not think that it makes sense to me, there is no discussion to
>> announce the feature before, and users never realized it in any doc so far.
>>
>> Thanks
>> Yubiao Feng
>>
>> On Tue, Jan 14, 2025 at 3:56 PM Lari Hotari <lh...@apache.org> wrote:
>>
>>> On Tue, 14 Jan 2025 at 09:26, Baodi Shi <ba...@apache.org> wrote:
>>> >
>>> > hi, @lari and @enrico thanks for discuss.
>>> >
>>> > Yes, I agree that PR 9292 is a useful feature.
>>> >
>>> >
>>> https://pulsar.apache.org/contribute/release-policy/#compatibility-between-releases
>>> >
>>> > When I look at our compatibility strategy, what we promised is
>>> > compatibility from 3.0 -> 4.0 -> 3.0.
>>> >
>>> > Note that when the documentation mentions 3.0 and 4.0, I understand
>>> > that they are the latest versions of 3.0 and 4.0, not the earlier
>>> > ones.
>>> >
>>> > We need to ensure this, right?
>>>
>>> When we say there's support for downgrading (rollback), it doesn't
>>> mean users don't need to take any action.
>>>
>>> We ensure downgrade compatibility by providing an upgrade guide that
>>> explains how to configure the system to allow rollbacks without losing
>>> state information. If we didn't do this, we'd always be stuck in the
>>> same situation whenever a new LTS version comes out.
>>>
>>> Let's start making progress:
>>> - keep PR 9292 as default
>>> - implement PR 23759 without changing the PR 9292 default
>>> - write a proper upgrade guide for Pulsar 4.0 where rollback
>>> considerations are explained
>>>
>>> Makes sense?
>>>
>>>
>>> -Lari
>>>
>>
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Yubiao Feng <yu...@streamnative.io.INVALID>.
Hi all
Since no more arguments, I merged the PR
On Tue, Jan 14, 2025 at 5:05 PM Yubiao Feng <yu...@streamnative.io>
wrote:
> Hi Lari
>
> > Let's start making progress:
> > - keep PR 9292 as default
> > - implement PR 23759 without changing the PR 9292 default
> > - write a proper upgrade guide for Pulsar 4.0 where rollback
> > considerations are explained
> > Makes sense?
>
> I do not think that it makes sense to me, there is no discussion to
> announce the feature before, and users never realized it in any doc so far.
>
> Thanks
> Yubiao Feng
>
> On Tue, Jan 14, 2025 at 3:56 PM Lari Hotari <lh...@apache.org> wrote:
>
>> On Tue, 14 Jan 2025 at 09:26, Baodi Shi <ba...@apache.org> wrote:
>> >
>> > hi, @lari and @enrico thanks for discuss.
>> >
>> > Yes, I agree that PR 9292 is a useful feature.
>> >
>> >
>> https://pulsar.apache.org/contribute/release-policy/#compatibility-between-releases
>> >
>> > When I look at our compatibility strategy, what we promised is
>> > compatibility from 3.0 -> 4.0 -> 3.0.
>> >
>> > Note that when the documentation mentions 3.0 and 4.0, I understand
>> > that they are the latest versions of 3.0 and 4.0, not the earlier
>> > ones.
>> >
>> > We need to ensure this, right?
>>
>> When we say there's support for downgrading (rollback), it doesn't
>> mean users don't need to take any action.
>>
>> We ensure downgrade compatibility by providing an upgrade guide that
>> explains how to configure the system to allow rollbacks without losing
>> state information. If we didn't do this, we'd always be stuck in the
>> same situation whenever a new LTS version comes out.
>>
>> Let's start making progress:
>> - keep PR 9292 as default
>> - implement PR 23759 without changing the PR 9292 default
>> - write a proper upgrade guide for Pulsar 4.0 where rollback
>> considerations are explained
>>
>> Makes sense?
>>
>>
>> -Lari
>>
>
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Yubiao Feng <yu...@streamnative.io.INVALID>.
Hi Lari
> Let's start making progress:
> - keep PR 9292 as default
> - implement PR 23759 without changing the PR 9292 default
> - write a proper upgrade guide for Pulsar 4.0 where rollback
> considerations are explained
> Makes sense?
I do not think that it makes sense to me, there is no discussion to
announce the feature before, and users never realized it in any doc so far.
Thanks
Yubiao Feng
On Tue, Jan 14, 2025 at 3:56 PM Lari Hotari <lh...@apache.org> wrote:
> On Tue, 14 Jan 2025 at 09:26, Baodi Shi <ba...@apache.org> wrote:
> >
> > hi, @lari and @enrico thanks for discuss.
> >
> > Yes, I agree that PR 9292 is a useful feature.
> >
> >
> https://pulsar.apache.org/contribute/release-policy/#compatibility-between-releases
> >
> > When I look at our compatibility strategy, what we promised is
> > compatibility from 3.0 -> 4.0 -> 3.0.
> >
> > Note that when the documentation mentions 3.0 and 4.0, I understand
> > that they are the latest versions of 3.0 and 4.0, not the earlier
> > ones.
> >
> > We need to ensure this, right?
>
> When we say there's support for downgrading (rollback), it doesn't
> mean users don't need to take any action.
>
> We ensure downgrade compatibility by providing an upgrade guide that
> explains how to configure the system to allow rollbacks without losing
> state information. If we didn't do this, we'd always be stuck in the
> same situation whenever a new LTS version comes out.
>
> Let's start making progress:
> - keep PR 9292 as default
> - implement PR 23759 without changing the PR 9292 default
> - write a proper upgrade guide for Pulsar 4.0 where rollback
> considerations are explained
>
> Makes sense?
>
>
> -Lari
>
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Lari Hotari <lh...@apache.org>.
On Tue, 14 Jan 2025 at 09:26, Baodi Shi <ba...@apache.org> wrote:
>
> hi, @lari and @enrico thanks for discuss.
>
> Yes, I agree that PR 9292 is a useful feature.
>
> https://pulsar.apache.org/contribute/release-policy/#compatibility-between-releases
>
> When I look at our compatibility strategy, what we promised is
> compatibility from 3.0 -> 4.0 -> 3.0.
>
> Note that when the documentation mentions 3.0 and 4.0, I understand
> that they are the latest versions of 3.0 and 4.0, not the earlier
> ones.
>
> We need to ensure this, right?
When we say there's support for downgrading (rollback), it doesn't
mean users don't need to take any action.
We ensure downgrade compatibility by providing an upgrade guide that
explains how to configure the system to allow rollbacks without losing
state information. If we didn't do this, we'd always be stuck in the
same situation whenever a new LTS version comes out.
Let's start making progress:
- keep PR 9292 as default
- implement PR 23759 without changing the PR 9292 default
- write a proper upgrade guide for Pulsar 4.0 where rollback
considerations are explained
Makes sense?
-Lari
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Baodi Shi <ba...@apache.org>.
hi, @lari and @enrico thanks for discuss.
Yes, I agree that PR 9292 is a useful feature.
https://pulsar.apache.org/contribute/release-policy/#compatibility-between-releases
When I look at our compatibility strategy, what we promised is
compatibility from 3.0 -> 4.0 -> 3.0.
Note that when the documentation mentions 3.0 and 4.0, I understand
that they are the latest versions of 3.0 and 4.0, not the earlier
ones.
We need to ensure this, right?
Thanks,
Baodi Shi
Lari Hotari <La...@hotari.net> 于2025年1月14日周二 14:42写道：
>
> On Tue, 14 Jan 2025 at 04:05, Baodi Shi <ba...@apache.org> wrote:
> >
> > I believe that in the early stages of a major version, some breaking
> > changes are unavoidable. We can't ensure 100% prevention.
> >
> > However, once an issue arises, we need to fix and resolve it promptly,
> > rather than maintaining the breaking change.
> >
> > In other words, how many users are currently using version 4.0 in
> > production? We should aim to deliver future 4.0.x versions with high
> > stability and compatibility with version 3.0.
>
> Thank you for raising these important considerations, Baodi.
>
> PR 9292 "Support large number of unack message store for cursor
> recovery" improvements deliver significant benefits for users running
> Pulsar at scale - which is a core use case for Pulsar. PR 9292
> functionality should remain enabled by default. This helps keep Pulsar
> competitive in the high-scale data streaming space.
>
> To address your concerns, we can follow Enrico's suggestion to provide
> clear upgrade guide instructions for:
> - Handling upgrade and downgrade (rollback) scenarios
> - Disabling these features if needed
>
> While PR 23759 (https://github.com/apache/pulsar/pull/23759) enhances
> PR 9292's functionality and addresses issues, it shouldn't disable the
> "Support large number of unack message store for cursor recovery"
> functionality by default. This lets us continue advancing Pulsar's
> capabilities for high-scale deployments.
>
> -Lari
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Lari Hotari <La...@hotari.net>.
On Tue, 14 Jan 2025 at 04:05, Baodi Shi <ba...@apache.org> wrote:
>
> I believe that in the early stages of a major version, some breaking
> changes are unavoidable. We can't ensure 100% prevention.
>
> However, once an issue arises, we need to fix and resolve it promptly,
> rather than maintaining the breaking change.
>
> In other words, how many users are currently using version 4.0 in
> production? We should aim to deliver future 4.0.x versions with high
> stability and compatibility with version 3.0.
Thank you for raising these important considerations, Baodi.
PR 9292 "Support large number of unack message store for cursor
recovery" improvements deliver significant benefits for users running
Pulsar at scale - which is a core use case for Pulsar. PR 9292
functionality should remain enabled by default. This helps keep Pulsar
competitive in the high-scale data streaming space.
To address your concerns, we can follow Enrico's suggestion to provide
clear upgrade guide instructions for:
- Handling upgrade and downgrade (rollback) scenarios
- Disabling these features if needed
While PR 23759 (https://github.com/apache/pulsar/pull/23759) enhances
PR 9292's functionality and addresses issues, it shouldn't disable the
"Support large number of unack message store for cursor recovery"
functionality by default. This lets us continue advancing Pulsar's
capabilities for high-scale deployments.
-Lari
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Baodi Shi <ba...@apache.org>.
I believe that in the early stages of a major version, some breaking
changes are unavoidable. We can't ensure 100% prevention.
However, once an issue arises, we need to fix and resolve it promptly,
rather than maintaining the breaking change.
In other words, how many users are currently using version 4.0 in
production? We should aim to deliver future 4.0.x versions with high
stability and compatibility with version 3.0.
We can mention in the documentation or community that versions 4.0.0
to 4.0.2 have compatibility issues, which will be fixed in 4.0.3 and
future versions.
Thanks,
Baodi Shi
Enrico Olivelli <eo...@gmail.com> 于2025年1月13日周一 22:51写道：
>
> Given that Pulsar 4.0 has been out for a while and it is considered
> production ready it is better to keep compatibility with Pulsar 4 (set the
> flag to 'true') and document this flag in the release notes or any upgrade
> guide from 3 to 4.
>
> So:
> - keep the current behavior in Pulsar 4.0.2
> - document that upgrading from 3.x to 4.x requires to set that flag in
> order to support rollbacks
>
> I know that this is not ideal, but current users of Pulsar 4 (that maybe
> started on 4) will see bad surprises when upgrading to a "point release".
> You can expect big surprises when you switch to a major release (2 -> 3, 3
> -> 4) but you really don't expect any surprises when you upgrade/rolback
> from x.y.z to x.y.k
>
> I am sorry that we did not catch this behavior change, we should add some
> "rollback testing" to our CI or at least to some checklist we run while
> voting a new "major release".
>
>
>
> Enrico
>
>
> Il giorno lun 13 gen 2025 alle ore 11:42 Lari Hotari <lh...@apache.org>
> ha scritto:
>
> > I hope to get broader opinions from the community before any such changes
> > are made.
> > Especially from Rajan and Enrico. Rajan is the author of
> > https://github.com/apache/pulsar/pull/9292 and Enrico has approved the PR.
> > There was a dev mailing list discussion regarding PIP-381 where PR 9292
> > was discussed. I found these messages by searching:
> > https://lists.apache.org/thread/dvhwo1jp5opyqkfspl3lynm6w4y9wg5g.
> >
> > -Lari
> >
> > On 2025/01/13 09:54:14 Baodi Shi wrote:
> > > hi,
> > >
> > > We should ensure forward and backward compatibility between 3.0 and
> > > 4.0. We can enable
> > > configuration(managedLedgerPersistIndividualAckAsLongArray) in 5.0.
> > >
> > > Therefore, I believe the default value should be false.
> > >
> > > We shouldn't be constrained by the already released 4.0.x versions. I
> > > believe users prefer to choose the newer 4.0.x versions and have
> > > seamless switching with 3.0.x.
> > >
> > >
> > > Thanks,
> > > Baodi Shi
> > >
> > > Yubiao Feng <yu...@streamnative.io.invalid> 于2025年1月13日周一 09:50写道：
> > > >
> > > > Hi Lari
> > > >
> > > > > We have already made releases 4.0.0 and 4.0.1
> > > > > with the changed defaults. Changing it now is too late.
> > > >
> > > > I think it is a fault and we should correct it, So let set the default
> > > > value to `false`.
> > > >
> > > > > PR 9292 has been approved by the community and there
> > > > > have been discussions in the past.
> > > >
> > > > I have never seen there is a discussion in the email list, and it was
> > > > merged with a unique approvement.
> > > >
> > > > Since it already introduced a regression, and no others mentioned how
> > to
> > > > upgrade/downgrade gracefully so far, we should set the default value to
> > > > `false`.
> > > >
> > > > On Thu, Jan 9, 2025 at 9:15 PM Lari Hotari <lh...@apache.org> wrote:
> > > >
> > > > > On Thu, 9 Jan 2025 at 14:26, Yubiao Feng
> > > > > <yu...@streamnative.io.invalid> wrote:
> > > > > > > Isn't the typical way to handle regressions and bugs to fix them?
> > > > > >
> > > > > > It may have other potential issues.
> > > > > >
> > > > > > > I don't see a sensible argument for changing configuration
> > defaults for
> > > > > > 4.0.2 .
> > > > > >
> > > > > > Is there a PIP related to #9292, which defined the default
> > behavior has
> > > > > > been changed?
> > > > >
> > > > > We have already made releases 4.0.0 and 4.0.1 with the changed
> > > > > defaults. Changing it now is too late.
> > > > > PR 9292 has been approved by the community and there have been
> > > > > discussions in the past.
> > > > > I don't see why we would be postponing such an important improvement
> > > > > to Pulsar 5.0 . Let's fix any possible regressions asap and adapt.
> > > > >
> > > > > -Lari
> > > > >
> > >
> >
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Lari Hotari <La...@hotari.net>.
On Mon, 13 Jan 2025 at 16:51, Enrico Olivelli <eo...@gmail.com> wrote:
>
> Given that Pulsar 4.0 has been out for a while and it is considered
> production ready it is better to keep compatibility with Pulsar 4 (set the
> flag to 'true') and document this flag in the release notes or any upgrade
> guide from 3 to 4.
>
> So:
> - keep the current behavior in Pulsar 4.0.2
> - document that upgrading from 3.x to 4.x requires to set that flag in
> order to support rollbacks
Thanks for sharing your thoughts, Enrico.
This is a really good point. Documenting this in the upgrade guide
should be addressing Baodi's and Yubiao's concerns about "breaking
changes".
>
> I know that this is not ideal, but current users of Pulsar 4 (that maybe
> started on 4) will see bad surprises when upgrading to a "point release".
> You can expect big surprises when you switch to a major release (2 -> 3, 3
> -> 4) but you really don't expect any surprises when you upgrade/rolback
> from x.y.z to x.y.k
>
> I am sorry that we did not catch this behavior change, we should add some
> "rollback testing" to our CI or at least to some checklist we run while
> voting a new "major release".
In general there isn't sufficient community activity to make such
activities possible. We need more active contributors that are able to
handle such improvements in our release process.
-Lari
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Enrico Olivelli <eo...@gmail.com>.
Given that Pulsar 4.0 has been out for a while and it is considered
production ready it is better to keep compatibility with Pulsar 4 (set the
flag to 'true') and document this flag in the release notes or any upgrade
guide from 3 to 4.
So:
- keep the current behavior in Pulsar 4.0.2
- document that upgrading from 3.x to 4.x requires to set that flag in
order to support rollbacks
I know that this is not ideal, but current users of Pulsar 4 (that maybe
started on 4) will see bad surprises when upgrading to a "point release".
You can expect big surprises when you switch to a major release (2 -> 3, 3
-> 4) but you really don't expect any surprises when you upgrade/rolback
from x.y.z to x.y.k
I am sorry that we did not catch this behavior change, we should add some
"rollback testing" to our CI or at least to some checklist we run while
voting a new "major release".
Enrico
Il giorno lun 13 gen 2025 alle ore 11:42 Lari Hotari <lh...@apache.org>
ha scritto:
> I hope to get broader opinions from the community before any such changes
> are made.
> Especially from Rajan and Enrico. Rajan is the author of
> https://github.com/apache/pulsar/pull/9292 and Enrico has approved the PR.
> There was a dev mailing list discussion regarding PIP-381 where PR 9292
> was discussed. I found these messages by searching:
> https://lists.apache.org/thread/dvhwo1jp5opyqkfspl3lynm6w4y9wg5g.
>
> -Lari
>
> On 2025/01/13 09:54:14 Baodi Shi wrote:
> > hi,
> >
> > We should ensure forward and backward compatibility between 3.0 and
> > 4.0. We can enable
> > configuration(managedLedgerPersistIndividualAckAsLongArray) in 5.0.
> >
> > Therefore, I believe the default value should be false.
> >
> > We shouldn't be constrained by the already released 4.0.x versions. I
> > believe users prefer to choose the newer 4.0.x versions and have
> > seamless switching with 3.0.x.
> >
> >
> > Thanks,
> > Baodi Shi
> >
> > Yubiao Feng <yu...@streamnative.io.invalid> 于2025年1月13日周一 09:50写道：
> > >
> > > Hi Lari
> > >
> > > > We have already made releases 4.0.0 and 4.0.1
> > > > with the changed defaults. Changing it now is too late.
> > >
> > > I think it is a fault and we should correct it, So let set the default
> > > value to `false`.
> > >
> > > > PR 9292 has been approved by the community and there
> > > > have been discussions in the past.
> > >
> > > I have never seen there is a discussion in the email list, and it was
> > > merged with a unique approvement.
> > >
> > > Since it already introduced a regression, and no others mentioned how
> to
> > > upgrade/downgrade gracefully so far, we should set the default value to
> > > `false`.
> > >
> > > On Thu, Jan 9, 2025 at 9:15 PM Lari Hotari <lh...@apache.org> wrote:
> > >
> > > > On Thu, 9 Jan 2025 at 14:26, Yubiao Feng
> > > > <yu...@streamnative.io.invalid> wrote:
> > > > > > Isn't the typical way to handle regressions and bugs to fix them?
> > > > >
> > > > > It may have other potential issues.
> > > > >
> > > > > > I don't see a sensible argument for changing configuration
> defaults for
> > > > > 4.0.2 .
> > > > >
> > > > > Is there a PIP related to #9292, which defined the default
> behavior has
> > > > > been changed?
> > > >
> > > > We have already made releases 4.0.0 and 4.0.1 with the changed
> > > > defaults. Changing it now is too late.
> > > > PR 9292 has been approved by the community and there have been
> > > > discussions in the past.
> > > > I don't see why we would be postponing such an important improvement
> > > > to Pulsar 5.0 . Let's fix any possible regressions asap and adapt.
> > > >
> > > > -Lari
> > > >
> >
>
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Lari Hotari <lh...@apache.org>.
I hope to get broader opinions from the community before any such changes are made.
Especially from Rajan and Enrico. Rajan is the author of https://github.com/apache/pulsar/pull/9292 and Enrico has approved the PR.
There was a dev mailing list discussion regarding PIP-381 where PR 9292 was discussed. I found these messages by searching: https://lists.apache.org/thread/dvhwo1jp5opyqkfspl3lynm6w4y9wg5g.
-Lari
On 2025/01/13 09:54:14 Baodi Shi wrote:
> hi,
>
> We should ensure forward and backward compatibility between 3.0 and
> 4.0. We can enable
> configuration(managedLedgerPersistIndividualAckAsLongArray) in 5.0.
>
> Therefore, I believe the default value should be false.
>
> We shouldn't be constrained by the already released 4.0.x versions. I
> believe users prefer to choose the newer 4.0.x versions and have
> seamless switching with 3.0.x.
>
>
> Thanks,
> Baodi Shi
>
> Yubiao Feng <yu...@streamnative.io.invalid> 于2025年1月13日周一 09:50写道：
> >
> > Hi Lari
> >
> > > We have already made releases 4.0.0 and 4.0.1
> > > with the changed defaults. Changing it now is too late.
> >
> > I think it is a fault and we should correct it, So let set the default
> > value to `false`.
> >
> > > PR 9292 has been approved by the community and there
> > > have been discussions in the past.
> >
> > I have never seen there is a discussion in the email list, and it was
> > merged with a unique approvement.
> >
> > Since it already introduced a regression, and no others mentioned how to
> > upgrade/downgrade gracefully so far, we should set the default value to
> > `false`.
> >
> > On Thu, Jan 9, 2025 at 9:15 PM Lari Hotari <lh...@apache.org> wrote:
> >
> > > On Thu, 9 Jan 2025 at 14:26, Yubiao Feng
> > > <yu...@streamnative.io.invalid> wrote:
> > > > > Isn't the typical way to handle regressions and bugs to fix them?
> > > >
> > > > It may have other potential issues.
> > > >
> > > > > I don't see a sensible argument for changing configuration defaults for
> > > > 4.0.2 .
> > > >
> > > > Is there a PIP related to #9292, which defined the default behavior has
> > > > been changed?
> > >
> > > We have already made releases 4.0.0 and 4.0.1 with the changed
> > > defaults. Changing it now is too late.
> > > PR 9292 has been approved by the community and there have been
> > > discussions in the past.
> > > I don't see why we would be postponing such an important improvement
> > > to Pulsar 5.0 . Let's fix any possible regressions asap and adapt.
> > >
> > > -Lari
> > >
>
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Baodi Shi <ba...@apache.org>.
hi,
We should ensure forward and backward compatibility between 3.0 and
4.0. We can enable
configuration(managedLedgerPersistIndividualAckAsLongArray) in 5.0.
Therefore, I believe the default value should be false.
We shouldn't be constrained by the already released 4.0.x versions. I
believe users prefer to choose the newer 4.0.x versions and have
seamless switching with 3.0.x.
Thanks,
Baodi Shi
Yubiao Feng <yu...@streamnative.io.invalid> 于2025年1月13日周一 09:50写道：
>
> Hi Lari
>
> > We have already made releases 4.0.0 and 4.0.1
> > with the changed defaults. Changing it now is too late.
>
> I think it is a fault and we should correct it, So let set the default
> value to `false`.
>
> > PR 9292 has been approved by the community and there
> > have been discussions in the past.
>
> I have never seen there is a discussion in the email list, and it was
> merged with a unique approvement.
>
> Since it already introduced a regression, and no others mentioned how to
> upgrade/downgrade gracefully so far, we should set the default value to
> `false`.
>
> On Thu, Jan 9, 2025 at 9:15 PM Lari Hotari <lh...@apache.org> wrote:
>
> > On Thu, 9 Jan 2025 at 14:26, Yubiao Feng
> > <yu...@streamnative.io.invalid> wrote:
> > > > Isn't the typical way to handle regressions and bugs to fix them?
> > >
> > > It may have other potential issues.
> > >
> > > > I don't see a sensible argument for changing configuration defaults for
> > > 4.0.2 .
> > >
> > > Is there a PIP related to #9292, which defined the default behavior has
> > > been changed?
> >
> > We have already made releases 4.0.0 and 4.0.1 with the changed
> > defaults. Changing it now is too late.
> > PR 9292 has been approved by the community and there have been
> > discussions in the past.
> > I don't see why we would be postponing such an important improvement
> > to Pulsar 5.0 . Let's fix any possible regressions asap and adapt.
> >
> > -Lari
> >
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Yubiao Feng <yu...@streamnative.io.INVALID>.
Hi Lari
> We have already made releases 4.0.0 and 4.0.1
> with the changed defaults. Changing it now is too late.
I think it is a fault and we should correct it, So let set the default
value to `false`.
> PR 9292 has been approved by the community and there
> have been discussions in the past.
I have never seen there is a discussion in the email list, and it was
merged with a unique approvement.
Since it already introduced a regression, and no others mentioned how to
upgrade/downgrade gracefully so far, we should set the default value to
`false`.
On Thu, Jan 9, 2025 at 9:15 PM Lari Hotari <lh...@apache.org> wrote:
> On Thu, 9 Jan 2025 at 14:26, Yubiao Feng
> <yu...@streamnative.io.invalid> wrote:
> > > Isn't the typical way to handle regressions and bugs to fix them?
> >
> > It may have other potential issues.
> >
> > > I don't see a sensible argument for changing configuration defaults for
> > 4.0.2 .
> >
> > Is there a PIP related to #9292, which defined the default behavior has
> > been changed?
>
> We have already made releases 4.0.0 and 4.0.1 with the changed
> defaults. Changing it now is too late.
> PR 9292 has been approved by the community and there have been
> discussions in the past.
> I don't see why we would be postponing such an important improvement
> to Pulsar 5.0 . Let's fix any possible regressions asap and adapt.
>
> -Lari
>
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Lari Hotari <lh...@apache.org>.
On Thu, 9 Jan 2025 at 14:26, Yubiao Feng
<yu...@streamnative.io.invalid> wrote:
> > Isn't the typical way to handle regressions and bugs to fix them?
>
> It may have other potential issues.
>
> > I don't see a sensible argument for changing configuration defaults for
> 4.0.2 .
>
> Is there a PIP related to #9292, which defined the default behavior has
> been changed?
We have already made releases 4.0.0 and 4.0.1 with the changed
defaults. Changing it now is too late.
PR 9292 has been approved by the community and there have been
discussions in the past.
I don't see why we would be postponing such an important improvement
to Pulsar 5.0 . Let's fix any possible regressions asap and adapt.
-Lari
Re: [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which contributed by #9292
Posted by Yubiao Feng <yu...@streamnative.io.INVALID>.
Hi Lari
> Isn't the typical way to handle regressions and bugs to fix them?
It may have other potential issues.
> I don't see a sensible argument for changing configuration defaults for
4.0.2 .
Is there a PIP related to #9292, which defined the default behavior has
been changed?
On Thu, Jan 2, 2025 at 3:25 PM Lari Hotari <lh...@apache.org> wrote:
> On 2024/12/30 02:15:52 Yubiao Feng wrote:
> > Hi Lari
> >
> > > I disagree on making a change to the Pulsar 4.0 default value since
> > Pulsar 4.0 has already been released.
> >
> > Since PR #9292 introduced a regression, we'd better make it the default
> > carefully. We can change it as default-enabled at the following feature
> > releases
>
> Isn't the typical way to handle regressions and bugs to fix them?
> I don't see a sensible argument for changing configuration defaults for
> 4.0.2 .
> Instead, let's document configuration recommendations in a separate Pulsar
> 4.0 upgrade guide for users that are concerned about losing individual ack
> state if there's a need to downgrade from 4.0 to 3.0.
>
> -Lari
>