[DISCUSS] Java 23 Support for 3.9.x

[Greg Harris <gr...@aiven.io.INVALID>]

Hi all,

Now that 3.9.0 is released and 4.0.x is progressing, I'd like to understand
everyone's expectations about the 3.9.x branch, and ask for a specific
consensus on Java 23 support.

Some context that I think is relevant to the discussion:
* KIP-1006 [1] proposes a backwards-compatible strategy for handling the
ongoing removal of the SecurityManager, which is merged and due to release
in 4.0.0 [2].
* KIP-1012 [3] rejected ongoing parallel feature development on a 3.x
branch while having trunk on 4.x.
* During the 3.9.0 release, the patch [2] was rejected [4] due to being a
new feature which did not meet the feature freeze deadline.
* Other than the SecurityManager removal, there are additional PRs which
would also need to be backported for full Java 23 support [5] including a
Scala patch upgrade.
* Downstream users are asking for a backport [6] because adding support for
Java 23 would obligate them to also include the 4.0 breaking changes.

So while adding Java version support in the past has been a KIP-less
feature and normally only appears in the next version, it happens to align
with a major version bump this time. This will cause additional pain for
users if we do not elect to backport this.

Thanks,
Greg

[1]
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
[2] https://github.com/apache/kafka/pull/16522
[3]
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
[4] https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
[5] https://issues.apache.org/jira/browse/KAFKA-17638
[6] https://github.com/apache/kafka/pull/16522#issuecomment-2488340682

Re: [DISCUSS] Java 23 Support for 3.9.x

[Anton Agestam <an...@aiven.io.INVALID>]

Hello,

I also think this sounds like a reasonable suggestion, also given that 3.9
was the first version to make KRaft feature complete, which means it's the
first version to bring production readiness for KRaft, in many contexts.

+1 from me.

BR,

Den tors 21 nov. 2024 kl 10:51 skrev Josep Prat <josep.prat@aiven.io.invalid
>:

> Hi all,
> Given 3.9 is the latest version before a major, and 4.0.0 comes with
> breaking changes (Zookeeper removal, Scala and Java version drops...) I
> think it's fair to assume that a considerable subset of Kafka users will be
> using 3.9 for a little longer than we usually support versions (roughly 1
> year). I would be in favour of considering 3.9 as and LTS and offer support
> for, let's say 18 to 24 months.
> In that spirit, I would be in favour of releasing a future 3.9 version that
> can run with JDK 23.
>
> Best,
>
> On Thu, Nov 21, 2024 at 3:46 AM Chia-Ping Tsai <ch...@gmail.com> wrote:
>
> > Dear all,
> >
> > It seems the main question is whether we should consider version 3.9 as
> an
> > LTS release. For example, will we continue with versions like 3.9.1,
> 3.9.2,
> > ... 3.9.100?
> >
> > If yes, we should backport both KIP-1006 and support for future JDKs.
> >
> > If not, backporting KIP-1006 to 3.9 would be sufficient to fix the issue
> of
> > running version 3.9 under JDK 23, even if JDK 23 is still not officially
> > supported by 3.9.
> >
> >
> > Best,
> >
> > Chia-Ping
> >
> >
> >
> > Greg Harris <gr...@aiven.io.invalid> 於 2024年11月21日 週四 上午4:37寫道：
> >
> > > > Has the SecurityManager been fully removed in JDK 23?
> > > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > >
> > > The SecurityManager has been degraded, so by default our users
> experience
> > > an UnsupportedOperationException. They can work-around this by setting
> a
> > > system property.
> > > In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched
> 3.9.x
> > > will experience an UnsupportedOperationException unconditionally.
> > >
> > > > I see https://issues.apache.org/jira/browse/KAFKA-17638
> > > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> > Lack
> > > of
> > > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as
> far
> > > as
> > > > I can tell).
> > >
> > > Originally we were planning to get this change into 3.9.0, but we
> missed
> > > the merge deadline. I opened that ticket afterwards to be fixed in
> 4.0.0
> > > because that's the next release.
> > > The patch was always intended to be backportable, and I intended to
> > > backport it [2].
> > >
> > > I understand that if we consider Java 23 support to be a feature (which
> > is
> > > the standing decision), this is a pretty obvious case of missing
> feature
> > > freeze, and the current course of action (releasing in 4.0.0) is how we
> > > would handle it.
> > > I'm asking for this to be reconsidered as a bug fix, because it allows
> us
> > > to backport the change, which is what our users are asking for [3].
> > >
> > > Thanks,
> > > Greg
> > >
> > > [1] https://openjdk.org/jeps/486
> > > [2] https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
> > > [3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> > >
> > > On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com>
> wrote:
> > >
> > > > Greg,
> > > >
> > > > I have not been following this closely, so apologies for some basic
> > > > questions.
> > > >
> > > > Has the SecurityManager been fully removed in JDK 23?
> > > >
> > > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > > >
> > > > By "4.0 breaking changes" do you mean changes to our JDK/Scala
> > supported
> > > > versions, removal or ZK, Kafka API changes, or something else?
> > > >
> > > > In general, I do not think we should change our supported JDK
> versions
> > > in a
> > > > hotfix release. I see
> > https://issues.apache.org/jira/browse/KAFKA-17638
> > > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> > Lack
> > > of
> > > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as
> far
> > > as
> > > > I can tell).
> > > >
> > > > Also, I feel that we should not add too much to 3.9.x aside from
> actual
> > > > bugs. If we backport things into 3.9.x, it will slow adoption of 4.x
> > and
> > > > increase our maintenance burden over time.
> > > >
> > > > Just my $0.02
> > > >
> > > > Thanks!
> > > > David A
> > > >
> > > > On Wed, Nov 20, 2024 at 12:22 PM Greg Harris
> > > <greg.harris@aiven.io.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> > > > understand
> > > > > everyone's expectations about the 3.9.x branch, and ask for a
> > specific
> > > > > consensus on Java 23 support.
> > > > >
> > > > > Some context that I think is relevant to the discussion:
> > > > > * KIP-1006 [1] proposes a backwards-compatible strategy for
> handling
> > > the
> > > > > ongoing removal of the SecurityManager, which is merged and due to
> > > > release
> > > > > in 4.0.0 [2].
> > > > > * KIP-1012 [3] rejected ongoing parallel feature development on a
> 3.x
> > > > > branch while having trunk on 4.x.
> > > > > * During the 3.9.0 release, the patch [2] was rejected [4] due to
> > > being a
> > > > > new feature which did not meet the feature freeze deadline.
> > > > > * Other than the SecurityManager removal, there are additional PRs
> > > which
> > > > > would also need to be backported for full Java 23 support [5]
> > > including a
> > > > > Scala patch upgrade.
> > > > > * Downstream users are asking for a backport [6] because adding
> > support
> > > > for
> > > > > Java 23 would obligate them to also include the 4.0 breaking
> changes.
> > > > >
> > > > > So while adding Java version support in the past has been a
> KIP-less
> > > > > feature and normally only appears in the next version, it happens
> to
> > > > align
> > > > > with a major version bump this time. This will cause additional
> pain
> > > for
> > > > > users if we do not elect to backport this.
> > > > >
> > > > > Thanks,
> > > > > Greg
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > > > > [2] https://github.com/apache/kafka/pull/16522
> > > > > [3]
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > > > > [4]
> https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > > > > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > > > > [6]
> > https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> > > > >
> > > >
> > > >
> > > > --
> > > > David Arthur
> > > >
> > >
> >
>
>
> --
> [image: Aiven] <https://www.aiven.io>
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.prat@aiven.io   |   +491715557497
> aiven.io <https://www.aiven.io>   |   <https://www.facebook.com/aivencloud
> >
>   <https://www.linkedin.com/company/aiven/>   <
> https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> Anna Richardson, Kenneth Chen
> Amtsgericht Charlottenburg, HRB 209739 B
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Anton Agestam <an...@aiven.io.INVALID>]

Hello,

I also think this sounds like a reasonable suggestion, also given that 3.9
was the first version to make KRaft feature complete, which means it's the
first version to bring production readiness for KRaft, in many contexts.

+1 from me.

BR,

Den tors 21 nov. 2024 kl 10:51 skrev Josep Prat <josep.prat@aiven.io.invalid
>:

> Hi all,
> Given 3.9 is the latest version before a major, and 4.0.0 comes with
> breaking changes (Zookeeper removal, Scala and Java version drops...) I
> think it's fair to assume that a considerable subset of Kafka users will be
> using 3.9 for a little longer than we usually support versions (roughly 1
> year). I would be in favour of considering 3.9 as and LTS and offer support
> for, let's say 18 to 24 months.
> In that spirit, I would be in favour of releasing a future 3.9 version that
> can run with JDK 23.
>
> Best,
>
> On Thu, Nov 21, 2024 at 3:46 AM Chia-Ping Tsai <ch...@gmail.com> wrote:
>
> > Dear all,
> >
> > It seems the main question is whether we should consider version 3.9 as
> an
> > LTS release. For example, will we continue with versions like 3.9.1,
> 3.9.2,
> > ... 3.9.100?
> >
> > If yes, we should backport both KIP-1006 and support for future JDKs.
> >
> > If not, backporting KIP-1006 to 3.9 would be sufficient to fix the issue
> of
> > running version 3.9 under JDK 23, even if JDK 23 is still not officially
> > supported by 3.9.
> >
> >
> > Best,
> >
> > Chia-Ping
> >
> >
> >
> > Greg Harris <gr...@aiven.io.invalid> 於 2024年11月21日 週四 上午4:37寫道：
> >
> > > > Has the SecurityManager been fully removed in JDK 23?
> > > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > >
> > > The SecurityManager has been degraded, so by default our users
> experience
> > > an UnsupportedOperationException. They can work-around this by setting
> a
> > > system property.
> > > In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched
> 3.9.x
> > > will experience an UnsupportedOperationException unconditionally.
> > >
> > > > I see https://issues.apache.org/jira/browse/KAFKA-17638
> > > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> > Lack
> > > of
> > > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as
> far
> > > as
> > > > I can tell).
> > >
> > > Originally we were planning to get this change into 3.9.0, but we
> missed
> > > the merge deadline. I opened that ticket afterwards to be fixed in
> 4.0.0
> > > because that's the next release.
> > > The patch was always intended to be backportable, and I intended to
> > > backport it [2].
> > >
> > > I understand that if we consider Java 23 support to be a feature (which
> > is
> > > the standing decision), this is a pretty obvious case of missing
> feature
> > > freeze, and the current course of action (releasing in 4.0.0) is how we
> > > would handle it.
> > > I'm asking for this to be reconsidered as a bug fix, because it allows
> us
> > > to backport the change, which is what our users are asking for [3].
> > >
> > > Thanks,
> > > Greg
> > >
> > > [1] https://openjdk.org/jeps/486
> > > [2] https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
> > > [3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> > >
> > > On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com>
> wrote:
> > >
> > > > Greg,
> > > >
> > > > I have not been following this closely, so apologies for some basic
> > > > questions.
> > > >
> > > > Has the SecurityManager been fully removed in JDK 23?
> > > >
> > > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > > >
> > > > By "4.0 breaking changes" do you mean changes to our JDK/Scala
> > supported
> > > > versions, removal or ZK, Kafka API changes, or something else?
> > > >
> > > > In general, I do not think we should change our supported JDK
> versions
> > > in a
> > > > hotfix release. I see
> > https://issues.apache.org/jira/browse/KAFKA-17638
> > > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> > Lack
> > > of
> > > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as
> far
> > > as
> > > > I can tell).
> > > >
> > > > Also, I feel that we should not add too much to 3.9.x aside from
> actual
> > > > bugs. If we backport things into 3.9.x, it will slow adoption of 4.x
> > and
> > > > increase our maintenance burden over time.
> > > >
> > > > Just my $0.02
> > > >
> > > > Thanks!
> > > > David A
> > > >
> > > > On Wed, Nov 20, 2024 at 12:22 PM Greg Harris
> > > <greg.harris@aiven.io.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> > > > understand
> > > > > everyone's expectations about the 3.9.x branch, and ask for a
> > specific
> > > > > consensus on Java 23 support.
> > > > >
> > > > > Some context that I think is relevant to the discussion:
> > > > > * KIP-1006 [1] proposes a backwards-compatible strategy for
> handling
> > > the
> > > > > ongoing removal of the SecurityManager, which is merged and due to
> > > > release
> > > > > in 4.0.0 [2].
> > > > > * KIP-1012 [3] rejected ongoing parallel feature development on a
> 3.x
> > > > > branch while having trunk on 4.x.
> > > > > * During the 3.9.0 release, the patch [2] was rejected [4] due to
> > > being a
> > > > > new feature which did not meet the feature freeze deadline.
> > > > > * Other than the SecurityManager removal, there are additional PRs
> > > which
> > > > > would also need to be backported for full Java 23 support [5]
> > > including a
> > > > > Scala patch upgrade.
> > > > > * Downstream users are asking for a backport [6] because adding
> > support
> > > > for
> > > > > Java 23 would obligate them to also include the 4.0 breaking
> changes.
> > > > >
> > > > > So while adding Java version support in the past has been a
> KIP-less
> > > > > feature and normally only appears in the next version, it happens
> to
> > > > align
> > > > > with a major version bump this time. This will cause additional
> pain
> > > for
> > > > > users if we do not elect to backport this.
> > > > >
> > > > > Thanks,
> > > > > Greg
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > > > > [2] https://github.com/apache/kafka/pull/16522
> > > > > [3]
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > > > > [4]
> https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > > > > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > > > > [6]
> > https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> > > > >
> > > >
> > > >
> > > > --
> > > > David Arthur
> > > >
> > >
> >
>
>
> --
> [image: Aiven] <https://www.aiven.io>
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.prat@aiven.io   |   +491715557497
> aiven.io <https://www.aiven.io>   |   <https://www.facebook.com/aivencloud
> >
>   <https://www.linkedin.com/company/aiven/>   <
> https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> Anna Richardson, Kenneth Chen
> Amtsgericht Charlottenburg, HRB 209739 B
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Mickael Maison <mi...@gmail.com>]

Hi Greg,

Thanks for following up on this issue.
I'd be supportive of backporting this specific change to 3.9.

Thanks,
Mickael

On Tue, Dec 17, 2024 at 6:02 PM Greg Harris
<gr...@aiven.io.invalid> wrote:
>
> Hey Luke and Divij,
>
> Updating our LTS policy, and/or declaring 3.9 an LTS version is a much
> larger topic than I intended to bring up here.
>
> And I don't believe LTS is necessary or sufficient to perform this backport:
> * 3.9 is currently supported: it's our latest release.
> * Restrictions like the ones that Divij mentions would exclude this
> backport.
>
> This thread is discussing a specific patch and a specific version, not
> asking for us to make a judgment on patches unseen, which appears unpopular.
>
> > Otherwise, if we only have v3.9.1 release and no more maintenance for
> > v3.9.x, why should we accept this "medium risk" decision?
>
> Being compatible with Java 23+ on 3.9.x would allow downstream projects
> depending on clients to use Java 23 while still remaining compatible with
> all previous broker versions.
> It would allow them to postpone the breaking changes present in the 4.0
> release until their own major release if desired.
> Even if 3.9.2 is never released, 3.9.1 will still provide value to the
> community.
>
> Thanks,
> Greg
>
>
> On Tue, Dec 17, 2024 at 12:25 AM Divij Vaidya <di...@gmail.com>
> wrote:
>
> > https://lists.apache.org/thread/tzx4zkhfz26joq5ydq70bxcfr3zwy1hk is a
> > discussion we had in the community a few years ago about the LTS / end of
> > life topic. Some of the conversation touched about a longer
> > maintenance window for last minor versions (and hence, treat it as LTS).
> >
> > I think it is perhaps time to restart that conversation in the community. I
> > do see a benefit in having a LTS for 3.9.x with some constraints such as
> > only data durability and security patches are fixed etc. but I also
> > understand the operational cost it comes with. It's a healthy debate to
> > have with a wider section of folks (perhaps include users@ list and have a
> > new subject line for the discussion).
> >
> > Greg, would you be willing to kick-off that discussion? We can park this
> > thread until we conclude on the end of life story for 3.9.x.
> >
> > --
> > Divij Vaidya
> >
> >
> >
> > On Tue, Dec 17, 2024 at 4:11 AM Luke Chen <sh...@gmail.com> wrote:
> >
> > > Hi Greg,
> > >
> > > Thanks for raising this.
> > > My main question is: Is Kafka v3.9 a LTS release?
> > > It seems many people think it is, but I never see this kind of discussion
> > > anywhere.
> > > In Kafka, IIRC, we have never had this kind of LTS release before.
> > > We need to clearly define the meaning of LTS release for v3.9.x and get
> > the
> > > consensus from the community.
> > >
> > > And once we all agree v3.9.x is a LTS release, I will be +1 for
> > backporting
> > > the patch to support jdk 23.
> > > Otherwise, if we only have v3.9.1 release and no more maintenance for
> > > v3.9.x, why should we accept this "medium risk" decision?
> > >
> > > Thanks.
> > > Luke
> > >
> > >
> > > On Tue, Dec 17, 2024 at 1:06 AM Greg Harris <greg.harris@aiven.io.invalid
> > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Another downstream project has reported upgrade difficulties because of
> > > our
> > > > Java support policy:
> > > > https://github.com/quarkusio/quarkus/issues/39634
> > > >
> > > > Thanks,
> > > > Greg
> > > >
> > > > On Tue, Dec 3, 2024 at 7:33 AM Chia-Ping Tsai <ch...@apache.org>
> > > wrote:
> > > >
> > > > > hi all,
> > > > >
> > > > > I believe this is an NP-hard challenge because we aim to release
> > > version
> > > > > 4.0 as quickly, stably, and effectively as possible. In the meantime,
> > > the
> > > > > progress of 4.0 increases the gap between 4.0 and 3.9. This means
> > that
> > > > > backporting fixes and improvements to 3.9 will require additional
> > > > resources
> > > > > from the community.
> > > > >
> > > > > However, I can imagine that many Kafka users and customers will
> > remain
> > > on
> > > > > version 3.x for a while. Therefore, it is worthwhile to keep
> > releasing
> > > > > 3.9.x for bug fixes. For improvements or new features, we should keep
> > > > them
> > > > > in 4.x to encourage users to advance alongside the community.
> > > > >
> > > > > Best,
> > > > > Chia-Ping
> > > > >
> > > > > On 2024/12/02 21:42:23 Swikar Patel wrote:
> > > > > > Java 23 is not LTS (Long Term Support) by Oracle
> > > > > > Do we still want to proceed?
> > > > > >
> > > > > > Thanks
> > > > > > Swikar
> > > > > >
> > > > > > > On Dec 2, 2024, at 1:07 PM, Greg Harris
> > > <greg.harris@aiven.io.invalid
> > > > >
> > > > > wrote:
> > > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > Thanks for the discussion so far, but I would like to refocus on
> > > the
> > > > > > > problem at hand.
> > > > > > >
> > > > > > > Our downstream users are asking for things that come at a cost to
> > > us
> > > > as
> > > > > > > maintainers: An exception to our processes, better support for
> > the
> > > > > final
> > > > > > > bridge release, and an easier maintenance burden.
> > > > > > >
> > > > > > > What is our consensus here? Should we disregard our users, and
> > > > > prioritize
> > > > > > > ourselves first?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Greg
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >> On Thu, Nov 21, 2024 at 12:52 PM Greg Harris <
> > > greg.harris@aiven.io>
> > > > > wrote:
> > > > > > >>
> > > > > > >> Hi Ismael,
> > > > > > >>
> > > > > > >> Thanks for your responses.
> > > > > > >>
> > > > > > >>> At the same time, where do we draw the line when it
> > > > > > >>> comes to future unreleased Java versions?
> > > > > > >>
> > > > > > >> I don't think we need to make a determination on that at this
> > > time.
> > > > > We are
> > > > > > >> making a one-time judgement about backporting a specific patch
> > to
> > > a
> > > > > > >> specific branch. We should have that discussion as a follow-up.
> > > > > > >>
> > > > > > >>> Is there evidence of this pain? That would help motivate the
> > > case.
> > > > > > >>
> > > > > > >> A downstream user has already replied to this thread, I'll
> > relink
> > > it
> > > > > here
> > > > > > >> [1] and the issue they were working on [2].
> > > > > > >>
> > > > > > >>> I don't think this is how we evaluate risk/reward, so I
> > disagree
> > > > with
> > > > > > >> your
> > > > > > >>> position here.
> > > > > > >>> Requiring users to set a system property is far from "harming"
> > > > them.
> > > > > > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > > > > > >>
> > > > > > >> I think you missed the point of my statement, which was that we
> > > > can't
> > > > > > >> justify our behavior by hiding among other libraries.
> > > > > > >> If everyone did that, nobody would ever respond to deprecations.
> > > > This
> > > > > is
> > > > > > >> an antisocial justification, I don't find it convincing, and we
> > > > > should not
> > > > > > >> rely on it.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Greg
> > > > > > >>
> > > > > > >> [1]
> > > > https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> > > > > > >> [2] https://github.com/trinodb/trino/issues/23498
> > > > > > >>
> > > > > > >>> On Thu, Nov 21, 2024 at 12:16 PM Ismael Juma <
> > me@ismaeljuma.com>
> > > > > wrote:
> > > > > > >>>
> > > > > > >>> Hi Greg,
> > > > > > >>>
> > > > > > >>> Comments below.
> > > > > > >>>
> > > > > > >>> On Thu, Nov 21, 2024 at 8:07 AM Greg Harris
> > > > > <greg.harris@aiven.io.invalid
> > > > > > >>>>
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>>>> Greg, why can't we set the relevant system property
> > > > > > >>>>> automatically for the broker/connect in 3.9.x?
> > > > > > >>>>
> > > > > > >>>> The System property workaround is not effective for Java 24
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> This is a fair point. At the same time, where do we draw the
> > line
> > > > > when it
> > > > > > >>> comes to future unreleased Java versions? There are two
> > possible
> > > > > paths:
> > > > > > >>> (1)
> > > > > > >>> we focus on the released versions, (2) we have a clear policy
> > for
> > > > > future
> > > > > > >>> versions (the upcoming version, the upcoming LTS version,
> > etc.).
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>> and doesn't
> > > > > > >>>> cover the kafka-as-a-library use-case, which is the one which
> > is
> > > > > > >>> generating
> > > > > > >>>> pain for our users.
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>> Is there evidence of this pain? That would help motivate the
> > > case.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>> We already have a patch reviewed and merged which can fix this
> > > > > generally
> > > > > > >>>> (23/24, clients/streams/brokers/connect), and is low risk.
> > > Please
> > > > > review
> > > > > > >>>> the PR to convince yourself of this.
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>> I did take a look at the PR and it did not seem low risk to me
> > > > (it's
> > > > > > >>> larger
> > > > > > >>> than we'd typically consider for bug fix releases, it includes
> > > > > reflection,
> > > > > > >>> etc.). And hence why I was exploring other options. But it's
> > not
> > > > > > >>> necessarily high risk either, it's somewhere in between.
> > > > > > >>>
> > > > > > >>>> When it comes to usage of
> > > > > > >>>>> Kafka as a library (eg clients), I very much doubt kafka will
> > > be
> > > > > the
> > > > > > >>> only
> > > > > > >>>>> system that requires this system property to be set, so not
> > > sure
> > > > > how
> > > > > > >>> much
> > > > > > >>>>> it matters.
> > > > > > >>>>
> > > > > > >>>> I don't think other libraries' support for SecurityManager is
> > > > > relevant
> > > > > > >>> to
> > > > > > >>>> this discussion, as there almost certainly is a downstream
> > > project
> > > > > out
> > > > > > >>>> there for which Kafka is the only dependency blocking their
> > > > upgrade.
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>> I don't think this is how we evaluate risk/reward, so I
> > disagree
> > > > > with your
> > > > > > >>> position here.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>> And if all libraries took that strategy, that would harm the
> > > > > adoption of
> > > > > > >>>> Java 23 and users, so we should avoid implementing that
> > strategy
> > > > > > >>> ourselves.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Requiring users to set a system property is far from "harming"
> > > > them.
> > > > > > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > > > > > >>>
> > > > > > >>> Ismael
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >

Re: [DISCUSS] Java 23 Support for 3.9.x

[Greg Harris <gr...@aiven.io.INVALID>]

Hey Luke and Divij,

Updating our LTS policy, and/or declaring 3.9 an LTS version is a much
larger topic than I intended to bring up here.

And I don't believe LTS is necessary or sufficient to perform this backport:
* 3.9 is currently supported: it's our latest release.
* Restrictions like the ones that Divij mentions would exclude this
backport.

This thread is discussing a specific patch and a specific version, not
asking for us to make a judgment on patches unseen, which appears unpopular.

> Otherwise, if we only have v3.9.1 release and no more maintenance for
> v3.9.x, why should we accept this "medium risk" decision?

Being compatible with Java 23+ on 3.9.x would allow downstream projects
depending on clients to use Java 23 while still remaining compatible with
all previous broker versions.
It would allow them to postpone the breaking changes present in the 4.0
release until their own major release if desired.
Even if 3.9.2 is never released, 3.9.1 will still provide value to the
community.

Thanks,
Greg


On Tue, Dec 17, 2024 at 12:25 AM Divij Vaidya <di...@gmail.com>
wrote:

> https://lists.apache.org/thread/tzx4zkhfz26joq5ydq70bxcfr3zwy1hk is a
> discussion we had in the community a few years ago about the LTS / end of
> life topic. Some of the conversation touched about a longer
> maintenance window for last minor versions (and hence, treat it as LTS).
>
> I think it is perhaps time to restart that conversation in the community. I
> do see a benefit in having a LTS for 3.9.x with some constraints such as
> only data durability and security patches are fixed etc. but I also
> understand the operational cost it comes with. It's a healthy debate to
> have with a wider section of folks (perhaps include users@ list and have a
> new subject line for the discussion).
>
> Greg, would you be willing to kick-off that discussion? We can park this
> thread until we conclude on the end of life story for 3.9.x.
>
> --
> Divij Vaidya
>
>
>
> On Tue, Dec 17, 2024 at 4:11 AM Luke Chen <sh...@gmail.com> wrote:
>
> > Hi Greg,
> >
> > Thanks for raising this.
> > My main question is: Is Kafka v3.9 a LTS release?
> > It seems many people think it is, but I never see this kind of discussion
> > anywhere.
> > In Kafka, IIRC, we have never had this kind of LTS release before.
> > We need to clearly define the meaning of LTS release for v3.9.x and get
> the
> > consensus from the community.
> >
> > And once we all agree v3.9.x is a LTS release, I will be +1 for
> backporting
> > the patch to support jdk 23.
> > Otherwise, if we only have v3.9.1 release and no more maintenance for
> > v3.9.x, why should we accept this "medium risk" decision?
> >
> > Thanks.
> > Luke
> >
> >
> > On Tue, Dec 17, 2024 at 1:06 AM Greg Harris <greg.harris@aiven.io.invalid
> >
> > wrote:
> >
> > > Hi all,
> > >
> > > Another downstream project has reported upgrade difficulties because of
> > our
> > > Java support policy:
> > > https://github.com/quarkusio/quarkus/issues/39634
> > >
> > > Thanks,
> > > Greg
> > >
> > > On Tue, Dec 3, 2024 at 7:33 AM Chia-Ping Tsai <ch...@apache.org>
> > wrote:
> > >
> > > > hi all,
> > > >
> > > > I believe this is an NP-hard challenge because we aim to release
> > version
> > > > 4.0 as quickly, stably, and effectively as possible. In the meantime,
> > the
> > > > progress of 4.0 increases the gap between 4.0 and 3.9. This means
> that
> > > > backporting fixes and improvements to 3.9 will require additional
> > > resources
> > > > from the community.
> > > >
> > > > However, I can imagine that many Kafka users and customers will
> remain
> > on
> > > > version 3.x for a while. Therefore, it is worthwhile to keep
> releasing
> > > > 3.9.x for bug fixes. For improvements or new features, we should keep
> > > them
> > > > in 4.x to encourage users to advance alongside the community.
> > > >
> > > > Best,
> > > > Chia-Ping
> > > >
> > > > On 2024/12/02 21:42:23 Swikar Patel wrote:
> > > > > Java 23 is not LTS (Long Term Support) by Oracle
> > > > > Do we still want to proceed?
> > > > >
> > > > > Thanks
> > > > > Swikar
> > > > >
> > > > > > On Dec 2, 2024, at 1:07 PM, Greg Harris
> > <greg.harris@aiven.io.invalid
> > > >
> > > > wrote:
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Thanks for the discussion so far, but I would like to refocus on
> > the
> > > > > > problem at hand.
> > > > > >
> > > > > > Our downstream users are asking for things that come at a cost to
> > us
> > > as
> > > > > > maintainers: An exception to our processes, better support for
> the
> > > > final
> > > > > > bridge release, and an easier maintenance burden.
> > > > > >
> > > > > > What is our consensus here? Should we disregard our users, and
> > > > prioritize
> > > > > > ourselves first?
> > > > > >
> > > > > > Thanks,
> > > > > > Greg
> > > > > >
> > > > > >
> > > > > >
> > > > > >> On Thu, Nov 21, 2024 at 12:52 PM Greg Harris <
> > greg.harris@aiven.io>
> > > > wrote:
> > > > > >>
> > > > > >> Hi Ismael,
> > > > > >>
> > > > > >> Thanks for your responses.
> > > > > >>
> > > > > >>> At the same time, where do we draw the line when it
> > > > > >>> comes to future unreleased Java versions?
> > > > > >>
> > > > > >> I don't think we need to make a determination on that at this
> > time.
> > > > We are
> > > > > >> making a one-time judgement about backporting a specific patch
> to
> > a
> > > > > >> specific branch. We should have that discussion as a follow-up.
> > > > > >>
> > > > > >>> Is there evidence of this pain? That would help motivate the
> > case.
> > > > > >>
> > > > > >> A downstream user has already replied to this thread, I'll
> relink
> > it
> > > > here
> > > > > >> [1] and the issue they were working on [2].
> > > > > >>
> > > > > >>> I don't think this is how we evaluate risk/reward, so I
> disagree
> > > with
> > > > > >> your
> > > > > >>> position here.
> > > > > >>> Requiring users to set a system property is far from "harming"
> > > them.
> > > > > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > > > > >>
> > > > > >> I think you missed the point of my statement, which was that we
> > > can't
> > > > > >> justify our behavior by hiding among other libraries.
> > > > > >> If everyone did that, nobody would ever respond to deprecations.
> > > This
> > > > is
> > > > > >> an antisocial justification, I don't find it convincing, and we
> > > > should not
> > > > > >> rely on it.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Greg
> > > > > >>
> > > > > >> [1]
> > > https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> > > > > >> [2] https://github.com/trinodb/trino/issues/23498
> > > > > >>
> > > > > >>> On Thu, Nov 21, 2024 at 12:16 PM Ismael Juma <
> me@ismaeljuma.com>
> > > > wrote:
> > > > > >>>
> > > > > >>> Hi Greg,
> > > > > >>>
> > > > > >>> Comments below.
> > > > > >>>
> > > > > >>> On Thu, Nov 21, 2024 at 8:07 AM Greg Harris
> > > > <greg.harris@aiven.io.invalid
> > > > > >>>>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>>> Greg, why can't we set the relevant system property
> > > > > >>>>> automatically for the broker/connect in 3.9.x?
> > > > > >>>>
> > > > > >>>> The System property workaround is not effective for Java 24
> > > > > >>>
> > > > > >>>
> > > > > >>> This is a fair point. At the same time, where do we draw the
> line
> > > > when it
> > > > > >>> comes to future unreleased Java versions? There are two
> possible
> > > > paths:
> > > > > >>> (1)
> > > > > >>> we focus on the released versions, (2) we have a clear policy
> for
> > > > future
> > > > > >>> versions (the upcoming version, the upcoming LTS version,
> etc.).
> > > > > >>>
> > > > > >>>
> > > > > >>>> and doesn't
> > > > > >>>> cover the kafka-as-a-library use-case, which is the one which
> is
> > > > > >>> generating
> > > > > >>>> pain for our users.
> > > > > >>>>
> > > > > >>>
> > > > > >>> Is there evidence of this pain? That would help motivate the
> > case.
> > > > > >>>
> > > > > >>>
> > > > > >>>> We already have a patch reviewed and merged which can fix this
> > > > generally
> > > > > >>>> (23/24, clients/streams/brokers/connect), and is low risk.
> > Please
> > > > review
> > > > > >>>> the PR to convince yourself of this.
> > > > > >>>>
> > > > > >>>
> > > > > >>> I did take a look at the PR and it did not seem low risk to me
> > > (it's
> > > > > >>> larger
> > > > > >>> than we'd typically consider for bug fix releases, it includes
> > > > reflection,
> > > > > >>> etc.). And hence why I was exploring other options. But it's
> not
> > > > > >>> necessarily high risk either, it's somewhere in between.
> > > > > >>>
> > > > > >>>> When it comes to usage of
> > > > > >>>>> Kafka as a library (eg clients), I very much doubt kafka will
> > be
> > > > the
> > > > > >>> only
> > > > > >>>>> system that requires this system property to be set, so not
> > sure
> > > > how
> > > > > >>> much
> > > > > >>>>> it matters.
> > > > > >>>>
> > > > > >>>> I don't think other libraries' support for SecurityManager is
> > > > relevant
> > > > > >>> to
> > > > > >>>> this discussion, as there almost certainly is a downstream
> > project
> > > > out
> > > > > >>>> there for which Kafka is the only dependency blocking their
> > > upgrade.
> > > > > >>>>
> > > > > >>>
> > > > > >>> I don't think this is how we evaluate risk/reward, so I
> disagree
> > > > with your
> > > > > >>> position here.
> > > > > >>>
> > > > > >>>
> > > > > >>>> And if all libraries took that strategy, that would harm the
> > > > adoption of
> > > > > >>>> Java 23 and users, so we should avoid implementing that
> strategy
> > > > > >>> ourselves.
> > > > > >>>
> > > > > >>>
> > > > > >>> Requiring users to set a system property is far from "harming"
> > > them.
> > > > > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > > > > >>>
> > > > > >>> Ismael
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Divij Vaidya <di...@gmail.com>]

https://lists.apache.org/thread/tzx4zkhfz26joq5ydq70bxcfr3zwy1hk is a
discussion we had in the community a few years ago about the LTS / end of
life topic. Some of the conversation touched about a longer
maintenance window for last minor versions (and hence, treat it as LTS).

I think it is perhaps time to restart that conversation in the community. I
do see a benefit in having a LTS for 3.9.x with some constraints such as
only data durability and security patches are fixed etc. but I also
understand the operational cost it comes with. It's a healthy debate to
have with a wider section of folks (perhaps include users@ list and have a
new subject line for the discussion).

Greg, would you be willing to kick-off that discussion? We can park this
thread until we conclude on the end of life story for 3.9.x.

--
Divij Vaidya



On Tue, Dec 17, 2024 at 4:11 AM Luke Chen <sh...@gmail.com> wrote:

> Hi Greg,
>
> Thanks for raising this.
> My main question is: Is Kafka v3.9 a LTS release?
> It seems many people think it is, but I never see this kind of discussion
> anywhere.
> In Kafka, IIRC, we have never had this kind of LTS release before.
> We need to clearly define the meaning of LTS release for v3.9.x and get the
> consensus from the community.
>
> And once we all agree v3.9.x is a LTS release, I will be +1 for backporting
> the patch to support jdk 23.
> Otherwise, if we only have v3.9.1 release and no more maintenance for
> v3.9.x, why should we accept this "medium risk" decision?
>
> Thanks.
> Luke
>
>
> On Tue, Dec 17, 2024 at 1:06 AM Greg Harris <gr...@aiven.io.invalid>
> wrote:
>
> > Hi all,
> >
> > Another downstream project has reported upgrade difficulties because of
> our
> > Java support policy:
> > https://github.com/quarkusio/quarkus/issues/39634
> >
> > Thanks,
> > Greg
> >
> > On Tue, Dec 3, 2024 at 7:33 AM Chia-Ping Tsai <ch...@apache.org>
> wrote:
> >
> > > hi all,
> > >
> > > I believe this is an NP-hard challenge because we aim to release
> version
> > > 4.0 as quickly, stably, and effectively as possible. In the meantime,
> the
> > > progress of 4.0 increases the gap between 4.0 and 3.9. This means that
> > > backporting fixes and improvements to 3.9 will require additional
> > resources
> > > from the community.
> > >
> > > However, I can imagine that many Kafka users and customers will remain
> on
> > > version 3.x for a while. Therefore, it is worthwhile to keep releasing
> > > 3.9.x for bug fixes. For improvements or new features, we should keep
> > them
> > > in 4.x to encourage users to advance alongside the community.
> > >
> > > Best,
> > > Chia-Ping
> > >
> > > On 2024/12/02 21:42:23 Swikar Patel wrote:
> > > > Java 23 is not LTS (Long Term Support) by Oracle
> > > > Do we still want to proceed?
> > > >
> > > > Thanks
> > > > Swikar
> > > >
> > > > > On Dec 2, 2024, at 1:07 PM, Greg Harris
> <greg.harris@aiven.io.invalid
> > >
> > > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Thanks for the discussion so far, but I would like to refocus on
> the
> > > > > problem at hand.
> > > > >
> > > > > Our downstream users are asking for things that come at a cost to
> us
> > as
> > > > > maintainers: An exception to our processes, better support for the
> > > final
> > > > > bridge release, and an easier maintenance burden.
> > > > >
> > > > > What is our consensus here? Should we disregard our users, and
> > > prioritize
> > > > > ourselves first?
> > > > >
> > > > > Thanks,
> > > > > Greg
> > > > >
> > > > >
> > > > >
> > > > >> On Thu, Nov 21, 2024 at 12:52 PM Greg Harris <
> greg.harris@aiven.io>
> > > wrote:
> > > > >>
> > > > >> Hi Ismael,
> > > > >>
> > > > >> Thanks for your responses.
> > > > >>
> > > > >>> At the same time, where do we draw the line when it
> > > > >>> comes to future unreleased Java versions?
> > > > >>
> > > > >> I don't think we need to make a determination on that at this
> time.
> > > We are
> > > > >> making a one-time judgement about backporting a specific patch to
> a
> > > > >> specific branch. We should have that discussion as a follow-up.
> > > > >>
> > > > >>> Is there evidence of this pain? That would help motivate the
> case.
> > > > >>
> > > > >> A downstream user has already replied to this thread, I'll relink
> it
> > > here
> > > > >> [1] and the issue they were working on [2].
> > > > >>
> > > > >>> I don't think this is how we evaluate risk/reward, so I disagree
> > with
> > > > >> your
> > > > >>> position here.
> > > > >>> Requiring users to set a system property is far from "harming"
> > them.
> > > > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > > > >>
> > > > >> I think you missed the point of my statement, which was that we
> > can't
> > > > >> justify our behavior by hiding among other libraries.
> > > > >> If everyone did that, nobody would ever respond to deprecations.
> > This
> > > is
> > > > >> an antisocial justification, I don't find it convincing, and we
> > > should not
> > > > >> rely on it.
> > > > >>
> > > > >> Thanks,
> > > > >> Greg
> > > > >>
> > > > >> [1]
> > https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> > > > >> [2] https://github.com/trinodb/trino/issues/23498
> > > > >>
> > > > >>> On Thu, Nov 21, 2024 at 12:16 PM Ismael Juma <me...@ismaeljuma.com>
> > > wrote:
> > > > >>>
> > > > >>> Hi Greg,
> > > > >>>
> > > > >>> Comments below.
> > > > >>>
> > > > >>> On Thu, Nov 21, 2024 at 8:07 AM Greg Harris
> > > <greg.harris@aiven.io.invalid
> > > > >>>>
> > > > >>> wrote:
> > > > >>>
> > > > >>>>> Greg, why can't we set the relevant system property
> > > > >>>>> automatically for the broker/connect in 3.9.x?
> > > > >>>>
> > > > >>>> The System property workaround is not effective for Java 24
> > > > >>>
> > > > >>>
> > > > >>> This is a fair point. At the same time, where do we draw the line
> > > when it
> > > > >>> comes to future unreleased Java versions? There are two possible
> > > paths:
> > > > >>> (1)
> > > > >>> we focus on the released versions, (2) we have a clear policy for
> > > future
> > > > >>> versions (the upcoming version, the upcoming LTS version, etc.).
> > > > >>>
> > > > >>>
> > > > >>>> and doesn't
> > > > >>>> cover the kafka-as-a-library use-case, which is the one which is
> > > > >>> generating
> > > > >>>> pain for our users.
> > > > >>>>
> > > > >>>
> > > > >>> Is there evidence of this pain? That would help motivate the
> case.
> > > > >>>
> > > > >>>
> > > > >>>> We already have a patch reviewed and merged which can fix this
> > > generally
> > > > >>>> (23/24, clients/streams/brokers/connect), and is low risk.
> Please
> > > review
> > > > >>>> the PR to convince yourself of this.
> > > > >>>>
> > > > >>>
> > > > >>> I did take a look at the PR and it did not seem low risk to me
> > (it's
> > > > >>> larger
> > > > >>> than we'd typically consider for bug fix releases, it includes
> > > reflection,
> > > > >>> etc.). And hence why I was exploring other options. But it's not
> > > > >>> necessarily high risk either, it's somewhere in between.
> > > > >>>
> > > > >>>> When it comes to usage of
> > > > >>>>> Kafka as a library (eg clients), I very much doubt kafka will
> be
> > > the
> > > > >>> only
> > > > >>>>> system that requires this system property to be set, so not
> sure
> > > how
> > > > >>> much
> > > > >>>>> it matters.
> > > > >>>>
> > > > >>>> I don't think other libraries' support for SecurityManager is
> > > relevant
> > > > >>> to
> > > > >>>> this discussion, as there almost certainly is a downstream
> project
> > > out
> > > > >>>> there for which Kafka is the only dependency blocking their
> > upgrade.
> > > > >>>>
> > > > >>>
> > > > >>> I don't think this is how we evaluate risk/reward, so I disagree
> > > with your
> > > > >>> position here.
> > > > >>>
> > > > >>>
> > > > >>>> And if all libraries took that strategy, that would harm the
> > > adoption of
> > > > >>>> Java 23 and users, so we should avoid implementing that strategy
> > > > >>> ourselves.
> > > > >>>
> > > > >>>
> > > > >>> Requiring users to set a system property is far from "harming"
> > them.
> > > > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > > > >>>
> > > > >>> Ismael
> > > > >>>
> > > > >>
> > > >
> > >
> >
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Luke Chen <sh...@gmail.com>]

Hi Greg,

Thanks for raising this.
My main question is: Is Kafka v3.9 a LTS release?
It seems many people think it is, but I never see this kind of discussion
anywhere.
In Kafka, IIRC, we have never had this kind of LTS release before.
We need to clearly define the meaning of LTS release for v3.9.x and get the
consensus from the community.

And once we all agree v3.9.x is a LTS release, I will be +1 for backporting
the patch to support jdk 23.
Otherwise, if we only have v3.9.1 release and no more maintenance for
v3.9.x, why should we accept this "medium risk" decision?

Thanks.
Luke


On Tue, Dec 17, 2024 at 1:06 AM Greg Harris <gr...@aiven.io.invalid>
wrote:

> Hi all,
>
> Another downstream project has reported upgrade difficulties because of our
> Java support policy:
> https://github.com/quarkusio/quarkus/issues/39634
>
> Thanks,
> Greg
>
> On Tue, Dec 3, 2024 at 7:33 AM Chia-Ping Tsai <ch...@apache.org> wrote:
>
> > hi all,
> >
> > I believe this is an NP-hard challenge because we aim to release version
> > 4.0 as quickly, stably, and effectively as possible. In the meantime, the
> > progress of 4.0 increases the gap between 4.0 and 3.9. This means that
> > backporting fixes and improvements to 3.9 will require additional
> resources
> > from the community.
> >
> > However, I can imagine that many Kafka users and customers will remain on
> > version 3.x for a while. Therefore, it is worthwhile to keep releasing
> > 3.9.x for bug fixes. For improvements or new features, we should keep
> them
> > in 4.x to encourage users to advance alongside the community.
> >
> > Best,
> > Chia-Ping
> >
> > On 2024/12/02 21:42:23 Swikar Patel wrote:
> > > Java 23 is not LTS (Long Term Support) by Oracle
> > > Do we still want to proceed?
> > >
> > > Thanks
> > > Swikar
> > >
> > > > On Dec 2, 2024, at 1:07 PM, Greg Harris <greg.harris@aiven.io.invalid
> >
> > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > Thanks for the discussion so far, but I would like to refocus on the
> > > > problem at hand.
> > > >
> > > > Our downstream users are asking for things that come at a cost to us
> as
> > > > maintainers: An exception to our processes, better support for the
> > final
> > > > bridge release, and an easier maintenance burden.
> > > >
> > > > What is our consensus here? Should we disregard our users, and
> > prioritize
> > > > ourselves first?
> > > >
> > > > Thanks,
> > > > Greg
> > > >
> > > >
> > > >
> > > >> On Thu, Nov 21, 2024 at 12:52 PM Greg Harris <gr...@aiven.io>
> > wrote:
> > > >>
> > > >> Hi Ismael,
> > > >>
> > > >> Thanks for your responses.
> > > >>
> > > >>> At the same time, where do we draw the line when it
> > > >>> comes to future unreleased Java versions?
> > > >>
> > > >> I don't think we need to make a determination on that at this time.
> > We are
> > > >> making a one-time judgement about backporting a specific patch to a
> > > >> specific branch. We should have that discussion as a follow-up.
> > > >>
> > > >>> Is there evidence of this pain? That would help motivate the case.
> > > >>
> > > >> A downstream user has already replied to this thread, I'll relink it
> > here
> > > >> [1] and the issue they were working on [2].
> > > >>
> > > >>> I don't think this is how we evaluate risk/reward, so I disagree
> with
> > > >> your
> > > >>> position here.
> > > >>> Requiring users to set a system property is far from "harming"
> them.
> > > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > > >>
> > > >> I think you missed the point of my statement, which was that we
> can't
> > > >> justify our behavior by hiding among other libraries.
> > > >> If everyone did that, nobody would ever respond to deprecations.
> This
> > is
> > > >> an antisocial justification, I don't find it convincing, and we
> > should not
> > > >> rely on it.
> > > >>
> > > >> Thanks,
> > > >> Greg
> > > >>
> > > >> [1]
> https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> > > >> [2] https://github.com/trinodb/trino/issues/23498
> > > >>
> > > >>> On Thu, Nov 21, 2024 at 12:16 PM Ismael Juma <me...@ismaeljuma.com>
> > wrote:
> > > >>>
> > > >>> Hi Greg,
> > > >>>
> > > >>> Comments below.
> > > >>>
> > > >>> On Thu, Nov 21, 2024 at 8:07 AM Greg Harris
> > <greg.harris@aiven.io.invalid
> > > >>>>
> > > >>> wrote:
> > > >>>
> > > >>>>> Greg, why can't we set the relevant system property
> > > >>>>> automatically for the broker/connect in 3.9.x?
> > > >>>>
> > > >>>> The System property workaround is not effective for Java 24
> > > >>>
> > > >>>
> > > >>> This is a fair point. At the same time, where do we draw the line
> > when it
> > > >>> comes to future unreleased Java versions? There are two possible
> > paths:
> > > >>> (1)
> > > >>> we focus on the released versions, (2) we have a clear policy for
> > future
> > > >>> versions (the upcoming version, the upcoming LTS version, etc.).
> > > >>>
> > > >>>
> > > >>>> and doesn't
> > > >>>> cover the kafka-as-a-library use-case, which is the one which is
> > > >>> generating
> > > >>>> pain for our users.
> > > >>>>
> > > >>>
> > > >>> Is there evidence of this pain? That would help motivate the case.
> > > >>>
> > > >>>
> > > >>>> We already have a patch reviewed and merged which can fix this
> > generally
> > > >>>> (23/24, clients/streams/brokers/connect), and is low risk. Please
> > review
> > > >>>> the PR to convince yourself of this.
> > > >>>>
> > > >>>
> > > >>> I did take a look at the PR and it did not seem low risk to me
> (it's
> > > >>> larger
> > > >>> than we'd typically consider for bug fix releases, it includes
> > reflection,
> > > >>> etc.). And hence why I was exploring other options. But it's not
> > > >>> necessarily high risk either, it's somewhere in between.
> > > >>>
> > > >>>> When it comes to usage of
> > > >>>>> Kafka as a library (eg clients), I very much doubt kafka will be
> > the
> > > >>> only
> > > >>>>> system that requires this system property to be set, so not sure
> > how
> > > >>> much
> > > >>>>> it matters.
> > > >>>>
> > > >>>> I don't think other libraries' support for SecurityManager is
> > relevant
> > > >>> to
> > > >>>> this discussion, as there almost certainly is a downstream project
> > out
> > > >>>> there for which Kafka is the only dependency blocking their
> upgrade.
> > > >>>>
> > > >>>
> > > >>> I don't think this is how we evaluate risk/reward, so I disagree
> > with your
> > > >>> position here.
> > > >>>
> > > >>>
> > > >>>> And if all libraries took that strategy, that would harm the
> > adoption of
> > > >>>> Java 23 and users, so we should avoid implementing that strategy
> > > >>> ourselves.
> > > >>>
> > > >>>
> > > >>> Requiring users to set a system property is far from "harming"
> them.
> > > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > > >>>
> > > >>> Ismael
> > > >>>
> > > >>
> > >
> >
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Greg Harris <gr...@aiven.io.INVALID>]

Hi all,

Another downstream project has reported upgrade difficulties because of our
Java support policy:
https://github.com/quarkusio/quarkus/issues/39634

Thanks,
Greg

On Tue, Dec 3, 2024 at 7:33 AM Chia-Ping Tsai <ch...@apache.org> wrote:

> hi all,
>
> I believe this is an NP-hard challenge because we aim to release version
> 4.0 as quickly, stably, and effectively as possible. In the meantime, the
> progress of 4.0 increases the gap between 4.0 and 3.9. This means that
> backporting fixes and improvements to 3.9 will require additional resources
> from the community.
>
> However, I can imagine that many Kafka users and customers will remain on
> version 3.x for a while. Therefore, it is worthwhile to keep releasing
> 3.9.x for bug fixes. For improvements or new features, we should keep them
> in 4.x to encourage users to advance alongside the community.
>
> Best,
> Chia-Ping
>
> On 2024/12/02 21:42:23 Swikar Patel wrote:
> > Java 23 is not LTS (Long Term Support) by Oracle
> > Do we still want to proceed?
> >
> > Thanks
> > Swikar
> >
> > > On Dec 2, 2024, at 1:07 PM, Greg Harris <gr...@aiven.io.invalid>
> wrote:
> > >
> > > Hi all,
> > >
> > > Thanks for the discussion so far, but I would like to refocus on the
> > > problem at hand.
> > >
> > > Our downstream users are asking for things that come at a cost to us as
> > > maintainers: An exception to our processes, better support for the
> final
> > > bridge release, and an easier maintenance burden.
> > >
> > > What is our consensus here? Should we disregard our users, and
> prioritize
> > > ourselves first?
> > >
> > > Thanks,
> > > Greg
> > >
> > >
> > >
> > >> On Thu, Nov 21, 2024 at 12:52 PM Greg Harris <gr...@aiven.io>
> wrote:
> > >>
> > >> Hi Ismael,
> > >>
> > >> Thanks for your responses.
> > >>
> > >>> At the same time, where do we draw the line when it
> > >>> comes to future unreleased Java versions?
> > >>
> > >> I don't think we need to make a determination on that at this time.
> We are
> > >> making a one-time judgement about backporting a specific patch to a
> > >> specific branch. We should have that discussion as a follow-up.
> > >>
> > >>> Is there evidence of this pain? That would help motivate the case.
> > >>
> > >> A downstream user has already replied to this thread, I'll relink it
> here
> > >> [1] and the issue they were working on [2].
> > >>
> > >>> I don't think this is how we evaluate risk/reward, so I disagree with
> > >> your
> > >>> position here.
> > >>> Requiring users to set a system property is far from "harming" them.
> > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > >>
> > >> I think you missed the point of my statement, which was that we can't
> > >> justify our behavior by hiding among other libraries.
> > >> If everyone did that, nobody would ever respond to deprecations. This
> is
> > >> an antisocial justification, I don't find it convincing, and we
> should not
> > >> rely on it.
> > >>
> > >> Thanks,
> > >> Greg
> > >>
> > >> [1] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> > >> [2] https://github.com/trinodb/trino/issues/23498
> > >>
> > >>> On Thu, Nov 21, 2024 at 12:16 PM Ismael Juma <me...@ismaeljuma.com>
> wrote:
> > >>>
> > >>> Hi Greg,
> > >>>
> > >>> Comments below.
> > >>>
> > >>> On Thu, Nov 21, 2024 at 8:07 AM Greg Harris
> <greg.harris@aiven.io.invalid
> > >>>>
> > >>> wrote:
> > >>>
> > >>>>> Greg, why can't we set the relevant system property
> > >>>>> automatically for the broker/connect in 3.9.x?
> > >>>>
> > >>>> The System property workaround is not effective for Java 24
> > >>>
> > >>>
> > >>> This is a fair point. At the same time, where do we draw the line
> when it
> > >>> comes to future unreleased Java versions? There are two possible
> paths:
> > >>> (1)
> > >>> we focus on the released versions, (2) we have a clear policy for
> future
> > >>> versions (the upcoming version, the upcoming LTS version, etc.).
> > >>>
> > >>>
> > >>>> and doesn't
> > >>>> cover the kafka-as-a-library use-case, which is the one which is
> > >>> generating
> > >>>> pain for our users.
> > >>>>
> > >>>
> > >>> Is there evidence of this pain? That would help motivate the case.
> > >>>
> > >>>
> > >>>> We already have a patch reviewed and merged which can fix this
> generally
> > >>>> (23/24, clients/streams/brokers/connect), and is low risk. Please
> review
> > >>>> the PR to convince yourself of this.
> > >>>>
> > >>>
> > >>> I did take a look at the PR and it did not seem low risk to me (it's
> > >>> larger
> > >>> than we'd typically consider for bug fix releases, it includes
> reflection,
> > >>> etc.). And hence why I was exploring other options. But it's not
> > >>> necessarily high risk either, it's somewhere in between.
> > >>>
> > >>>> When it comes to usage of
> > >>>>> Kafka as a library (eg clients), I very much doubt kafka will be
> the
> > >>> only
> > >>>>> system that requires this system property to be set, so not sure
> how
> > >>> much
> > >>>>> it matters.
> > >>>>
> > >>>> I don't think other libraries' support for SecurityManager is
> relevant
> > >>> to
> > >>>> this discussion, as there almost certainly is a downstream project
> out
> > >>>> there for which Kafka is the only dependency blocking their upgrade.
> > >>>>
> > >>>
> > >>> I don't think this is how we evaluate risk/reward, so I disagree
> with your
> > >>> position here.
> > >>>
> > >>>
> > >>>> And if all libraries took that strategy, that would harm the
> adoption of
> > >>>> Java 23 and users, so we should avoid implementing that strategy
> > >>> ourselves.
> > >>>
> > >>>
> > >>> Requiring users to set a system property is far from "harming" them.
> > >>> Nonetheless, if we can reduce friction, it's a good thing,
> > >>>
> > >>> Ismael
> > >>>
> > >>
> >
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Chia-Ping Tsai <ch...@apache.org>]

hi all,

I believe this is an NP-hard challenge because we aim to release version 4.0 as quickly, stably, and effectively as possible. In the meantime, the progress of 4.0 increases the gap between 4.0 and 3.9. This means that backporting fixes and improvements to 3.9 will require additional resources from the community.

However, I can imagine that many Kafka users and customers will remain on version 3.x for a while. Therefore, it is worthwhile to keep releasing 3.9.x for bug fixes. For improvements or new features, we should keep them in 4.x to encourage users to advance alongside the community.

Best,
Chia-Ping

On 2024/12/02 21:42:23 Swikar Patel wrote:
> Java 23 is not LTS (Long Term Support) by Oracle
> Do we still want to proceed?
> 
> Thanks
> Swikar 
> 
> > On Dec 2, 2024, at 1:07 PM, Greg Harris <gr...@aiven.io.invalid> wrote:
> > 
> > Hi all,
> > 
> > Thanks for the discussion so far, but I would like to refocus on the
> > problem at hand.
> > 
> > Our downstream users are asking for things that come at a cost to us as
> > maintainers: An exception to our processes, better support for the final
> > bridge release, and an easier maintenance burden.
> > 
> > What is our consensus here? Should we disregard our users, and prioritize
> > ourselves first?
> > 
> > Thanks,
> > Greg
> > 
> > 
> > 
> >> On Thu, Nov 21, 2024 at 12:52 PM Greg Harris <gr...@aiven.io> wrote:
> >> 
> >> Hi Ismael,
> >> 
> >> Thanks for your responses.
> >> 
> >>> At the same time, where do we draw the line when it
> >>> comes to future unreleased Java versions?
> >> 
> >> I don't think we need to make a determination on that at this time. We are
> >> making a one-time judgement about backporting a specific patch to a
> >> specific branch. We should have that discussion as a follow-up.
> >> 
> >>> Is there evidence of this pain? That would help motivate the case.
> >> 
> >> A downstream user has already replied to this thread, I'll relink it here
> >> [1] and the issue they were working on [2].
> >> 
> >>> I don't think this is how we evaluate risk/reward, so I disagree with
> >> your
> >>> position here.
> >>> Requiring users to set a system property is far from "harming" them.
> >>> Nonetheless, if we can reduce friction, it's a good thing,
> >> 
> >> I think you missed the point of my statement, which was that we can't
> >> justify our behavior by hiding among other libraries.
> >> If everyone did that, nobody would ever respond to deprecations. This is
> >> an antisocial justification, I don't find it convincing, and we should not
> >> rely on it.
> >> 
> >> Thanks,
> >> Greg
> >> 
> >> [1] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> >> [2] https://github.com/trinodb/trino/issues/23498
> >> 
> >>> On Thu, Nov 21, 2024 at 12:16 PM Ismael Juma <me...@ismaeljuma.com> wrote:
> >>> 
> >>> Hi Greg,
> >>> 
> >>> Comments below.
> >>> 
> >>> On Thu, Nov 21, 2024 at 8:07 AM Greg Harris <greg.harris@aiven.io.invalid
> >>>> 
> >>> wrote:
> >>> 
> >>>>> Greg, why can't we set the relevant system property
> >>>>> automatically for the broker/connect in 3.9.x?
> >>>> 
> >>>> The System property workaround is not effective for Java 24
> >>> 
> >>> 
> >>> This is a fair point. At the same time, where do we draw the line when it
> >>> comes to future unreleased Java versions? There are two possible paths:
> >>> (1)
> >>> we focus on the released versions, (2) we have a clear policy for future
> >>> versions (the upcoming version, the upcoming LTS version, etc.).
> >>> 
> >>> 
> >>>> and doesn't
> >>>> cover the kafka-as-a-library use-case, which is the one which is
> >>> generating
> >>>> pain for our users.
> >>>> 
> >>> 
> >>> Is there evidence of this pain? That would help motivate the case.
> >>> 
> >>> 
> >>>> We already have a patch reviewed and merged which can fix this generally
> >>>> (23/24, clients/streams/brokers/connect), and is low risk. Please review
> >>>> the PR to convince yourself of this.
> >>>> 
> >>> 
> >>> I did take a look at the PR and it did not seem low risk to me (it's
> >>> larger
> >>> than we'd typically consider for bug fix releases, it includes reflection,
> >>> etc.). And hence why I was exploring other options. But it's not
> >>> necessarily high risk either, it's somewhere in between.
> >>> 
> >>>> When it comes to usage of
> >>>>> Kafka as a library (eg clients), I very much doubt kafka will be the
> >>> only
> >>>>> system that requires this system property to be set, so not sure how
> >>> much
> >>>>> it matters.
> >>>> 
> >>>> I don't think other libraries' support for SecurityManager is relevant
> >>> to
> >>>> this discussion, as there almost certainly is a downstream project out
> >>>> there for which Kafka is the only dependency blocking their upgrade.
> >>>> 
> >>> 
> >>> I don't think this is how we evaluate risk/reward, so I disagree with your
> >>> position here.
> >>> 
> >>> 
> >>>> And if all libraries took that strategy, that would harm the adoption of
> >>>> Java 23 and users, so we should avoid implementing that strategy
> >>> ourselves.
> >>> 
> >>> 
> >>> Requiring users to set a system property is far from "harming" them.
> >>> Nonetheless, if we can reduce friction, it's a good thing,
> >>> 
> >>> Ismael
> >>> 
> >> 
> 

Re: [DISCUSS] Java 23 Support for 3.9.x

[Swikar Patel <sw...@gmail.com>]

Java 23 is not LTS (Long Term Support) by Oracle
Do we still want to proceed?

Thanks
Swikar 

> On Dec 2, 2024, at 1:07 PM, Greg Harris <gr...@aiven.io.invalid> wrote:
> 
> Hi all,
> 
> Thanks for the discussion so far, but I would like to refocus on the
> problem at hand.
> 
> Our downstream users are asking for things that come at a cost to us as
> maintainers: An exception to our processes, better support for the final
> bridge release, and an easier maintenance burden.
> 
> What is our consensus here? Should we disregard our users, and prioritize
> ourselves first?
> 
> Thanks,
> Greg
> 
> 
> 
>> On Thu, Nov 21, 2024 at 12:52 PM Greg Harris <gr...@aiven.io> wrote:
>> 
>> Hi Ismael,
>> 
>> Thanks for your responses.
>> 
>>> At the same time, where do we draw the line when it
>>> comes to future unreleased Java versions?
>> 
>> I don't think we need to make a determination on that at this time. We are
>> making a one-time judgement about backporting a specific patch to a
>> specific branch. We should have that discussion as a follow-up.
>> 
>>> Is there evidence of this pain? That would help motivate the case.
>> 
>> A downstream user has already replied to this thread, I'll relink it here
>> [1] and the issue they were working on [2].
>> 
>>> I don't think this is how we evaluate risk/reward, so I disagree with
>> your
>>> position here.
>>> Requiring users to set a system property is far from "harming" them.
>>> Nonetheless, if we can reduce friction, it's a good thing,
>> 
>> I think you missed the point of my statement, which was that we can't
>> justify our behavior by hiding among other libraries.
>> If everyone did that, nobody would ever respond to deprecations. This is
>> an antisocial justification, I don't find it convincing, and we should not
>> rely on it.
>> 
>> Thanks,
>> Greg
>> 
>> [1] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
>> [2] https://github.com/trinodb/trino/issues/23498
>> 
>>> On Thu, Nov 21, 2024 at 12:16 PM Ismael Juma <me...@ismaeljuma.com> wrote:
>>> 
>>> Hi Greg,
>>> 
>>> Comments below.
>>> 
>>> On Thu, Nov 21, 2024 at 8:07 AM Greg Harris <greg.harris@aiven.io.invalid
>>>> 
>>> wrote:
>>> 
>>>>> Greg, why can't we set the relevant system property
>>>>> automatically for the broker/connect in 3.9.x?
>>>> 
>>>> The System property workaround is not effective for Java 24
>>> 
>>> 
>>> This is a fair point. At the same time, where do we draw the line when it
>>> comes to future unreleased Java versions? There are two possible paths:
>>> (1)
>>> we focus on the released versions, (2) we have a clear policy for future
>>> versions (the upcoming version, the upcoming LTS version, etc.).
>>> 
>>> 
>>>> and doesn't
>>>> cover the kafka-as-a-library use-case, which is the one which is
>>> generating
>>>> pain for our users.
>>>> 
>>> 
>>> Is there evidence of this pain? That would help motivate the case.
>>> 
>>> 
>>>> We already have a patch reviewed and merged which can fix this generally
>>>> (23/24, clients/streams/brokers/connect), and is low risk. Please review
>>>> the PR to convince yourself of this.
>>>> 
>>> 
>>> I did take a look at the PR and it did not seem low risk to me (it's
>>> larger
>>> than we'd typically consider for bug fix releases, it includes reflection,
>>> etc.). And hence why I was exploring other options. But it's not
>>> necessarily high risk either, it's somewhere in between.
>>> 
>>>> When it comes to usage of
>>>>> Kafka as a library (eg clients), I very much doubt kafka will be the
>>> only
>>>>> system that requires this system property to be set, so not sure how
>>> much
>>>>> it matters.
>>>> 
>>>> I don't think other libraries' support for SecurityManager is relevant
>>> to
>>>> this discussion, as there almost certainly is a downstream project out
>>>> there for which Kafka is the only dependency blocking their upgrade.
>>>> 
>>> 
>>> I don't think this is how we evaluate risk/reward, so I disagree with your
>>> position here.
>>> 
>>> 
>>>> And if all libraries took that strategy, that would harm the adoption of
>>>> Java 23 and users, so we should avoid implementing that strategy
>>> ourselves.
>>> 
>>> 
>>> Requiring users to set a system property is far from "harming" them.
>>> Nonetheless, if we can reduce friction, it's a good thing,
>>> 
>>> Ismael
>>> 
>> 

Re: [DISCUSS] Java 23 Support for 3.9.x

[Greg Harris <gr...@aiven.io.INVALID>]

Hi all,

Thanks for the discussion so far, but I would like to refocus on the
problem at hand.

Our downstream users are asking for things that come at a cost to us as
maintainers: An exception to our processes, better support for the final
bridge release, and an easier maintenance burden.

What is our consensus here? Should we disregard our users, and prioritize
ourselves first?

Thanks,
Greg



On Thu, Nov 21, 2024 at 12:52 PM Greg Harris <gr...@aiven.io> wrote:

> Hi Ismael,
>
> Thanks for your responses.
>
> > At the same time, where do we draw the line when it
> > comes to future unreleased Java versions?
>
> I don't think we need to make a determination on that at this time. We are
> making a one-time judgement about backporting a specific patch to a
> specific branch. We should have that discussion as a follow-up.
>
> > Is there evidence of this pain? That would help motivate the case.
>
> A downstream user has already replied to this thread, I'll relink it here
> [1] and the issue they were working on [2].
>
> > I don't think this is how we evaluate risk/reward, so I disagree with
> your
> > position here.
> > Requiring users to set a system property is far from "harming" them.
> > Nonetheless, if we can reduce friction, it's a good thing,
>
> I think you missed the point of my statement, which was that we can't
> justify our behavior by hiding among other libraries.
> If everyone did that, nobody would ever respond to deprecations. This is
> an antisocial justification, I don't find it convincing, and we should not
> rely on it.
>
> Thanks,
> Greg
>
> [1] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> [2] https://github.com/trinodb/trino/issues/23498
>
> On Thu, Nov 21, 2024 at 12:16 PM Ismael Juma <me...@ismaeljuma.com> wrote:
>
>> Hi Greg,
>>
>> Comments below.
>>
>> On Thu, Nov 21, 2024 at 8:07 AM Greg Harris <greg.harris@aiven.io.invalid
>> >
>> wrote:
>>
>> > > Greg, why can't we set the relevant system property
>> > > automatically for the broker/connect in 3.9.x?
>> >
>> > The System property workaround is not effective for Java 24
>>
>>
>> This is a fair point. At the same time, where do we draw the line when it
>> comes to future unreleased Java versions? There are two possible paths:
>> (1)
>> we focus on the released versions, (2) we have a clear policy for future
>> versions (the upcoming version, the upcoming LTS version, etc.).
>>
>>
>> > and doesn't
>> > cover the kafka-as-a-library use-case, which is the one which is
>> generating
>> > pain for our users.
>> >
>>
>> Is there evidence of this pain? That would help motivate the case.
>>
>>
>> > We already have a patch reviewed and merged which can fix this generally
>> > (23/24, clients/streams/brokers/connect), and is low risk. Please review
>> > the PR to convince yourself of this.
>> >
>>
>> I did take a look at the PR and it did not seem low risk to me (it's
>> larger
>> than we'd typically consider for bug fix releases, it includes reflection,
>> etc.). And hence why I was exploring other options. But it's not
>> necessarily high risk either, it's somewhere in between.
>>
>> > When it comes to usage of
>> > > Kafka as a library (eg clients), I very much doubt kafka will be the
>> only
>> > > system that requires this system property to be set, so not sure how
>> much
>> > > it matters.
>> >
>> > I don't think other libraries' support for SecurityManager is relevant
>> to
>> > this discussion, as there almost certainly is a downstream project out
>> > there for which Kafka is the only dependency blocking their upgrade.
>> >
>>
>> I don't think this is how we evaluate risk/reward, so I disagree with your
>> position here.
>>
>>
>> > And if all libraries took that strategy, that would harm the adoption of
>> > Java 23 and users, so we should avoid implementing that strategy
>> ourselves.
>>
>>
>> Requiring users to set a system property is far from "harming" them.
>> Nonetheless, if we can reduce friction, it's a good thing,
>>
>> Ismael
>>
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Greg Harris <gr...@aiven.io.INVALID>]

Hi Ismael,

Thanks for your responses.

> At the same time, where do we draw the line when it
> comes to future unreleased Java versions?

I don't think we need to make a determination on that at this time. We are
making a one-time judgement about backporting a specific patch to a
specific branch. We should have that discussion as a follow-up.

> Is there evidence of this pain? That would help motivate the case.

A downstream user has already replied to this thread, I'll relink it here
[1] and the issue they were working on [2].

> I don't think this is how we evaluate risk/reward, so I disagree with your
> position here.
> Requiring users to set a system property is far from "harming" them.
> Nonetheless, if we can reduce friction, it's a good thing,

I think you missed the point of my statement, which was that we can't
justify our behavior by hiding among other libraries.
If everyone did that, nobody would ever respond to deprecations. This is an
antisocial justification, I don't find it convincing, and we should not
rely on it.

Thanks,
Greg

[1] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
[2] https://github.com/trinodb/trino/issues/23498

On Thu, Nov 21, 2024 at 12:16 PM Ismael Juma <me...@ismaeljuma.com> wrote:

> Hi Greg,
>
> Comments below.
>
> On Thu, Nov 21, 2024 at 8:07 AM Greg Harris <gr...@aiven.io.invalid>
> wrote:
>
> > > Greg, why can't we set the relevant system property
> > > automatically for the broker/connect in 3.9.x?
> >
> > The System property workaround is not effective for Java 24
>
>
> This is a fair point. At the same time, where do we draw the line when it
> comes to future unreleased Java versions? There are two possible paths: (1)
> we focus on the released versions, (2) we have a clear policy for future
> versions (the upcoming version, the upcoming LTS version, etc.).
>
>
> > and doesn't
> > cover the kafka-as-a-library use-case, which is the one which is
> generating
> > pain for our users.
> >
>
> Is there evidence of this pain? That would help motivate the case.
>
>
> > We already have a patch reviewed and merged which can fix this generally
> > (23/24, clients/streams/brokers/connect), and is low risk. Please review
> > the PR to convince yourself of this.
> >
>
> I did take a look at the PR and it did not seem low risk to me (it's larger
> than we'd typically consider for bug fix releases, it includes reflection,
> etc.). And hence why I was exploring other options. But it's not
> necessarily high risk either, it's somewhere in between.
>
> > When it comes to usage of
> > > Kafka as a library (eg clients), I very much doubt kafka will be the
> only
> > > system that requires this system property to be set, so not sure how
> much
> > > it matters.
> >
> > I don't think other libraries' support for SecurityManager is relevant to
> > this discussion, as there almost certainly is a downstream project out
> > there for which Kafka is the only dependency blocking their upgrade.
> >
>
> I don't think this is how we evaluate risk/reward, so I disagree with your
> position here.
>
>
> > And if all libraries took that strategy, that would harm the adoption of
> > Java 23 and users, so we should avoid implementing that strategy
> ourselves.
>
>
> Requiring users to set a system property is far from "harming" them.
> Nonetheless, if we can reduce friction, it's a good thing,
>
> Ismael
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Ismael Juma <me...@ismaeljuma.com>]

Hi Greg,

Comments below.

On Thu, Nov 21, 2024 at 8:07 AM Greg Harris <gr...@aiven.io.invalid>
wrote:

> > Greg, why can't we set the relevant system property
> > automatically for the broker/connect in 3.9.x?
>
> The System property workaround is not effective for Java 24


This is a fair point. At the same time, where do we draw the line when it
comes to future unreleased Java versions? There are two possible paths: (1)
we focus on the released versions, (2) we have a clear policy for future
versions (the upcoming version, the upcoming LTS version, etc.).


> and doesn't
> cover the kafka-as-a-library use-case, which is the one which is generating
> pain for our users.
>

Is there evidence of this pain? That would help motivate the case.


> We already have a patch reviewed and merged which can fix this generally
> (23/24, clients/streams/brokers/connect), and is low risk. Please review
> the PR to convince yourself of this.
>

I did take a look at the PR and it did not seem low risk to me (it's larger
than we'd typically consider for bug fix releases, it includes reflection,
etc.). And hence why I was exploring other options. But it's not
necessarily high risk either, it's somewhere in between.

> When it comes to usage of
> > Kafka as a library (eg clients), I very much doubt kafka will be the only
> > system that requires this system property to be set, so not sure how much
> > it matters.
>
> I don't think other libraries' support for SecurityManager is relevant to
> this discussion, as there almost certainly is a downstream project out
> there for which Kafka is the only dependency blocking their upgrade.
>

I don't think this is how we evaluate risk/reward, so I disagree with your
position here.


> And if all libraries took that strategy, that would harm the adoption of
> Java 23 and users, so we should avoid implementing that strategy ourselves.


Requiring users to set a system property is far from "harming" them.
Nonetheless, if we can reduce friction, it's a good thing,

Ismael

Re: [DISCUSS] Java 23 Support for 3.9.x

[Greg Harris <gr...@aiven.io.INVALID>]

Hi Ismael,

Thanks for your question

> Greg, why can't we set the relevant system property
> automatically for the broker/connect in 3.9.x?

The System property workaround is not effective for Java 24, and doesn't
cover the kafka-as-a-library use-case, which is the one which is generating
pain for our users.
We already have a patch reviewed and merged which can fix this generally
(23/24, clients/streams/brokers/connect), and is low risk. Please review
the PR to convince yourself of this.

> When it comes to usage of
> Kafka as a library (eg clients), I very much doubt kafka will be the only
> system that requires this system property to be set, so not sure how much
> it matters.

I don't think other libraries' support for SecurityManager is relevant to
this discussion, as there almost certainly is a downstream project out
there for which Kafka is the only dependency blocking their upgrade.
And if all libraries took that strategy, that would harm the adoption of
Java 23 and users, so we should avoid implementing that strategy ourselves.

Thanks,
Greg


On Thu, Nov 21, 2024 at 7:31 AM Ismael Juma <me...@ismaeljuma.com> wrote:

> We had agreed to extend _critical bug fixes_ for one year for 3.9.
> Something beyond that would require its own separate discuss thread, it
> should not be in the middle of a thread about Java 23 support in 3.9. I am
> personally not in favor of a LTS or 18-24 months cycle - it would divert a
> lot of resources from the community (vendors are welcome to do that, of
> course).
>
> With regards to having Java 23 work in 3.9.x, that doesn't seem
> unreasonable if we can do so via a low risk change (especially since 3.9.0
> was just released). Greg, why can't we set the relevant system property
> automatically for the broker/connect in 3.9.x? When it comes to usage of
> Kafka as a library (eg clients), I very much doubt kafka will be the only
> system that requires this system property to be set, so not sure how much
> it matters.
>
> Ismael
>
> On Thu, Nov 21, 2024 at 1:51 AM Josep Prat <jo...@aiven.io.invalid>
> wrote:
>
> > Hi all,
> > Given 3.9 is the latest version before a major, and 4.0.0 comes with
> > breaking changes (Zookeeper removal, Scala and Java version drops...) I
> > think it's fair to assume that a considerable subset of Kafka users will
> be
> > using 3.9 for a little longer than we usually support versions (roughly 1
> > year). I would be in favour of considering 3.9 as and LTS and offer
> support
> > for, let's say 18 to 24 months.
> > In that spirit, I would be in favour of releasing a future 3.9 version
> that
> > can run with JDK 23.
> >
> > Best,
> >
> > On Thu, Nov 21, 2024 at 3:46 AM Chia-Ping Tsai <ch...@gmail.com>
> wrote:
> >
> > > Dear all,
> > >
> > > It seems the main question is whether we should consider version 3.9 as
> > an
> > > LTS release. For example, will we continue with versions like 3.9.1,
> > 3.9.2,
> > > ... 3.9.100?
> > >
> > > If yes, we should backport both KIP-1006 and support for future JDKs.
> > >
> > > If not, backporting KIP-1006 to 3.9 would be sufficient to fix the
> issue
> > of
> > > running version 3.9 under JDK 23, even if JDK 23 is still not
> officially
> > > supported by 3.9.
> > >
> > >
> > > Best,
> > >
> > > Chia-Ping
> > >
> > >
> > >
> > > Greg Harris <gr...@aiven.io.invalid> 於 2024年11月21日 週四 上午4:37寫道：
> > >
> > > > > Has the SecurityManager been fully removed in JDK 23?
> > > > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > > >
> > > > The SecurityManager has been degraded, so by default our users
> > experience
> > > > an UnsupportedOperationException. They can work-around this by
> setting
> > a
> > > > system property.
> > > > In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched
> > 3.9.x
> > > > will experience an UnsupportedOperationException unconditionally.
> > > >
> > > > > I see https://issues.apache.org/jira/browse/KAFKA-17638
> > > > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> > > Lack
> > > > of
> > > > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as
> > far
> > > > as
> > > > > I can tell).
> > > >
> > > > Originally we were planning to get this change into 3.9.0, but we
> > missed
> > > > the merge deadline. I opened that ticket afterwards to be fixed in
> > 4.0.0
> > > > because that's the next release.
> > > > The patch was always intended to be backportable, and I intended to
> > > > backport it [2].
> > > >
> > > > I understand that if we consider Java 23 support to be a feature
> (which
> > > is
> > > > the standing decision), this is a pretty obvious case of missing
> > feature
> > > > freeze, and the current course of action (releasing in 4.0.0) is how
> we
> > > > would handle it.
> > > > I'm asking for this to be reconsidered as a bug fix, because it
> allows
> > us
> > > > to backport the change, which is what our users are asking for [3].
> > > >
> > > > Thanks,
> > > > Greg
> > > >
> > > > [1] https://openjdk.org/jeps/486
> > > > [2]
> https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
> > > > [3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> > > >
> > > > On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com>
> > wrote:
> > > >
> > > > > Greg,
> > > > >
> > > > > I have not been following this closely, so apologies for some basic
> > > > > questions.
> > > > >
> > > > > Has the SecurityManager been fully removed in JDK 23?
> > > > >
> > > > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > > > >
> > > > > By "4.0 breaking changes" do you mean changes to our JDK/Scala
> > > supported
> > > > > versions, removal or ZK, Kafka API changes, or something else?
> > > > >
> > > > > In general, I do not think we should change our supported JDK
> > versions
> > > > in a
> > > > > hotfix release. I see
> > > https://issues.apache.org/jira/browse/KAFKA-17638
> > > > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> > > Lack
> > > > of
> > > > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as
> > far
> > > > as
> > > > > I can tell).
> > > > >
> > > > > Also, I feel that we should not add too much to 3.9.x aside from
> > actual
> > > > > bugs. If we backport things into 3.9.x, it will slow adoption of
> 4.x
> > > and
> > > > > increase our maintenance burden over time.
> > > > >
> > > > > Just my $0.02
> > > > >
> > > > > Thanks!
> > > > > David A
> > > > >
> > > > > On Wed, Nov 20, 2024 at 12:22 PM Greg Harris
> > > > <greg.harris@aiven.io.invalid
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> > > > > understand
> > > > > > everyone's expectations about the 3.9.x branch, and ask for a
> > > specific
> > > > > > consensus on Java 23 support.
> > > > > >
> > > > > > Some context that I think is relevant to the discussion:
> > > > > > * KIP-1006 [1] proposes a backwards-compatible strategy for
> > handling
> > > > the
> > > > > > ongoing removal of the SecurityManager, which is merged and due
> to
> > > > > release
> > > > > > in 4.0.0 [2].
> > > > > > * KIP-1012 [3] rejected ongoing parallel feature development on a
> > 3.x
> > > > > > branch while having trunk on 4.x.
> > > > > > * During the 3.9.0 release, the patch [2] was rejected [4] due to
> > > > being a
> > > > > > new feature which did not meet the feature freeze deadline.
> > > > > > * Other than the SecurityManager removal, there are additional
> PRs
> > > > which
> > > > > > would also need to be backported for full Java 23 support [5]
> > > > including a
> > > > > > Scala patch upgrade.
> > > > > > * Downstream users are asking for a backport [6] because adding
> > > support
> > > > > for
> > > > > > Java 23 would obligate them to also include the 4.0 breaking
> > changes.
> > > > > >
> > > > > > So while adding Java version support in the past has been a
> > KIP-less
> > > > > > feature and normally only appears in the next version, it happens
> > to
> > > > > align
> > > > > > with a major version bump this time. This will cause additional
> > pain
> > > > for
> > > > > > users if we do not elect to backport this.
> > > > > >
> > > > > > Thanks,
> > > > > > Greg
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > > > > > [2] https://github.com/apache/kafka/pull/16522
> > > > > > [3]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > > > > > [4]
> > https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > > > > > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > > > > > [6]
> > > https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > David Arthur
> > > > >
> > > >
> > >
> >
> >
> > --
> > [image: Aiven] <https://www.aiven.io>
> >
> > *Josep Prat*
> > Open Source Engineering Director, *Aiven*
> > josep.prat@aiven.io   |   +491715557497
> > aiven.io <https://www.aiven.io>   |   <
> https://www.facebook.com/aivencloud
> > >
> >   <https://www.linkedin.com/company/aiven/>   <
> > https://twitter.com/aiven_io>
> > *Aiven Deutschland GmbH*
> > Alexanderufer 3-7, 10117 Berlin
> > Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> > Anna Richardson, Kenneth Chen
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Ismael Juma <me...@ismaeljuma.com>]

We had agreed to extend _critical bug fixes_ for one year for 3.9.
Something beyond that would require its own separate discuss thread, it
should not be in the middle of a thread about Java 23 support in 3.9. I am
personally not in favor of a LTS or 18-24 months cycle - it would divert a
lot of resources from the community (vendors are welcome to do that, of
course).

With regards to having Java 23 work in 3.9.x, that doesn't seem
unreasonable if we can do so via a low risk change (especially since 3.9.0
was just released). Greg, why can't we set the relevant system property
automatically for the broker/connect in 3.9.x? When it comes to usage of
Kafka as a library (eg clients), I very much doubt kafka will be the only
system that requires this system property to be set, so not sure how much
it matters.

Ismael

On Thu, Nov 21, 2024 at 1:51 AM Josep Prat <jo...@aiven.io.invalid>
wrote:

> Hi all,
> Given 3.9 is the latest version before a major, and 4.0.0 comes with
> breaking changes (Zookeeper removal, Scala and Java version drops...) I
> think it's fair to assume that a considerable subset of Kafka users will be
> using 3.9 for a little longer than we usually support versions (roughly 1
> year). I would be in favour of considering 3.9 as and LTS and offer support
> for, let's say 18 to 24 months.
> In that spirit, I would be in favour of releasing a future 3.9 version that
> can run with JDK 23.
>
> Best,
>
> On Thu, Nov 21, 2024 at 3:46 AM Chia-Ping Tsai <ch...@gmail.com> wrote:
>
> > Dear all,
> >
> > It seems the main question is whether we should consider version 3.9 as
> an
> > LTS release. For example, will we continue with versions like 3.9.1,
> 3.9.2,
> > ... 3.9.100?
> >
> > If yes, we should backport both KIP-1006 and support for future JDKs.
> >
> > If not, backporting KIP-1006 to 3.9 would be sufficient to fix the issue
> of
> > running version 3.9 under JDK 23, even if JDK 23 is still not officially
> > supported by 3.9.
> >
> >
> > Best,
> >
> > Chia-Ping
> >
> >
> >
> > Greg Harris <gr...@aiven.io.invalid> 於 2024年11月21日 週四 上午4:37寫道：
> >
> > > > Has the SecurityManager been fully removed in JDK 23?
> > > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > >
> > > The SecurityManager has been degraded, so by default our users
> experience
> > > an UnsupportedOperationException. They can work-around this by setting
> a
> > > system property.
> > > In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched
> 3.9.x
> > > will experience an UnsupportedOperationException unconditionally.
> > >
> > > > I see https://issues.apache.org/jira/browse/KAFKA-17638
> > > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> > Lack
> > > of
> > > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as
> far
> > > as
> > > > I can tell).
> > >
> > > Originally we were planning to get this change into 3.9.0, but we
> missed
> > > the merge deadline. I opened that ticket afterwards to be fixed in
> 4.0.0
> > > because that's the next release.
> > > The patch was always intended to be backportable, and I intended to
> > > backport it [2].
> > >
> > > I understand that if we consider Java 23 support to be a feature (which
> > is
> > > the standing decision), this is a pretty obvious case of missing
> feature
> > > freeze, and the current course of action (releasing in 4.0.0) is how we
> > > would handle it.
> > > I'm asking for this to be reconsidered as a bug fix, because it allows
> us
> > > to backport the change, which is what our users are asking for [3].
> > >
> > > Thanks,
> > > Greg
> > >
> > > [1] https://openjdk.org/jeps/486
> > > [2] https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
> > > [3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> > >
> > > On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com>
> wrote:
> > >
> > > > Greg,
> > > >
> > > > I have not been following this closely, so apologies for some basic
> > > > questions.
> > > >
> > > > Has the SecurityManager been fully removed in JDK 23?
> > > >
> > > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > > >
> > > > By "4.0 breaking changes" do you mean changes to our JDK/Scala
> > supported
> > > > versions, removal or ZK, Kafka API changes, or something else?
> > > >
> > > > In general, I do not think we should change our supported JDK
> versions
> > > in a
> > > > hotfix release. I see
> > https://issues.apache.org/jira/browse/KAFKA-17638
> > > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> > Lack
> > > of
> > > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as
> far
> > > as
> > > > I can tell).
> > > >
> > > > Also, I feel that we should not add too much to 3.9.x aside from
> actual
> > > > bugs. If we backport things into 3.9.x, it will slow adoption of 4.x
> > and
> > > > increase our maintenance burden over time.
> > > >
> > > > Just my $0.02
> > > >
> > > > Thanks!
> > > > David A
> > > >
> > > > On Wed, Nov 20, 2024 at 12:22 PM Greg Harris
> > > <greg.harris@aiven.io.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> > > > understand
> > > > > everyone's expectations about the 3.9.x branch, and ask for a
> > specific
> > > > > consensus on Java 23 support.
> > > > >
> > > > > Some context that I think is relevant to the discussion:
> > > > > * KIP-1006 [1] proposes a backwards-compatible strategy for
> handling
> > > the
> > > > > ongoing removal of the SecurityManager, which is merged and due to
> > > > release
> > > > > in 4.0.0 [2].
> > > > > * KIP-1012 [3] rejected ongoing parallel feature development on a
> 3.x
> > > > > branch while having trunk on 4.x.
> > > > > * During the 3.9.0 release, the patch [2] was rejected [4] due to
> > > being a
> > > > > new feature which did not meet the feature freeze deadline.
> > > > > * Other than the SecurityManager removal, there are additional PRs
> > > which
> > > > > would also need to be backported for full Java 23 support [5]
> > > including a
> > > > > Scala patch upgrade.
> > > > > * Downstream users are asking for a backport [6] because adding
> > support
> > > > for
> > > > > Java 23 would obligate them to also include the 4.0 breaking
> changes.
> > > > >
> > > > > So while adding Java version support in the past has been a
> KIP-less
> > > > > feature and normally only appears in the next version, it happens
> to
> > > > align
> > > > > with a major version bump this time. This will cause additional
> pain
> > > for
> > > > > users if we do not elect to backport this.
> > > > >
> > > > > Thanks,
> > > > > Greg
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > > > > [2] https://github.com/apache/kafka/pull/16522
> > > > > [3]
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > > > > [4]
> https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > > > > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > > > > [6]
> > https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> > > > >
> > > >
> > > >
> > > > --
> > > > David Arthur
> > > >
> > >
> >
>
>
> --
> [image: Aiven] <https://www.aiven.io>
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.prat@aiven.io   |   +491715557497
> aiven.io <https://www.aiven.io>   |   <https://www.facebook.com/aivencloud
> >
>   <https://www.linkedin.com/company/aiven/>   <
> https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> Anna Richardson, Kenneth Chen
> Amtsgericht Charlottenburg, HRB 209739 B
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Josep Prat <jo...@aiven.io.INVALID>]

Hi all,
Given 3.9 is the latest version before a major, and 4.0.0 comes with
breaking changes (Zookeeper removal, Scala and Java version drops...) I
think it's fair to assume that a considerable subset of Kafka users will be
using 3.9 for a little longer than we usually support versions (roughly 1
year). I would be in favour of considering 3.9 as and LTS and offer support
for, let's say 18 to 24 months.
In that spirit, I would be in favour of releasing a future 3.9 version that
can run with JDK 23.

Best,

On Thu, Nov 21, 2024 at 3:46 AM Chia-Ping Tsai <ch...@gmail.com> wrote:

> Dear all,
>
> It seems the main question is whether we should consider version 3.9 as an
> LTS release. For example, will we continue with versions like 3.9.1, 3.9.2,
> ... 3.9.100?
>
> If yes, we should backport both KIP-1006 and support for future JDKs.
>
> If not, backporting KIP-1006 to 3.9 would be sufficient to fix the issue of
> running version 3.9 under JDK 23, even if JDK 23 is still not officially
> supported by 3.9.
>
>
> Best,
>
> Chia-Ping
>
>
>
> Greg Harris <gr...@aiven.io.invalid> 於 2024年11月21日 週四 上午4:37寫道：
>
> > > Has the SecurityManager been fully removed in JDK 23?
> > > What is the effect of running Kafka 3.9.0 with JDK 23?
> >
> > The SecurityManager has been degraded, so by default our users experience
> > an UnsupportedOperationException. They can work-around this by setting a
> > system property.
> > In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched 3.9.x
> > will experience an UnsupportedOperationException unconditionally.
> >
> > > I see https://issues.apache.org/jira/browse/KAFKA-17638
> > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> Lack
> > of
> > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far
> > as
> > > I can tell).
> >
> > Originally we were planning to get this change into 3.9.0, but we missed
> > the merge deadline. I opened that ticket afterwards to be fixed in 4.0.0
> > because that's the next release.
> > The patch was always intended to be backportable, and I intended to
> > backport it [2].
> >
> > I understand that if we consider Java 23 support to be a feature (which
> is
> > the standing decision), this is a pretty obvious case of missing feature
> > freeze, and the current course of action (releasing in 4.0.0) is how we
> > would handle it.
> > I'm asking for this to be reconsidered as a bug fix, because it allows us
> > to backport the change, which is what our users are asking for [3].
> >
> > Thanks,
> > Greg
> >
> > [1] https://openjdk.org/jeps/486
> > [2] https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
> > [3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> >
> > On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com> wrote:
> >
> > > Greg,
> > >
> > > I have not been following this closely, so apologies for some basic
> > > questions.
> > >
> > > Has the SecurityManager been fully removed in JDK 23?
> > >
> > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > >
> > > By "4.0 breaking changes" do you mean changes to our JDK/Scala
> supported
> > > versions, removal or ZK, Kafka API changes, or something else?
> > >
> > > In general, I do not think we should change our supported JDK versions
> > in a
> > > hotfix release. I see
> https://issues.apache.org/jira/browse/KAFKA-17638
> > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> Lack
> > of
> > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far
> > as
> > > I can tell).
> > >
> > > Also, I feel that we should not add too much to 3.9.x aside from actual
> > > bugs. If we backport things into 3.9.x, it will slow adoption of 4.x
> and
> > > increase our maintenance burden over time.
> > >
> > > Just my $0.02
> > >
> > > Thanks!
> > > David A
> > >
> > > On Wed, Nov 20, 2024 at 12:22 PM Greg Harris
> > <greg.harris@aiven.io.invalid
> > > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> > > understand
> > > > everyone's expectations about the 3.9.x branch, and ask for a
> specific
> > > > consensus on Java 23 support.
> > > >
> > > > Some context that I think is relevant to the discussion:
> > > > * KIP-1006 [1] proposes a backwards-compatible strategy for handling
> > the
> > > > ongoing removal of the SecurityManager, which is merged and due to
> > > release
> > > > in 4.0.0 [2].
> > > > * KIP-1012 [3] rejected ongoing parallel feature development on a 3.x
> > > > branch while having trunk on 4.x.
> > > > * During the 3.9.0 release, the patch [2] was rejected [4] due to
> > being a
> > > > new feature which did not meet the feature freeze deadline.
> > > > * Other than the SecurityManager removal, there are additional PRs
> > which
> > > > would also need to be backported for full Java 23 support [5]
> > including a
> > > > Scala patch upgrade.
> > > > * Downstream users are asking for a backport [6] because adding
> support
> > > for
> > > > Java 23 would obligate them to also include the 4.0 breaking changes.
> > > >
> > > > So while adding Java version support in the past has been a KIP-less
> > > > feature and normally only appears in the next version, it happens to
> > > align
> > > > with a major version bump this time. This will cause additional pain
> > for
> > > > users if we do not elect to backport this.
> > > >
> > > > Thanks,
> > > > Greg
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > > > [2] https://github.com/apache/kafka/pull/16522
> > > > [3]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > > > [4] https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > > > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > > > [6]
> https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> > > >
> > >
> > >
> > > --
> > > David Arthur
> > >
> >
>


-- 
[image: Aiven] <https://www.aiven.io>

*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.prat@aiven.io   |   +491715557497
aiven.io <https://www.aiven.io>   |   <https://www.facebook.com/aivencloud>
  <https://www.linkedin.com/company/aiven/>   <https://twitter.com/aiven_io>
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
Anna Richardson, Kenneth Chen
Amtsgericht Charlottenburg, HRB 209739 B

Re: [DISCUSS] Java 23 Support for 3.9.x

[Josep Prat <jo...@aiven.io.INVALID>]

Hi all,
Given 3.9 is the latest version before a major, and 4.0.0 comes with
breaking changes (Zookeeper removal, Scala and Java version drops...) I
think it's fair to assume that a considerable subset of Kafka users will be
using 3.9 for a little longer than we usually support versions (roughly 1
year). I would be in favour of considering 3.9 as and LTS and offer support
for, let's say 18 to 24 months.
In that spirit, I would be in favour of releasing a future 3.9 version that
can run with JDK 23.

Best,

On Thu, Nov 21, 2024 at 3:46 AM Chia-Ping Tsai <ch...@gmail.com> wrote:

> Dear all,
>
> It seems the main question is whether we should consider version 3.9 as an
> LTS release. For example, will we continue with versions like 3.9.1, 3.9.2,
> ... 3.9.100?
>
> If yes, we should backport both KIP-1006 and support for future JDKs.
>
> If not, backporting KIP-1006 to 3.9 would be sufficient to fix the issue of
> running version 3.9 under JDK 23, even if JDK 23 is still not officially
> supported by 3.9.
>
>
> Best,
>
> Chia-Ping
>
>
>
> Greg Harris <gr...@aiven.io.invalid> 於 2024年11月21日 週四 上午4:37寫道：
>
> > > Has the SecurityManager been fully removed in JDK 23?
> > > What is the effect of running Kafka 3.9.0 with JDK 23?
> >
> > The SecurityManager has been degraded, so by default our users experience
> > an UnsupportedOperationException. They can work-around this by setting a
> > system property.
> > In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched 3.9.x
> > will experience an UnsupportedOperationException unconditionally.
> >
> > > I see https://issues.apache.org/jira/browse/KAFKA-17638
> > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> Lack
> > of
> > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far
> > as
> > > I can tell).
> >
> > Originally we were planning to get this change into 3.9.0, but we missed
> > the merge deadline. I opened that ticket afterwards to be fixed in 4.0.0
> > because that's the next release.
> > The patch was always intended to be backportable, and I intended to
> > backport it [2].
> >
> > I understand that if we consider Java 23 support to be a feature (which
> is
> > the standing decision), this is a pretty obvious case of missing feature
> > freeze, and the current course of action (releasing in 4.0.0) is how we
> > would handle it.
> > I'm asking for this to be reconsidered as a bug fix, because it allows us
> > to backport the change, which is what our users are asking for [3].
> >
> > Thanks,
> > Greg
> >
> > [1] https://openjdk.org/jeps/486
> > [2] https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
> > [3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
> >
> > On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com> wrote:
> >
> > > Greg,
> > >
> > > I have not been following this closely, so apologies for some basic
> > > questions.
> > >
> > > Has the SecurityManager been fully removed in JDK 23?
> > >
> > > What is the effect of running Kafka 3.9.0 with JDK 23?
> > >
> > > By "4.0 breaking changes" do you mean changes to our JDK/Scala
> supported
> > > versions, removal or ZK, Kafka API changes, or something else?
> > >
> > > In general, I do not think we should change our supported JDK versions
> > in a
> > > hotfix release. I see
> https://issues.apache.org/jira/browse/KAFKA-17638
> > > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0.
> Lack
> > of
> > > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far
> > as
> > > I can tell).
> > >
> > > Also, I feel that we should not add too much to 3.9.x aside from actual
> > > bugs. If we backport things into 3.9.x, it will slow adoption of 4.x
> and
> > > increase our maintenance burden over time.
> > >
> > > Just my $0.02
> > >
> > > Thanks!
> > > David A
> > >
> > > On Wed, Nov 20, 2024 at 12:22 PM Greg Harris
> > <greg.harris@aiven.io.invalid
> > > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> > > understand
> > > > everyone's expectations about the 3.9.x branch, and ask for a
> specific
> > > > consensus on Java 23 support.
> > > >
> > > > Some context that I think is relevant to the discussion:
> > > > * KIP-1006 [1] proposes a backwards-compatible strategy for handling
> > the
> > > > ongoing removal of the SecurityManager, which is merged and due to
> > > release
> > > > in 4.0.0 [2].
> > > > * KIP-1012 [3] rejected ongoing parallel feature development on a 3.x
> > > > branch while having trunk on 4.x.
> > > > * During the 3.9.0 release, the patch [2] was rejected [4] due to
> > being a
> > > > new feature which did not meet the feature freeze deadline.
> > > > * Other than the SecurityManager removal, there are additional PRs
> > which
> > > > would also need to be backported for full Java 23 support [5]
> > including a
> > > > Scala patch upgrade.
> > > > * Downstream users are asking for a backport [6] because adding
> support
> > > for
> > > > Java 23 would obligate them to also include the 4.0 breaking changes.
> > > >
> > > > So while adding Java version support in the past has been a KIP-less
> > > > feature and normally only appears in the next version, it happens to
> > > align
> > > > with a major version bump this time. This will cause additional pain
> > for
> > > > users if we do not elect to backport this.
> > > >
> > > > Thanks,
> > > > Greg
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > > > [2] https://github.com/apache/kafka/pull/16522
> > > > [3]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > > > [4] https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > > > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > > > [6]
> https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> > > >
> > >
> > >
> > > --
> > > David Arthur
> > >
> >
>


-- 
[image: Aiven] <https://www.aiven.io>

*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.prat@aiven.io   |   +491715557497
aiven.io <https://www.aiven.io>   |   <https://www.facebook.com/aivencloud>
  <https://www.linkedin.com/company/aiven/>   <https://twitter.com/aiven_io>
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
Anna Richardson, Kenneth Chen
Amtsgericht Charlottenburg, HRB 209739 B

Re: [DISCUSS] Java 23 Support for 3.9.x

[Chia-Ping Tsai <ch...@gmail.com>]

Dear all,

It seems the main question is whether we should consider version 3.9 as an
LTS release. For example, will we continue with versions like 3.9.1, 3.9.2,
... 3.9.100?

If yes, we should backport both KIP-1006 and support for future JDKs.

If not, backporting KIP-1006 to 3.9 would be sufficient to fix the issue of
running version 3.9 under JDK 23, even if JDK 23 is still not officially
supported by 3.9.


Best,

Chia-Ping



Greg Harris <gr...@aiven.io.invalid> 於 2024年11月21日 週四 上午4:37寫道：

> > Has the SecurityManager been fully removed in JDK 23?
> > What is the effect of running Kafka 3.9.0 with JDK 23?
>
> The SecurityManager has been degraded, so by default our users experience
> an UnsupportedOperationException. They can work-around this by setting a
> system property.
> In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched 3.9.x
> will experience an UnsupportedOperationException unconditionally.
>
> > I see https://issues.apache.org/jira/browse/KAFKA-17638
> > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack
> of
> > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far
> as
> > I can tell).
>
> Originally we were planning to get this change into 3.9.0, but we missed
> the merge deadline. I opened that ticket afterwards to be fixed in 4.0.0
> because that's the next release.
> The patch was always intended to be backportable, and I intended to
> backport it [2].
>
> I understand that if we consider Java 23 support to be a feature (which is
> the standing decision), this is a pretty obvious case of missing feature
> freeze, and the current course of action (releasing in 4.0.0) is how we
> would handle it.
> I'm asking for this to be reconsidered as a bug fix, because it allows us
> to backport the change, which is what our users are asking for [3].
>
> Thanks,
> Greg
>
> [1] https://openjdk.org/jeps/486
> [2] https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
> [3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
>
> On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com> wrote:
>
> > Greg,
> >
> > I have not been following this closely, so apologies for some basic
> > questions.
> >
> > Has the SecurityManager been fully removed in JDK 23?
> >
> > What is the effect of running Kafka 3.9.0 with JDK 23?
> >
> > By "4.0 breaking changes" do you mean changes to our JDK/Scala supported
> > versions, removal or ZK, Kafka API changes, or something else?
> >
> > In general, I do not think we should change our supported JDK versions
> in a
> > hotfix release. I see https://issues.apache.org/jira/browse/KAFKA-17638
> > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack
> of
> > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far
> as
> > I can tell).
> >
> > Also, I feel that we should not add too much to 3.9.x aside from actual
> > bugs. If we backport things into 3.9.x, it will slow adoption of 4.x and
> > increase our maintenance burden over time.
> >
> > Just my $0.02
> >
> > Thanks!
> > David A
> >
> > On Wed, Nov 20, 2024 at 12:22 PM Greg Harris
> <greg.harris@aiven.io.invalid
> > >
> > wrote:
> >
> > > Hi all,
> > >
> > > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> > understand
> > > everyone's expectations about the 3.9.x branch, and ask for a specific
> > > consensus on Java 23 support.
> > >
> > > Some context that I think is relevant to the discussion:
> > > * KIP-1006 [1] proposes a backwards-compatible strategy for handling
> the
> > > ongoing removal of the SecurityManager, which is merged and due to
> > release
> > > in 4.0.0 [2].
> > > * KIP-1012 [3] rejected ongoing parallel feature development on a 3.x
> > > branch while having trunk on 4.x.
> > > * During the 3.9.0 release, the patch [2] was rejected [4] due to
> being a
> > > new feature which did not meet the feature freeze deadline.
> > > * Other than the SecurityManager removal, there are additional PRs
> which
> > > would also need to be backported for full Java 23 support [5]
> including a
> > > Scala patch upgrade.
> > > * Downstream users are asking for a backport [6] because adding support
> > for
> > > Java 23 would obligate them to also include the 4.0 breaking changes.
> > >
> > > So while adding Java version support in the past has been a KIP-less
> > > feature and normally only appears in the next version, it happens to
> > align
> > > with a major version bump this time. This will cause additional pain
> for
> > > users if we do not elect to backport this.
> > >
> > > Thanks,
> > > Greg
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > > [2] https://github.com/apache/kafka/pull/16522
> > > [3]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > > [4] https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > > [6] https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> > >
> >
> >
> > --
> > David Arthur
> >
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Chia-Ping Tsai <ch...@gmail.com>]

Dear all,

It seems the main question is whether we should consider version 3.9 as an
LTS release. For example, will we continue with versions like 3.9.1, 3.9.2,
... 3.9.100?

If yes, we should backport both KIP-1006 and support for future JDKs.

If not, backporting KIP-1006 to 3.9 would be sufficient to fix the issue of
running version 3.9 under JDK 23, even if JDK 23 is still not officially
supported by 3.9.


Best,

Chia-Ping



Greg Harris <gr...@aiven.io.invalid> 於 2024年11月21日 週四 上午4:37寫道：

> > Has the SecurityManager been fully removed in JDK 23?
> > What is the effect of running Kafka 3.9.0 with JDK 23?
>
> The SecurityManager has been degraded, so by default our users experience
> an UnsupportedOperationException. They can work-around this by setting a
> system property.
> In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched 3.9.x
> will experience an UnsupportedOperationException unconditionally.
>
> > I see https://issues.apache.org/jira/browse/KAFKA-17638
> > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack
> of
> > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far
> as
> > I can tell).
>
> Originally we were planning to get this change into 3.9.0, but we missed
> the merge deadline. I opened that ticket afterwards to be fixed in 4.0.0
> because that's the next release.
> The patch was always intended to be backportable, and I intended to
> backport it [2].
>
> I understand that if we consider Java 23 support to be a feature (which is
> the standing decision), this is a pretty obvious case of missing feature
> freeze, and the current course of action (releasing in 4.0.0) is how we
> would handle it.
> I'm asking for this to be reconsidered as a bug fix, because it allows us
> to backport the change, which is what our users are asking for [3].
>
> Thanks,
> Greg
>
> [1] https://openjdk.org/jeps/486
> [2] https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
> [3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g
>
> On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com> wrote:
>
> > Greg,
> >
> > I have not been following this closely, so apologies for some basic
> > questions.
> >
> > Has the SecurityManager been fully removed in JDK 23?
> >
> > What is the effect of running Kafka 3.9.0 with JDK 23?
> >
> > By "4.0 breaking changes" do you mean changes to our JDK/Scala supported
> > versions, removal or ZK, Kafka API changes, or something else?
> >
> > In general, I do not think we should change our supported JDK versions
> in a
> > hotfix release. I see https://issues.apache.org/jira/browse/KAFKA-17638
> > which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack
> of
> > support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far
> as
> > I can tell).
> >
> > Also, I feel that we should not add too much to 3.9.x aside from actual
> > bugs. If we backport things into 3.9.x, it will slow adoption of 4.x and
> > increase our maintenance burden over time.
> >
> > Just my $0.02
> >
> > Thanks!
> > David A
> >
> > On Wed, Nov 20, 2024 at 12:22 PM Greg Harris
> <greg.harris@aiven.io.invalid
> > >
> > wrote:
> >
> > > Hi all,
> > >
> > > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> > understand
> > > everyone's expectations about the 3.9.x branch, and ask for a specific
> > > consensus on Java 23 support.
> > >
> > > Some context that I think is relevant to the discussion:
> > > * KIP-1006 [1] proposes a backwards-compatible strategy for handling
> the
> > > ongoing removal of the SecurityManager, which is merged and due to
> > release
> > > in 4.0.0 [2].
> > > * KIP-1012 [3] rejected ongoing parallel feature development on a 3.x
> > > branch while having trunk on 4.x.
> > > * During the 3.9.0 release, the patch [2] was rejected [4] due to
> being a
> > > new feature which did not meet the feature freeze deadline.
> > > * Other than the SecurityManager removal, there are additional PRs
> which
> > > would also need to be backported for full Java 23 support [5]
> including a
> > > Scala patch upgrade.
> > > * Downstream users are asking for a backport [6] because adding support
> > for
> > > Java 23 would obligate them to also include the 4.0 breaking changes.
> > >
> > > So while adding Java version support in the past has been a KIP-less
> > > feature and normally only appears in the next version, it happens to
> > align
> > > with a major version bump this time. This will cause additional pain
> for
> > > users if we do not elect to backport this.
> > >
> > > Thanks,
> > > Greg
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > > [2] https://github.com/apache/kafka/pull/16522
> > > [3]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > > [4] https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > > [6] https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> > >
> >
> >
> > --
> > David Arthur
> >
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Greg Harris <gr...@aiven.io.INVALID>]

> Has the SecurityManager been fully removed in JDK 23?
> What is the effect of running Kafka 3.9.0 with JDK 23?

The SecurityManager has been degraded, so by default our users experience
an UnsupportedOperationException. They can work-around this by setting a
system property.
In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched 3.9.x
will experience an UnsupportedOperationException unconditionally.

> I see https://issues.apache.org/jira/browse/KAFKA-17638
> which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack
of
> support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far as
> I can tell).

Originally we were planning to get this change into 3.9.0, but we missed
the merge deadline. I opened that ticket afterwards to be fixed in 4.0.0
because that's the next release.
The patch was always intended to be backportable, and I intended to
backport it [2].

I understand that if we consider Java 23 support to be a feature (which is
the standing decision), this is a pretty obvious case of missing feature
freeze, and the current course of action (releasing in 4.0.0) is how we
would handle it.
I'm asking for this to be reconsidered as a bug fix, because it allows us
to backport the change, which is what our users are asking for [3].

Thanks,
Greg

[1] https://openjdk.org/jeps/486
[2] https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
[3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g

On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com> wrote:

> Greg,
>
> I have not been following this closely, so apologies for some basic
> questions.
>
> Has the SecurityManager been fully removed in JDK 23?
>
> What is the effect of running Kafka 3.9.0 with JDK 23?
>
> By "4.0 breaking changes" do you mean changes to our JDK/Scala supported
> versions, removal or ZK, Kafka API changes, or something else?
>
> In general, I do not think we should change our supported JDK versions in a
> hotfix release. I see https://issues.apache.org/jira/browse/KAFKA-17638
> which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack of
> support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far as
> I can tell).
>
> Also, I feel that we should not add too much to 3.9.x aside from actual
> bugs. If we backport things into 3.9.x, it will slow adoption of 4.x and
> increase our maintenance burden over time.
>
> Just my $0.02
>
> Thanks!
> David A
>
> On Wed, Nov 20, 2024 at 12:22 PM Greg Harris <greg.harris@aiven.io.invalid
> >
> wrote:
>
> > Hi all,
> >
> > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> understand
> > everyone's expectations about the 3.9.x branch, and ask for a specific
> > consensus on Java 23 support.
> >
> > Some context that I think is relevant to the discussion:
> > * KIP-1006 [1] proposes a backwards-compatible strategy for handling the
> > ongoing removal of the SecurityManager, which is merged and due to
> release
> > in 4.0.0 [2].
> > * KIP-1012 [3] rejected ongoing parallel feature development on a 3.x
> > branch while having trunk on 4.x.
> > * During the 3.9.0 release, the patch [2] was rejected [4] due to being a
> > new feature which did not meet the feature freeze deadline.
> > * Other than the SecurityManager removal, there are additional PRs which
> > would also need to be backported for full Java 23 support [5] including a
> > Scala patch upgrade.
> > * Downstream users are asking for a backport [6] because adding support
> for
> > Java 23 would obligate them to also include the 4.0 breaking changes.
> >
> > So while adding Java version support in the past has been a KIP-less
> > feature and normally only appears in the next version, it happens to
> align
> > with a major version bump this time. This will cause additional pain for
> > users if we do not elect to backport this.
> >
> > Thanks,
> > Greg
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > [2] https://github.com/apache/kafka/pull/16522
> > [3]
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > [4] https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > [6] https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> >
>
>
> --
> David Arthur
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[Greg Harris <gr...@aiven.io.INVALID>]

> Has the SecurityManager been fully removed in JDK 23?
> What is the effect of running Kafka 3.9.0 with JDK 23?

The SecurityManager has been degraded, so by default our users experience
an UnsupportedOperationException. They can work-around this by setting a
system property.
In JRE 24, JEP-486 [1] has removed this workaround, so an unpatched 3.9.x
will experience an UnsupportedOperationException unconditionally.

> I see https://issues.apache.org/jira/browse/KAFKA-17638
> which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack
of
> support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far as
> I can tell).

Originally we were planning to get this change into 3.9.0, but we missed
the merge deadline. I opened that ticket afterwards to be fixed in 4.0.0
because that's the next release.
The patch was always intended to be backportable, and I intended to
backport it [2].

I understand that if we consider Java 23 support to be a feature (which is
the standing decision), this is a pretty obvious case of missing feature
freeze, and the current course of action (releasing in 4.0.0) is how we
would handle it.
I'm asking for this to be reconsidered as a bug fix, because it allows us
to backport the change, which is what our users are asking for [3].

Thanks,
Greg

[1] https://openjdk.org/jeps/486
[2] https://github.com/apache/kafka/pull/16522#issuecomment-2377340024
[3] https://lists.apache.org/thread/312lm617q05k87kxsrwlqhk8rfg29t7g

On Wed, Nov 20, 2024 at 11:50 AM David Arthur <mu...@gmail.com> wrote:

> Greg,
>
> I have not been following this closely, so apologies for some basic
> questions.
>
> Has the SecurityManager been fully removed in JDK 23?
>
> What is the effect of running Kafka 3.9.0 with JDK 23?
>
> By "4.0 breaking changes" do you mean changes to our JDK/Scala supported
> versions, removal or ZK, Kafka API changes, or something else?
>
> In general, I do not think we should change our supported JDK versions in a
> hotfix release. I see https://issues.apache.org/jira/browse/KAFKA-17638
> which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack of
> support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far as
> I can tell).
>
> Also, I feel that we should not add too much to 3.9.x aside from actual
> bugs. If we backport things into 3.9.x, it will slow adoption of 4.x and
> increase our maintenance burden over time.
>
> Just my $0.02
>
> Thanks!
> David A
>
> On Wed, Nov 20, 2024 at 12:22 PM Greg Harris <greg.harris@aiven.io.invalid
> >
> wrote:
>
> > Hi all,
> >
> > Now that 3.9.0 is released and 4.0.x is progressing, I'd like to
> understand
> > everyone's expectations about the 3.9.x branch, and ask for a specific
> > consensus on Java 23 support.
> >
> > Some context that I think is relevant to the discussion:
> > * KIP-1006 [1] proposes a backwards-compatible strategy for handling the
> > ongoing removal of the SecurityManager, which is merged and due to
> release
> > in 4.0.0 [2].
> > * KIP-1012 [3] rejected ongoing parallel feature development on a 3.x
> > branch while having trunk on 4.x.
> > * During the 3.9.0 release, the patch [2] was rejected [4] due to being a
> > new feature which did not meet the feature freeze deadline.
> > * Other than the SecurityManager removal, there are additional PRs which
> > would also need to be backported for full Java 23 support [5] including a
> > Scala patch upgrade.
> > * Downstream users are asking for a backport [6] because adding support
> for
> > Java 23 would obligate them to also include the 4.0 breaking changes.
> >
> > So while adding Java version support in the past has been a KIP-less
> > feature and normally only appears in the next version, it happens to
> align
> > with a major version bump this time. This will cause additional pain for
> > users if we do not elect to backport this.
> >
> > Thanks,
> > Greg
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> > [2] https://github.com/apache/kafka/pull/16522
> > [3]
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> > [4] https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> > [5] https://issues.apache.org/jira/browse/KAFKA-17638
> > [6] https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
> >
>
>
> --
> David Arthur
>

Re: [DISCUSS] Java 23 Support for 3.9.x

[David Arthur <mu...@gmail.com>]

Greg,

I have not been following this closely, so apologies for some basic
questions.

Has the SecurityManager been fully removed in JDK 23?

What is the effect of running Kafka 3.9.0 with JDK 23?

By "4.0 breaking changes" do you mean changes to our JDK/Scala supported
versions, removal or ZK, Kafka API changes, or something else?

In general, I do not think we should change our supported JDK versions in a
hotfix release. I see https://issues.apache.org/jira/browse/KAFKA-17638
which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack of
support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far as
I can tell).

Also, I feel that we should not add too much to 3.9.x aside from actual
bugs. If we backport things into 3.9.x, it will slow adoption of 4.x and
increase our maintenance burden over time.

Just my $0.02

Thanks!
David A

On Wed, Nov 20, 2024 at 12:22 PM Greg Harris <gr...@aiven.io.invalid>
wrote:

> Hi all,
>
> Now that 3.9.0 is released and 4.0.x is progressing, I'd like to understand
> everyone's expectations about the 3.9.x branch, and ask for a specific
> consensus on Java 23 support.
>
> Some context that I think is relevant to the discussion:
> * KIP-1006 [1] proposes a backwards-compatible strategy for handling the
> ongoing removal of the SecurityManager, which is merged and due to release
> in 4.0.0 [2].
> * KIP-1012 [3] rejected ongoing parallel feature development on a 3.x
> branch while having trunk on 4.x.
> * During the 3.9.0 release, the patch [2] was rejected [4] due to being a
> new feature which did not meet the feature freeze deadline.
> * Other than the SecurityManager removal, there are additional PRs which
> would also need to be backported for full Java 23 support [5] including a
> Scala patch upgrade.
> * Downstream users are asking for a backport [6] because adding support for
> Java 23 would obligate them to also include the 4.0 breaking changes.
>
> So while adding Java version support in the past has been a KIP-less
> feature and normally only appears in the next version, it happens to align
> with a major version bump this time. This will cause additional pain for
> users if we do not elect to backport this.
>
> Thanks,
> Greg
>
> [1]
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> [2] https://github.com/apache/kafka/pull/16522
> [3]
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> [4] https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> [5] https://issues.apache.org/jira/browse/KAFKA-17638
> [6] https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
>


-- 
David Arthur

Re: [DISCUSS] Java 23 Support for 3.9.x

[David Arthur <mu...@gmail.com>]

Greg,

I have not been following this closely, so apologies for some basic
questions.

Has the SecurityManager been fully removed in JDK 23?

What is the effect of running Kafka 3.9.0 with JDK 23?

By "4.0 breaking changes" do you mean changes to our JDK/Scala supported
versions, removal or ZK, Kafka API changes, or something else?

In general, I do not think we should change our supported JDK versions in a
hotfix release. I see https://issues.apache.org/jira/browse/KAFKA-17638
which explicitly adds JDK 23 to our CI with a fix version of 4.0.0. Lack of
support for JDK 23 in 3.9.x is not a bug, it is what we planned (as far as
I can tell).

Also, I feel that we should not add too much to 3.9.x aside from actual
bugs. If we backport things into 3.9.x, it will slow adoption of 4.x and
increase our maintenance burden over time.

Just my $0.02

Thanks!
David A

On Wed, Nov 20, 2024 at 12:22 PM Greg Harris <gr...@aiven.io.invalid>
wrote:

> Hi all,
>
> Now that 3.9.0 is released and 4.0.x is progressing, I'd like to understand
> everyone's expectations about the 3.9.x branch, and ask for a specific
> consensus on Java 23 support.
>
> Some context that I think is relevant to the discussion:
> * KIP-1006 [1] proposes a backwards-compatible strategy for handling the
> ongoing removal of the SecurityManager, which is merged and due to release
> in 4.0.0 [2].
> * KIP-1012 [3] rejected ongoing parallel feature development on a 3.x
> branch while having trunk on 4.x.
> * During the 3.9.0 release, the patch [2] was rejected [4] due to being a
> new feature which did not meet the feature freeze deadline.
> * Other than the SecurityManager removal, there are additional PRs which
> would also need to be backported for full Java 23 support [5] including a
> Scala patch upgrade.
> * Downstream users are asking for a backport [6] because adding support for
> Java 23 would obligate them to also include the 4.0 breaking changes.
>
> So while adding Java version support in the past has been a KIP-less
> feature and normally only appears in the next version, it happens to align
> with a major version bump this time. This will cause additional pain for
> users if we do not elect to backport this.
>
> Thanks,
> Greg
>
> [1]
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1006%3A+Remove+SecurityManager+Support
> [2] https://github.com/apache/kafka/pull/16522
> [3]
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8+and+3.9+release
> [4] https://lists.apache.org/thread/xy5rwd1w274qgpwf3qxxnzlqpoly5d4p
> [5] https://issues.apache.org/jira/browse/KAFKA-17638
> [6] https://github.com/apache/kafka/pull/16522#issuecomment-2488340682
>


-- 
David Arthur