You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by David Jacot <da...@apache.org> on 2025/02/22 10:16:43 UTC
[VOTE] 4.0.0 RC0
Hello Kafka users, developers and client-developers,
This is the first candidate for release of Apache Kafka 4.0.0. We
still have some remaining blockers but we figured that getting a first
release candidate will help the community to test this major release.
- This is the first release without Apache Zookeeper
- The Next Generation of the Consumer Rebalance Protocol is Generally Available
- The Transactions Server-Side Defense (Phase 2) is Generally Available
- Queues for Kafka is in Early Access
- Kafka uses log4j2
- Drop broker and tools support for Java 11
- Remove old client protocol API versions
Release notes for the 4.0.0 release:
https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
*** Please download and test the release. Voting is not necessary as
we still have blockers.
Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS
* Release artifacts to be voted upon (source and binary):
https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
* Docker release artifacts to be voted upon:
apache/kafka:4.0.0-rc0
apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
need to investigate it)
* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/
* Javadoc:
https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
* Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
https://github.com/apache/kafka/releases/tag/4.0.0-rc0
* Documentation:
https://kafka.apache.org/40/documentation.html
* Protocol:
https://kafka.apache.org/40/protocol.html
* Successful CI builds for the 4.0 branch:
Unit/integration tests: https://github.com/apache/kafka/actions/runs/13459676207
System tests: TBD
* Successful Docker Image Github Actions Pipeline for 4.0 branch:
Docker Build Test Pipeline (JVM):
https://github.com/apache/kafka/actions/runs/13471603921
Docker Build Test Pipeline (Native):
https://github.com/apache/kafka/actions/runs/13471605941
/**************************************
Thanks,
David
Re: [VOTE] 4.0.0 RC0
Posted by Andrew Schofield <an...@outlook.com>.
Hi David,
Thanks for running the release.
I performed the following steps:
* Built from source using Java 17
* Enabled KIP-932 and ran through some basic actions to make sure it worked based on
https://cwiki.apache.org/confluence/display/KAFKA/Queues+for+Kafka+%28KIP-932%29+-+Early+Access+Release+Notes
* Validated the APIs enabled using kafka-broker-api-versions.sh
* Stopped the broker and removed the config changes so KIP-932 was no longer enabled
* Restarted the broker and validated the APIs enabled did not include KIP-932
All good.
I also looked at the protocol docs and observe the following:
* A bunch of APIs were removed in 4.0 (correct, of course) - these are not in the protocol docs
* The APIs which are supported by the controller, such as Vote, are omitted from the protocol docs
My view is that these APIs are supported in 4.0 and probably should be in the protocol docs.
I haven't created an issue at this point because I'm not sure whether others will agree.
Thanks,
Andrew
________________________________________
From: Ismael Juma <me...@ismaeljuma.com>
Sent: 26 February 2025 00:08
To: dev@kafka.apache.org <de...@kafka.apache.org>
Subject: Re: [VOTE] 4.0.0 RC0
Hi Jun,
When it comes to the upgrade documentation, a couple of changes landed
after the RC was generated:
*
https://github.com/apache/kafka/commit/da8f390c4599d7199c4cdf2bb85441146e859b17
*
https://github.com/apache/kafka/commit/da3b8e88dc61a1b749895866394cab68410e0eda
Regarding the message format, I agree we can improve the documentation - I
will submit a PR for that momentarily.
Ismael
On Tue, Feb 25, 2025 at 3:58 PM Jun Rao <ju...@confluent.io.invalid> wrote:
> Hi, David,
>
> Thanks for preparing RC0.
>
> A few comments on the documentation.
>
> 1.5 Upgrading
> There is no documentation on which previous releases can be upgraded to
> 4.0.
>
> 5.3.3 Old Message Format
> Should we remove this section since 4.0 only supports the V2 message
> format?
>
> 6.10 Consumer Rebalance Protocol
> It's missing from the index on the left.
>
> ConsumerGroupPartitionAssignor is cut off in
> org.apache.kafka.coordinator.group.api.assignor.ConsumerGroupPartitionA.
>
> 6.11 Transaction Protocol
> It's missing from the index on the left.
>
> Jun
>
>
> On Tue, Feb 25, 2025 at 11:28 AM José Armando García Sancio
> <js...@confluent.io.invalid> wrote:
>
> > Hi David and all,
> >
> > I am about to merge https://github.com/apache/kafka/pull/18852 to
> > trunk: https://issues.apache.org/jira/browse/KAFKA-18723. I would like
> > to cherry-pick it to 4.0.0. What do you think?
> >
> > I will also be cherry picking the change back to 3.7.x, 3.8.x and
> > 3.9.x. If it is not included in the 4.0.0 release, we will need to
> > include it in the 4.0.1 release.
> >
> > Thanks,
> > --
> > -José
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by Ismael Juma <me...@ismaeljuma.com>.
Hi Jun,
When it comes to the upgrade documentation, a couple of changes landed
after the RC was generated:
*
https://github.com/apache/kafka/commit/da8f390c4599d7199c4cdf2bb85441146e859b17
*
https://github.com/apache/kafka/commit/da3b8e88dc61a1b749895866394cab68410e0eda
Regarding the message format, I agree we can improve the documentation - I
will submit a PR for that momentarily.
Ismael
On Tue, Feb 25, 2025 at 3:58 PM Jun Rao <ju...@confluent.io.invalid> wrote:
> Hi, David,
>
> Thanks for preparing RC0.
>
> A few comments on the documentation.
>
> 1.5 Upgrading
> There is no documentation on which previous releases can be upgraded to
> 4.0.
>
> 5.3.3 Old Message Format
> Should we remove this section since 4.0 only supports the V2 message
> format?
>
> 6.10 Consumer Rebalance Protocol
> It's missing from the index on the left.
>
> ConsumerGroupPartitionAssignor is cut off in
> org.apache.kafka.coordinator.group.api.assignor.ConsumerGroupPartitionA.
>
> 6.11 Transaction Protocol
> It's missing from the index on the left.
>
> Jun
>
>
> On Tue, Feb 25, 2025 at 11:28 AM José Armando García Sancio
> <js...@confluent.io.invalid> wrote:
>
> > Hi David and all,
> >
> > I am about to merge https://github.com/apache/kafka/pull/18852 to
> > trunk: https://issues.apache.org/jira/browse/KAFKA-18723. I would like
> > to cherry-pick it to 4.0.0. What do you think?
> >
> > I will also be cherry picking the change back to 3.7.x, 3.8.x and
> > 3.9.x. If it is not included in the 4.0.0 release, we will need to
> > include it in the 4.0.1 release.
> >
> > Thanks,
> > --
> > -José
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by Jun Rao <ju...@confluent.io.INVALID>.
Hi, David,
Thanks for preparing RC0.
A few comments on the documentation.
1.5 Upgrading
There is no documentation on which previous releases can be upgraded to 4.0.
5.3.3 Old Message Format
Should we remove this section since 4.0 only supports the V2 message format?
6.10 Consumer Rebalance Protocol
It's missing from the index on the left.
ConsumerGroupPartitionAssignor is cut off in
org.apache.kafka.coordinator.group.api.assignor.ConsumerGroupPartitionA.
6.11 Transaction Protocol
It's missing from the index on the left.
Jun
On Tue, Feb 25, 2025 at 11:28 AM José Armando García Sancio
<js...@confluent.io.invalid> wrote:
> Hi David and all,
>
> I am about to merge https://github.com/apache/kafka/pull/18852 to
> trunk: https://issues.apache.org/jira/browse/KAFKA-18723. I would like
> to cherry-pick it to 4.0.0. What do you think?
>
> I will also be cherry picking the change back to 3.7.x, 3.8.x and
> 3.9.x. If it is not included in the 4.0.0 release, we will need to
> include it in the 4.0.1 release.
>
> Thanks,
> --
> -José
>
Re: [VOTE] 4.0.0 RC0
Posted by Ismael Juma <me...@ismaeljuma.com>.
Thanks for the clarification Jose. Makes sense.
Ismael
On Wed, Feb 26, 2025 at 9:22 AM José Armando García Sancio
<js...@confluent.io.invalid> wrote:
> Hi Ismael,
>
> On Wed, Feb 26, 2025 at 7:51 AM Ismael Juma <me...@ismaeljuma.com> wrote:
> >
> > Hi Jose,
> >
> > Is it a low risk change? If not, I would suggest giving it a bit of
> > stabilization time before cherry-picking to older branches including 4.0
> > and 3.9. I don't think it's necessary to backport to other older branches
> > given that it's not a regression and I don't believe there are any cases
> of
> > users running into it.
>
> We do have users that have encountered this issue in KRaft. I
> documented two cases in the Jira.
>
> Since this is not a regression, I am okay not including this in the
> 4.0.0 release. We should include it in the 4.0.1 release. I updated
> the Jira so that the fixed version is 4.0.1.
>
> Thanks,
> --
> -José
>
Re: [VOTE] 4.0.0 RC0
Posted by José Armando García Sancio <js...@confluent.io.INVALID>.
Hi Ismael,
On Wed, Feb 26, 2025 at 7:51 AM Ismael Juma <me...@ismaeljuma.com> wrote:
>
> Hi Jose,
>
> Is it a low risk change? If not, I would suggest giving it a bit of
> stabilization time before cherry-picking to older branches including 4.0
> and 3.9. I don't think it's necessary to backport to other older branches
> given that it's not a regression and I don't believe there are any cases of
> users running into it.
We do have users that have encountered this issue in KRaft. I
documented two cases in the Jira.
Since this is not a regression, I am okay not including this in the
4.0.0 release. We should include it in the 4.0.1 release. I updated
the Jira so that the fixed version is 4.0.1.
Thanks,
--
-José
Re: [VOTE] 4.0.0 RC0
Posted by Ismael Juma <me...@ismaeljuma.com>.
Hi Jose,
Is it a low risk change? If not, I would suggest giving it a bit of
stabilization time before cherry-picking to older branches including 4.0
and 3.9. I don't think it's necessary to backport to other older branches
given that it's not a regression and I don't believe there are any cases of
users running into it.
Ismael
On Tue, Feb 25, 2025 at 11:30 AM José Armando García Sancio
<js...@confluent.io.invalid> wrote:
> Hi David and all,
>
> I am about to merge https://github.com/apache/kafka/pull/18852 to
> trunk: https://issues.apache.org/jira/browse/KAFKA-18723. I would like
> to cherry-pick it to 4.0.0. What do you think?
>
> I will also be cherry picking the change back to 3.7.x, 3.8.x and
> 3.9.x. If it is not included in the 4.0.0 release, we will need to
> include it in the 4.0.1 release.
>
> Thanks,
> --
> -José
>
Re: [VOTE] 4.0.0 RC0
Posted by José Armando García Sancio <js...@confluent.io.INVALID>.
Hi David and all,
I am about to merge https://github.com/apache/kafka/pull/18852 to
trunk: https://issues.apache.org/jira/browse/KAFKA-18723. I would like
to cherry-pick it to 4.0.0. What do you think?
I will also be cherry picking the change back to 3.7.x, 3.8.x and
3.9.x. If it is not included in the 4.0.0 release, we will need to
include it in the 4.0.1 release.
Thanks,
--
-José
Re: [VOTE] 4.0.0 RC0
Posted by 黃竣陽 <s7...@gmail.com>.
Hello David,
I've completed the following tests on my local machine:
- Built from source using JDK 21 & JDK 17 (Zulu, Amazon, Eclipse)
- Executed all unit and integration tests
- Ran the quickstart successfully
- Tested dynamic voter with 4.0 and 3.9 controllers
- Tested 4.0 controller with a 3.9 broker
Best Regards
Jiunn-Yang
> Stanislav Kozlovski <st...@apache.org> 於 2025年2月25日 下午6:55 寫道：
>
> +1
>
> On 2025/02/24 23:07:12 Christopher Shannon wrote:
>> If the purpose of this is not to vote but just to evaluate the RC then I
>> think it would be better to not use the [VOTE] tag for the subject as that
>> is super confusing.
>>
>> On Mon, Feb 24, 2025 at 10:45 AM Ismael Juma <me...@ismaeljuma.com> wrote:
>>
>>> Hi Mickael,
>>>
>>> This was addressed in the original message:
>>>
>>> "This is the first candidate for release of Apache Kafka 4.0.0. We
>>> still have some remaining blockers but we figured that getting a first
>>> release candidate will help the community to test this major release."
>>>
>>> There are a large number of changes and the remaining blockers don't
>>> prevent broader testing from the community.
>>>
>>> Ismael
>>>
>>> On Mon, Feb 24, 2025, 7:22 AM Mickael Maison <mi...@gmail.com>
>>> wrote:
>>>
>>>> Hi David,
>>>>
>>>> I see we still have a few open Jiras marked as blockers for 4.0.0:
>>>>
>>>>
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=325421614#ReleasePlan4.0.0-OpenIssues
>>>>
>>>> Are we expecting another RC soon? Or can this vote pass and these
>>>> Jiras are moved to 4.1.0?
>>>>
>>>> Thanks,
>>>> Mickael
>>>>
>>>> On Sun, Feb 23, 2025 at 10:46 PM Ismael Juma <me...@ismaeljuma.com> wrote:
>>>>>
>>>>> Thanks David! A few ideas of things to test:
>>>>>
>>>>> 1. Test clients and streams with Java 11: this is the first time we
>>> use a
>>>>> different Java version for clients/streams vs the rest, so it would be
>>>> good
>>>>> to ensure the generated artifacts are good when it comes to this.
>>>>> 2. Similarly, test brokers and connect (including some community
>>>>> connectors) with Java 17.
>>>>> 3. Test community clients (librdkafka & related, kafka-python, sarama,
>>>>> segment io, etc.) with Apache 4.0.
>>>>> 4. Check that logging is as you expect both with your existing log4j
>>>> config
>>>>> and the new config that is now included with the distribution - the new
>>>>> config uses a native log4j2 config while the existing one would rely on
>>>> the
>>>>> compatibility layer.
>>>>> 5. Run the quickstarts and look for zk specific things in the
>>> quickstarts
>>>>> or other documentation.
>>>>> 6. Follow the upgrade documentation and verify that upgrades work from
>>>>> older versions (3.3 or higher).
>>>>> 7. Check that the examples still work (i.e. they were not relying on
>>>>> deprecated apis that have since been removed).
>>>>> 8. Verify that performance of your applications remains the same or has
>>>>> improved (both client and server-side).
>>>>>
>>>>> Ismael
>>>>>
>>>>> On Sat, Feb 22, 2025 at 2:17 AM David Jacot <da...@apache.org> wrote:
>>>>>
>>>>>> Hello Kafka users, developers and client-developers,
>>>>>>
>>>>>> This is the first candidate for release of Apache Kafka 4.0.0. We
>>>>>> still have some remaining blockers but we figured that getting a
>>> first
>>>>>> release candidate will help the community to test this major release.
>>>>>>
>>>>>> - This is the first release without Apache Zookeeper
>>>>>> - The Next Generation of the Consumer Rebalance Protocol is Generally
>>>>>> Available
>>>>>> - The Transactions Server-Side Defense (Phase 2) is Generally
>>> Available
>>>>>> - Queues for Kafka is in Early Access
>>>>>> - Kafka uses log4j2
>>>>>> - Drop broker and tools support for Java 11
>>>>>> - Remove old client protocol API versions
>>>>>>
>>>>>> Release notes for the 4.0.0 release:
>>>>>>
>>>>
>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
>>>>>>
>>>>>> *** Please download and test the release. Voting is not necessary as
>>>>>> we still have blockers.
>>>>>>
>>>>>> Kafka's KEYS file containing PGP keys we use to sign the release:
>>>>>> https://kafka.apache.org/KEYS
>>>>>>
>>>>>> * Release artifacts to be voted upon (source and binary):
>>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
>>>>>>
>>>>>> * Docker release artifacts to be voted upon:
>>>>>> apache/kafka:4.0.0-rc0
>>>>>> apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
>>>>>> need to investigate it)
>>>>>>
>>>>>> * Maven artifacts to be voted upon:
>>>>>>
>>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>>>>>
>>>>>> * Javadoc:
>>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
>>>>>>
>>>>>> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
>>>>>> https://github.com/apache/kafka/releases/tag/4.0.0-rc0
>>>>>>
>>>>>> * Documentation:
>>>>>> https://kafka.apache.org/40/documentation.html
>>>>>>
>>>>>> * Protocol:
>>>>>> https://kafka.apache.org/40/protocol.html
>>>>>>
>>>>>> * Successful CI builds for the 4.0 branch:
>>>>>> Unit/integration tests:
>>>>>> https://github.com/apache/kafka/actions/runs/13459676207
>>>>>> System tests: TBD
>>>>>>
>>>>>> * Successful Docker Image Github Actions Pipeline for 4.0 branch:
>>>>>> Docker Build Test Pipeline (JVM):
>>>>>> https://github.com/apache/kafka/actions/runs/13471603921
>>>>>> Docker Build Test Pipeline (Native):
>>>>>> https://github.com/apache/kafka/actions/runs/13471605941
>>>>>>
>>>>>> /**************************************
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>
>>>
>>
Re: [VOTE] 4.0.0 RC0
Posted by Stanislav Kozlovski <st...@apache.org>.
+1
On 2025/02/24 23:07:12 Christopher Shannon wrote:
> If the purpose of this is not to vote but just to evaluate the RC then I
> think it would be better to not use the [VOTE] tag for the subject as that
> is super confusing.
>
> On Mon, Feb 24, 2025 at 10:45 AM Ismael Juma <me...@ismaeljuma.com> wrote:
>
> > Hi Mickael,
> >
> > This was addressed in the original message:
> >
> > "This is the first candidate for release of Apache Kafka 4.0.0. We
> > still have some remaining blockers but we figured that getting a first
> > release candidate will help the community to test this major release."
> >
> > There are a large number of changes and the remaining blockers don't
> > prevent broader testing from the community.
> >
> > Ismael
> >
> > On Mon, Feb 24, 2025, 7:22 AM Mickael Maison <mi...@gmail.com>
> > wrote:
> >
> > > Hi David,
> > >
> > > I see we still have a few open Jiras marked as blockers for 4.0.0:
> > >
> > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=325421614#ReleasePlan4.0.0-OpenIssues
> > >
> > > Are we expecting another RC soon? Or can this vote pass and these
> > > Jiras are moved to 4.1.0?
> > >
> > > Thanks,
> > > Mickael
> > >
> > > On Sun, Feb 23, 2025 at 10:46 PM Ismael Juma <me...@ismaeljuma.com> wrote:
> > > >
> > > > Thanks David! A few ideas of things to test:
> > > >
> > > > 1. Test clients and streams with Java 11: this is the first time we
> > use a
> > > > different Java version for clients/streams vs the rest, so it would be
> > > good
> > > > to ensure the generated artifacts are good when it comes to this.
> > > > 2. Similarly, test brokers and connect (including some community
> > > > connectors) with Java 17.
> > > > 3. Test community clients (librdkafka & related, kafka-python, sarama,
> > > > segment io, etc.) with Apache 4.0.
> > > > 4. Check that logging is as you expect both with your existing log4j
> > > config
> > > > and the new config that is now included with the distribution - the new
> > > > config uses a native log4j2 config while the existing one would rely on
> > > the
> > > > compatibility layer.
> > > > 5. Run the quickstarts and look for zk specific things in the
> > quickstarts
> > > > or other documentation.
> > > > 6. Follow the upgrade documentation and verify that upgrades work from
> > > > older versions (3.3 or higher).
> > > > 7. Check that the examples still work (i.e. they were not relying on
> > > > deprecated apis that have since been removed).
> > > > 8. Verify that performance of your applications remains the same or has
> > > > improved (both client and server-side).
> > > >
> > > > Ismael
> > > >
> > > > On Sat, Feb 22, 2025 at 2:17 AM David Jacot <da...@apache.org> wrote:
> > > >
> > > > > Hello Kafka users, developers and client-developers,
> > > > >
> > > > > This is the first candidate for release of Apache Kafka 4.0.0. We
> > > > > still have some remaining blockers but we figured that getting a
> > first
> > > > > release candidate will help the community to test this major release.
> > > > >
> > > > > - This is the first release without Apache Zookeeper
> > > > > - The Next Generation of the Consumer Rebalance Protocol is Generally
> > > > > Available
> > > > > - The Transactions Server-Side Defense (Phase 2) is Generally
> > Available
> > > > > - Queues for Kafka is in Early Access
> > > > > - Kafka uses log4j2
> > > > > - Drop broker and tools support for Java 11
> > > > > - Remove old client protocol API versions
> > > > >
> > > > > Release notes for the 4.0.0 release:
> > > > >
> > >
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> > > > >
> > > > > *** Please download and test the release. Voting is not necessary as
> > > > > we still have blockers.
> > > > >
> > > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > https://kafka.apache.org/KEYS
> > > > >
> > > > > * Release artifacts to be voted upon (source and binary):
> > > > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> > > > >
> > > > > * Docker release artifacts to be voted upon:
> > > > > apache/kafka:4.0.0-rc0
> > > > > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > > > > need to investigate it)
> > > > >
> > > > > * Maven artifacts to be voted upon:
> > > > >
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > >
> > > > > * Javadoc:
> > > > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> > > > >
> > > > > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > > > > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> > > > >
> > > > > * Documentation:
> > > > > https://kafka.apache.org/40/documentation.html
> > > > >
> > > > > * Protocol:
> > > > > https://kafka.apache.org/40/protocol.html
> > > > >
> > > > > * Successful CI builds for the 4.0 branch:
> > > > > Unit/integration tests:
> > > > > https://github.com/apache/kafka/actions/runs/13459676207
> > > > > System tests: TBD
> > > > >
> > > > > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > > > > Docker Build Test Pipeline (JVM):
> > > > > https://github.com/apache/kafka/actions/runs/13471603921
> > > > > Docker Build Test Pipeline (Native):
> > > > > https://github.com/apache/kafka/actions/runs/13471605941
> > > > >
> > > > > /**************************************
> > > > >
> > > > > Thanks,
> > > > > David
> > > > >
> > >
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by Christopher Shannon <ch...@gmail.com>.
If the purpose of this is not to vote but just to evaluate the RC then I
think it would be better to not use the [VOTE] tag for the subject as that
is super confusing.
On Mon, Feb 24, 2025 at 10:45 AM Ismael Juma <me...@ismaeljuma.com> wrote:
> Hi Mickael,
>
> This was addressed in the original message:
>
> "This is the first candidate for release of Apache Kafka 4.0.0. We
> still have some remaining blockers but we figured that getting a first
> release candidate will help the community to test this major release."
>
> There are a large number of changes and the remaining blockers don't
> prevent broader testing from the community.
>
> Ismael
>
> On Mon, Feb 24, 2025, 7:22 AM Mickael Maison <mi...@gmail.com>
> wrote:
>
> > Hi David,
> >
> > I see we still have a few open Jiras marked as blockers for 4.0.0:
> >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=325421614#ReleasePlan4.0.0-OpenIssues
> >
> > Are we expecting another RC soon? Or can this vote pass and these
> > Jiras are moved to 4.1.0?
> >
> > Thanks,
> > Mickael
> >
> > On Sun, Feb 23, 2025 at 10:46 PM Ismael Juma <me...@ismaeljuma.com> wrote:
> > >
> > > Thanks David! A few ideas of things to test:
> > >
> > > 1. Test clients and streams with Java 11: this is the first time we
> use a
> > > different Java version for clients/streams vs the rest, so it would be
> > good
> > > to ensure the generated artifacts are good when it comes to this.
> > > 2. Similarly, test brokers and connect (including some community
> > > connectors) with Java 17.
> > > 3. Test community clients (librdkafka & related, kafka-python, sarama,
> > > segment io, etc.) with Apache 4.0.
> > > 4. Check that logging is as you expect both with your existing log4j
> > config
> > > and the new config that is now included with the distribution - the new
> > > config uses a native log4j2 config while the existing one would rely on
> > the
> > > compatibility layer.
> > > 5. Run the quickstarts and look for zk specific things in the
> quickstarts
> > > or other documentation.
> > > 6. Follow the upgrade documentation and verify that upgrades work from
> > > older versions (3.3 or higher).
> > > 7. Check that the examples still work (i.e. they were not relying on
> > > deprecated apis that have since been removed).
> > > 8. Verify that performance of your applications remains the same or has
> > > improved (both client and server-side).
> > >
> > > Ismael
> > >
> > > On Sat, Feb 22, 2025 at 2:17 AM David Jacot <da...@apache.org> wrote:
> > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the first candidate for release of Apache Kafka 4.0.0. We
> > > > still have some remaining blockers but we figured that getting a
> first
> > > > release candidate will help the community to test this major release.
> > > >
> > > > - This is the first release without Apache Zookeeper
> > > > - The Next Generation of the Consumer Rebalance Protocol is Generally
> > > > Available
> > > > - The Transactions Server-Side Defense (Phase 2) is Generally
> Available
> > > > - Queues for Kafka is in Early Access
> > > > - Kafka uses log4j2
> > > > - Drop broker and tools support for Java 11
> > > > - Remove old client protocol API versions
> > > >
> > > > Release notes for the 4.0.0 release:
> > > >
> >
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> > > >
> > > > *** Please download and test the release. Voting is not necessary as
> > > > we still have blockers.
> > > >
> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > https://kafka.apache.org/KEYS
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> > > >
> > > > * Docker release artifacts to be voted upon:
> > > > apache/kafka:4.0.0-rc0
> > > > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > > > need to investigate it)
> > > >
> > > > * Maven artifacts to be voted upon:
> > > >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >
> > > > * Javadoc:
> > > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> > > >
> > > > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > > > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> > > >
> > > > * Documentation:
> > > > https://kafka.apache.org/40/documentation.html
> > > >
> > > > * Protocol:
> > > > https://kafka.apache.org/40/protocol.html
> > > >
> > > > * Successful CI builds for the 4.0 branch:
> > > > Unit/integration tests:
> > > > https://github.com/apache/kafka/actions/runs/13459676207
> > > > System tests: TBD
> > > >
> > > > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > > > Docker Build Test Pipeline (JVM):
> > > > https://github.com/apache/kafka/actions/runs/13471603921
> > > > Docker Build Test Pipeline (Native):
> > > > https://github.com/apache/kafka/actions/runs/13471605941
> > > >
> > > > /**************************************
> > > >
> > > > Thanks,
> > > > David
> > > >
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by Ismael Juma <me...@ismaeljuma.com>.
Hi Mickael,
This was addressed in the original message:
"This is the first candidate for release of Apache Kafka 4.0.0. We
still have some remaining blockers but we figured that getting a first
release candidate will help the community to test this major release."
There are a large number of changes and the remaining blockers don't
prevent broader testing from the community.
Ismael
On Mon, Feb 24, 2025, 7:22 AM Mickael Maison <mi...@gmail.com>
wrote:
> Hi David,
>
> I see we still have a few open Jiras marked as blockers for 4.0.0:
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=325421614#ReleasePlan4.0.0-OpenIssues
>
> Are we expecting another RC soon? Or can this vote pass and these
> Jiras are moved to 4.1.0?
>
> Thanks,
> Mickael
>
> On Sun, Feb 23, 2025 at 10:46 PM Ismael Juma <me...@ismaeljuma.com> wrote:
> >
> > Thanks David! A few ideas of things to test:
> >
> > 1. Test clients and streams with Java 11: this is the first time we use a
> > different Java version for clients/streams vs the rest, so it would be
> good
> > to ensure the generated artifacts are good when it comes to this.
> > 2. Similarly, test brokers and connect (including some community
> > connectors) with Java 17.
> > 3. Test community clients (librdkafka & related, kafka-python, sarama,
> > segment io, etc.) with Apache 4.0.
> > 4. Check that logging is as you expect both with your existing log4j
> config
> > and the new config that is now included with the distribution - the new
> > config uses a native log4j2 config while the existing one would rely on
> the
> > compatibility layer.
> > 5. Run the quickstarts and look for zk specific things in the quickstarts
> > or other documentation.
> > 6. Follow the upgrade documentation and verify that upgrades work from
> > older versions (3.3 or higher).
> > 7. Check that the examples still work (i.e. they were not relying on
> > deprecated apis that have since been removed).
> > 8. Verify that performance of your applications remains the same or has
> > improved (both client and server-side).
> >
> > Ismael
> >
> > On Sat, Feb 22, 2025 at 2:17 AM David Jacot <da...@apache.org> wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the first candidate for release of Apache Kafka 4.0.0. We
> > > still have some remaining blockers but we figured that getting a first
> > > release candidate will help the community to test this major release.
> > >
> > > - This is the first release without Apache Zookeeper
> > > - The Next Generation of the Consumer Rebalance Protocol is Generally
> > > Available
> > > - The Transactions Server-Side Defense (Phase 2) is Generally Available
> > > - Queues for Kafka is in Early Access
> > > - Kafka uses log4j2
> > > - Drop broker and tools support for Java 11
> > > - Remove old client protocol API versions
> > >
> > > Release notes for the 4.0.0 release:
> > >
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download and test the release. Voting is not necessary as
> > > we still have blockers.
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > https://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> > >
> > > * Docker release artifacts to be voted upon:
> > > apache/kafka:4.0.0-rc0
> > > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > > need to investigate it)
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >
> > > * Javadoc:
> > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> > >
> > > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> > >
> > > * Documentation:
> > > https://kafka.apache.org/40/documentation.html
> > >
> > > * Protocol:
> > > https://kafka.apache.org/40/protocol.html
> > >
> > > * Successful CI builds for the 4.0 branch:
> > > Unit/integration tests:
> > > https://github.com/apache/kafka/actions/runs/13459676207
> > > System tests: TBD
> > >
> > > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > > Docker Build Test Pipeline (JVM):
> > > https://github.com/apache/kafka/actions/runs/13471603921
> > > Docker Build Test Pipeline (Native):
> > > https://github.com/apache/kafka/actions/runs/13471605941
> > >
> > > /**************************************
> > >
> > > Thanks,
> > > David
> > >
>
Re: [VOTE] 4.0.0 RC0
Posted by Mickael Maison <mi...@gmail.com>.
Hi David,
I see we still have a few open Jiras marked as blockers for 4.0.0:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=325421614#ReleasePlan4.0.0-OpenIssues
Are we expecting another RC soon? Or can this vote pass and these
Jiras are moved to 4.1.0?
Thanks,
Mickael
On Sun, Feb 23, 2025 at 10:46 PM Ismael Juma <me...@ismaeljuma.com> wrote:
>
> Thanks David! A few ideas of things to test:
>
> 1. Test clients and streams with Java 11: this is the first time we use a
> different Java version for clients/streams vs the rest, so it would be good
> to ensure the generated artifacts are good when it comes to this.
> 2. Similarly, test brokers and connect (including some community
> connectors) with Java 17.
> 3. Test community clients (librdkafka & related, kafka-python, sarama,
> segment io, etc.) with Apache 4.0.
> 4. Check that logging is as you expect both with your existing log4j config
> and the new config that is now included with the distribution - the new
> config uses a native log4j2 config while the existing one would rely on the
> compatibility layer.
> 5. Run the quickstarts and look for zk specific things in the quickstarts
> or other documentation.
> 6. Follow the upgrade documentation and verify that upgrades work from
> older versions (3.3 or higher).
> 7. Check that the examples still work (i.e. they were not relying on
> deprecated apis that have since been removed).
> 8. Verify that performance of your applications remains the same or has
> improved (both client and server-side).
>
> Ismael
>
> On Sat, Feb 22, 2025 at 2:17 AM David Jacot <da...@apache.org> wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for release of Apache Kafka 4.0.0. We
> > still have some remaining blockers but we figured that getting a first
> > release candidate will help the community to test this major release.
> >
> > - This is the first release without Apache Zookeeper
> > - The Next Generation of the Consumer Rebalance Protocol is Generally
> > Available
> > - The Transactions Server-Side Defense (Phase 2) is Generally Available
> > - Queues for Kafka is in Early Access
> > - Kafka uses log4j2
> > - Drop broker and tools support for Java 11
> > - Remove old client protocol API versions
> >
> > Release notes for the 4.0.0 release:
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> >
> > *** Please download and test the release. Voting is not necessary as
> > we still have blockers.
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> >
> > * Docker release artifacts to be voted upon:
> > apache/kafka:4.0.0-rc0
> > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > need to investigate it)
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> >
> > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> >
> > * Documentation:
> > https://kafka.apache.org/40/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/40/protocol.html
> >
> > * Successful CI builds for the 4.0 branch:
> > Unit/integration tests:
> > https://github.com/apache/kafka/actions/runs/13459676207
> > System tests: TBD
> >
> > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > Docker Build Test Pipeline (JVM):
> > https://github.com/apache/kafka/actions/runs/13471603921
> > Docker Build Test Pipeline (Native):
> > https://github.com/apache/kafka/actions/runs/13471605941
> >
> > /**************************************
> >
> > Thanks,
> > David
> >
Re: [VOTE] 4.0.0 RC0
Posted by Ismael Juma <me...@ismaeljuma.com>.
Thanks David! A few ideas of things to test:
1. Test clients and streams with Java 11: this is the first time we use a
different Java version for clients/streams vs the rest, so it would be good
to ensure the generated artifacts are good when it comes to this.
2. Similarly, test brokers and connect (including some community
connectors) with Java 17.
3. Test community clients (librdkafka & related, kafka-python, sarama,
segment io, etc.) with Apache 4.0.
4. Check that logging is as you expect both with your existing log4j config
and the new config that is now included with the distribution - the new
config uses a native log4j2 config while the existing one would rely on the
compatibility layer.
5. Run the quickstarts and look for zk specific things in the quickstarts
or other documentation.
6. Follow the upgrade documentation and verify that upgrades work from
older versions (3.3 or higher).
7. Check that the examples still work (i.e. they were not relying on
deprecated apis that have since been removed).
8. Verify that performance of your applications remains the same or has
improved (both client and server-side).
Ismael
On Sat, Feb 22, 2025 at 2:17 AM David Jacot <da...@apache.org> wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 4.0.0. We
> still have some remaining blockers but we figured that getting a first
> release candidate will help the community to test this major release.
>
> - This is the first release without Apache Zookeeper
> - The Next Generation of the Consumer Rebalance Protocol is Generally
> Available
> - The Transactions Server-Side Defense (Phase 2) is Generally Available
> - Queues for Kafka is in Early Access
> - Kafka uses log4j2
> - Drop broker and tools support for Java 11
> - Remove old client protocol API versions
>
> Release notes for the 4.0.0 release:
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
>
> *** Please download and test the release. Voting is not necessary as
> we still have blockers.
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
>
> * Docker release artifacts to be voted upon:
> apache/kafka:4.0.0-rc0
> apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> need to investigate it)
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
>
> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> https://github.com/apache/kafka/releases/tag/4.0.0-rc0
>
> * Documentation:
> https://kafka.apache.org/40/documentation.html
>
> * Protocol:
> https://kafka.apache.org/40/protocol.html
>
> * Successful CI builds for the 4.0 branch:
> Unit/integration tests:
> https://github.com/apache/kafka/actions/runs/13459676207
> System tests: TBD
>
> * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> Docker Build Test Pipeline (JVM):
> https://github.com/apache/kafka/actions/runs/13471603921
> Docker Build Test Pipeline (Native):
> https://github.com/apache/kafka/actions/runs/13471605941
>
> /**************************************
>
> Thanks,
> David
>
Re: [VOTE] 4.0.0 RC0
Posted by Ismael Juma <me...@ismaeljuma.com>.
Thanks David! A few ideas of things to test:
1. Test clients and streams with Java 11: this is the first time we use a
different Java version for clients/streams vs the rest, so it would be good
to ensure the generated artifacts are good when it comes to this.
2. Similarly, test brokers and connect (including some community
connectors) with Java 17.
3. Test community clients (librdkafka & related, kafka-python, sarama,
segment io, etc.) with Apache 4.0.
4. Check that logging is as you expect both with your existing log4j config
and the new config that is now included with the distribution - the new
config uses a native log4j2 config while the existing one would rely on the
compatibility layer.
5. Run the quickstarts and look for zk specific things in the quickstarts
or other documentation.
6. Follow the upgrade documentation and verify that upgrades work from
older versions (3.3 or higher).
7. Check that the examples still work (i.e. they were not relying on
deprecated apis that have since been removed).
8. Verify that performance of your applications remains the same or has
improved (both client and server-side).
Ismael
On Sat, Feb 22, 2025 at 2:17 AM David Jacot <da...@apache.org> wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 4.0.0. We
> still have some remaining blockers but we figured that getting a first
> release candidate will help the community to test this major release.
>
> - This is the first release without Apache Zookeeper
> - The Next Generation of the Consumer Rebalance Protocol is Generally
> Available
> - The Transactions Server-Side Defense (Phase 2) is Generally Available
> - Queues for Kafka is in Early Access
> - Kafka uses log4j2
> - Drop broker and tools support for Java 11
> - Remove old client protocol API versions
>
> Release notes for the 4.0.0 release:
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
>
> *** Please download and test the release. Voting is not necessary as
> we still have blockers.
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
>
> * Docker release artifacts to be voted upon:
> apache/kafka:4.0.0-rc0
> apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> need to investigate it)
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
>
> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> https://github.com/apache/kafka/releases/tag/4.0.0-rc0
>
> * Documentation:
> https://kafka.apache.org/40/documentation.html
>
> * Protocol:
> https://kafka.apache.org/40/protocol.html
>
> * Successful CI builds for the 4.0 branch:
> Unit/integration tests:
> https://github.com/apache/kafka/actions/runs/13459676207
> System tests: TBD
>
> * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> Docker Build Test Pipeline (JVM):
> https://github.com/apache/kafka/actions/runs/13471603921
> Docker Build Test Pipeline (Native):
> https://github.com/apache/kafka/actions/runs/13471605941
>
> /**************************************
>
> Thanks,
> David
>
Re: [VOTE] 4.0.0 RC0
Posted by TengYao Chi <ki...@gmail.com>.
Hi David,
So far, I have completed the following validations on my local machine:
- Built from source with JDK21-Zulu
- Ran all unit and integration tests
- Ran the KRaft quickstart using kafka_2.13-4.0.0.tgz
- Built JVM/native images with RC archive
- Tested JVM/native containers by following the Multiple Nodes example from
Docker Hub
All validations passed successfully on my machine.
Best Regards,
TengYao
David Jacot <da...@gmail.com> 於 2025年2月22日 週六 下午7:36寫道：
> Hi all,
>
> The native image has been published:
> https://github.com/apache/kafka/actions/runs/13471585477/job/37646356143.
> I thought that the job was stuck but it just took longer than what I
> expected. Sorry for the confusion.
>
> Best,
> David
>
> On Sat, Feb 22, 2025 at 12:10 PM TengYao Chi <ki...@gmail.com> wrote:
> >
> > Hi David,
> > Thanks for running the 4.0 release!
> >
> > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I need
> to
> > > investigate it)
> >
> >
> > I have just tried to build 4.0 native images on my local machine and it
> > worked normally.
> > (command: `python3 docker_build_test.py kafka/test --image-tag kafka-4.0
> > --image-type native --kafka-url
> >
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/kafka_2.13-4.0.0.tgz
> > --build`)
> >
> > I also test it by going through the example of the Multiple Nodes
> section.
> > Could you share more information about how it failed?
> >
> > Many thanks
> >
> > Best Regards,
> > TengYao
> >
> > David Jacot <da...@apache.org> 於 2025年2月22日 週六 下午6:17寫道：
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the first candidate for release of Apache Kafka 4.0.0. We
> > > still have some remaining blockers but we figured that getting a first
> > > release candidate will help the community to test this major release.
> > >
> > > - This is the first release without Apache Zookeeper
> > > - The Next Generation of the Consumer Rebalance Protocol is Generally
> > > Available
> > > - The Transactions Server-Side Defense (Phase 2) is Generally Available
> > > - Queues for Kafka is in Early Access
> > > - Kafka uses log4j2
> > > - Drop broker and tools support for Java 11
> > > - Remove old client protocol API versions
> > >
> > > Release notes for the 4.0.0 release:
> > >
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download and test the release. Voting is not necessary as
> > > we still have blockers.
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > https://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> > >
> > > * Docker release artifacts to be voted upon:
> > > apache/kafka:4.0.0-rc0
> > > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > > need to investigate it)
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >
> > > * Javadoc:
> > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> > >
> > > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> > >
> > > * Documentation:
> > > https://kafka.apache.org/40/documentation.html
> > >
> > > * Protocol:
> > > https://kafka.apache.org/40/protocol.html
> > >
> > > * Successful CI builds for the 4.0 branch:
> > > Unit/integration tests:
> > > https://github.com/apache/kafka/actions/runs/13459676207
> > > System tests: TBD
> > >
> > > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > > Docker Build Test Pipeline (JVM):
> > > https://github.com/apache/kafka/actions/runs/13471603921
> > > Docker Build Test Pipeline (Native):
> > > https://github.com/apache/kafka/actions/runs/13471605941
> > >
> > > /**************************************
> > >
> > > Thanks,
> > > David
> > >
>
Re: [VOTE] 4.0.0 RC0
Posted by David Jacot <da...@gmail.com>.
Hi all,
The native image has been published:
https://github.com/apache/kafka/actions/runs/13471585477/job/37646356143.
I thought that the job was stuck but it just took longer than what I
expected. Sorry for the confusion.
Best,
David
On Sat, Feb 22, 2025 at 12:10 PM TengYao Chi <ki...@gmail.com> wrote:
>
> Hi David,
> Thanks for running the 4.0 release!
>
> apache/kafka-native:4.0.0-rc0 (Building the native image failed, I need to
> > investigate it)
>
>
> I have just tried to build 4.0 native images on my local machine and it
> worked normally.
> (command: `python3 docker_build_test.py kafka/test --image-tag kafka-4.0
> --image-type native --kafka-url
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/kafka_2.13-4.0.0.tgz
> --build`)
>
> I also test it by going through the example of the Multiple Nodes section.
> Could you share more information about how it failed?
>
> Many thanks
>
> Best Regards,
> TengYao
>
> David Jacot <da...@apache.org> 於 2025年2月22日 週六 下午6:17寫道：
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for release of Apache Kafka 4.0.0. We
> > still have some remaining blockers but we figured that getting a first
> > release candidate will help the community to test this major release.
> >
> > - This is the first release without Apache Zookeeper
> > - The Next Generation of the Consumer Rebalance Protocol is Generally
> > Available
> > - The Transactions Server-Side Defense (Phase 2) is Generally Available
> > - Queues for Kafka is in Early Access
> > - Kafka uses log4j2
> > - Drop broker and tools support for Java 11
> > - Remove old client protocol API versions
> >
> > Release notes for the 4.0.0 release:
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> >
> > *** Please download and test the release. Voting is not necessary as
> > we still have blockers.
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> >
> > * Docker release artifacts to be voted upon:
> > apache/kafka:4.0.0-rc0
> > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > need to investigate it)
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> >
> > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> >
> > * Documentation:
> > https://kafka.apache.org/40/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/40/protocol.html
> >
> > * Successful CI builds for the 4.0 branch:
> > Unit/integration tests:
> > https://github.com/apache/kafka/actions/runs/13459676207
> > System tests: TBD
> >
> > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > Docker Build Test Pipeline (JVM):
> > https://github.com/apache/kafka/actions/runs/13471603921
> > Docker Build Test Pipeline (Native):
> > https://github.com/apache/kafka/actions/runs/13471605941
> >
> > /**************************************
> >
> > Thanks,
> > David
> >
Re: [VOTE] 4.0.0 RC0
Posted by TengYao Chi <ki...@gmail.com>.
Hi David,
Thanks for running the 4.0 release!
apache/kafka-native:4.0.0-rc0 (Building the native image failed, I need to
> investigate it)
I have just tried to build 4.0 native images on my local machine and it
worked normally.
(command: `python3 docker_build_test.py kafka/test --image-tag kafka-4.0
--image-type native --kafka-url
https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/kafka_2.13-4.0.0.tgz
--build`)
I also test it by going through the example of the Multiple Nodes section.
Could you share more information about how it failed?
Many thanks
Best Regards,
TengYao
David Jacot <da...@apache.org> 於 2025年2月22日 週六 下午6:17寫道：
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 4.0.0. We
> still have some remaining blockers but we figured that getting a first
> release candidate will help the community to test this major release.
>
> - This is the first release without Apache Zookeeper
> - The Next Generation of the Consumer Rebalance Protocol is Generally
> Available
> - The Transactions Server-Side Defense (Phase 2) is Generally Available
> - Queues for Kafka is in Early Access
> - Kafka uses log4j2
> - Drop broker and tools support for Java 11
> - Remove old client protocol API versions
>
> Release notes for the 4.0.0 release:
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
>
> *** Please download and test the release. Voting is not necessary as
> we still have blockers.
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
>
> * Docker release artifacts to be voted upon:
> apache/kafka:4.0.0-rc0
> apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> need to investigate it)
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
>
> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> https://github.com/apache/kafka/releases/tag/4.0.0-rc0
>
> * Documentation:
> https://kafka.apache.org/40/documentation.html
>
> * Protocol:
> https://kafka.apache.org/40/protocol.html
>
> * Successful CI builds for the 4.0 branch:
> Unit/integration tests:
> https://github.com/apache/kafka/actions/runs/13459676207
> System tests: TBD
>
> * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> Docker Build Test Pipeline (JVM):
> https://github.com/apache/kafka/actions/runs/13471603921
> Docker Build Test Pipeline (Native):
> https://github.com/apache/kafka/actions/runs/13471605941
>
> /**************************************
>
> Thanks,
> David
>
Re: [VOTE] 4.0.0 RC0
Posted by David Jacot <dj...@confluent.io.INVALID>.
Hi Jun,
Thanks. I have cherry-picked
https://issues.apache.org/jira/browse/KAFKA-18864 to 4.0.
Best,
David
On Mon, Mar 3, 2025 at 3:18 PM David Jacot <dj...@confluent.io> wrote:
> Hi all,
>
> I have opened https://github.com/apache/kafka/pull/19080 for Kirk's
> logging observation.
>
> Best,
> David
>
> On Mon, Mar 3, 2025 at 12:22 PM David Jacot <da...@gmail.com> wrote:
>
>> Hi Bruno,
>>
>> Please go ahead and cherry-pick it to 4.0.
>>
>> Best,
>> David
>>
>> On Mon, Mar 3, 2025 at 10:12 AM Bruno Cadonna <ca...@apache.org> wrote:
>> >
>> > Hi David,
>> >
>> > we found an issue with Kafka Streams that definitely is a regression. A
>> > commit fixed a leak, introduced a bug that prevented Kafka Streams from
>> > re-initializing its transactional producer in case the producer is
>> > fenced. The consequence of not re-initializing the transactional
>> > producer is an infinite loop of ProducerFencedExceptions with
>> > corresponding rebalances (approx every min).
>> >
>> > I opened a PR that reverts the defective commit:
>> > https://github.com/apache/kafka/pull/19078
>> >
>> > We tested the revert in our internal soak and everything seems fine
>> again.
>> >
>> > May I cherry-pick the revert to 4.0?
>> >
>> > Best,
>> > Bruno
>> >
>> > On 28.02.25 22:59, Ismael Juma wrote:
>> > > Given that this is effectively a documentation fix, a cherry-pick
>> makes
>> > > sense.
>> > >
>> > > Ismael
>> > >
>> > > On Fri, Feb 28, 2025 at 1:30 PM Jun Rao <ju...@confluent.io.invalid>
>> wrote:
>> > >
>> > >> Hi, David,
>> > >>
>> > >> We just fixed https://issues.apache.org/jira/browse/KAFKA-18864.
>> Not sure
>> > >> if you want to cherry-pick it to 4.0.
>> > >>
>> > >> Thanks,
>> > >>
>> > >> Jun
>> > >>
>> > >> On Thu, Feb 27, 2025 at 7:22 PM Ismael Juma <me...@ismaeljuma.com>
>> wrote:
>> > >>
>> > >>> Thanks for the testing Kirk. Can you please file an issue for the
>> > >> repeated
>> > >>> log to make sure it doesn't get lost in this thread?
>> > >>>
>> > >>> Ismael
>> > >>>
>> > >>> On Thu, Feb 27, 2025 at 2:54 PM Kirk True <ki...@kirktrue.pro>
>> wrote:
>> > >>>
>> > >>>> Hi David,
>> > >>>>
>> > >>>> I focused on testing the clients using Java 11:
>> > >>>>
>> > >>>> My steps:
>> > >>>>
>> > >>>> 1. Start the broker via the native Docker image
>> > >>>> (apache/kafka-native:4.0.0-rc0)
>> > >>>> 2. Wrote a simple client to write and read messages, using
>> > >> kafka-clients
>> > >>>> 4.0.0 from https://repository.apache.org/content/groups/staging
>> > >>>> 3. Ran my client with group.protocol set to CLASSIC
>> > >>>> 4. I then ran the client with CONSUMER
>> > >>>> 5. Finally, I ran two clients instances in the same consumer
>> group, one
>> > >>>> with a group.protocol set to CLASSIC and the other with CONSUMER
>> > >>>>
>> > >>>> Everything worked as expected.
>> > >>>>
>> > >>>> I did notice the following message showing up repeatedly in the
>> broker
>> > >>>> logs after my tests completed:
>> > >>>>
>> > >>>> Generated 0 tombstone records while cleaning up group metadata in 0
>> > >>>> milliseconds.
>> > >> (org.apache.kafka.coordinator.group.GroupCoordinatorShard)
>> > >>>>
>> > >>>> Not sure if that's expected or not.
>> > >>>>
>> > >>>> Thanks,
>> > >>>> Kirk
>> > >>>>
>> > >>>> On Wed, Feb 26, 2025, at 5:15 PM, Chia-Ping Tsai wrote:
>> > >>>>> hi David
>> > >>>>>
>> > >>>>> I apologize for not testing the release candidate (RC) as I
>> focused
>> > >> on
>> > >>>> reviewing the documentation. I have created tickets to address
>> issues
>> > >> in
>> > >>>> the documentation [0][1][2][3], which will be merged into 4.0 as
>> the
>> > >> risk
>> > >>>> is minimal.
>> > >>>>>
>> > >>>>> Please let me know if you have any other options for those
>> tickets.
>> > >>>>>
>> > >>>>> [0] https://issues.apache.org/jira/browse/KAFKA-18869
>> > >>>>> [1] https://issues.apache.org/jira/browse/KAFKA-18868
>> > >>>>> [2] https://issues.apache.org/jira/browse/KAFKA-18850
>> > >>>>> [3] https://issues.apache.org/jira/browse/KAFKA-18849
>> > >>>>>
>> > >>>>> Best,
>> > >>>>> Chia-Ping
>> > >>>>>
>> > >>>>> On 2025/02/22 10:16:43 David Jacot wrote:
>> > >>>>>> Hello Kafka users, developers and client-developers,
>> > >>>>>>
>> > >>>>>> This is the first candidate for release of Apache Kafka 4.0.0. We
>> > >>>>>> still have some remaining blockers but we figured that getting a
>> > >>> first
>> > >>>>>> release candidate will help the community to test this major
>> > >> release.
>> > >>>>>>
>> > >>>>>> - This is the first release without Apache Zookeeper
>> > >>>>>> - The Next Generation of the Consumer Rebalance Protocol is
>> > >> Generally
>> > >>>> Available
>> > >>>>>> - The Transactions Server-Side Defense (Phase 2) is Generally
>> > >>> Available
>> > >>>>>> - Queues for Kafka is in Early Access
>> > >>>>>> - Kafka uses log4j2
>> > >>>>>> - Drop broker and tools support for Java 11
>> > >>>>>> - Remove old client protocol API versions
>> > >>>>>>
>> > >>>>>> Release notes for the 4.0.0 release:
>> > >>>>>>
>> > >>>>
>> > >>>
>> > >>
>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
>> > >>>>>>
>> > >>>>>> *** Please download and test the release. Voting is not necessary
>> > >> as
>> > >>>>>> we still have blockers.
>> > >>>>>>
>> > >>>>>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> > >>>>>> https://kafka.apache.org/KEYS
>> > >>>>>>
>> > >>>>>> * Release artifacts to be voted upon (source and binary):
>> > >>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
>> > >>>>>>
>> > >>>>>> * Docker release artifacts to be voted upon:
>> > >>>>>> apache/kafka:4.0.0-rc0
>> > >>>>>> apache/kafka-native:4.0.0-rc0 (Building the native image failed,
>> I
>> > >>>>>> need to investigate it)
>> > >>>>>>
>> > >>>>>> * Maven artifacts to be voted upon:
>> > >>>>>>
>> > >>>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> > >>>>>>
>> > >>>>>> * Javadoc:
>> > >>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
>> > >>>>>>
>> > >>>>>> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
>> > >>>>>> https://github.com/apache/kafka/releases/tag/4.0.0-rc0
>> > >>>>>>
>> > >>>>>> * Documentation:
>> > >>>>>> https://kafka.apache.org/40/documentation.html
>> > >>>>>>
>> > >>>>>> * Protocol:
>> > >>>>>> https://kafka.apache.org/40/protocol.html
>> > >>>>>>
>> > >>>>>> * Successful CI builds for the 4.0 branch:
>> > >>>>>> Unit/integration tests:
>> > >>>> https://github.com/apache/kafka/actions/runs/13459676207
>> > >>>>>> System tests: TBD
>> > >>>>>>
>> > >>>>>> * Successful Docker Image Github Actions Pipeline for 4.0 branch:
>> > >>>>>> Docker Build Test Pipeline (JVM):
>> > >>>>>> https://github.com/apache/kafka/actions/runs/13471603921
>> > >>>>>> Docker Build Test Pipeline (Native):
>> > >>>>>> https://github.com/apache/kafka/actions/runs/13471605941
>> > >>>>>>
>> > >>>>>> /**************************************
>> > >>>>>>
>> > >>>>>> Thanks,
>> > >>>>>> David
>> > >>>>>>
>> > >>>>>
>> > >>>>
>> > >>>
>> > >>
>> > >
>> >
>>
>
Re: [VOTE] 4.0.0 RC0
Posted by David Jacot <dj...@confluent.io.INVALID>.
Hi all,
I have opened https://github.com/apache/kafka/pull/19080 for Kirk's logging
observation.
Best,
David
On Mon, Mar 3, 2025 at 12:22 PM David Jacot <da...@gmail.com> wrote:
> Hi Bruno,
>
> Please go ahead and cherry-pick it to 4.0.
>
> Best,
> David
>
> On Mon, Mar 3, 2025 at 10:12 AM Bruno Cadonna <ca...@apache.org> wrote:
> >
> > Hi David,
> >
> > we found an issue with Kafka Streams that definitely is a regression. A
> > commit fixed a leak, introduced a bug that prevented Kafka Streams from
> > re-initializing its transactional producer in case the producer is
> > fenced. The consequence of not re-initializing the transactional
> > producer is an infinite loop of ProducerFencedExceptions with
> > corresponding rebalances (approx every min).
> >
> > I opened a PR that reverts the defective commit:
> > https://github.com/apache/kafka/pull/19078
> >
> > We tested the revert in our internal soak and everything seems fine
> again.
> >
> > May I cherry-pick the revert to 4.0?
> >
> > Best,
> > Bruno
> >
> > On 28.02.25 22:59, Ismael Juma wrote:
> > > Given that this is effectively a documentation fix, a cherry-pick makes
> > > sense.
> > >
> > > Ismael
> > >
> > > On Fri, Feb 28, 2025 at 1:30 PM Jun Rao <ju...@confluent.io.invalid>
> wrote:
> > >
> > >> Hi, David,
> > >>
> > >> We just fixed https://issues.apache.org/jira/browse/KAFKA-18864. Not
> sure
> > >> if you want to cherry-pick it to 4.0.
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >> On Thu, Feb 27, 2025 at 7:22 PM Ismael Juma <me...@ismaeljuma.com>
> wrote:
> > >>
> > >>> Thanks for the testing Kirk. Can you please file an issue for the
> > >> repeated
> > >>> log to make sure it doesn't get lost in this thread?
> > >>>
> > >>> Ismael
> > >>>
> > >>> On Thu, Feb 27, 2025 at 2:54 PM Kirk True <ki...@kirktrue.pro> wrote:
> > >>>
> > >>>> Hi David,
> > >>>>
> > >>>> I focused on testing the clients using Java 11:
> > >>>>
> > >>>> My steps:
> > >>>>
> > >>>> 1. Start the broker via the native Docker image
> > >>>> (apache/kafka-native:4.0.0-rc0)
> > >>>> 2. Wrote a simple client to write and read messages, using
> > >> kafka-clients
> > >>>> 4.0.0 from https://repository.apache.org/content/groups/staging
> > >>>> 3. Ran my client with group.protocol set to CLASSIC
> > >>>> 4. I then ran the client with CONSUMER
> > >>>> 5. Finally, I ran two clients instances in the same consumer group,
> one
> > >>>> with a group.protocol set to CLASSIC and the other with CONSUMER
> > >>>>
> > >>>> Everything worked as expected.
> > >>>>
> > >>>> I did notice the following message showing up repeatedly in the
> broker
> > >>>> logs after my tests completed:
> > >>>>
> > >>>> Generated 0 tombstone records while cleaning up group metadata in 0
> > >>>> milliseconds.
> > >> (org.apache.kafka.coordinator.group.GroupCoordinatorShard)
> > >>>>
> > >>>> Not sure if that's expected or not.
> > >>>>
> > >>>> Thanks,
> > >>>> Kirk
> > >>>>
> > >>>> On Wed, Feb 26, 2025, at 5:15 PM, Chia-Ping Tsai wrote:
> > >>>>> hi David
> > >>>>>
> > >>>>> I apologize for not testing the release candidate (RC) as I focused
> > >> on
> > >>>> reviewing the documentation. I have created tickets to address
> issues
> > >> in
> > >>>> the documentation [0][1][2][3], which will be merged into 4.0 as the
> > >> risk
> > >>>> is minimal.
> > >>>>>
> > >>>>> Please let me know if you have any other options for those tickets.
> > >>>>>
> > >>>>> [0] https://issues.apache.org/jira/browse/KAFKA-18869
> > >>>>> [1] https://issues.apache.org/jira/browse/KAFKA-18868
> > >>>>> [2] https://issues.apache.org/jira/browse/KAFKA-18850
> > >>>>> [3] https://issues.apache.org/jira/browse/KAFKA-18849
> > >>>>>
> > >>>>> Best,
> > >>>>> Chia-Ping
> > >>>>>
> > >>>>> On 2025/02/22 10:16:43 David Jacot wrote:
> > >>>>>> Hello Kafka users, developers and client-developers,
> > >>>>>>
> > >>>>>> This is the first candidate for release of Apache Kafka 4.0.0. We
> > >>>>>> still have some remaining blockers but we figured that getting a
> > >>> first
> > >>>>>> release candidate will help the community to test this major
> > >> release.
> > >>>>>>
> > >>>>>> - This is the first release without Apache Zookeeper
> > >>>>>> - The Next Generation of the Consumer Rebalance Protocol is
> > >> Generally
> > >>>> Available
> > >>>>>> - The Transactions Server-Side Defense (Phase 2) is Generally
> > >>> Available
> > >>>>>> - Queues for Kafka is in Early Access
> > >>>>>> - Kafka uses log4j2
> > >>>>>> - Drop broker and tools support for Java 11
> > >>>>>> - Remove old client protocol API versions
> > >>>>>>
> > >>>>>> Release notes for the 4.0.0 release:
> > >>>>>>
> > >>>>
> > >>>
> > >>
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> > >>>>>>
> > >>>>>> *** Please download and test the release. Voting is not necessary
> > >> as
> > >>>>>> we still have blockers.
> > >>>>>>
> > >>>>>> Kafka's KEYS file containing PGP keys we use to sign the release:
> > >>>>>> https://kafka.apache.org/KEYS
> > >>>>>>
> > >>>>>> * Release artifacts to be voted upon (source and binary):
> > >>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> > >>>>>>
> > >>>>>> * Docker release artifacts to be voted upon:
> > >>>>>> apache/kafka:4.0.0-rc0
> > >>>>>> apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > >>>>>> need to investigate it)
> > >>>>>>
> > >>>>>> * Maven artifacts to be voted upon:
> > >>>>>>
> > >>>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >>>>>>
> > >>>>>> * Javadoc:
> > >>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> > >>>>>>
> > >>>>>> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > >>>>>> https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> > >>>>>>
> > >>>>>> * Documentation:
> > >>>>>> https://kafka.apache.org/40/documentation.html
> > >>>>>>
> > >>>>>> * Protocol:
> > >>>>>> https://kafka.apache.org/40/protocol.html
> > >>>>>>
> > >>>>>> * Successful CI builds for the 4.0 branch:
> > >>>>>> Unit/integration tests:
> > >>>> https://github.com/apache/kafka/actions/runs/13459676207
> > >>>>>> System tests: TBD
> > >>>>>>
> > >>>>>> * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > >>>>>> Docker Build Test Pipeline (JVM):
> > >>>>>> https://github.com/apache/kafka/actions/runs/13471603921
> > >>>>>> Docker Build Test Pipeline (Native):
> > >>>>>> https://github.com/apache/kafka/actions/runs/13471605941
> > >>>>>>
> > >>>>>> /**************************************
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> David
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by David Jacot <da...@gmail.com>.
Hi Bruno,
Please go ahead and cherry-pick it to 4.0.
Best,
David
On Mon, Mar 3, 2025 at 10:12 AM Bruno Cadonna <ca...@apache.org> wrote:
>
> Hi David,
>
> we found an issue with Kafka Streams that definitely is a regression. A
> commit fixed a leak, introduced a bug that prevented Kafka Streams from
> re-initializing its transactional producer in case the producer is
> fenced. The consequence of not re-initializing the transactional
> producer is an infinite loop of ProducerFencedExceptions with
> corresponding rebalances (approx every min).
>
> I opened a PR that reverts the defective commit:
> https://github.com/apache/kafka/pull/19078
>
> We tested the revert in our internal soak and everything seems fine again.
>
> May I cherry-pick the revert to 4.0?
>
> Best,
> Bruno
>
> On 28.02.25 22:59, Ismael Juma wrote:
> > Given that this is effectively a documentation fix, a cherry-pick makes
> > sense.
> >
> > Ismael
> >
> > On Fri, Feb 28, 2025 at 1:30 PM Jun Rao <ju...@confluent.io.invalid> wrote:
> >
> >> Hi, David,
> >>
> >> We just fixed https://issues.apache.org/jira/browse/KAFKA-18864. Not sure
> >> if you want to cherry-pick it to 4.0.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Thu, Feb 27, 2025 at 7:22 PM Ismael Juma <me...@ismaeljuma.com> wrote:
> >>
> >>> Thanks for the testing Kirk. Can you please file an issue for the
> >> repeated
> >>> log to make sure it doesn't get lost in this thread?
> >>>
> >>> Ismael
> >>>
> >>> On Thu, Feb 27, 2025 at 2:54 PM Kirk True <ki...@kirktrue.pro> wrote:
> >>>
> >>>> Hi David,
> >>>>
> >>>> I focused on testing the clients using Java 11:
> >>>>
> >>>> My steps:
> >>>>
> >>>> 1. Start the broker via the native Docker image
> >>>> (apache/kafka-native:4.0.0-rc0)
> >>>> 2. Wrote a simple client to write and read messages, using
> >> kafka-clients
> >>>> 4.0.0 from https://repository.apache.org/content/groups/staging
> >>>> 3. Ran my client with group.protocol set to CLASSIC
> >>>> 4. I then ran the client with CONSUMER
> >>>> 5. Finally, I ran two clients instances in the same consumer group, one
> >>>> with a group.protocol set to CLASSIC and the other with CONSUMER
> >>>>
> >>>> Everything worked as expected.
> >>>>
> >>>> I did notice the following message showing up repeatedly in the broker
> >>>> logs after my tests completed:
> >>>>
> >>>> Generated 0 tombstone records while cleaning up group metadata in 0
> >>>> milliseconds.
> >> (org.apache.kafka.coordinator.group.GroupCoordinatorShard)
> >>>>
> >>>> Not sure if that's expected or not.
> >>>>
> >>>> Thanks,
> >>>> Kirk
> >>>>
> >>>> On Wed, Feb 26, 2025, at 5:15 PM, Chia-Ping Tsai wrote:
> >>>>> hi David
> >>>>>
> >>>>> I apologize for not testing the release candidate (RC) as I focused
> >> on
> >>>> reviewing the documentation. I have created tickets to address issues
> >> in
> >>>> the documentation [0][1][2][3], which will be merged into 4.0 as the
> >> risk
> >>>> is minimal.
> >>>>>
> >>>>> Please let me know if you have any other options for those tickets.
> >>>>>
> >>>>> [0] https://issues.apache.org/jira/browse/KAFKA-18869
> >>>>> [1] https://issues.apache.org/jira/browse/KAFKA-18868
> >>>>> [2] https://issues.apache.org/jira/browse/KAFKA-18850
> >>>>> [3] https://issues.apache.org/jira/browse/KAFKA-18849
> >>>>>
> >>>>> Best,
> >>>>> Chia-Ping
> >>>>>
> >>>>> On 2025/02/22 10:16:43 David Jacot wrote:
> >>>>>> Hello Kafka users, developers and client-developers,
> >>>>>>
> >>>>>> This is the first candidate for release of Apache Kafka 4.0.0. We
> >>>>>> still have some remaining blockers but we figured that getting a
> >>> first
> >>>>>> release candidate will help the community to test this major
> >> release.
> >>>>>>
> >>>>>> - This is the first release without Apache Zookeeper
> >>>>>> - The Next Generation of the Consumer Rebalance Protocol is
> >> Generally
> >>>> Available
> >>>>>> - The Transactions Server-Side Defense (Phase 2) is Generally
> >>> Available
> >>>>>> - Queues for Kafka is in Early Access
> >>>>>> - Kafka uses log4j2
> >>>>>> - Drop broker and tools support for Java 11
> >>>>>> - Remove old client protocol API versions
> >>>>>>
> >>>>>> Release notes for the 4.0.0 release:
> >>>>>>
> >>>>
> >>>
> >> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> >>>>>>
> >>>>>> *** Please download and test the release. Voting is not necessary
> >> as
> >>>>>> we still have blockers.
> >>>>>>
> >>>>>> Kafka's KEYS file containing PGP keys we use to sign the release:
> >>>>>> https://kafka.apache.org/KEYS
> >>>>>>
> >>>>>> * Release artifacts to be voted upon (source and binary):
> >>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> >>>>>>
> >>>>>> * Docker release artifacts to be voted upon:
> >>>>>> apache/kafka:4.0.0-rc0
> >>>>>> apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> >>>>>> need to investigate it)
> >>>>>>
> >>>>>> * Maven artifacts to be voted upon:
> >>>>>>
> >>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >>>>>>
> >>>>>> * Javadoc:
> >>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> >>>>>>
> >>>>>> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> >>>>>> https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> >>>>>>
> >>>>>> * Documentation:
> >>>>>> https://kafka.apache.org/40/documentation.html
> >>>>>>
> >>>>>> * Protocol:
> >>>>>> https://kafka.apache.org/40/protocol.html
> >>>>>>
> >>>>>> * Successful CI builds for the 4.0 branch:
> >>>>>> Unit/integration tests:
> >>>> https://github.com/apache/kafka/actions/runs/13459676207
> >>>>>> System tests: TBD
> >>>>>>
> >>>>>> * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> >>>>>> Docker Build Test Pipeline (JVM):
> >>>>>> https://github.com/apache/kafka/actions/runs/13471603921
> >>>>>> Docker Build Test Pipeline (Native):
> >>>>>> https://github.com/apache/kafka/actions/runs/13471605941
> >>>>>>
> >>>>>> /**************************************
> >>>>>>
> >>>>>> Thanks,
> >>>>>> David
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by Bruno Cadonna <ca...@apache.org>.
Hi David,
we found an issue with Kafka Streams that definitely is a regression. A
commit fixed a leak, introduced a bug that prevented Kafka Streams from
re-initializing its transactional producer in case the producer is
fenced. The consequence of not re-initializing the transactional
producer is an infinite loop of ProducerFencedExceptions with
corresponding rebalances (approx every min).
I opened a PR that reverts the defective commit:
https://github.com/apache/kafka/pull/19078
We tested the revert in our internal soak and everything seems fine again.
May I cherry-pick the revert to 4.0?
Best,
Bruno
On 28.02.25 22:59, Ismael Juma wrote:
> Given that this is effectively a documentation fix, a cherry-pick makes
> sense.
>
> Ismael
>
> On Fri, Feb 28, 2025 at 1:30 PM Jun Rao <ju...@confluent.io.invalid> wrote:
>
>> Hi, David,
>>
>> We just fixed https://issues.apache.org/jira/browse/KAFKA-18864. Not sure
>> if you want to cherry-pick it to 4.0.
>>
>> Thanks,
>>
>> Jun
>>
>> On Thu, Feb 27, 2025 at 7:22 PM Ismael Juma <me...@ismaeljuma.com> wrote:
>>
>>> Thanks for the testing Kirk. Can you please file an issue for the
>> repeated
>>> log to make sure it doesn't get lost in this thread?
>>>
>>> Ismael
>>>
>>> On Thu, Feb 27, 2025 at 2:54 PM Kirk True <ki...@kirktrue.pro> wrote:
>>>
>>>> Hi David,
>>>>
>>>> I focused on testing the clients using Java 11:
>>>>
>>>> My steps:
>>>>
>>>> 1. Start the broker via the native Docker image
>>>> (apache/kafka-native:4.0.0-rc0)
>>>> 2. Wrote a simple client to write and read messages, using
>> kafka-clients
>>>> 4.0.0 from https://repository.apache.org/content/groups/staging
>>>> 3. Ran my client with group.protocol set to CLASSIC
>>>> 4. I then ran the client with CONSUMER
>>>> 5. Finally, I ran two clients instances in the same consumer group, one
>>>> with a group.protocol set to CLASSIC and the other with CONSUMER
>>>>
>>>> Everything worked as expected.
>>>>
>>>> I did notice the following message showing up repeatedly in the broker
>>>> logs after my tests completed:
>>>>
>>>> Generated 0 tombstone records while cleaning up group metadata in 0
>>>> milliseconds.
>> (org.apache.kafka.coordinator.group.GroupCoordinatorShard)
>>>>
>>>> Not sure if that's expected or not.
>>>>
>>>> Thanks,
>>>> Kirk
>>>>
>>>> On Wed, Feb 26, 2025, at 5:15 PM, Chia-Ping Tsai wrote:
>>>>> hi David
>>>>>
>>>>> I apologize for not testing the release candidate (RC) as I focused
>> on
>>>> reviewing the documentation. I have created tickets to address issues
>> in
>>>> the documentation [0][1][2][3], which will be merged into 4.0 as the
>> risk
>>>> is minimal.
>>>>>
>>>>> Please let me know if you have any other options for those tickets.
>>>>>
>>>>> [0] https://issues.apache.org/jira/browse/KAFKA-18869
>>>>> [1] https://issues.apache.org/jira/browse/KAFKA-18868
>>>>> [2] https://issues.apache.org/jira/browse/KAFKA-18850
>>>>> [3] https://issues.apache.org/jira/browse/KAFKA-18849
>>>>>
>>>>> Best,
>>>>> Chia-Ping
>>>>>
>>>>> On 2025/02/22 10:16:43 David Jacot wrote:
>>>>>> Hello Kafka users, developers and client-developers,
>>>>>>
>>>>>> This is the first candidate for release of Apache Kafka 4.0.0. We
>>>>>> still have some remaining blockers but we figured that getting a
>>> first
>>>>>> release candidate will help the community to test this major
>> release.
>>>>>>
>>>>>> - This is the first release without Apache Zookeeper
>>>>>> - The Next Generation of the Consumer Rebalance Protocol is
>> Generally
>>>> Available
>>>>>> - The Transactions Server-Side Defense (Phase 2) is Generally
>>> Available
>>>>>> - Queues for Kafka is in Early Access
>>>>>> - Kafka uses log4j2
>>>>>> - Drop broker and tools support for Java 11
>>>>>> - Remove old client protocol API versions
>>>>>>
>>>>>> Release notes for the 4.0.0 release:
>>>>>>
>>>>
>>>
>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
>>>>>>
>>>>>> *** Please download and test the release. Voting is not necessary
>> as
>>>>>> we still have blockers.
>>>>>>
>>>>>> Kafka's KEYS file containing PGP keys we use to sign the release:
>>>>>> https://kafka.apache.org/KEYS
>>>>>>
>>>>>> * Release artifacts to be voted upon (source and binary):
>>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
>>>>>>
>>>>>> * Docker release artifacts to be voted upon:
>>>>>> apache/kafka:4.0.0-rc0
>>>>>> apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
>>>>>> need to investigate it)
>>>>>>
>>>>>> * Maven artifacts to be voted upon:
>>>>>>
>>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>>>>>
>>>>>> * Javadoc:
>>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
>>>>>>
>>>>>> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
>>>>>> https://github.com/apache/kafka/releases/tag/4.0.0-rc0
>>>>>>
>>>>>> * Documentation:
>>>>>> https://kafka.apache.org/40/documentation.html
>>>>>>
>>>>>> * Protocol:
>>>>>> https://kafka.apache.org/40/protocol.html
>>>>>>
>>>>>> * Successful CI builds for the 4.0 branch:
>>>>>> Unit/integration tests:
>>>> https://github.com/apache/kafka/actions/runs/13459676207
>>>>>> System tests: TBD
>>>>>>
>>>>>> * Successful Docker Image Github Actions Pipeline for 4.0 branch:
>>>>>> Docker Build Test Pipeline (JVM):
>>>>>> https://github.com/apache/kafka/actions/runs/13471603921
>>>>>> Docker Build Test Pipeline (Native):
>>>>>> https://github.com/apache/kafka/actions/runs/13471605941
>>>>>>
>>>>>> /**************************************
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [VOTE] 4.0.0 RC0
Posted by Ismael Juma <me...@ismaeljuma.com>.
Given that this is effectively a documentation fix, a cherry-pick makes
sense.
Ismael
On Fri, Feb 28, 2025 at 1:30 PM Jun Rao <ju...@confluent.io.invalid> wrote:
> Hi, David,
>
> We just fixed https://issues.apache.org/jira/browse/KAFKA-18864. Not sure
> if you want to cherry-pick it to 4.0.
>
> Thanks,
>
> Jun
>
> On Thu, Feb 27, 2025 at 7:22 PM Ismael Juma <me...@ismaeljuma.com> wrote:
>
> > Thanks for the testing Kirk. Can you please file an issue for the
> repeated
> > log to make sure it doesn't get lost in this thread?
> >
> > Ismael
> >
> > On Thu, Feb 27, 2025 at 2:54 PM Kirk True <ki...@kirktrue.pro> wrote:
> >
> > > Hi David,
> > >
> > > I focused on testing the clients using Java 11:
> > >
> > > My steps:
> > >
> > > 1. Start the broker via the native Docker image
> > > (apache/kafka-native:4.0.0-rc0)
> > > 2. Wrote a simple client to write and read messages, using
> kafka-clients
> > > 4.0.0 from https://repository.apache.org/content/groups/staging
> > > 3. Ran my client with group.protocol set to CLASSIC
> > > 4. I then ran the client with CONSUMER
> > > 5. Finally, I ran two clients instances in the same consumer group, one
> > > with a group.protocol set to CLASSIC and the other with CONSUMER
> > >
> > > Everything worked as expected.
> > >
> > > I did notice the following message showing up repeatedly in the broker
> > > logs after my tests completed:
> > >
> > > Generated 0 tombstone records while cleaning up group metadata in 0
> > > milliseconds.
> (org.apache.kafka.coordinator.group.GroupCoordinatorShard)
> > >
> > > Not sure if that's expected or not.
> > >
> > > Thanks,
> > > Kirk
> > >
> > > On Wed, Feb 26, 2025, at 5:15 PM, Chia-Ping Tsai wrote:
> > > > hi David
> > > >
> > > > I apologize for not testing the release candidate (RC) as I focused
> on
> > > reviewing the documentation. I have created tickets to address issues
> in
> > > the documentation [0][1][2][3], which will be merged into 4.0 as the
> risk
> > > is minimal.
> > > >
> > > > Please let me know if you have any other options for those tickets.
> > > >
> > > > [0] https://issues.apache.org/jira/browse/KAFKA-18869
> > > > [1] https://issues.apache.org/jira/browse/KAFKA-18868
> > > > [2] https://issues.apache.org/jira/browse/KAFKA-18850
> > > > [3] https://issues.apache.org/jira/browse/KAFKA-18849
> > > >
> > > > Best,
> > > > Chia-Ping
> > > >
> > > > On 2025/02/22 10:16:43 David Jacot wrote:
> > > > > Hello Kafka users, developers and client-developers,
> > > > >
> > > > > This is the first candidate for release of Apache Kafka 4.0.0. We
> > > > > still have some remaining blockers but we figured that getting a
> > first
> > > > > release candidate will help the community to test this major
> release.
> > > > >
> > > > > - This is the first release without Apache Zookeeper
> > > > > - The Next Generation of the Consumer Rebalance Protocol is
> Generally
> > > Available
> > > > > - The Transactions Server-Side Defense (Phase 2) is Generally
> > Available
> > > > > - Queues for Kafka is in Early Access
> > > > > - Kafka uses log4j2
> > > > > - Drop broker and tools support for Java 11
> > > > > - Remove old client protocol API versions
> > > > >
> > > > > Release notes for the 4.0.0 release:
> > > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> > > > >
> > > > > *** Please download and test the release. Voting is not necessary
> as
> > > > > we still have blockers.
> > > > >
> > > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > https://kafka.apache.org/KEYS
> > > > >
> > > > > * Release artifacts to be voted upon (source and binary):
> > > > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> > > > >
> > > > > * Docker release artifacts to be voted upon:
> > > > > apache/kafka:4.0.0-rc0
> > > > > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > > > > need to investigate it)
> > > > >
> > > > > * Maven artifacts to be voted upon:
> > > > >
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > >
> > > > > * Javadoc:
> > > > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> > > > >
> > > > > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > > > > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> > > > >
> > > > > * Documentation:
> > > > > https://kafka.apache.org/40/documentation.html
> > > > >
> > > > > * Protocol:
> > > > > https://kafka.apache.org/40/protocol.html
> > > > >
> > > > > * Successful CI builds for the 4.0 branch:
> > > > > Unit/integration tests:
> > > https://github.com/apache/kafka/actions/runs/13459676207
> > > > > System tests: TBD
> > > > >
> > > > > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > > > > Docker Build Test Pipeline (JVM):
> > > > > https://github.com/apache/kafka/actions/runs/13471603921
> > > > > Docker Build Test Pipeline (Native):
> > > > > https://github.com/apache/kafka/actions/runs/13471605941
> > > > >
> > > > > /**************************************
> > > > >
> > > > > Thanks,
> > > > > David
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by Jun Rao <ju...@confluent.io.INVALID>.
Hi, David,
We just fixed https://issues.apache.org/jira/browse/KAFKA-18864. Not sure
if you want to cherry-pick it to 4.0.
Thanks,
Jun
On Thu, Feb 27, 2025 at 7:22 PM Ismael Juma <me...@ismaeljuma.com> wrote:
> Thanks for the testing Kirk. Can you please file an issue for the repeated
> log to make sure it doesn't get lost in this thread?
>
> Ismael
>
> On Thu, Feb 27, 2025 at 2:54 PM Kirk True <ki...@kirktrue.pro> wrote:
>
> > Hi David,
> >
> > I focused on testing the clients using Java 11:
> >
> > My steps:
> >
> > 1. Start the broker via the native Docker image
> > (apache/kafka-native:4.0.0-rc0)
> > 2. Wrote a simple client to write and read messages, using kafka-clients
> > 4.0.0 from https://repository.apache.org/content/groups/staging
> > 3. Ran my client with group.protocol set to CLASSIC
> > 4. I then ran the client with CONSUMER
> > 5. Finally, I ran two clients instances in the same consumer group, one
> > with a group.protocol set to CLASSIC and the other with CONSUMER
> >
> > Everything worked as expected.
> >
> > I did notice the following message showing up repeatedly in the broker
> > logs after my tests completed:
> >
> > Generated 0 tombstone records while cleaning up group metadata in 0
> > milliseconds. (org.apache.kafka.coordinator.group.GroupCoordinatorShard)
> >
> > Not sure if that's expected or not.
> >
> > Thanks,
> > Kirk
> >
> > On Wed, Feb 26, 2025, at 5:15 PM, Chia-Ping Tsai wrote:
> > > hi David
> > >
> > > I apologize for not testing the release candidate (RC) as I focused on
> > reviewing the documentation. I have created tickets to address issues in
> > the documentation [0][1][2][3], which will be merged into 4.0 as the risk
> > is minimal.
> > >
> > > Please let me know if you have any other options for those tickets.
> > >
> > > [0] https://issues.apache.org/jira/browse/KAFKA-18869
> > > [1] https://issues.apache.org/jira/browse/KAFKA-18868
> > > [2] https://issues.apache.org/jira/browse/KAFKA-18850
> > > [3] https://issues.apache.org/jira/browse/KAFKA-18849
> > >
> > > Best,
> > > Chia-Ping
> > >
> > > On 2025/02/22 10:16:43 David Jacot wrote:
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the first candidate for release of Apache Kafka 4.0.0. We
> > > > still have some remaining blockers but we figured that getting a
> first
> > > > release candidate will help the community to test this major release.
> > > >
> > > > - This is the first release without Apache Zookeeper
> > > > - The Next Generation of the Consumer Rebalance Protocol is Generally
> > Available
> > > > - The Transactions Server-Side Defense (Phase 2) is Generally
> Available
> > > > - Queues for Kafka is in Early Access
> > > > - Kafka uses log4j2
> > > > - Drop broker and tools support for Java 11
> > > > - Remove old client protocol API versions
> > > >
> > > > Release notes for the 4.0.0 release:
> > > >
> >
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> > > >
> > > > *** Please download and test the release. Voting is not necessary as
> > > > we still have blockers.
> > > >
> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > https://kafka.apache.org/KEYS
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> > > >
> > > > * Docker release artifacts to be voted upon:
> > > > apache/kafka:4.0.0-rc0
> > > > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > > > need to investigate it)
> > > >
> > > > * Maven artifacts to be voted upon:
> > > >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >
> > > > * Javadoc:
> > > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> > > >
> > > > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > > > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> > > >
> > > > * Documentation:
> > > > https://kafka.apache.org/40/documentation.html
> > > >
> > > > * Protocol:
> > > > https://kafka.apache.org/40/protocol.html
> > > >
> > > > * Successful CI builds for the 4.0 branch:
> > > > Unit/integration tests:
> > https://github.com/apache/kafka/actions/runs/13459676207
> > > > System tests: TBD
> > > >
> > > > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > > > Docker Build Test Pipeline (JVM):
> > > > https://github.com/apache/kafka/actions/runs/13471603921
> > > > Docker Build Test Pipeline (Native):
> > > > https://github.com/apache/kafka/actions/runs/13471605941
> > > >
> > > > /**************************************
> > > >
> > > > Thanks,
> > > > David
> > > >
> > >
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by Ismael Juma <me...@ismaeljuma.com>.
Thanks for the testing Kirk. Can you please file an issue for the repeated
log to make sure it doesn't get lost in this thread?
Ismael
On Thu, Feb 27, 2025 at 2:54 PM Kirk True <ki...@kirktrue.pro> wrote:
> Hi David,
>
> I focused on testing the clients using Java 11:
>
> My steps:
>
> 1. Start the broker via the native Docker image
> (apache/kafka-native:4.0.0-rc0)
> 2. Wrote a simple client to write and read messages, using kafka-clients
> 4.0.0 from https://repository.apache.org/content/groups/staging
> 3. Ran my client with group.protocol set to CLASSIC
> 4. I then ran the client with CONSUMER
> 5. Finally, I ran two clients instances in the same consumer group, one
> with a group.protocol set to CLASSIC and the other with CONSUMER
>
> Everything worked as expected.
>
> I did notice the following message showing up repeatedly in the broker
> logs after my tests completed:
>
> Generated 0 tombstone records while cleaning up group metadata in 0
> milliseconds. (org.apache.kafka.coordinator.group.GroupCoordinatorShard)
>
> Not sure if that's expected or not.
>
> Thanks,
> Kirk
>
> On Wed, Feb 26, 2025, at 5:15 PM, Chia-Ping Tsai wrote:
> > hi David
> >
> > I apologize for not testing the release candidate (RC) as I focused on
> reviewing the documentation. I have created tickets to address issues in
> the documentation [0][1][2][3], which will be merged into 4.0 as the risk
> is minimal.
> >
> > Please let me know if you have any other options for those tickets.
> >
> > [0] https://issues.apache.org/jira/browse/KAFKA-18869
> > [1] https://issues.apache.org/jira/browse/KAFKA-18868
> > [2] https://issues.apache.org/jira/browse/KAFKA-18850
> > [3] https://issues.apache.org/jira/browse/KAFKA-18849
> >
> > Best,
> > Chia-Ping
> >
> > On 2025/02/22 10:16:43 David Jacot wrote:
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the first candidate for release of Apache Kafka 4.0.0. We
> > > still have some remaining blockers but we figured that getting a first
> > > release candidate will help the community to test this major release.
> > >
> > > - This is the first release without Apache Zookeeper
> > > - The Next Generation of the Consumer Rebalance Protocol is Generally
> Available
> > > - The Transactions Server-Side Defense (Phase 2) is Generally Available
> > > - Queues for Kafka is in Early Access
> > > - Kafka uses log4j2
> > > - Drop broker and tools support for Java 11
> > > - Remove old client protocol API versions
> > >
> > > Release notes for the 4.0.0 release:
> > >
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download and test the release. Voting is not necessary as
> > > we still have blockers.
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > https://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> > >
> > > * Docker release artifacts to be voted upon:
> > > apache/kafka:4.0.0-rc0
> > > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > > need to investigate it)
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >
> > > * Javadoc:
> > > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> > >
> > > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> > >
> > > * Documentation:
> > > https://kafka.apache.org/40/documentation.html
> > >
> > > * Protocol:
> > > https://kafka.apache.org/40/protocol.html
> > >
> > > * Successful CI builds for the 4.0 branch:
> > > Unit/integration tests:
> https://github.com/apache/kafka/actions/runs/13459676207
> > > System tests: TBD
> > >
> > > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > > Docker Build Test Pipeline (JVM):
> > > https://github.com/apache/kafka/actions/runs/13471603921
> > > Docker Build Test Pipeline (Native):
> > > https://github.com/apache/kafka/actions/runs/13471605941
> > >
> > > /**************************************
> > >
> > > Thanks,
> > > David
> > >
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by Kirk True <ki...@kirktrue.pro>.
Hi David,
I focused on testing the clients using Java 11:
My steps:
1. Start the broker via the native Docker image (apache/kafka-native:4.0.0-rc0)
2. Wrote a simple client to write and read messages, using kafka-clients 4.0.0 from https://repository.apache.org/content/groups/staging
3. Ran my client with group.protocol set to CLASSIC
4. I then ran the client with CONSUMER
5. Finally, I ran two clients instances in the same consumer group, one with a group.protocol set to CLASSIC and the other with CONSUMER
Everything worked as expected.
I did notice the following message showing up repeatedly in the broker logs after my tests completed:
Generated 0 tombstone records while cleaning up group metadata in 0 milliseconds. (org.apache.kafka.coordinator.group.GroupCoordinatorShard)
Not sure if that's expected or not.
Thanks,
Kirk
On Wed, Feb 26, 2025, at 5:15 PM, Chia-Ping Tsai wrote:
> hi David
>
> I apologize for not testing the release candidate (RC) as I focused on reviewing the documentation. I have created tickets to address issues in the documentation [0][1][2][3], which will be merged into 4.0 as the risk is minimal.
>
> Please let me know if you have any other options for those tickets.
>
> [0] https://issues.apache.org/jira/browse/KAFKA-18869
> [1] https://issues.apache.org/jira/browse/KAFKA-18868
> [2] https://issues.apache.org/jira/browse/KAFKA-18850
> [3] https://issues.apache.org/jira/browse/KAFKA-18849
>
> Best,
> Chia-Ping
>
> On 2025/02/22 10:16:43 David Jacot wrote:
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for release of Apache Kafka 4.0.0. We
> > still have some remaining blockers but we figured that getting a first
> > release candidate will help the community to test this major release.
> >
> > - This is the first release without Apache Zookeeper
> > - The Next Generation of the Consumer Rebalance Protocol is Generally Available
> > - The Transactions Server-Side Defense (Phase 2) is Generally Available
> > - Queues for Kafka is in Early Access
> > - Kafka uses log4j2
> > - Drop broker and tools support for Java 11
> > - Remove old client protocol API versions
> >
> > Release notes for the 4.0.0 release:
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
> >
> > *** Please download and test the release. Voting is not necessary as
> > we still have blockers.
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
> >
> > * Docker release artifacts to be voted upon:
> > apache/kafka:4.0.0-rc0
> > apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> > need to investigate it)
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
> >
> > * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> > https://github.com/apache/kafka/releases/tag/4.0.0-rc0
> >
> > * Documentation:
> > https://kafka.apache.org/40/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/40/protocol.html
> >
> > * Successful CI builds for the 4.0 branch:
> > Unit/integration tests: https://github.com/apache/kafka/actions/runs/13459676207
> > System tests: TBD
> >
> > * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> > Docker Build Test Pipeline (JVM):
> > https://github.com/apache/kafka/actions/runs/13471603921
> > Docker Build Test Pipeline (Native):
> > https://github.com/apache/kafka/actions/runs/13471605941
> >
> > /**************************************
> >
> > Thanks,
> > David
> >
>
Re: [VOTE] 4.0.0 RC0
Posted by Chia-Ping Tsai <ch...@apache.org>.
hi David
I apologize for not testing the release candidate (RC) as I focused on reviewing the documentation. I have created tickets to address issues in the documentation [0][1][2][3], which will be merged into 4.0 as the risk is minimal.
Please let me know if you have any other options for those tickets.
[0] https://issues.apache.org/jira/browse/KAFKA-18869
[1] https://issues.apache.org/jira/browse/KAFKA-18868
[2] https://issues.apache.org/jira/browse/KAFKA-18850
[3] https://issues.apache.org/jira/browse/KAFKA-18849
Best,
Chia-Ping
On 2025/02/22 10:16:43 David Jacot wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 4.0.0. We
> still have some remaining blockers but we figured that getting a first
> release candidate will help the community to test this major release.
>
> - This is the first release without Apache Zookeeper
> - The Next Generation of the Consumer Rebalance Protocol is Generally Available
> - The Transactions Server-Side Defense (Phase 2) is Generally Available
> - Queues for Kafka is in Early Access
> - Kafka uses log4j2
> - Drop broker and tools support for Java 11
> - Remove old client protocol API versions
>
> Release notes for the 4.0.0 release:
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
>
> *** Please download and test the release. Voting is not necessary as
> we still have blockers.
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
>
> * Docker release artifacts to be voted upon:
> apache/kafka:4.0.0-rc0
> apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
> need to investigate it)
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
>
> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
> https://github.com/apache/kafka/releases/tag/4.0.0-rc0
>
> * Documentation:
> https://kafka.apache.org/40/documentation.html
>
> * Protocol:
> https://kafka.apache.org/40/protocol.html
>
> * Successful CI builds for the 4.0 branch:
> Unit/integration tests: https://github.com/apache/kafka/actions/runs/13459676207
> System tests: TBD
>
> * Successful Docker Image Github Actions Pipeline for 4.0 branch:
> Docker Build Test Pipeline (JVM):
> https://github.com/apache/kafka/actions/runs/13471603921
> Docker Build Test Pipeline (Native):
> https://github.com/apache/kafka/actions/runs/13471605941
>
> /**************************************
>
> Thanks,
> David
>