You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Luke Chen <sh...@gmail.com> on 2023/05/22 12:53:27 UTC
[VOTE] 3.4.1 RC1
Hello Kafka users, developers and client-developers,
This is the 2nd candidate for release of Apache Kafka 3.4.1.
This is a bugfix release with several fixes since the release of 3.4.0. A
few of the major issues include:
- core
KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644> Process
should stop after failure in raft IO thread
KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
controller node shutting down while renouncing leadership
KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK session
timeout can cause broker to shutdown
- client
KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
CooperativeStickyAssignor revokes/assigns partition in one rebalance cycle
- connect
KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2 may not
sync partition offsets correctly
KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2 should
translate consumer group offsets behind replication flow
- stream
KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug: State
stores lose state when tasks are reassigned under EOS
Release notes for the 3.4.1 release:
https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
*** Please download, test and vote by May 29, 2023
Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS
* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~showuon/kafka-3.4.1-rc0/
* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/
* Javadoc:
https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
* Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
https://github.com/apache/kafka/releases/tag/3.4.1-rc0
* Documentation: (will be updated after released)
https://kafka.apache.org/34/documentation.html
* Protocol: (will be updated after released)
https://kafka.apache.org/34/protocol.html
The most recent build has had test failures. These all appear to be due to
flakiness, but it would be nice if someone more familiar with the failed
tests could confirm this. I may update this thread with passing build links
if I can get one, or start a new release vote thread if test failures must
be addressed beyond re-running builds until they pass.
Unit/integration tests:
https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
<https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
System tests:
Will update the results later
Confirmed Maven artifacts are in staging repository.
Thank you.
Luke
Re: [VOTE] 3.4.1 RC1
Posted by Luke Chen <sh...@gmail.com>.
Hi Mickael,
Thanks for pointing this out.
Yes, I forgot to update the LICENSE-binary file.
I'll create another RC build to patch it.
Thanks Chris and Greg for pointing out these 2 issues.
I've backported into v3.4 branch. Will include them in next RC build.
Thanks Fede and Josep for helping verify this RC build.
Luke
On Tue, May 23, 2023 at 7:13 PM Mickael Maison <mi...@gmail.com>
wrote:
> Hi Luke,
>
> I've had to update the LICENSE-binary file when running the 3.5.0
> release and I noticed the file also needs updates in 3.4. If I run the
> validation logic described in
> https://issues.apache.org/jira/browse/KAFKA-12622, I get the
> following:
> audience-annotations-0.13.0 is missing in license file
> jackson-annotations-2.13.5 is missing in license file
> jackson-core-2.13.5 is missing in license file
> jackson-databind-2.13.5 is missing in license file
> jackson-dataformat-csv-2.13.5 is missing in license file
> jackson-datatype-jdk8-2.13.5 is missing in license file
> jackson-jaxrs-base-2.13.5 is missing in license file
> jackson-jaxrs-json-provider-2.13.5 is missing in license file
> jackson-module-jaxb-annotations-2.13.5 is missing in license file
> jackson-module-scala_2.13-2.13.5 is missing in license file
> javassist-3.29.2-GA is missing in license file
> javax.activation-api-1.2.0 is missing in license file
> jaxb-api-2.3.1 is missing in license file
> jersey-client-2.39.1 is missing in license file
> jersey-common-2.39.1 is missing in license file
> jersey-container-servlet-2.39.1 is missing in license file
> jersey-container-servlet-core-2.39.1 is missing in license file
> jersey-hk2-2.39.1 is missing in license file
> jersey-server-2.39.1 is missing in license file
> jetty-client-9.4.51.v20230217 is missing in license file
> jetty-continuation-9.4.51.v20230217 is missing in license file
> jetty-http-9.4.51.v20230217 is missing in license file
> jetty-io-9.4.51.v20230217 is missing in license file
> jetty-security-9.4.51.v20230217 is missing in license file
> jetty-server-9.4.51.v20230217 is missing in license file
> jetty-servlet-9.4.51.v20230217 is missing in license file
> jetty-servlets-9.4.51.v20230217 is missing in license file
> jetty-util-9.4.51.v20230217 is missing in license file
> jetty-util-ajax-9.4.51.v20230217 is missing in license file
> jline-3.22.0 is missing in license file
> netty-buffer-4.1.92.Final is missing in license file
> netty-codec-4.1.92.Final is missing in license file
> netty-common-4.1.92.Final is missing in license file
> netty-handler-4.1.92.Final is missing in license file
> netty-resolver-4.1.92.Final is missing in license file
> netty-transport-4.1.92.Final is missing in license file
> netty-transport-classes-epoll-4.1.92.Final is missing in license file
> netty-transport-native-epoll-4.1.92.Final is missing in license file
> netty-transport-native-unix-common-4.1.92.Final is missing in license file
> reload4j-1.2.25 is missing in license file
> scala-collection-compat_2.13-2.10.0 is missing in license file
>
> I think this needs to be addressed before releasing 3.4.1.
>
> Thanks,
> Mickael
>
> On Mon, May 22, 2023 at 11:08 PM Greg Harris
> <gr...@aiven.io.invalid> wrote:
> >
> > Also I'll point it out here so that everyone is aware, but I don't
> > think it necessarily warrants a new RC on it's own:
> >
> > This patch: https://github.com/apache/kafka/pull/13005 is present on
> > trunk/3.5.0, but is not present in 3.4.0/3.4.1-RC1.
> > It is not a regression strictly, since it was already a problem in 3.4.0.
> >
> > Impact: when any MM2 connector starts, it attempts to create a topic.
> > if that topic exists (as it will on every connector restart), it
> > prints an ERROR log with a TopicExistsException.
> > This is annoying and causes ERROR log noise, but does not otherwise
> > affect the operation or correctness of the connectors.
> >
> > If something else forces a new RC, perhaps consider rolling this into
> > the next RC.
> >
> > Thanks,
> > Greg
> >
> > On Mon, May 22, 2023 at 1:47 PM Greg Harris <gr...@aiven.io>
> wrote:
> > >
> > > Hi Luke,
> > >
> > > I performed a test upgrade of MM2 from 3.4.0 to 3.4.1-RC1, and
> > > verified that the new offset translation logic worked as intended.
> > >
> > > Steps I took to verify:
> > > - Downloaded 3.4.0_2.13 from the Kafka website
> > > - Formatted and started two 3.4.0 1-node clusters, and configured MM2
> > > to mirror data and consumer group offsets between those clusters.
> > > - I used the console producer to produce data in batches to the source
> > > cluster, and verified that similar amounts of data were present in the
> > > destination.
> > > - Verified that pre-upgrade, offset translation occurs smoothly.
> > > - Stopped MM2 3.4.0 and started 3.4.1-RC1 with the same configuration.
> > > - Verified that post-upgrade, offset translation occurs in steps and
> > > can be performed prior to the latest offset sync.
> > >
> > > Thanks!
> > > Greg
> > >
> > > On Mon, May 22, 2023 at 11:37 AM Josep Prat
> <jo...@aiven.io.invalid> wrote:
> > > >
> > > > Hi Luke,
> > > >
> > > > I can confirm the
> > > > *org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeFormat*
> > > > consistent failure when not running on a system set on UTC-0
> timezone.
> > > >
> > > > I verified:
> > > > - Built from source with Java 11 and Scala 2.13
> > > > - Checked signatures and hashes
> > > > - Check Javadoc and checked links to JDK javadoc are functional
> > > > - Run the unit tests (works on UTC-0 environments)
> > > > - Run integration tests
> > > > - Run the quickstart in KRaft and Zookeeper mode
> > > >
> > > > Best,
> > > >
> > > > On Mon, May 22, 2023 at 7:04 PM Chris Egerton <
> fearthecellos@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Luke,
> > > > >
> > > > > Thanks for running the release!
> > > > >
> > > > > Steps I took to verify:
> > > > >
> > > > > - Built from source with Java 11
> > > > > - Checked signatures and checksums
> > > > > - Ran the quickstart with Java 11 in KRaft mode
> > > > > - Ran all unit tests
> > > > > - - The
> org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
> > > > > test case failed consistently. It appears this is because we
> haven't
> > > > > backported a fix for KAFKA-14836 onto the 3.4 branch; after
> applying those
> > > > > changes to my local copy of the RC's source code, the test began
> to pass. I
> > > > > don't know if we want to count this as a blocker since the test
> failure is
> > > > > not indicative of actual issues with the main code base, but it
> does seem
> > > > > like a smooth backport is possible and would fix these test
> failure if we
> > > > > want to generate a new RC.
> > > > > - Ran all integration tests for Connect and MM2
> > > > >
> > > > > Aside from the noted unit test failure, evening else looks good.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Chris
> > > > >
> > > > > On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com>
> wrote:
> > > > >
> > > > > > Hi Luke,
> > > > > >
> > > > > > - Source signature and checksum
> > > > > > - Build from source with Java 17 and Scala 2.13
> > > > > > - Full unit and integration test suite
> > > > > > - Java app with staging Maven artifacts
> > > > > >
> > > > > > +1 (non binding)
> > > > > >
> > > > > > Thanks
> > > > > > Fede
> > > > > >
> > > > > > PS: Links still point to RC0, but I checked and RC1 artifacts are
> > > > > > there, including Maven. The only risk I see is that you may
> actually
> > > > > > test with the wrong artifacts. To avoid any confusion, I would
> suggest
> > > > > > to resend them on this thread.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com>
> wrote:
> > > > > > >
> > > > > > > Hello Kafka users, developers and client-developers,
> > > > > > >
> > > > > > > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> > > > > > >
> > > > > > > This is a bugfix release with several fixes since the release
> of
> > > > > 3.4.0. A
> > > > > > > few of the major issues include:
> > > > > > > - core
> > > > > > > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644
> >
> > > > > Process
> > > > > > > should stop after failure in raft IO thread
> > > > > > > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946>
> KRaft
> > > > > > > controller node shutting down while renouncing leadership
> > > > > > > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887>
> ZK
> > > > > > session
> > > > > > > timeout can cause broker to shutdown
> > > > > > > - client
> > > > > > > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639>
> Kafka
> > > > > > > CooperativeStickyAssignor revokes/assigns partition in one
> rebalance
> > > > > > cycle
> > > > > > > - connect
> > > > > > > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558>
> MM2
> > > > > may
> > > > > > not
> > > > > > > sync partition offsets correctly
> > > > > > > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666>
> MM2
> > > > > > should
> > > > > > > translate consumer group offsets behind replication flow
> > > > > > > - stream
> > > > > > > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172>
> bug:
> > > > > > State
> > > > > > > stores lose state when tasks are reassigned under EOS
> > > > > > >
> > > > > > > Release notes for the 3.4.1 release:
> > > > > > >
> https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> > > > > > >
> > > > > > > *** Please download, test and vote by May 29, 2023
> > > > > > > Kafka's KEYS file containing PGP keys we use to sign the
> release:
> > > > > > > https://kafka.apache.org/KEYS
> > > > > > >
> > > > > > > * Release artifacts to be voted upon (source and binary):
> > > > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> > > > > > >
> > > > > > > * Maven artifacts to be voted upon:
> > > > > > >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > > > >
> > > > > > > * Javadoc:
> > > > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> > > > > > >
> > > > > > > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > > > > > > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> > > > > > >
> > > > > > > * Documentation: (will be updated after released)
> > > > > > > https://kafka.apache.org/34/documentation.html
> > > > > > >
> > > > > > > * Protocol: (will be updated after released)
> > > > > > > https://kafka.apache.org/34/protocol.html
> > > > > > >
> > > > > > > The most recent build has had test failures. These all appear
> to be due
> > > > > > to
> > > > > > > flakiness, but it would be nice if someone more familiar with
> the
> > > > > failed
> > > > > > > tests could confirm this. I may update this thread with
> passing build
> > > > > > links
> > > > > > > if I can get one, or start a new release vote thread if test
> failures
> > > > > > must
> > > > > > > be addressed beyond re-running builds until they pass.
> > > > > > >
> > > > > > > Unit/integration tests:
> > > > > > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > > > > > > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/
> >
> > > > > > >
> > > > > > > System tests:
> > > > > > > Will update the results later
> > > > > > >
> > > > > > > Confirmed Maven artifacts are in staging repository.
> > > > > > >
> > > > > > > Thank you.
> > > > > > > Luke
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > [image: Aiven] <https://www.aiven.io>
> > > >
> > > > *Josep Prat*
> > > > Open Source Engineering Director, *Aiven*
> > > > josep.prat@aiven.io | +491715557497
> > > > aiven.io <https://www.aiven.io> | <
> https://www.facebook.com/aivencloud>
> > > > <https://www.linkedin.com/company/aiven/> <
> https://twitter.com/aiven_io>
> > > > *Aiven Deutschland GmbH*
> > > > Alexanderufer 3-7, 10117 Berlin
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > Amtsgericht Charlottenburg, HRB 209739 B
>
Re: [VOTE] 3.4.1 RC1
Posted by Mickael Maison <mi...@gmail.com>.
Hi Luke,
I've had to update the LICENSE-binary file when running the 3.5.0
release and I noticed the file also needs updates in 3.4. If I run the
validation logic described in
https://issues.apache.org/jira/browse/KAFKA-12622, I get the
following:
audience-annotations-0.13.0 is missing in license file
jackson-annotations-2.13.5 is missing in license file
jackson-core-2.13.5 is missing in license file
jackson-databind-2.13.5 is missing in license file
jackson-dataformat-csv-2.13.5 is missing in license file
jackson-datatype-jdk8-2.13.5 is missing in license file
jackson-jaxrs-base-2.13.5 is missing in license file
jackson-jaxrs-json-provider-2.13.5 is missing in license file
jackson-module-jaxb-annotations-2.13.5 is missing in license file
jackson-module-scala_2.13-2.13.5 is missing in license file
javassist-3.29.2-GA is missing in license file
javax.activation-api-1.2.0 is missing in license file
jaxb-api-2.3.1 is missing in license file
jersey-client-2.39.1 is missing in license file
jersey-common-2.39.1 is missing in license file
jersey-container-servlet-2.39.1 is missing in license file
jersey-container-servlet-core-2.39.1 is missing in license file
jersey-hk2-2.39.1 is missing in license file
jersey-server-2.39.1 is missing in license file
jetty-client-9.4.51.v20230217 is missing in license file
jetty-continuation-9.4.51.v20230217 is missing in license file
jetty-http-9.4.51.v20230217 is missing in license file
jetty-io-9.4.51.v20230217 is missing in license file
jetty-security-9.4.51.v20230217 is missing in license file
jetty-server-9.4.51.v20230217 is missing in license file
jetty-servlet-9.4.51.v20230217 is missing in license file
jetty-servlets-9.4.51.v20230217 is missing in license file
jetty-util-9.4.51.v20230217 is missing in license file
jetty-util-ajax-9.4.51.v20230217 is missing in license file
jline-3.22.0 is missing in license file
netty-buffer-4.1.92.Final is missing in license file
netty-codec-4.1.92.Final is missing in license file
netty-common-4.1.92.Final is missing in license file
netty-handler-4.1.92.Final is missing in license file
netty-resolver-4.1.92.Final is missing in license file
netty-transport-4.1.92.Final is missing in license file
netty-transport-classes-epoll-4.1.92.Final is missing in license file
netty-transport-native-epoll-4.1.92.Final is missing in license file
netty-transport-native-unix-common-4.1.92.Final is missing in license file
reload4j-1.2.25 is missing in license file
scala-collection-compat_2.13-2.10.0 is missing in license file
I think this needs to be addressed before releasing 3.4.1.
Thanks,
Mickael
On Mon, May 22, 2023 at 11:08 PM Greg Harris
<gr...@aiven.io.invalid> wrote:
>
> Also I'll point it out here so that everyone is aware, but I don't
> think it necessarily warrants a new RC on it's own:
>
> This patch: https://github.com/apache/kafka/pull/13005 is present on
> trunk/3.5.0, but is not present in 3.4.0/3.4.1-RC1.
> It is not a regression strictly, since it was already a problem in 3.4.0.
>
> Impact: when any MM2 connector starts, it attempts to create a topic.
> if that topic exists (as it will on every connector restart), it
> prints an ERROR log with a TopicExistsException.
> This is annoying and causes ERROR log noise, but does not otherwise
> affect the operation or correctness of the connectors.
>
> If something else forces a new RC, perhaps consider rolling this into
> the next RC.
>
> Thanks,
> Greg
>
> On Mon, May 22, 2023 at 1:47 PM Greg Harris <gr...@aiven.io> wrote:
> >
> > Hi Luke,
> >
> > I performed a test upgrade of MM2 from 3.4.0 to 3.4.1-RC1, and
> > verified that the new offset translation logic worked as intended.
> >
> > Steps I took to verify:
> > - Downloaded 3.4.0_2.13 from the Kafka website
> > - Formatted and started two 3.4.0 1-node clusters, and configured MM2
> > to mirror data and consumer group offsets between those clusters.
> > - I used the console producer to produce data in batches to the source
> > cluster, and verified that similar amounts of data were present in the
> > destination.
> > - Verified that pre-upgrade, offset translation occurs smoothly.
> > - Stopped MM2 3.4.0 and started 3.4.1-RC1 with the same configuration.
> > - Verified that post-upgrade, offset translation occurs in steps and
> > can be performed prior to the latest offset sync.
> >
> > Thanks!
> > Greg
> >
> > On Mon, May 22, 2023 at 11:37 AM Josep Prat <jo...@aiven.io.invalid> wrote:
> > >
> > > Hi Luke,
> > >
> > > I can confirm the
> > > *org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeFormat*
> > > consistent failure when not running on a system set on UTC-0 timezone.
> > >
> > > I verified:
> > > - Built from source with Java 11 and Scala 2.13
> > > - Checked signatures and hashes
> > > - Check Javadoc and checked links to JDK javadoc are functional
> > > - Run the unit tests (works on UTC-0 environments)
> > > - Run integration tests
> > > - Run the quickstart in KRaft and Zookeeper mode
> > >
> > > Best,
> > >
> > > On Mon, May 22, 2023 at 7:04 PM Chris Egerton <fe...@gmail.com>
> > > wrote:
> > >
> > > > Hi Luke,
> > > >
> > > > Thanks for running the release!
> > > >
> > > > Steps I took to verify:
> > > >
> > > > - Built from source with Java 11
> > > > - Checked signatures and checksums
> > > > - Ran the quickstart with Java 11 in KRaft mode
> > > > - Ran all unit tests
> > > > - - The org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
> > > > test case failed consistently. It appears this is because we haven't
> > > > backported a fix for KAFKA-14836 onto the 3.4 branch; after applying those
> > > > changes to my local copy of the RC's source code, the test began to pass. I
> > > > don't know if we want to count this as a blocker since the test failure is
> > > > not indicative of actual issues with the main code base, but it does seem
> > > > like a smooth backport is possible and would fix these test failure if we
> > > > want to generate a new RC.
> > > > - Ran all integration tests for Connect and MM2
> > > >
> > > > Aside from the noted unit test failure, evening else looks good.
> > > >
> > > > Cheers,
> > > >
> > > > Chris
> > > >
> > > > On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com> wrote:
> > > >
> > > > > Hi Luke,
> > > > >
> > > > > - Source signature and checksum
> > > > > - Build from source with Java 17 and Scala 2.13
> > > > > - Full unit and integration test suite
> > > > > - Java app with staging Maven artifacts
> > > > >
> > > > > +1 (non binding)
> > > > >
> > > > > Thanks
> > > > > Fede
> > > > >
> > > > > PS: Links still point to RC0, but I checked and RC1 artifacts are
> > > > > there, including Maven. The only risk I see is that you may actually
> > > > > test with the wrong artifacts. To avoid any confusion, I would suggest
> > > > > to resend them on this thread.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
> > > > > >
> > > > > > Hello Kafka users, developers and client-developers,
> > > > > >
> > > > > > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> > > > > >
> > > > > > This is a bugfix release with several fixes since the release of
> > > > 3.4.0. A
> > > > > > few of the major issues include:
> > > > > > - core
> > > > > > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644>
> > > > Process
> > > > > > should stop after failure in raft IO thread
> > > > > > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> > > > > > controller node shutting down while renouncing leadership
> > > > > > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK
> > > > > session
> > > > > > timeout can cause broker to shutdown
> > > > > > - client
> > > > > > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> > > > > > CooperativeStickyAssignor revokes/assigns partition in one rebalance
> > > > > cycle
> > > > > > - connect
> > > > > > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2
> > > > may
> > > > > not
> > > > > > sync partition offsets correctly
> > > > > > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2
> > > > > should
> > > > > > translate consumer group offsets behind replication flow
> > > > > > - stream
> > > > > > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug:
> > > > > State
> > > > > > stores lose state when tasks are reassigned under EOS
> > > > > >
> > > > > > Release notes for the 3.4.1 release:
> > > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> > > > > >
> > > > > > *** Please download, test and vote by May 29, 2023
> > > > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > > https://kafka.apache.org/KEYS
> > > > > >
> > > > > > * Release artifacts to be voted upon (source and binary):
> > > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> > > > > >
> > > > > > * Maven artifacts to be voted upon:
> > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > > >
> > > > > > * Javadoc:
> > > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> > > > > >
> > > > > > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > > > > > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> > > > > >
> > > > > > * Documentation: (will be updated after released)
> > > > > > https://kafka.apache.org/34/documentation.html
> > > > > >
> > > > > > * Protocol: (will be updated after released)
> > > > > > https://kafka.apache.org/34/protocol.html
> > > > > >
> > > > > > The most recent build has had test failures. These all appear to be due
> > > > > to
> > > > > > flakiness, but it would be nice if someone more familiar with the
> > > > failed
> > > > > > tests could confirm this. I may update this thread with passing build
> > > > > links
> > > > > > if I can get one, or start a new release vote thread if test failures
> > > > > must
> > > > > > be addressed beyond re-running builds until they pass.
> > > > > >
> > > > > > Unit/integration tests:
> > > > > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > > > > > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
> > > > > >
> > > > > > System tests:
> > > > > > Will update the results later
> > > > > >
> > > > > > Confirmed Maven artifacts are in staging repository.
> > > > > >
> > > > > > Thank you.
> > > > > > Luke
> > > > >
> > > >
> > >
> > >
> > > --
> > > [image: Aiven] <https://www.aiven.io>
> > >
> > > *Josep Prat*
> > > Open Source Engineering Director, *Aiven*
> > > josep.prat@aiven.io | +491715557497
> > > aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud>
> > > <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io>
> > > *Aiven Deutschland GmbH*
> > > Alexanderufer 3-7, 10117 Berlin
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.4.1 RC1
Posted by Greg Harris <gr...@aiven.io.INVALID>.
Also I'll point it out here so that everyone is aware, but I don't
think it necessarily warrants a new RC on it's own:
This patch: https://github.com/apache/kafka/pull/13005 is present on
trunk/3.5.0, but is not present in 3.4.0/3.4.1-RC1.
It is not a regression strictly, since it was already a problem in 3.4.0.
Impact: when any MM2 connector starts, it attempts to create a topic.
if that topic exists (as it will on every connector restart), it
prints an ERROR log with a TopicExistsException.
This is annoying and causes ERROR log noise, but does not otherwise
affect the operation or correctness of the connectors.
If something else forces a new RC, perhaps consider rolling this into
the next RC.
Thanks,
Greg
On Mon, May 22, 2023 at 1:47 PM Greg Harris <gr...@aiven.io> wrote:
>
> Hi Luke,
>
> I performed a test upgrade of MM2 from 3.4.0 to 3.4.1-RC1, and
> verified that the new offset translation logic worked as intended.
>
> Steps I took to verify:
> - Downloaded 3.4.0_2.13 from the Kafka website
> - Formatted and started two 3.4.0 1-node clusters, and configured MM2
> to mirror data and consumer group offsets between those clusters.
> - I used the console producer to produce data in batches to the source
> cluster, and verified that similar amounts of data were present in the
> destination.
> - Verified that pre-upgrade, offset translation occurs smoothly.
> - Stopped MM2 3.4.0 and started 3.4.1-RC1 with the same configuration.
> - Verified that post-upgrade, offset translation occurs in steps and
> can be performed prior to the latest offset sync.
>
> Thanks!
> Greg
>
> On Mon, May 22, 2023 at 11:37 AM Josep Prat <jo...@aiven.io.invalid> wrote:
> >
> > Hi Luke,
> >
> > I can confirm the
> > *org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeFormat*
> > consistent failure when not running on a system set on UTC-0 timezone.
> >
> > I verified:
> > - Built from source with Java 11 and Scala 2.13
> > - Checked signatures and hashes
> > - Check Javadoc and checked links to JDK javadoc are functional
> > - Run the unit tests (works on UTC-0 environments)
> > - Run integration tests
> > - Run the quickstart in KRaft and Zookeeper mode
> >
> > Best,
> >
> > On Mon, May 22, 2023 at 7:04 PM Chris Egerton <fe...@gmail.com>
> > wrote:
> >
> > > Hi Luke,
> > >
> > > Thanks for running the release!
> > >
> > > Steps I took to verify:
> > >
> > > - Built from source with Java 11
> > > - Checked signatures and checksums
> > > - Ran the quickstart with Java 11 in KRaft mode
> > > - Ran all unit tests
> > > - - The org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
> > > test case failed consistently. It appears this is because we haven't
> > > backported a fix for KAFKA-14836 onto the 3.4 branch; after applying those
> > > changes to my local copy of the RC's source code, the test began to pass. I
> > > don't know if we want to count this as a blocker since the test failure is
> > > not indicative of actual issues with the main code base, but it does seem
> > > like a smooth backport is possible and would fix these test failure if we
> > > want to generate a new RC.
> > > - Ran all integration tests for Connect and MM2
> > >
> > > Aside from the noted unit test failure, evening else looks good.
> > >
> > > Cheers,
> > >
> > > Chris
> > >
> > > On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com> wrote:
> > >
> > > > Hi Luke,
> > > >
> > > > - Source signature and checksum
> > > > - Build from source with Java 17 and Scala 2.13
> > > > - Full unit and integration test suite
> > > > - Java app with staging Maven artifacts
> > > >
> > > > +1 (non binding)
> > > >
> > > > Thanks
> > > > Fede
> > > >
> > > > PS: Links still point to RC0, but I checked and RC1 artifacts are
> > > > there, including Maven. The only risk I see is that you may actually
> > > > test with the wrong artifacts. To avoid any confusion, I would suggest
> > > > to resend them on this thread.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
> > > > >
> > > > > Hello Kafka users, developers and client-developers,
> > > > >
> > > > > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> > > > >
> > > > > This is a bugfix release with several fixes since the release of
> > > 3.4.0. A
> > > > > few of the major issues include:
> > > > > - core
> > > > > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644>
> > > Process
> > > > > should stop after failure in raft IO thread
> > > > > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> > > > > controller node shutting down while renouncing leadership
> > > > > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK
> > > > session
> > > > > timeout can cause broker to shutdown
> > > > > - client
> > > > > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> > > > > CooperativeStickyAssignor revokes/assigns partition in one rebalance
> > > > cycle
> > > > > - connect
> > > > > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2
> > > may
> > > > not
> > > > > sync partition offsets correctly
> > > > > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2
> > > > should
> > > > > translate consumer group offsets behind replication flow
> > > > > - stream
> > > > > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug:
> > > > State
> > > > > stores lose state when tasks are reassigned under EOS
> > > > >
> > > > > Release notes for the 3.4.1 release:
> > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> > > > >
> > > > > *** Please download, test and vote by May 29, 2023
> > > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > https://kafka.apache.org/KEYS
> > > > >
> > > > > * Release artifacts to be voted upon (source and binary):
> > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> > > > >
> > > > > * Maven artifacts to be voted upon:
> > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > >
> > > > > * Javadoc:
> > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> > > > >
> > > > > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > > > > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> > > > >
> > > > > * Documentation: (will be updated after released)
> > > > > https://kafka.apache.org/34/documentation.html
> > > > >
> > > > > * Protocol: (will be updated after released)
> > > > > https://kafka.apache.org/34/protocol.html
> > > > >
> > > > > The most recent build has had test failures. These all appear to be due
> > > > to
> > > > > flakiness, but it would be nice if someone more familiar with the
> > > failed
> > > > > tests could confirm this. I may update this thread with passing build
> > > > links
> > > > > if I can get one, or start a new release vote thread if test failures
> > > > must
> > > > > be addressed beyond re-running builds until they pass.
> > > > >
> > > > > Unit/integration tests:
> > > > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > > > > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
> > > > >
> > > > > System tests:
> > > > > Will update the results later
> > > > >
> > > > > Confirmed Maven artifacts are in staging repository.
> > > > >
> > > > > Thank you.
> > > > > Luke
> > > >
> > >
> >
> >
> > --
> > [image: Aiven] <https://www.aiven.io>
> >
> > *Josep Prat*
> > Open Source Engineering Director, *Aiven*
> > josep.prat@aiven.io | +491715557497
> > aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud>
> > <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io>
> > *Aiven Deutschland GmbH*
> > Alexanderufer 3-7, 10117 Berlin
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.4.1 RC1
Posted by Greg Harris <gr...@aiven.io.INVALID>.
Also I'll point it out here so that everyone is aware, but I don't
think it necessarily warrants a new RC on it's own:
This patch: https://github.com/apache/kafka/pull/13005 is present on
trunk/3.5.0, but is not present in 3.4.0/3.4.1-RC1.
It is not a regression strictly, since it was already a problem in 3.4.0.
Impact: when any MM2 connector starts, it attempts to create a topic.
if that topic exists (as it will on every connector restart), it
prints an ERROR log with a TopicExistsException.
This is annoying and causes ERROR log noise, but does not otherwise
affect the operation or correctness of the connectors.
If something else forces a new RC, perhaps consider rolling this into
the next RC.
Thanks,
Greg
On Mon, May 22, 2023 at 1:47 PM Greg Harris <gr...@aiven.io> wrote:
>
> Hi Luke,
>
> I performed a test upgrade of MM2 from 3.4.0 to 3.4.1-RC1, and
> verified that the new offset translation logic worked as intended.
>
> Steps I took to verify:
> - Downloaded 3.4.0_2.13 from the Kafka website
> - Formatted and started two 3.4.0 1-node clusters, and configured MM2
> to mirror data and consumer group offsets between those clusters.
> - I used the console producer to produce data in batches to the source
> cluster, and verified that similar amounts of data were present in the
> destination.
> - Verified that pre-upgrade, offset translation occurs smoothly.
> - Stopped MM2 3.4.0 and started 3.4.1-RC1 with the same configuration.
> - Verified that post-upgrade, offset translation occurs in steps and
> can be performed prior to the latest offset sync.
>
> Thanks!
> Greg
>
> On Mon, May 22, 2023 at 11:37 AM Josep Prat <jo...@aiven.io.invalid> wrote:
> >
> > Hi Luke,
> >
> > I can confirm the
> > *org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeFormat*
> > consistent failure when not running on a system set on UTC-0 timezone.
> >
> > I verified:
> > - Built from source with Java 11 and Scala 2.13
> > - Checked signatures and hashes
> > - Check Javadoc and checked links to JDK javadoc are functional
> > - Run the unit tests (works on UTC-0 environments)
> > - Run integration tests
> > - Run the quickstart in KRaft and Zookeeper mode
> >
> > Best,
> >
> > On Mon, May 22, 2023 at 7:04 PM Chris Egerton <fe...@gmail.com>
> > wrote:
> >
> > > Hi Luke,
> > >
> > > Thanks for running the release!
> > >
> > > Steps I took to verify:
> > >
> > > - Built from source with Java 11
> > > - Checked signatures and checksums
> > > - Ran the quickstart with Java 11 in KRaft mode
> > > - Ran all unit tests
> > > - - The org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
> > > test case failed consistently. It appears this is because we haven't
> > > backported a fix for KAFKA-14836 onto the 3.4 branch; after applying those
> > > changes to my local copy of the RC's source code, the test began to pass. I
> > > don't know if we want to count this as a blocker since the test failure is
> > > not indicative of actual issues with the main code base, but it does seem
> > > like a smooth backport is possible and would fix these test failure if we
> > > want to generate a new RC.
> > > - Ran all integration tests for Connect and MM2
> > >
> > > Aside from the noted unit test failure, evening else looks good.
> > >
> > > Cheers,
> > >
> > > Chris
> > >
> > > On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com> wrote:
> > >
> > > > Hi Luke,
> > > >
> > > > - Source signature and checksum
> > > > - Build from source with Java 17 and Scala 2.13
> > > > - Full unit and integration test suite
> > > > - Java app with staging Maven artifacts
> > > >
> > > > +1 (non binding)
> > > >
> > > > Thanks
> > > > Fede
> > > >
> > > > PS: Links still point to RC0, but I checked and RC1 artifacts are
> > > > there, including Maven. The only risk I see is that you may actually
> > > > test with the wrong artifacts. To avoid any confusion, I would suggest
> > > > to resend them on this thread.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
> > > > >
> > > > > Hello Kafka users, developers and client-developers,
> > > > >
> > > > > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> > > > >
> > > > > This is a bugfix release with several fixes since the release of
> > > 3.4.0. A
> > > > > few of the major issues include:
> > > > > - core
> > > > > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644>
> > > Process
> > > > > should stop after failure in raft IO thread
> > > > > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> > > > > controller node shutting down while renouncing leadership
> > > > > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK
> > > > session
> > > > > timeout can cause broker to shutdown
> > > > > - client
> > > > > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> > > > > CooperativeStickyAssignor revokes/assigns partition in one rebalance
> > > > cycle
> > > > > - connect
> > > > > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2
> > > may
> > > > not
> > > > > sync partition offsets correctly
> > > > > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2
> > > > should
> > > > > translate consumer group offsets behind replication flow
> > > > > - stream
> > > > > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug:
> > > > State
> > > > > stores lose state when tasks are reassigned under EOS
> > > > >
> > > > > Release notes for the 3.4.1 release:
> > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> > > > >
> > > > > *** Please download, test and vote by May 29, 2023
> > > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > https://kafka.apache.org/KEYS
> > > > >
> > > > > * Release artifacts to be voted upon (source and binary):
> > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> > > > >
> > > > > * Maven artifacts to be voted upon:
> > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > >
> > > > > * Javadoc:
> > > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> > > > >
> > > > > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > > > > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> > > > >
> > > > > * Documentation: (will be updated after released)
> > > > > https://kafka.apache.org/34/documentation.html
> > > > >
> > > > > * Protocol: (will be updated after released)
> > > > > https://kafka.apache.org/34/protocol.html
> > > > >
> > > > > The most recent build has had test failures. These all appear to be due
> > > > to
> > > > > flakiness, but it would be nice if someone more familiar with the
> > > failed
> > > > > tests could confirm this. I may update this thread with passing build
> > > > links
> > > > > if I can get one, or start a new release vote thread if test failures
> > > > must
> > > > > be addressed beyond re-running builds until they pass.
> > > > >
> > > > > Unit/integration tests:
> > > > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > > > > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
> > > > >
> > > > > System tests:
> > > > > Will update the results later
> > > > >
> > > > > Confirmed Maven artifacts are in staging repository.
> > > > >
> > > > > Thank you.
> > > > > Luke
> > > >
> > >
> >
> >
> > --
> > [image: Aiven] <https://www.aiven.io>
> >
> > *Josep Prat*
> > Open Source Engineering Director, *Aiven*
> > josep.prat@aiven.io | +491715557497
> > aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud>
> > <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io>
> > *Aiven Deutschland GmbH*
> > Alexanderufer 3-7, 10117 Berlin
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.4.1 RC1
Posted by Greg Harris <gr...@aiven.io.INVALID>.
Hi Luke,
I performed a test upgrade of MM2 from 3.4.0 to 3.4.1-RC1, and
verified that the new offset translation logic worked as intended.
Steps I took to verify:
- Downloaded 3.4.0_2.13 from the Kafka website
- Formatted and started two 3.4.0 1-node clusters, and configured MM2
to mirror data and consumer group offsets between those clusters.
- I used the console producer to produce data in batches to the source
cluster, and verified that similar amounts of data were present in the
destination.
- Verified that pre-upgrade, offset translation occurs smoothly.
- Stopped MM2 3.4.0 and started 3.4.1-RC1 with the same configuration.
- Verified that post-upgrade, offset translation occurs in steps and
can be performed prior to the latest offset sync.
Thanks!
Greg
On Mon, May 22, 2023 at 11:37 AM Josep Prat <jo...@aiven.io.invalid> wrote:
>
> Hi Luke,
>
> I can confirm the
> *org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeFormat*
> consistent failure when not running on a system set on UTC-0 timezone.
>
> I verified:
> - Built from source with Java 11 and Scala 2.13
> - Checked signatures and hashes
> - Check Javadoc and checked links to JDK javadoc are functional
> - Run the unit tests (works on UTC-0 environments)
> - Run integration tests
> - Run the quickstart in KRaft and Zookeeper mode
>
> Best,
>
> On Mon, May 22, 2023 at 7:04 PM Chris Egerton <fe...@gmail.com>
> wrote:
>
> > Hi Luke,
> >
> > Thanks for running the release!
> >
> > Steps I took to verify:
> >
> > - Built from source with Java 11
> > - Checked signatures and checksums
> > - Ran the quickstart with Java 11 in KRaft mode
> > - Ran all unit tests
> > - - The org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
> > test case failed consistently. It appears this is because we haven't
> > backported a fix for KAFKA-14836 onto the 3.4 branch; after applying those
> > changes to my local copy of the RC's source code, the test began to pass. I
> > don't know if we want to count this as a blocker since the test failure is
> > not indicative of actual issues with the main code base, but it does seem
> > like a smooth backport is possible and would fix these test failure if we
> > want to generate a new RC.
> > - Ran all integration tests for Connect and MM2
> >
> > Aside from the noted unit test failure, evening else looks good.
> >
> > Cheers,
> >
> > Chris
> >
> > On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com> wrote:
> >
> > > Hi Luke,
> > >
> > > - Source signature and checksum
> > > - Build from source with Java 17 and Scala 2.13
> > > - Full unit and integration test suite
> > > - Java app with staging Maven artifacts
> > >
> > > +1 (non binding)
> > >
> > > Thanks
> > > Fede
> > >
> > > PS: Links still point to RC0, but I checked and RC1 artifacts are
> > > there, including Maven. The only risk I see is that you may actually
> > > test with the wrong artifacts. To avoid any confusion, I would suggest
> > > to resend them on this thread.
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
> > > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> > > >
> > > > This is a bugfix release with several fixes since the release of
> > 3.4.0. A
> > > > few of the major issues include:
> > > > - core
> > > > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644>
> > Process
> > > > should stop after failure in raft IO thread
> > > > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> > > > controller node shutting down while renouncing leadership
> > > > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK
> > > session
> > > > timeout can cause broker to shutdown
> > > > - client
> > > > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> > > > CooperativeStickyAssignor revokes/assigns partition in one rebalance
> > > cycle
> > > > - connect
> > > > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2
> > may
> > > not
> > > > sync partition offsets correctly
> > > > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2
> > > should
> > > > translate consumer group offsets behind replication flow
> > > > - stream
> > > > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug:
> > > State
> > > > stores lose state when tasks are reassigned under EOS
> > > >
> > > > Release notes for the 3.4.1 release:
> > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> > > >
> > > > *** Please download, test and vote by May 29, 2023
> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > https://kafka.apache.org/KEYS
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> > > >
> > > > * Maven artifacts to be voted upon:
> > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >
> > > > * Javadoc:
> > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> > > >
> > > > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > > > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> > > >
> > > > * Documentation: (will be updated after released)
> > > > https://kafka.apache.org/34/documentation.html
> > > >
> > > > * Protocol: (will be updated after released)
> > > > https://kafka.apache.org/34/protocol.html
> > > >
> > > > The most recent build has had test failures. These all appear to be due
> > > to
> > > > flakiness, but it would be nice if someone more familiar with the
> > failed
> > > > tests could confirm this. I may update this thread with passing build
> > > links
> > > > if I can get one, or start a new release vote thread if test failures
> > > must
> > > > be addressed beyond re-running builds until they pass.
> > > >
> > > > Unit/integration tests:
> > > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > > > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
> > > >
> > > > System tests:
> > > > Will update the results later
> > > >
> > > > Confirmed Maven artifacts are in staging repository.
> > > >
> > > > Thank you.
> > > > Luke
> > >
> >
>
>
> --
> [image: Aiven] <https://www.aiven.io>
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.prat@aiven.io | +491715557497
> aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud>
> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.4.1 RC1
Posted by Greg Harris <gr...@aiven.io.INVALID>.
Hi Luke,
I performed a test upgrade of MM2 from 3.4.0 to 3.4.1-RC1, and
verified that the new offset translation logic worked as intended.
Steps I took to verify:
- Downloaded 3.4.0_2.13 from the Kafka website
- Formatted and started two 3.4.0 1-node clusters, and configured MM2
to mirror data and consumer group offsets between those clusters.
- I used the console producer to produce data in batches to the source
cluster, and verified that similar amounts of data were present in the
destination.
- Verified that pre-upgrade, offset translation occurs smoothly.
- Stopped MM2 3.4.0 and started 3.4.1-RC1 with the same configuration.
- Verified that post-upgrade, offset translation occurs in steps and
can be performed prior to the latest offset sync.
Thanks!
Greg
On Mon, May 22, 2023 at 11:37 AM Josep Prat <jo...@aiven.io.invalid> wrote:
>
> Hi Luke,
>
> I can confirm the
> *org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeFormat*
> consistent failure when not running on a system set on UTC-0 timezone.
>
> I verified:
> - Built from source with Java 11 and Scala 2.13
> - Checked signatures and hashes
> - Check Javadoc and checked links to JDK javadoc are functional
> - Run the unit tests (works on UTC-0 environments)
> - Run integration tests
> - Run the quickstart in KRaft and Zookeeper mode
>
> Best,
>
> On Mon, May 22, 2023 at 7:04 PM Chris Egerton <fe...@gmail.com>
> wrote:
>
> > Hi Luke,
> >
> > Thanks for running the release!
> >
> > Steps I took to verify:
> >
> > - Built from source with Java 11
> > - Checked signatures and checksums
> > - Ran the quickstart with Java 11 in KRaft mode
> > - Ran all unit tests
> > - - The org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
> > test case failed consistently. It appears this is because we haven't
> > backported a fix for KAFKA-14836 onto the 3.4 branch; after applying those
> > changes to my local copy of the RC's source code, the test began to pass. I
> > don't know if we want to count this as a blocker since the test failure is
> > not indicative of actual issues with the main code base, but it does seem
> > like a smooth backport is possible and would fix these test failure if we
> > want to generate a new RC.
> > - Ran all integration tests for Connect and MM2
> >
> > Aside from the noted unit test failure, evening else looks good.
> >
> > Cheers,
> >
> > Chris
> >
> > On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com> wrote:
> >
> > > Hi Luke,
> > >
> > > - Source signature and checksum
> > > - Build from source with Java 17 and Scala 2.13
> > > - Full unit and integration test suite
> > > - Java app with staging Maven artifacts
> > >
> > > +1 (non binding)
> > >
> > > Thanks
> > > Fede
> > >
> > > PS: Links still point to RC0, but I checked and RC1 artifacts are
> > > there, including Maven. The only risk I see is that you may actually
> > > test with the wrong artifacts. To avoid any confusion, I would suggest
> > > to resend them on this thread.
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
> > > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> > > >
> > > > This is a bugfix release with several fixes since the release of
> > 3.4.0. A
> > > > few of the major issues include:
> > > > - core
> > > > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644>
> > Process
> > > > should stop after failure in raft IO thread
> > > > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> > > > controller node shutting down while renouncing leadership
> > > > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK
> > > session
> > > > timeout can cause broker to shutdown
> > > > - client
> > > > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> > > > CooperativeStickyAssignor revokes/assigns partition in one rebalance
> > > cycle
> > > > - connect
> > > > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2
> > may
> > > not
> > > > sync partition offsets correctly
> > > > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2
> > > should
> > > > translate consumer group offsets behind replication flow
> > > > - stream
> > > > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug:
> > > State
> > > > stores lose state when tasks are reassigned under EOS
> > > >
> > > > Release notes for the 3.4.1 release:
> > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> > > >
> > > > *** Please download, test and vote by May 29, 2023
> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > https://kafka.apache.org/KEYS
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> > > >
> > > > * Maven artifacts to be voted upon:
> > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >
> > > > * Javadoc:
> > > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> > > >
> > > > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > > > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> > > >
> > > > * Documentation: (will be updated after released)
> > > > https://kafka.apache.org/34/documentation.html
> > > >
> > > > * Protocol: (will be updated after released)
> > > > https://kafka.apache.org/34/protocol.html
> > > >
> > > > The most recent build has had test failures. These all appear to be due
> > > to
> > > > flakiness, but it would be nice if someone more familiar with the
> > failed
> > > > tests could confirm this. I may update this thread with passing build
> > > links
> > > > if I can get one, or start a new release vote thread if test failures
> > > must
> > > > be addressed beyond re-running builds until they pass.
> > > >
> > > > Unit/integration tests:
> > > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > > > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
> > > >
> > > > System tests:
> > > > Will update the results later
> > > >
> > > > Confirmed Maven artifacts are in staging repository.
> > > >
> > > > Thank you.
> > > > Luke
> > >
> >
>
>
> --
> [image: Aiven] <https://www.aiven.io>
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.prat@aiven.io | +491715557497
> aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud>
> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.4.1 RC1
Posted by Josep Prat <jo...@aiven.io.INVALID>.
Hi Luke,
I can confirm the
*org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeFormat*
consistent failure when not running on a system set on UTC-0 timezone.
I verified:
- Built from source with Java 11 and Scala 2.13
- Checked signatures and hashes
- Check Javadoc and checked links to JDK javadoc are functional
- Run the unit tests (works on UTC-0 environments)
- Run integration tests
- Run the quickstart in KRaft and Zookeeper mode
Best,
On Mon, May 22, 2023 at 7:04 PM Chris Egerton <fe...@gmail.com>
wrote:
> Hi Luke,
>
> Thanks for running the release!
>
> Steps I took to verify:
>
> - Built from source with Java 11
> - Checked signatures and checksums
> - Ran the quickstart with Java 11 in KRaft mode
> - Ran all unit tests
> - - The org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
> test case failed consistently. It appears this is because we haven't
> backported a fix for KAFKA-14836 onto the 3.4 branch; after applying those
> changes to my local copy of the RC's source code, the test began to pass. I
> don't know if we want to count this as a blocker since the test failure is
> not indicative of actual issues with the main code base, but it does seem
> like a smooth backport is possible and would fix these test failure if we
> want to generate a new RC.
> - Ran all integration tests for Connect and MM2
>
> Aside from the noted unit test failure, evening else looks good.
>
> Cheers,
>
> Chris
>
> On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com> wrote:
>
> > Hi Luke,
> >
> > - Source signature and checksum
> > - Build from source with Java 17 and Scala 2.13
> > - Full unit and integration test suite
> > - Java app with staging Maven artifacts
> >
> > +1 (non binding)
> >
> > Thanks
> > Fede
> >
> > PS: Links still point to RC0, but I checked and RC1 artifacts are
> > there, including Maven. The only risk I see is that you may actually
> > test with the wrong artifacts. To avoid any confusion, I would suggest
> > to resend them on this thread.
> >
> >
> >
> >
> >
> >
> > On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
> > >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> > >
> > > This is a bugfix release with several fixes since the release of
> 3.4.0. A
> > > few of the major issues include:
> > > - core
> > > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644>
> Process
> > > should stop after failure in raft IO thread
> > > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> > > controller node shutting down while renouncing leadership
> > > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK
> > session
> > > timeout can cause broker to shutdown
> > > - client
> > > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> > > CooperativeStickyAssignor revokes/assigns partition in one rebalance
> > cycle
> > > - connect
> > > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2
> may
> > not
> > > sync partition offsets correctly
> > > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2
> > should
> > > translate consumer group offsets behind replication flow
> > > - stream
> > > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug:
> > State
> > > stores lose state when tasks are reassigned under EOS
> > >
> > > Release notes for the 3.4.1 release:
> > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote by May 29, 2023
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > https://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >
> > > * Javadoc:
> > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> > >
> > > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> > >
> > > * Documentation: (will be updated after released)
> > > https://kafka.apache.org/34/documentation.html
> > >
> > > * Protocol: (will be updated after released)
> > > https://kafka.apache.org/34/protocol.html
> > >
> > > The most recent build has had test failures. These all appear to be due
> > to
> > > flakiness, but it would be nice if someone more familiar with the
> failed
> > > tests could confirm this. I may update this thread with passing build
> > links
> > > if I can get one, or start a new release vote thread if test failures
> > must
> > > be addressed beyond re-running builds until they pass.
> > >
> > > Unit/integration tests:
> > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
> > >
> > > System tests:
> > > Will update the results later
> > >
> > > Confirmed Maven artifacts are in staging repository.
> > >
> > > Thank you.
> > > Luke
> >
>
--
[image: Aiven] <https://www.aiven.io>
*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.prat@aiven.io | +491715557497
aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud>
<https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io>
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.4.1 RC1
Posted by Josep Prat <jo...@aiven.io.INVALID>.
Hi Luke,
I can confirm the
*org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeFormat*
consistent failure when not running on a system set on UTC-0 timezone.
I verified:
- Built from source with Java 11 and Scala 2.13
- Checked signatures and hashes
- Check Javadoc and checked links to JDK javadoc are functional
- Run the unit tests (works on UTC-0 environments)
- Run integration tests
- Run the quickstart in KRaft and Zookeeper mode
Best,
On Mon, May 22, 2023 at 7:04 PM Chris Egerton <fe...@gmail.com>
wrote:
> Hi Luke,
>
> Thanks for running the release!
>
> Steps I took to verify:
>
> - Built from source with Java 11
> - Checked signatures and checksums
> - Ran the quickstart with Java 11 in KRaft mode
> - Ran all unit tests
> - - The org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
> test case failed consistently. It appears this is because we haven't
> backported a fix for KAFKA-14836 onto the 3.4 branch; after applying those
> changes to my local copy of the RC's source code, the test began to pass. I
> don't know if we want to count this as a blocker since the test failure is
> not indicative of actual issues with the main code base, but it does seem
> like a smooth backport is possible and would fix these test failure if we
> want to generate a new RC.
> - Ran all integration tests for Connect and MM2
>
> Aside from the noted unit test failure, evening else looks good.
>
> Cheers,
>
> Chris
>
> On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com> wrote:
>
> > Hi Luke,
> >
> > - Source signature and checksum
> > - Build from source with Java 17 and Scala 2.13
> > - Full unit and integration test suite
> > - Java app with staging Maven artifacts
> >
> > +1 (non binding)
> >
> > Thanks
> > Fede
> >
> > PS: Links still point to RC0, but I checked and RC1 artifacts are
> > there, including Maven. The only risk I see is that you may actually
> > test with the wrong artifacts. To avoid any confusion, I would suggest
> > to resend them on this thread.
> >
> >
> >
> >
> >
> >
> > On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
> > >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> > >
> > > This is a bugfix release with several fixes since the release of
> 3.4.0. A
> > > few of the major issues include:
> > > - core
> > > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644>
> Process
> > > should stop after failure in raft IO thread
> > > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> > > controller node shutting down while renouncing leadership
> > > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK
> > session
> > > timeout can cause broker to shutdown
> > > - client
> > > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> > > CooperativeStickyAssignor revokes/assigns partition in one rebalance
> > cycle
> > > - connect
> > > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2
> may
> > not
> > > sync partition offsets correctly
> > > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2
> > should
> > > translate consumer group offsets behind replication flow
> > > - stream
> > > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug:
> > State
> > > stores lose state when tasks are reassigned under EOS
> > >
> > > Release notes for the 3.4.1 release:
> > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote by May 29, 2023
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > https://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >
> > > * Javadoc:
> > > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> > >
> > > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> > >
> > > * Documentation: (will be updated after released)
> > > https://kafka.apache.org/34/documentation.html
> > >
> > > * Protocol: (will be updated after released)
> > > https://kafka.apache.org/34/protocol.html
> > >
> > > The most recent build has had test failures. These all appear to be due
> > to
> > > flakiness, but it would be nice if someone more familiar with the
> failed
> > > tests could confirm this. I may update this thread with passing build
> > links
> > > if I can get one, or start a new release vote thread if test failures
> > must
> > > be addressed beyond re-running builds until they pass.
> > >
> > > Unit/integration tests:
> > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
> > >
> > > System tests:
> > > Will update the results later
> > >
> > > Confirmed Maven artifacts are in staging repository.
> > >
> > > Thank you.
> > > Luke
> >
>
--
[image: Aiven] <https://www.aiven.io>
*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.prat@aiven.io | +491715557497
aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud>
<https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io>
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.4.1 RC1
Posted by Chris Egerton <fe...@gmail.com>.
Hi Luke,
Thanks for running the release!
Steps I took to verify:
- Built from source with Java 11
- Checked signatures and checksums
- Ran the quickstart with Java 11 in KRaft mode
- Ran all unit tests
- - The org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
test case failed consistently. It appears this is because we haven't
backported a fix for KAFKA-14836 onto the 3.4 branch; after applying those
changes to my local copy of the RC's source code, the test began to pass. I
don't know if we want to count this as a blocker since the test failure is
not indicative of actual issues with the main code base, but it does seem
like a smooth backport is possible and would fix these test failure if we
want to generate a new RC.
- Ran all integration tests for Connect and MM2
Aside from the noted unit test failure, evening else looks good.
Cheers,
Chris
On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com> wrote:
> Hi Luke,
>
> - Source signature and checksum
> - Build from source with Java 17 and Scala 2.13
> - Full unit and integration test suite
> - Java app with staging Maven artifacts
>
> +1 (non binding)
>
> Thanks
> Fede
>
> PS: Links still point to RC0, but I checked and RC1 artifacts are
> there, including Maven. The only risk I see is that you may actually
> test with the wrong artifacts. To avoid any confusion, I would suggest
> to resend them on this thread.
>
>
>
>
>
>
> On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
> >
> > Hello Kafka users, developers and client-developers,
> >
> > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> >
> > This is a bugfix release with several fixes since the release of 3.4.0. A
> > few of the major issues include:
> > - core
> > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644> Process
> > should stop after failure in raft IO thread
> > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> > controller node shutting down while renouncing leadership
> > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK
> session
> > timeout can cause broker to shutdown
> > - client
> > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> > CooperativeStickyAssignor revokes/assigns partition in one rebalance
> cycle
> > - connect
> > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2 may
> not
> > sync partition offsets correctly
> > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2
> should
> > translate consumer group offsets behind replication flow
> > - stream
> > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug:
> State
> > stores lose state when tasks are reassigned under EOS
> >
> > Release notes for the 3.4.1 release:
> > https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by May 29, 2023
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> >
> > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> >
> > * Documentation: (will be updated after released)
> > https://kafka.apache.org/34/documentation.html
> >
> > * Protocol: (will be updated after released)
> > https://kafka.apache.org/34/protocol.html
> >
> > The most recent build has had test failures. These all appear to be due
> to
> > flakiness, but it would be nice if someone more familiar with the failed
> > tests could confirm this. I may update this thread with passing build
> links
> > if I can get one, or start a new release vote thread if test failures
> must
> > be addressed beyond re-running builds until they pass.
> >
> > Unit/integration tests:
> > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
> >
> > System tests:
> > Will update the results later
> >
> > Confirmed Maven artifacts are in staging repository.
> >
> > Thank you.
> > Luke
>
Re: [VOTE] 3.4.1 RC1
Posted by Chris Egerton <fe...@gmail.com>.
Hi Luke,
Thanks for running the release!
Steps I took to verify:
- Built from source with Java 11
- Checked signatures and checksums
- Ran the quickstart with Java 11 in KRaft mode
- Ran all unit tests
- - The org.apache.kafka.common.utils.UtilsTest.testToLogDateTimeDormat
test case failed consistently. It appears this is because we haven't
backported a fix for KAFKA-14836 onto the 3.4 branch; after applying those
changes to my local copy of the RC's source code, the test began to pass. I
don't know if we want to count this as a blocker since the test failure is
not indicative of actual issues with the main code base, but it does seem
like a smooth backport is possible and would fix these test failure if we
want to generate a new RC.
- Ran all integration tests for Connect and MM2
Aside from the noted unit test failure, evening else looks good.
Cheers,
Chris
On Mon, May 22, 2023, 10:50 Federico Valeri <fe...@gmail.com> wrote:
> Hi Luke,
>
> - Source signature and checksum
> - Build from source with Java 17 and Scala 2.13
> - Full unit and integration test suite
> - Java app with staging Maven artifacts
>
> +1 (non binding)
>
> Thanks
> Fede
>
> PS: Links still point to RC0, but I checked and RC1 artifacts are
> there, including Maven. The only risk I see is that you may actually
> test with the wrong artifacts. To avoid any confusion, I would suggest
> to resend them on this thread.
>
>
>
>
>
>
> On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
> >
> > Hello Kafka users, developers and client-developers,
> >
> > This is the 2nd candidate for release of Apache Kafka 3.4.1.
> >
> > This is a bugfix release with several fixes since the release of 3.4.0. A
> > few of the major issues include:
> > - core
> > KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644> Process
> > should stop after failure in raft IO thread
> > KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> > controller node shutting down while renouncing leadership
> > KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK
> session
> > timeout can cause broker to shutdown
> > - client
> > KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> > CooperativeStickyAssignor revokes/assigns partition in one rebalance
> cycle
> > - connect
> > KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2 may
> not
> > sync partition offsets correctly
> > KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2
> should
> > translate consumer group offsets behind replication flow
> > - stream
> > KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug:
> State
> > stores lose state when tasks are reassigned under EOS
> >
> > Release notes for the 3.4.1 release:
> > https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by May 29, 2023
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://home.apache.org/~showuon/kafka-3.4.1-rc0/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
> >
> > * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> > https://github.com/apache/kafka/releases/tag/3.4.1-rc0
> >
> > * Documentation: (will be updated after released)
> > https://kafka.apache.org/34/documentation.html
> >
> > * Protocol: (will be updated after released)
> > https://kafka.apache.org/34/protocol.html
> >
> > The most recent build has had test failures. These all appear to be due
> to
> > flakiness, but it would be nice if someone more familiar with the failed
> > tests could confirm this. I may update this thread with passing build
> links
> > if I can get one, or start a new release vote thread if test failures
> must
> > be addressed beyond re-running builds until they pass.
> >
> > Unit/integration tests:
> > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> > <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
> >
> > System tests:
> > Will update the results later
> >
> > Confirmed Maven artifacts are in staging repository.
> >
> > Thank you.
> > Luke
>
Re: [VOTE] 3.4.1 RC1
Posted by Federico Valeri <fe...@gmail.com>.
Hi Luke,
- Source signature and checksum
- Build from source with Java 17 and Scala 2.13
- Full unit and integration test suite
- Java app with staging Maven artifacts
+1 (non binding)
Thanks
Fede
PS: Links still point to RC0, but I checked and RC1 artifacts are
there, including Maven. The only risk I see is that you may actually
test with the wrong artifacts. To avoid any confusion, I would suggest
to resend them on this thread.
On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
>
> Hello Kafka users, developers and client-developers,
>
> This is the 2nd candidate for release of Apache Kafka 3.4.1.
>
> This is a bugfix release with several fixes since the release of 3.4.0. A
> few of the major issues include:
> - core
> KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644> Process
> should stop after failure in raft IO thread
> KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> controller node shutting down while renouncing leadership
> KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK session
> timeout can cause broker to shutdown
> - client
> KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> CooperativeStickyAssignor revokes/assigns partition in one rebalance cycle
> - connect
> KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2 may not
> sync partition offsets correctly
> KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2 should
> translate consumer group offsets behind replication flow
> - stream
> KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug: State
> stores lose state when tasks are reassigned under EOS
>
> Release notes for the 3.4.1 release:
> https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by May 29, 2023
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~showuon/kafka-3.4.1-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
>
> * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> https://github.com/apache/kafka/releases/tag/3.4.1-rc0
>
> * Documentation: (will be updated after released)
> https://kafka.apache.org/34/documentation.html
>
> * Protocol: (will be updated after released)
> https://kafka.apache.org/34/protocol.html
>
> The most recent build has had test failures. These all appear to be due to
> flakiness, but it would be nice if someone more familiar with the failed
> tests could confirm this. I may update this thread with passing build links
> if I can get one, or start a new release vote thread if test failures must
> be addressed beyond re-running builds until they pass.
>
> Unit/integration tests:
> https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
>
> System tests:
> Will update the results later
>
> Confirmed Maven artifacts are in staging repository.
>
> Thank you.
> Luke
Re: [VOTE] 3.4.1 RC1
Posted by Federico Valeri <fe...@gmail.com>.
Hi Luke,
- Source signature and checksum
- Build from source with Java 17 and Scala 2.13
- Full unit and integration test suite
- Java app with staging Maven artifacts
+1 (non binding)
Thanks
Fede
PS: Links still point to RC0, but I checked and RC1 artifacts are
there, including Maven. The only risk I see is that you may actually
test with the wrong artifacts. To avoid any confusion, I would suggest
to resend them on this thread.
On Mon, May 22, 2023 at 2:53 PM Luke Chen <sh...@gmail.com> wrote:
>
> Hello Kafka users, developers and client-developers,
>
> This is the 2nd candidate for release of Apache Kafka 3.4.1.
>
> This is a bugfix release with several fixes since the release of 3.4.0. A
> few of the major issues include:
> - core
> KAFKA-14644 <https://issues.apache.org/jira/browse/KAFKA-14644> Process
> should stop after failure in raft IO thread
> KAFKA-14946 <https://issues.apache.org/jira/browse/KAFKA-14946> KRaft
> controller node shutting down while renouncing leadership
> KAFKA-14887 <https://issues.apache.org/jira/browse/KAFKA-14887> ZK session
> timeout can cause broker to shutdown
> - client
> KAFKA-14639 <https://issues.apache.org/jira/browse/KAFKA-14639> Kafka
> CooperativeStickyAssignor revokes/assigns partition in one rebalance cycle
> - connect
> KAFKA-12558 <https://issues.apache.org/jira/browse/KAFKA-12558> MM2 may not
> sync partition offsets correctly
> KAFKA-14666 <https://issues.apache.org/jira/browse/KAFKA-14666> MM2 should
> translate consumer group offsets behind replication flow
> - stream
> KAFKA-14172 <https://issues.apache.org/jira/browse/KAFKA-14172> bug: State
> stores lose state when tasks are reassigned under EOS
>
> Release notes for the 3.4.1 release:
> https://home.apache.org/~showuon/kafka-3.4.1-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by May 29, 2023
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~showuon/kafka-3.4.1-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~showuon/kafka-3.4.1-rc0/javadoc/
>
> * Tag to be voted upon (off 3.4 branch) is the 3.4.1 tag:
> https://github.com/apache/kafka/releases/tag/3.4.1-rc0
>
> * Documentation: (will be updated after released)
> https://kafka.apache.org/34/documentation.html
>
> * Protocol: (will be updated after released)
> https://kafka.apache.org/34/protocol.html
>
> The most recent build has had test failures. These all appear to be due to
> flakiness, but it would be nice if someone more familiar with the failed
> tests could confirm this. I may update this thread with passing build links
> if I can get one, or start a new release vote thread if test failures must
> be addressed beyond re-running builds until they pass.
>
> Unit/integration tests:
> https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/135/
> <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.4/133/>
>
> System tests:
> Will update the results later
>
> Confirmed Maven artifacts are in staging repository.
>
> Thank you.
> Luke