You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by Luke Chen <sh...@gmail.com> on 2024/11/01 04:03:12 UTC

Re: [VOTE] 3.9.0 RC5

Hi Colin,

Thanks for the update.
I've ran a sanity check as Mickael suggested for the maven artifacts with
the one in https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/.
I used jarcomp <https://activityworkshop.net/software/jarcomp/index.html>
to verify:
1. kafka_2.13-3.9.0.jar
2. kafka-clients-3.9.0.jar
3. kafka-metadata-3.9.0.jar

I confirmed all the class files in the jars have the same content.

So +1 (binding) from me.

Thank you.
Luke

On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <mi...@gmail.com>
wrote:

> Hi,
>
> Can we add sanity checks to ensure all artifacts are present when
> running the release?
> Also should we use a different folder for each RC? This would
> eliminate the risk of artifacts from previous RCs hiding an issue.
> Luke spotted the old artifacts this time but this could have been
> missed.
>
> Thanks,
> Mickael
>
> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cm...@apache.org> wrote:
> >
> > Hi David,
> >
> > Thanks for checking. I took a look and yes, I did forget to upload the
> Scala 2.12 artifacts previously. The script runs gradlewAll, but when I was
> trying to do the steps manually, I missed that and just ran gradlew, which
> only did Scala 2.13.
> >
> > This should be fixed now!
> >
> > The scala 2.12 artifacts are here:
> >
> > https://repository.apache.org/content/repositories/orgapachekafka-1827/
> >
> > and also here:
> >
> >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/
> >
> > I wonder if we should change the release.py script to just run the
> gradlew steps manually. The gradlewAll script seems a bit confusing (or
> maybe that's just me?)
> >
> > best,
> > Colin
> >
> >
> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote:
> > > Should there be Scala 2.12 artifacts in the Maven staging?
> > >
> > > I see these for 2.13:
> > > *
> > >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/
> > > *
> > >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/
> > >
> > > but I don't see 2.12 for 3.9.0 release.
> > >
> > > I can confirm that the other artifacts (such as kafka-clients) do have
> > > today's timestamp for the 3.9.0 artifact.
> > >
> > > -David
> > >
> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <cm...@apache.org>
> wrote:
> > >
> > >> Thanks for checking, Luke. I did actually visit
> > >> https://repository.apache.org/#stagingRepositories and hit close on
> the
> > >> artifacts as the release.py script says to do. I don't know why they
> ended
> > >> up not showing up.
> > >>
> > >> Today I dropped all the maven artifacts for previous 3.9 RCs,
> switched to
> > >> java 8, and re-ran "gradlew publish" and "mvn deploy -Pgpg-signing".
> Then I
> > >> closed the two artifacts that were uploaded. You can see them here:
> > >>
> > >>
> https://repository.apache.org/content/repositories/orgapachekafka-1825/
> > >>
> https://repository.apache.org/content/repositories/orgapachekafka-1826/
> > >>
> > >> Hopefully these should now show up in Maven as expected.
> > >>
> > >> best,
> > >> Colin
> > >>
> > >>
> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote:
> > >> > Hi Colin,
> > >> >
> > >> > I was trying to test the RC, but I found the artifacts in Maven
> artifacts
> > >> > to be voted upon are not up-to-date.
> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are on
> Oct. 10,
> > >> > but also the source code in the artifacts didn't include the latest
> > >> commits
> > >> > in RC5 here <https://github.com/apache/kafka/commits/3.9.0-rc5/>.
> > >> >
> > >> > Could you help verify it?
> > >> >
> > >> > But I confirmed the release artifacts in "
> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/" is
> up-to-date.
> > >> > I did:
> > >> > - checked the checksums and signatures
> > >> > - ran upgrade from v3.8.1 to v3.9.0
> > >> > - ran quickstart with the 2.13 binaries
> > >> > - verify tiered storage and KIP-950 (dynamically disablement)
> > >> >
> > >> > I'll be +1 after the maven artifacts are updated.
> > >> >
> > >> > Thank you for running the release.
> > >> > Luke
> > >> >
> > >> >
> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan
> > >> <jo...@confluent.io.invalid>
> > >> > wrote:
> > >> >
> > >> >> Hey Colin,
> > >> >>
> > >> >> Thanks for the new RC. I've tested the --feature functionality
> that was
> > >> >> missing before. Seems to be working as expected now.
> > >> >> As a side note, I was looking into how to configure kip-853 and it
> took
> > >> me
> > >> >> a moment to find the instructions in the documentation, I wonder
> if it
> > >> >> could be included in the notable changes section of the upgrade
> notes.
> > >> Not
> > >> >> a blocker as I believe we can still update documentation.
> > >> >>
> > >> >> I did some other spot checks on the rest of the release. +1
> (binding)
> > >> from
> > >> >> me
> > >> >>
> > >> >> Justine
> > >> >>
> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <cm...@apache.org>
> > >> wrote:
> > >> >>
> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look at
> how we
> > >> can
> > >> >> > improve the docs here...
> > >> >> >
> > >> >> > best,
> > >> >> > Colin
> > >> >> >
> > >> >> >
> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote:
> > >> >> > > Hi Chia-Ping,
> > >> >> > >
> > >> >> > > Thanks for pointing those two fields out. I retract my -1.
> > >> >> > >
> > >> >> > > Cheers,
> > >> >> > > Anton
> > >> >> > >
> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai <
> > >> >> chia7712@apache.org
> > >> >> > >:
> > >> >> > >
> > >> >> > >> hi Anton
> > >> >> > >>
> > >> >> > >> Thanks for sharing your insights on Kafka serialization—it’s
> really
> > >> >> cool
> > >> >> > >> and interesting to me. Additionally, you inspired me to file
> a JIRA
> > >> >> > issue
> > >> >> > >> (KAFKA-17882) to improve the documentation.
> > >> >> > >>
> > >> >> > >> The most important aspect of Kafka is compatibility, and the
> > >> >> > undocumented
> > >> >> > >> behavior has been in place for some time [0][1]. This means
> > >> there’s no
> > >> >> > need
> > >> >> > >> to rush your improvement for 3.9, as we’ll need to explicitly
> add
> > >> >> > default
> > >> >> > >> values after applying your patch to ensure we generate the
> same
> > >> binary
> > >> >> > data.
> > >> >> > >>
> > >> >> > >> In short, we can improve the documentation first. In the
> meantime,
> > >> we
> > >> >> > can
> > >> >> > >> continue discussing behavior clarification for 4.0, and RM
> can keep
> > >> >> > running
> > >> >> > >> the RC for 3.9. Everything is on track.
> > >> >> > >>
> > >> >> > >> Best,
> > >> >> > >> Chia-Ping
> > >> >> > >>
> > >> >> > >> [0]
> > >> >> > >>
> > >> >> >
> > >> >>
> > >>
> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43
> > >> >> > >> [1]
> > >> >> > >>
> > >> >> >
> > >> >>
> > >>
> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39
> > >> >> > >>
> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote:
> > >> >> > >> > -1, refer to comments on the RC 2 thread.
> > >> >> > >> >
> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe <
> > >> >> cmccabe@apache.org
> > >> >> > >:
> > >> >> > >> >
> > >> >> > >> > > This is the RC5 candidate for the release of Apache Kafka
> > >> 3.9.0.
> > >> >> > >> > >
> > >> >> > >> > > - This is a major release, the final one in the 3.x line.
> > >> (There
> > >> >> > may of
> > >> >> > >> > > course be other minor releases in this line, such as
> 3.9.1.)
> > >> >> > >> > > - Tiered storage will be considered production-ready in
> this
> > >> >> > release.
> > >> >> > >> > > - This will be the final major release to feature the
> > >> deprecated
> > >> >> > >> ZooKeeper
> > >> >> > >> > > mode.
> > >> >> > >> > >
> > >> >> > >> > > This release includes the following KIPs:
> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft controller
> > >> >> membership
> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the dump log
> tool
> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to Kafka
> > >> Streams
> > >> >> > >> > > - KIP-1040: Improve handling of nullable values in
> InsertField,
> > >> >> > >> > > ExtractField, and other transformations
> > >> >> > >> > > - KIP-1031: Control offset translation in
> MirrorSourceConnector
> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for
> exceptions
> > >> >> > >> occurring
> > >> >> > >> > > during processing
> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect
> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and
> clientSecret in
> > >> >> > >> > > authorization header
> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and TieredOffset
> > >> >> > >> > > - KIP-950: Tiered Storage Disablement
> > >> >> > >> > > - KIP-956: Tiered Storage Quotas
> > >> >> > >> > >
> > >> >> > >> > > Release notes for the 3.9.0 release:
> > >> >> > >> > >
> > >> >> > >>
> > >> >> >
> > >> >>
> > >>
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html
> > >> >> > >> > >
> > >> >> > >> > > *** Please download, test and vote by October 30, 2024.
> > >> >> > >> > >
> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to sign the
> > >> release:
> > >> >> > >> > > https://kafka.apache.org/KEYS
> > >> >> > >> > >
> > >> >> > >> > > * Release artifacts to be voted upon (source and binary):
> > >> >> > >> > > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/
> > >> >> > >> > >
> > >> >> > >> > > * Docker release artifacts to be voted upon:
> > >> >> > >> > > apache/kafka:3.9.0-rc5
> > >> >> > >> > > apache/kafka-native:3.9.0-rc5
> > >> >> > >> > >
> > >> >> > >> > > * Maven artifacts to be voted upon:
> > >> >> > >> > >
> > >> >> >
> > >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >> >> > >> > >
> > >> >> > >> > > * Javadoc:
> > >> >> > >> > >
> > >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/
> > >> >> > >> > >
> > >> >> > >> > > * Documentation:
> > >> >> > >> > > https://kafka.apache.org/39/documentation.html
> > >> >> > >> > >
> > >> >> > >> > > * Protocol:
> > >> >> > >> > > https://kafka.apache.org/39/protocol.html
> > >> >> > >> > >
> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc5
> tag:
> > >> >> > >> > > https://github.com/apache/kafka/releases/tag/3.9.0-rc5
> > >> >> > >> > >
> > >> >> > >> > > * Successful Docker Image Github Actions Pipeline for 3.9
> > >> branch:
> > >> >> > >> > > Docker Build Test Pipeline (JVM):
> > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535300463
> > >> >> > >> > > Docker Build Test Pipeline (Native):
> > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535328957
> > >> >> > >> > >
> > >> >> > >> > > Thanks to everyone who helped with this release candidate,
> > >> either
> > >> >> by
> > >> >> > >> > > contributing code, testing, or documentation.
> > >> >> > >> > >
> > >> >> > >> > > Regards,
> > >> >> > >> > > Colin
> > >> >> > >> > >
> > >> >> > >> >
> > >> >> > >>
> > >> >> >
> > >> >>
> > >>
> > >
> > >
> > > --
> > > David Arthur
>

Re: [VOTE] 3.9.0 RC5

Posted by Chia-Ping Tsai <ch...@apache.org>.

hi Kamal

Nice find. I checked those tests, and they are unrelated to production code—they're just something we missed during the backport. I've filed KAFKA-17969 to fix it.

Thanks,
Chia-Ping

On 2024/11/08 10:40:07 Kamal Chandraprakash wrote:
> Hi all,
> 
> I took the 3.9 branch <https://github.com/apache/kafka/tree/3.9> from
> upstream and ran all the unit tests. The StorageToolTest unit test is
> failing consistently:
> 
>  % ./gradlew core:test --tests StorageToolTest
> 
> Gradle Test Run :core:test > Gradle Test Executor 20 > StorageToolTest >
> testFormatWithReleaseVersionAsFeature() FAILED
>     org.opentest4j.AssertionFailedError: Failed to find content in output:
> Formatting metadata directory
> /var/folders/bq/w6tnkbq964q8sqpvj4fbmjr40000gq/T/kafka-12278605357058017295
> with metadata.version 3.8-IV0.
>      ==> expected: <true> but was: <false>
>         at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
>         at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
>         at
> app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
>         at
> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
>         at
> app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
>         at
> app//kafka.tools.StorageToolTest.testFormatWithReleaseVersionAsFeature(StorageToolTest.scala:329)
> 
> 
> Gradle Test Run :core:test > Gradle Test Executor 20 > StorageToolTest >
> testFormatWithInvalidFeature() FAILED
>     org.opentest4j.AssertionFailedError: expected: <Unsupported feature:
> non.existent.feature. Supported features are: group.version, kraft.version,
> transaction.version> but was: <Unsupported feature: non.existent.feature.
> Supported features are: kraft.version, transaction.version>
>         at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
>         at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
>         at
> app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
>         at
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
>         at
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
>         at
> app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1145)
>         at
> app//kafka.tools.StorageToolTest.testFormatWithInvalidFeature(StorageToolTest.scala:340)
> 
> Am I missing something?
> 
> 
> On Fri, Nov 8, 2024 at 12:50 AM Colin McCabe <cm...@apache.org> wrote:
> 
> > Thanks, Josep.
> >
> > best,
> > Colin
> >
> > On Wed, Nov 6, 2024, at 23:58, Josep Prat wrote:
> > > Hi Colin,
> > > I see that you made 3.9 the default docs version in Kafka-Site. However,
> > as
> > > your doc changes were pushed before the 3.8.1 release, now the
> > upgrade.html
> > > page doesn't mention 3.8.1 at all. I'll update the site.
> > >
> > > On Wed, Nov 6, 2024 at 4:48 PM Viktor Somogyi-Vass
> > > <vi...@cloudera.com.invalid> wrote:
> > >
> > >> Ok, sorry for the noise. It seems like my setup was incorrectly
> > configured,
> > >> one controller had an incorrect voters config and that caused the issue.
> > >> Retested it and it works now.
> > >> But given this, I might as well just give a +1. :)
> > >>
> > >> On Wed, Nov 6, 2024 at 2:53 PM Viktor Somogyi-Vass <
> > >> viktor.somogyi@cloudera.com> wrote:
> > >>
> > >> > Hey All,
> > >> >
> > >> > It seems like I ran into an issue. While trying to do the migration
> > (3.9
> > >> > ZK -> 3.9 KRaft), at the very last step the whole cluster (except the
> > >> > controller I just restarted) fails with a fatal error where a
> > controller
> > >> (I
> > >> > think) requests a truncation that is below the HWM.
> > >> > I tried to capture everything but with the configs and the steps I
> > >> > described it can be reproduced every time.
> > >> >
> > >> > I started looking into it, but since I'm completely unfamiliar with
> > many
> > >> > parts of KRaft, I'd be happy if someone else more familiar with it
> > could
> > >> > look at it. Especially if I didn't follow the migration procedure well
> > >> (but
> > >> > I think I did so).
> > >> > All the info captured here:
> > >> > https://issues.apache.org/jira/browse/KAFKA-17950
> > >> >
> > >> > Viktor
> > >> >
> > >> >
> > >> > On Fri, Nov 1, 2024 at 11:00 PM Colin McCabe <cm...@apache.org>
> > wrote:
> > >> >
> > >> >> Argh. That should read "5 +1 votes". :)
> > >> >>
> > >> >> Thanks, everyone.
> > >> >> Colin
> > >> >>
> > >> >>
> > >> >> On Fri, Nov 1, 2024, at 14:58, Colin McCabe wrote:
> > >> >> > Hi all,
> > >> >> >
> > >> >> > This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes.
> > >> >> >
> > >> >> > +1 votes
> > >> >> > PMC Members:
> > >> >> > * Justine Olshan
> > >> >> > * Bill Bejeck
> > >> >> > * David Arthur
> > >> >> > * Luke Chen
> > >> >> >
> > >> >> > Community:
> > >> >> > * Federico Valeri
> > >> >> >
> > >> >> > 0 votes
> > >> >> > * No votes
> > >> >> >
> > >> >> > -1 votes
> > >> >> > * No votes
> > >> >> >
> > >> >> > Vote thread:
> > >> >> > https://lists.apache.org/thread/og8l24z8oj87x0zsoqddfxo54qsj4d1o
> > >> >> >
> > >> >> > I'll continue with the release process and the release announcement
> > >> >> > will follow in the next few days.
> > >> >> >
> > >> >> > best,
> > >> >> > Colin
> > >> >> >
> > >> >> >
> > >> >> > On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote:
> > >> >> >> Hi Colin,
> > >> >> >>
> > >> >> >> Thanks for the update.
> > >> >> >> I've ran a sanity check as Mickael suggested for the maven
> > artifacts
> > >> >> with
> > >> >> >> the one in
> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/.
> > >> >> >> I used jarcomp <
> > >> >> https://activityworkshop.net/software/jarcomp/index.html>
> > >> >> >> to verify:
> > >> >> >> 1. kafka_2.13-3.9.0.jar
> > >> >> >> 2. kafka-clients-3.9.0.jar
> > >> >> >> 3. kafka-metadata-3.9.0.jar
> > >> >> >>
> > >> >> >> I confirmed all the class files in the jars have the same content.
> > >> >> >>
> > >> >> >> So +1 (binding) from me.
> > >> >> >>
> > >> >> >> Thank you.
> > >> >> >> Luke
> > >> >> >>
> > >> >> >> On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <
> > >> >> mickael.maison@gmail.com>
> > >> >> >> wrote:
> > >> >> >>
> > >> >> >>> Hi,
> > >> >> >>>
> > >> >> >>> Can we add sanity checks to ensure all artifacts are present when
> > >> >> >>> running the release?
> > >> >> >>> Also should we use a different folder for each RC? This would
> > >> >> >>> eliminate the risk of artifacts from previous RCs hiding an
> > issue.
> > >> >> >>> Luke spotted the old artifacts this time but this could have been
> > >> >> >>> missed.
> > >> >> >>>
> > >> >> >>> Thanks,
> > >> >> >>> Mickael
> > >> >> >>>
> > >> >> >>> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cmccabe@apache.org
> > >
> > >> >> wrote:
> > >> >> >>> >
> > >> >> >>> > Hi David,
> > >> >> >>> >
> > >> >> >>> > Thanks for checking. I took a look and yes, I did forget to
> > upload
> > >> >> the
> > >> >> >>> Scala 2.12 artifacts previously. The script runs gradlewAll, but
> > >> when
> > >> >> I was
> > >> >> >>> trying to do the steps manually, I missed that and just ran
> > gradlew,
> > >> >> which
> > >> >> >>> only did Scala 2.13.
> > >> >> >>> >
> > >> >> >>> > This should be fixed now!
> > >> >> >>> >
> > >> >> >>> > The scala 2.12 artifacts are here:
> > >> >> >>> >
> > >> >> >>> >
> > >> >>
> > https://repository.apache.org/content/repositories/orgapachekafka-1827/
> > >> >> >>> >
> > >> >> >>> > and also here:
> > >> >> >>> >
> > >> >> >>> >
> > >> >> >>>
> > >> >>
> > >>
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/
> > >> >> >>> >
> > >> >> >>> > I wonder if we should change the release.py script to just run
> > the
> > >> >> >>> gradlew steps manually. The gradlewAll script seems a bit
> > confusing
> > >> >> (or
> > >> >> >>> maybe that's just me?)
> > >> >> >>> >
> > >> >> >>> > best,
> > >> >> >>> > Colin
> > >> >> >>> >
> > >> >> >>> >
> > >> >> >>> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote:
> > >> >> >>> > > Should there be Scala 2.12 artifacts in the Maven staging?
> > >> >> >>> > >
> > >> >> >>> > > I see these for 2.13:
> > >> >> >>> > > *
> > >> >> >>> > >
> > >> >> >>>
> > >> >>
> > >>
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/
> > >> >> >>> > > *
> > >> >> >>> > >
> > >> >> >>>
> > >> >>
> > >>
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/
> > >> >> >>> > >
> > >> >> >>> > > but I don't see 2.12 for 3.9.0 release.
> > >> >> >>> > >
> > >> >> >>> > > I can confirm that the other artifacts (such as
> > kafka-clients)
> > >> do
> > >> >> have
> > >> >> >>> > > today's timestamp for the 3.9.0 artifact.
> > >> >> >>> > >
> > >> >> >>> > > -David
> > >> >> >>> > >
> > >> >> >>> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <
> > >> cmccabe@apache.org
> > >> >> >
> > >> >> >>> wrote:
> > >> >> >>> > >
> > >> >> >>> > >> Thanks for checking, Luke. I did actually visit
> > >> >> >>> > >> https://repository.apache.org/#stagingRepositories and hit
> > >> >> close on
> > >> >> >>> the
> > >> >> >>> > >> artifacts as the release.py script says to do. I don't know
> > why
> > >> >> they
> > >> >> >>> ended
> > >> >> >>> > >> up not showing up.
> > >> >> >>> > >>
> > >> >> >>> > >> Today I dropped all the maven artifacts for previous 3.9
> > RCs,
> > >> >> >>> switched to
> > >> >> >>> > >> java 8, and re-ran "gradlew publish" and "mvn deploy
> > >> >> -Pgpg-signing".
> > >> >> >>> Then I
> > >> >> >>> > >> closed the two artifacts that were uploaded. You can see
> > them
> > >> >> here:
> > >> >> >>> > >>
> > >> >> >>> > >>
> > >> >> >>>
> > >> >>
> > https://repository.apache.org/content/repositories/orgapachekafka-1825/
> > >> >> >>> > >>
> > >> >> >>>
> > >> >>
> > https://repository.apache.org/content/repositories/orgapachekafka-1826/
> > >> >> >>> > >>
> > >> >> >>> > >> Hopefully these should now show up in Maven as expected.
> > >> >> >>> > >>
> > >> >> >>> > >> best,
> > >> >> >>> > >> Colin
> > >> >> >>> > >>
> > >> >> >>> > >>
> > >> >> >>> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote:
> > >> >> >>> > >> > Hi Colin,
> > >> >> >>> > >> >
> > >> >> >>> > >> > I was trying to test the RC, but I found the artifacts in
> > >> Maven
> > >> >> >>> artifacts
> > >> >> >>> > >> > to be voted upon are not up-to-date.
> > >> >> >>> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts
> > are
> > >> on
> > >> >> >>> Oct. 10,
> > >> >> >>> > >> > but also the source code in the artifacts didn't include
> > the
> > >> >> latest
> > >> >> >>> > >> commits
> > >> >> >>> > >> > in RC5 here <
> > >> >> https://github.com/apache/kafka/commits/3.9.0-rc5/>.
> > >> >> >>> > >> >
> > >> >> >>> > >> > Could you help verify it?
> > >> >> >>> > >> >
> > >> >> >>> > >> > But I confirmed the release artifacts in "
> > >> >> >>> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/"
> > is
> > >> >> >>> up-to-date.
> > >> >> >>> > >> > I did:
> > >> >> >>> > >> > - checked the checksums and signatures
> > >> >> >>> > >> > - ran upgrade from v3.8.1 to v3.9.0
> > >> >> >>> > >> > - ran quickstart with the 2.13 binaries
> > >> >> >>> > >> > - verify tiered storage and KIP-950 (dynamically
> > disablement)
> > >> >> >>> > >> >
> > >> >> >>> > >> > I'll be +1 after the maven artifacts are updated.
> > >> >> >>> > >> >
> > >> >> >>> > >> > Thank you for running the release.
> > >> >> >>> > >> > Luke
> > >> >> >>> > >> >
> > >> >> >>> > >> >
> > >> >> >>> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan
> > >> >> >>> > >> <jo...@confluent.io.invalid>
> > >> >> >>> > >> > wrote:
> > >> >> >>> > >> >
> > >> >> >>> > >> >> Hey Colin,
> > >> >> >>> > >> >>
> > >> >> >>> > >> >> Thanks for the new RC. I've tested the --feature
> > >> functionality
> > >> >> >>> that was
> > >> >> >>> > >> >> missing before. Seems to be working as expected now.
> > >> >> >>> > >> >> As a side note, I was looking into how to configure
> > kip-853
> > >> >> and it
> > >> >> >>> took
> > >> >> >>> > >> me
> > >> >> >>> > >> >> a moment to find the instructions in the documentation, I
> > >> >> wonder
> > >> >> >>> if it
> > >> >> >>> > >> >> could be included in the notable changes section of the
> > >> >> upgrade
> > >> >> >>> notes.
> > >> >> >>> > >> Not
> > >> >> >>> > >> >> a blocker as I believe we can still update documentation.
> > >> >> >>> > >> >>
> > >> >> >>> > >> >> I did some other spot checks on the rest of the release.
> > +1
> > >> >> >>> (binding)
> > >> >> >>> > >> from
> > >> >> >>> > >> >> me
> > >> >> >>> > >> >>
> > >> >> >>> > >> >> Justine
> > >> >> >>> > >> >>
> > >> >> >>> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <
> > >> >> cmccabe@apache.org>
> > >> >> >>> > >> wrote:
> > >> >> >>> > >> >>
> > >> >> >>> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a
> > look
> > >> >> at
> > >> >> >>> how we
> > >> >> >>> > >> can
> > >> >> >>> > >> >> > improve the docs here...
> > >> >> >>> > >> >> >
> > >> >> >>> > >> >> > best,
> > >> >> >>> > >> >> > Colin
> > >> >> >>> > >> >> >
> > >> >> >>> > >> >> >
> > >> >> >>> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote:
> > >> >> >>> > >> >> > > Hi Chia-Ping,
> > >> >> >>> > >> >> > >
> > >> >> >>> > >> >> > > Thanks for pointing those two fields out. I retract
> > my
> > >> -1.
> > >> >> >>> > >> >> > >
> > >> >> >>> > >> >> > > Cheers,
> > >> >> >>> > >> >> > > Anton
> > >> >> >>> > >> >> > >
> > >> >> >>> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai <
> > >> >> >>> > >> >> chia7712@apache.org
> > >> >> >>> > >> >> > >:
> > >> >> >>> > >> >> > >
> > >> >> >>> > >> >> > >> hi Anton
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> > >> Thanks for sharing your insights on Kafka
> > >> >> serialization—it’s
> > >> >> >>> really
> > >> >> >>> > >> >> cool
> > >> >> >>> > >> >> > >> and interesting to me. Additionally, you inspired
> > me to
> > >> >> file
> > >> >> >>> a JIRA
> > >> >> >>> > >> >> > issue
> > >> >> >>> > >> >> > >> (KAFKA-17882) to improve the documentation.
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> > >> The most important aspect of Kafka is compatibility,
> > >> and
> > >> >> the
> > >> >> >>> > >> >> > undocumented
> > >> >> >>> > >> >> > >> behavior has been in place for some time [0][1].
> > This
> > >> >> means
> > >> >> >>> > >> there’s no
> > >> >> >>> > >> >> > need
> > >> >> >>> > >> >> > >> to rush your improvement for 3.9, as we’ll need to
> > >> >> explicitly
> > >> >> >>> add
> > >> >> >>> > >> >> > default
> > >> >> >>> > >> >> > >> values after applying your patch to ensure we
> > generate
> > >> >> the
> > >> >> >>> same
> > >> >> >>> > >> binary
> > >> >> >>> > >> >> > data.
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> > >> In short, we can improve the documentation first. In
> > >> the
> > >> >> >>> meantime,
> > >> >> >>> > >> we
> > >> >> >>> > >> >> > can
> > >> >> >>> > >> >> > >> continue discussing behavior clarification for 4.0,
> > and
> > >> >> RM
> > >> >> >>> can keep
> > >> >> >>> > >> >> > running
> > >> >> >>> > >> >> > >> the RC for 3.9. Everything is on track.
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> > >> Best,
> > >> >> >>> > >> >> > >> Chia-Ping
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> > >> [0]
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> >
> > >> >> >>> > >> >>
> > >> >> >>> > >>
> > >> >> >>>
> > >> >>
> > >>
> > https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43
> > >> >> >>> > >> >> > >> [1]
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> >
> > >> >> >>> > >> >>
> > >> >> >>> > >>
> > >> >> >>>
> > >> >>
> > >>
> > https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote:
> > >> >> >>> > >> >> > >> > -1, refer to comments on the RC 2 thread.
> > >> >> >>> > >> >> > >> >
> > >> >> >>> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe <
> > >> >> >>> > >> >> cmccabe@apache.org
> > >> >> >>> > >> >> > >:
> > >> >> >>> > >> >> > >> >
> > >> >> >>> > >> >> > >> > > This is the RC5 candidate for the release of
> > Apache
> > >> >> Kafka
> > >> >> >>> > >> 3.9.0.
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > - This is a major release, the final one in the
> > 3.x
> > >> >> line.
> > >> >> >>> > >> (There
> > >> >> >>> > >> >> > may of
> > >> >> >>> > >> >> > >> > > course be other minor releases in this line,
> > such
> > >> as
> > >> >> >>> 3.9.1.)
> > >> >> >>> > >> >> > >> > > - Tiered storage will be considered
> > >> production-ready
> > >> >> in
> > >> >> >>> this
> > >> >> >>> > >> >> > release.
> > >> >> >>> > >> >> > >> > > - This will be the final major release to
> > feature
> > >> the
> > >> >> >>> > >> deprecated
> > >> >> >>> > >> >> > >> ZooKeeper
> > >> >> >>> > >> >> > >> > > mode.
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > This release includes the following KIPs:
> > >> >> >>> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft
> > >> >> controller
> > >> >> >>> > >> >> membership
> > >> >> >>> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the
> > >> dump
> > >> >> log
> > >> >> >>> tool
> > >> >> >>> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms
> > to
> > >> >> Kafka
> > >> >> >>> > >> Streams
> > >> >> >>> > >> >> > >> > > - KIP-1040: Improve handling of nullable values
> > in
> > >> >> >>> InsertField,
> > >> >> >>> > >> >> > >> > > ExtractField, and other transformations
> > >> >> >>> > >> >> > >> > > - KIP-1031: Control offset translation in
> > >> >> >>> MirrorSourceConnector
> > >> >> >>> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler
> > for
> > >> >> >>> exceptions
> > >> >> >>> > >> >> > >> occurring
> > >> >> >>> > >> >> > >> > > during processing
> > >> >> >>> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka
> > Connect
> > >> >> >>> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and
> > >> >> >>> clientSecret in
> > >> >> >>> > >> >> > >> > > authorization header
> > >> >> >>> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and
> > >> >> TieredOffset
> > >> >> >>> > >> >> > >> > > - KIP-950: Tiered Storage Disablement
> > >> >> >>> > >> >> > >> > > - KIP-956: Tiered Storage Quotas
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > Release notes for the 3.9.0 release:
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> >
> > >> >> >>> > >> >>
> > >> >> >>> > >>
> > >> >> >>>
> > >> >>
> > >>
> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > *** Please download, test and vote by October
> > 30,
> > >> >> 2024.
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to
> > >> sign
> > >> >> the
> > >> >> >>> > >> release:
> > >> >> >>> > >> >> > >> > > https://kafka.apache.org/KEYS
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > * Release artifacts to be voted upon (source and
> > >> >> binary):
> > >> >> >>> > >> >> > >> > >
> > >> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > * Docker release artifacts to be voted upon:
> > >> >> >>> > >> >> > >> > > apache/kafka:3.9.0-rc5
> > >> >> >>> > >> >> > >> > > apache/kafka-native:3.9.0-rc5
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > * Maven artifacts to be voted upon:
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> >
> > >> >> >>> > >>
> > >> >> >>>
> > >> >>
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > * Javadoc:
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >>
> > >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > * Documentation:
> > >> >> >>> > >> >> > >> > > https://kafka.apache.org/39/documentation.html
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > * Protocol:
> > >> >> >>> > >> >> > >> > > https://kafka.apache.org/39/protocol.html
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the
> > >> >> 3.9.0-rc5
> > >> >> >>> tag:
> > >> >> >>> > >> >> > >> > >
> > >> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc5
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > * Successful Docker Image Github Actions
> > Pipeline
> > >> >> for 3.9
> > >> >> >>> > >> branch:
> > >> >> >>> > >> >> > >> > > Docker Build Test Pipeline (JVM):
> > >> >> >>> > >> >> > >> > >
> > >> >> https://github.com/apache/kafka/actions/runs/11535300463
> > >> >> >>> > >> >> > >> > > Docker Build Test Pipeline (Native):
> > >> >> >>> > >> >> > >> > >
> > >> >> https://github.com/apache/kafka/actions/runs/11535328957
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > Thanks to everyone who helped with this release
> > >> >> candidate,
> > >> >> >>> > >> either
> > >> >> >>> > >> >> by
> > >> >> >>> > >> >> > >> > > contributing code, testing, or documentation.
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> > > Regards,
> > >> >> >>> > >> >> > >> > > Colin
> > >> >> >>> > >> >> > >> > >
> > >> >> >>> > >> >> > >> >
> > >> >> >>> > >> >> > >>
> > >> >> >>> > >> >> >
> > >> >> >>> > >> >>
> > >> >> >>> > >>
> > >> >> >>> > >
> > >> >> >>> > >
> > >> >> >>> > > --
> > >> >> >>> > > David Arthur
> > >> >> >>>
> > >> >>
> > >> >
> > >>
> > >
> > >
> > > --
> > > [image: Aiven] <https://www.aiven.io>
> > >
> > > *Josep Prat*
> > > Open Source Engineering Director, *Aiven*
> > > josep.prat@aiven.io   |   +491715557497
> > > aiven.io <https://www.aiven.io>   |   <
> > https://www.facebook.com/aivencloud>
> > >   <https://www.linkedin.com/company/aiven/>   <
> > https://twitter.com/aiven_io>
> > > *Aiven Deutschland GmbH*
> > > Alexanderufer 3-7, 10117 Berlin
> > > Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> > > Anna Richardson, Kenneth Chen
> > > Amtsgericht Charlottenburg, HRB 209739 B
> >
>

Re: [VOTE] 3.9.0 RC5

Posted by Kamal Chandraprakash <ka...@gmail.com>.

Hi all,

I took the 3.9 branch <https://github.com/apache/kafka/tree/3.9> from
upstream and ran all the unit tests. The StorageToolTest unit test is
failing consistently:

 % ./gradlew core:test --tests StorageToolTest

Gradle Test Run :core:test > Gradle Test Executor 20 > StorageToolTest >
testFormatWithReleaseVersionAsFeature() FAILED
    org.opentest4j.AssertionFailedError: Failed to find content in output:
Formatting metadata directory
/var/folders/bq/w6tnkbq964q8sqpvj4fbmjr40000gq/T/kafka-12278605357058017295
with metadata.version 3.8-IV0.
     ==> expected: <true> but was: <false>
        at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
        at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
        at
app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
        at
app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
        at
app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
        at
app//kafka.tools.StorageToolTest.testFormatWithReleaseVersionAsFeature(StorageToolTest.scala:329)


Gradle Test Run :core:test > Gradle Test Executor 20 > StorageToolTest >
testFormatWithInvalidFeature() FAILED
    org.opentest4j.AssertionFailedError: expected: <Unsupported feature:
non.existent.feature. Supported features are: group.version, kraft.version,
transaction.version> but was: <Unsupported feature: non.existent.feature.
Supported features are: kraft.version, transaction.version>
        at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
        at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
        at
app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
        at
app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
        at
app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
        at
app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1145)
        at
app//kafka.tools.StorageToolTest.testFormatWithInvalidFeature(StorageToolTest.scala:340)

Am I missing something?


On Fri, Nov 8, 2024 at 12:50 AM Colin McCabe <cm...@apache.org> wrote:

> Thanks, Josep.
>
> best,
> Colin
>
> On Wed, Nov 6, 2024, at 23:58, Josep Prat wrote:
> > Hi Colin,
> > I see that you made 3.9 the default docs version in Kafka-Site. However,
> as
> > your doc changes were pushed before the 3.8.1 release, now the
> upgrade.html
> > page doesn't mention 3.8.1 at all. I'll update the site.
> >
> > On Wed, Nov 6, 2024 at 4:48 PM Viktor Somogyi-Vass
> > <vi...@cloudera.com.invalid> wrote:
> >
> >> Ok, sorry for the noise. It seems like my setup was incorrectly
> configured,
> >> one controller had an incorrect voters config and that caused the issue.
> >> Retested it and it works now.
> >> But given this, I might as well just give a +1. :)
> >>
> >> On Wed, Nov 6, 2024 at 2:53 PM Viktor Somogyi-Vass <
> >> viktor.somogyi@cloudera.com> wrote:
> >>
> >> > Hey All,
> >> >
> >> > It seems like I ran into an issue. While trying to do the migration
> (3.9
> >> > ZK -> 3.9 KRaft), at the very last step the whole cluster (except the
> >> > controller I just restarted) fails with a fatal error where a
> controller
> >> (I
> >> > think) requests a truncation that is below the HWM.
> >> > I tried to capture everything but with the configs and the steps I
> >> > described it can be reproduced every time.
> >> >
> >> > I started looking into it, but since I'm completely unfamiliar with
> many
> >> > parts of KRaft, I'd be happy if someone else more familiar with it
> could
> >> > look at it. Especially if I didn't follow the migration procedure well
> >> (but
> >> > I think I did so).
> >> > All the info captured here:
> >> > https://issues.apache.org/jira/browse/KAFKA-17950
> >> >
> >> > Viktor
> >> >
> >> >
> >> > On Fri, Nov 1, 2024 at 11:00 PM Colin McCabe <cm...@apache.org>
> wrote:
> >> >
> >> >> Argh. That should read "5 +1 votes". :)
> >> >>
> >> >> Thanks, everyone.
> >> >> Colin
> >> >>
> >> >>
> >> >> On Fri, Nov 1, 2024, at 14:58, Colin McCabe wrote:
> >> >> > Hi all,
> >> >> >
> >> >> > This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes.
> >> >> >
> >> >> > +1 votes
> >> >> > PMC Members:
> >> >> > * Justine Olshan
> >> >> > * Bill Bejeck
> >> >> > * David Arthur
> >> >> > * Luke Chen
> >> >> >
> >> >> > Community:
> >> >> > * Federico Valeri
> >> >> >
> >> >> > 0 votes
> >> >> > * No votes
> >> >> >
> >> >> > -1 votes
> >> >> > * No votes
> >> >> >
> >> >> > Vote thread:
> >> >> > https://lists.apache.org/thread/og8l24z8oj87x0zsoqddfxo54qsj4d1o
> >> >> >
> >> >> > I'll continue with the release process and the release announcement
> >> >> > will follow in the next few days.
> >> >> >
> >> >> > best,
> >> >> > Colin
> >> >> >
> >> >> >
> >> >> > On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote:
> >> >> >> Hi Colin,
> >> >> >>
> >> >> >> Thanks for the update.
> >> >> >> I've ran a sanity check as Mickael suggested for the maven
> artifacts
> >> >> with
> >> >> >> the one in
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/.
> >> >> >> I used jarcomp <
> >> >> https://activityworkshop.net/software/jarcomp/index.html>
> >> >> >> to verify:
> >> >> >> 1. kafka_2.13-3.9.0.jar
> >> >> >> 2. kafka-clients-3.9.0.jar
> >> >> >> 3. kafka-metadata-3.9.0.jar
> >> >> >>
> >> >> >> I confirmed all the class files in the jars have the same content.
> >> >> >>
> >> >> >> So +1 (binding) from me.
> >> >> >>
> >> >> >> Thank you.
> >> >> >> Luke
> >> >> >>
> >> >> >> On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <
> >> >> mickael.maison@gmail.com>
> >> >> >> wrote:
> >> >> >>
> >> >> >>> Hi,
> >> >> >>>
> >> >> >>> Can we add sanity checks to ensure all artifacts are present when
> >> >> >>> running the release?
> >> >> >>> Also should we use a different folder for each RC? This would
> >> >> >>> eliminate the risk of artifacts from previous RCs hiding an
> issue.
> >> >> >>> Luke spotted the old artifacts this time but this could have been
> >> >> >>> missed.
> >> >> >>>
> >> >> >>> Thanks,
> >> >> >>> Mickael
> >> >> >>>
> >> >> >>> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cmccabe@apache.org
> >
> >> >> wrote:
> >> >> >>> >
> >> >> >>> > Hi David,
> >> >> >>> >
> >> >> >>> > Thanks for checking. I took a look and yes, I did forget to
> upload
> >> >> the
> >> >> >>> Scala 2.12 artifacts previously. The script runs gradlewAll, but
> >> when
> >> >> I was
> >> >> >>> trying to do the steps manually, I missed that and just ran
> gradlew,
> >> >> which
> >> >> >>> only did Scala 2.13.
> >> >> >>> >
> >> >> >>> > This should be fixed now!
> >> >> >>> >
> >> >> >>> > The scala 2.12 artifacts are here:
> >> >> >>> >
> >> >> >>> >
> >> >>
> https://repository.apache.org/content/repositories/orgapachekafka-1827/
> >> >> >>> >
> >> >> >>> > and also here:
> >> >> >>> >
> >> >> >>> >
> >> >> >>>
> >> >>
> >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/
> >> >> >>> >
> >> >> >>> > I wonder if we should change the release.py script to just run
> the
> >> >> >>> gradlew steps manually. The gradlewAll script seems a bit
> confusing
> >> >> (or
> >> >> >>> maybe that's just me?)
> >> >> >>> >
> >> >> >>> > best,
> >> >> >>> > Colin
> >> >> >>> >
> >> >> >>> >
> >> >> >>> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote:
> >> >> >>> > > Should there be Scala 2.12 artifacts in the Maven staging?
> >> >> >>> > >
> >> >> >>> > > I see these for 2.13:
> >> >> >>> > > *
> >> >> >>> > >
> >> >> >>>
> >> >>
> >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/
> >> >> >>> > > *
> >> >> >>> > >
> >> >> >>>
> >> >>
> >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/
> >> >> >>> > >
> >> >> >>> > > but I don't see 2.12 for 3.9.0 release.
> >> >> >>> > >
> >> >> >>> > > I can confirm that the other artifacts (such as
> kafka-clients)
> >> do
> >> >> have
> >> >> >>> > > today's timestamp for the 3.9.0 artifact.
> >> >> >>> > >
> >> >> >>> > > -David
> >> >> >>> > >
> >> >> >>> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <
> >> cmccabe@apache.org
> >> >> >
> >> >> >>> wrote:
> >> >> >>> > >
> >> >> >>> > >> Thanks for checking, Luke. I did actually visit
> >> >> >>> > >> https://repository.apache.org/#stagingRepositories and hit
> >> >> close on
> >> >> >>> the
> >> >> >>> > >> artifacts as the release.py script says to do. I don't know
> why
> >> >> they
> >> >> >>> ended
> >> >> >>> > >> up not showing up.
> >> >> >>> > >>
> >> >> >>> > >> Today I dropped all the maven artifacts for previous 3.9
> RCs,
> >> >> >>> switched to
> >> >> >>> > >> java 8, and re-ran "gradlew publish" and "mvn deploy
> >> >> -Pgpg-signing".
> >> >> >>> Then I
> >> >> >>> > >> closed the two artifacts that were uploaded. You can see
> them
> >> >> here:
> >> >> >>> > >>
> >> >> >>> > >>
> >> >> >>>
> >> >>
> https://repository.apache.org/content/repositories/orgapachekafka-1825/
> >> >> >>> > >>
> >> >> >>>
> >> >>
> https://repository.apache.org/content/repositories/orgapachekafka-1826/
> >> >> >>> > >>
> >> >> >>> > >> Hopefully these should now show up in Maven as expected.
> >> >> >>> > >>
> >> >> >>> > >> best,
> >> >> >>> > >> Colin
> >> >> >>> > >>
> >> >> >>> > >>
> >> >> >>> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote:
> >> >> >>> > >> > Hi Colin,
> >> >> >>> > >> >
> >> >> >>> > >> > I was trying to test the RC, but I found the artifacts in
> >> Maven
> >> >> >>> artifacts
> >> >> >>> > >> > to be voted upon are not up-to-date.
> >> >> >>> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts
> are
> >> on
> >> >> >>> Oct. 10,
> >> >> >>> > >> > but also the source code in the artifacts didn't include
> the
> >> >> latest
> >> >> >>> > >> commits
> >> >> >>> > >> > in RC5 here <
> >> >> https://github.com/apache/kafka/commits/3.9.0-rc5/>.
> >> >> >>> > >> >
> >> >> >>> > >> > Could you help verify it?
> >> >> >>> > >> >
> >> >> >>> > >> > But I confirmed the release artifacts in "
> >> >> >>> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/"
> is
> >> >> >>> up-to-date.
> >> >> >>> > >> > I did:
> >> >> >>> > >> > - checked the checksums and signatures
> >> >> >>> > >> > - ran upgrade from v3.8.1 to v3.9.0
> >> >> >>> > >> > - ran quickstart with the 2.13 binaries
> >> >> >>> > >> > - verify tiered storage and KIP-950 (dynamically
> disablement)
> >> >> >>> > >> >
> >> >> >>> > >> > I'll be +1 after the maven artifacts are updated.
> >> >> >>> > >> >
> >> >> >>> > >> > Thank you for running the release.
> >> >> >>> > >> > Luke
> >> >> >>> > >> >
> >> >> >>> > >> >
> >> >> >>> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan
> >> >> >>> > >> <jo...@confluent.io.invalid>
> >> >> >>> > >> > wrote:
> >> >> >>> > >> >
> >> >> >>> > >> >> Hey Colin,
> >> >> >>> > >> >>
> >> >> >>> > >> >> Thanks for the new RC. I've tested the --feature
> >> functionality
> >> >> >>> that was
> >> >> >>> > >> >> missing before. Seems to be working as expected now.
> >> >> >>> > >> >> As a side note, I was looking into how to configure
> kip-853
> >> >> and it
> >> >> >>> took
> >> >> >>> > >> me
> >> >> >>> > >> >> a moment to find the instructions in the documentation, I
> >> >> wonder
> >> >> >>> if it
> >> >> >>> > >> >> could be included in the notable changes section of the
> >> >> upgrade
> >> >> >>> notes.
> >> >> >>> > >> Not
> >> >> >>> > >> >> a blocker as I believe we can still update documentation.
> >> >> >>> > >> >>
> >> >> >>> > >> >> I did some other spot checks on the rest of the release.
> +1
> >> >> >>> (binding)
> >> >> >>> > >> from
> >> >> >>> > >> >> me
> >> >> >>> > >> >>
> >> >> >>> > >> >> Justine
> >> >> >>> > >> >>
> >> >> >>> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <
> >> >> cmccabe@apache.org>
> >> >> >>> > >> wrote:
> >> >> >>> > >> >>
> >> >> >>> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a
> look
> >> >> at
> >> >> >>> how we
> >> >> >>> > >> can
> >> >> >>> > >> >> > improve the docs here...
> >> >> >>> > >> >> >
> >> >> >>> > >> >> > best,
> >> >> >>> > >> >> > Colin
> >> >> >>> > >> >> >
> >> >> >>> > >> >> >
> >> >> >>> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote:
> >> >> >>> > >> >> > > Hi Chia-Ping,
> >> >> >>> > >> >> > >
> >> >> >>> > >> >> > > Thanks for pointing those two fields out. I retract
> my
> >> -1.
> >> >> >>> > >> >> > >
> >> >> >>> > >> >> > > Cheers,
> >> >> >>> > >> >> > > Anton
> >> >> >>> > >> >> > >
> >> >> >>> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai <
> >> >> >>> > >> >> chia7712@apache.org
> >> >> >>> > >> >> > >:
> >> >> >>> > >> >> > >
> >> >> >>> > >> >> > >> hi Anton
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> > >> Thanks for sharing your insights on Kafka
> >> >> serialization—it’s
> >> >> >>> really
> >> >> >>> > >> >> cool
> >> >> >>> > >> >> > >> and interesting to me. Additionally, you inspired
> me to
> >> >> file
> >> >> >>> a JIRA
> >> >> >>> > >> >> > issue
> >> >> >>> > >> >> > >> (KAFKA-17882) to improve the documentation.
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> > >> The most important aspect of Kafka is compatibility,
> >> and
> >> >> the
> >> >> >>> > >> >> > undocumented
> >> >> >>> > >> >> > >> behavior has been in place for some time [0][1].
> This
> >> >> means
> >> >> >>> > >> there’s no
> >> >> >>> > >> >> > need
> >> >> >>> > >> >> > >> to rush your improvement for 3.9, as we’ll need to
> >> >> explicitly
> >> >> >>> add
> >> >> >>> > >> >> > default
> >> >> >>> > >> >> > >> values after applying your patch to ensure we
> generate
> >> >> the
> >> >> >>> same
> >> >> >>> > >> binary
> >> >> >>> > >> >> > data.
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> > >> In short, we can improve the documentation first. In
> >> the
> >> >> >>> meantime,
> >> >> >>> > >> we
> >> >> >>> > >> >> > can
> >> >> >>> > >> >> > >> continue discussing behavior clarification for 4.0,
> and
> >> >> RM
> >> >> >>> can keep
> >> >> >>> > >> >> > running
> >> >> >>> > >> >> > >> the RC for 3.9. Everything is on track.
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> > >> Best,
> >> >> >>> > >> >> > >> Chia-Ping
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> > >> [0]
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> >
> >> >> >>> > >> >>
> >> >> >>> > >>
> >> >> >>>
> >> >>
> >>
> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43
> >> >> >>> > >> >> > >> [1]
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> >
> >> >> >>> > >> >>
> >> >> >>> > >>
> >> >> >>>
> >> >>
> >>
> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote:
> >> >> >>> > >> >> > >> > -1, refer to comments on the RC 2 thread.
> >> >> >>> > >> >> > >> >
> >> >> >>> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe <
> >> >> >>> > >> >> cmccabe@apache.org
> >> >> >>> > >> >> > >:
> >> >> >>> > >> >> > >> >
> >> >> >>> > >> >> > >> > > This is the RC5 candidate for the release of
> Apache
> >> >> Kafka
> >> >> >>> > >> 3.9.0.
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > - This is a major release, the final one in the
> 3.x
> >> >> line.
> >> >> >>> > >> (There
> >> >> >>> > >> >> > may of
> >> >> >>> > >> >> > >> > > course be other minor releases in this line,
> such
> >> as
> >> >> >>> 3.9.1.)
> >> >> >>> > >> >> > >> > > - Tiered storage will be considered
> >> production-ready
> >> >> in
> >> >> >>> this
> >> >> >>> > >> >> > release.
> >> >> >>> > >> >> > >> > > - This will be the final major release to
> feature
> >> the
> >> >> >>> > >> deprecated
> >> >> >>> > >> >> > >> ZooKeeper
> >> >> >>> > >> >> > >> > > mode.
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > This release includes the following KIPs:
> >> >> >>> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft
> >> >> controller
> >> >> >>> > >> >> membership
> >> >> >>> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the
> >> dump
> >> >> log
> >> >> >>> tool
> >> >> >>> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms
> to
> >> >> Kafka
> >> >> >>> > >> Streams
> >> >> >>> > >> >> > >> > > - KIP-1040: Improve handling of nullable values
> in
> >> >> >>> InsertField,
> >> >> >>> > >> >> > >> > > ExtractField, and other transformations
> >> >> >>> > >> >> > >> > > - KIP-1031: Control offset translation in
> >> >> >>> MirrorSourceConnector
> >> >> >>> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler
> for
> >> >> >>> exceptions
> >> >> >>> > >> >> > >> occurring
> >> >> >>> > >> >> > >> > > during processing
> >> >> >>> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka
> Connect
> >> >> >>> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and
> >> >> >>> clientSecret in
> >> >> >>> > >> >> > >> > > authorization header
> >> >> >>> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and
> >> >> TieredOffset
> >> >> >>> > >> >> > >> > > - KIP-950: Tiered Storage Disablement
> >> >> >>> > >> >> > >> > > - KIP-956: Tiered Storage Quotas
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > Release notes for the 3.9.0 release:
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> >
> >> >> >>> > >> >>
> >> >> >>> > >>
> >> >> >>>
> >> >>
> >>
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > *** Please download, test and vote by October
> 30,
> >> >> 2024.
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to
> >> sign
> >> >> the
> >> >> >>> > >> release:
> >> >> >>> > >> >> > >> > > https://kafka.apache.org/KEYS
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > * Release artifacts to be voted upon (source and
> >> >> binary):
> >> >> >>> > >> >> > >> > >
> >> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > * Docker release artifacts to be voted upon:
> >> >> >>> > >> >> > >> > > apache/kafka:3.9.0-rc5
> >> >> >>> > >> >> > >> > > apache/kafka-native:3.9.0-rc5
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > * Maven artifacts to be voted upon:
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> >
> >> >> >>> > >>
> >> >> >>>
> >> >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > * Javadoc:
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >>
> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > * Documentation:
> >> >> >>> > >> >> > >> > > https://kafka.apache.org/39/documentation.html
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > * Protocol:
> >> >> >>> > >> >> > >> > > https://kafka.apache.org/39/protocol.html
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the
> >> >> 3.9.0-rc5
> >> >> >>> tag:
> >> >> >>> > >> >> > >> > >
> >> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc5
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > * Successful Docker Image Github Actions
> Pipeline
> >> >> for 3.9
> >> >> >>> > >> branch:
> >> >> >>> > >> >> > >> > > Docker Build Test Pipeline (JVM):
> >> >> >>> > >> >> > >> > >
> >> >> https://github.com/apache/kafka/actions/runs/11535300463
> >> >> >>> > >> >> > >> > > Docker Build Test Pipeline (Native):
> >> >> >>> > >> >> > >> > >
> >> >> https://github.com/apache/kafka/actions/runs/11535328957
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > Thanks to everyone who helped with this release
> >> >> candidate,
> >> >> >>> > >> either
> >> >> >>> > >> >> by
> >> >> >>> > >> >> > >> > > contributing code, testing, or documentation.
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> > > Regards,
> >> >> >>> > >> >> > >> > > Colin
> >> >> >>> > >> >> > >> > >
> >> >> >>> > >> >> > >> >
> >> >> >>> > >> >> > >>
> >> >> >>> > >> >> >
> >> >> >>> > >> >>
> >> >> >>> > >>
> >> >> >>> > >
> >> >> >>> > >
> >> >> >>> > > --
> >> >> >>> > > David Arthur
> >> >> >>>
> >> >>
> >> >
> >>
> >
> >
> > --
> > [image: Aiven] <https://www.aiven.io>
> >
> > *Josep Prat*
> > Open Source Engineering Director, *Aiven*
> > josep.prat@aiven.io   |   +491715557497
> > aiven.io <https://www.aiven.io>   |   <
> https://www.facebook.com/aivencloud>
> >   <https://www.linkedin.com/company/aiven/>   <
> https://twitter.com/aiven_io>
> > *Aiven Deutschland GmbH*
> > Alexanderufer 3-7, 10117 Berlin
> > Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> > Anna Richardson, Kenneth Chen
> > Amtsgericht Charlottenburg, HRB 209739 B
>

Re: [VOTE] 3.9.0 RC5

Posted by Colin McCabe <cm...@apache.org>.

Thanks, Josep.

best,
Colin

On Wed, Nov 6, 2024, at 23:58, Josep Prat wrote:
> Hi Colin,
> I see that you made 3.9 the default docs version in Kafka-Site. However, as
> your doc changes were pushed before the 3.8.1 release, now the upgrade.html
> page doesn't mention 3.8.1 at all. I'll update the site.
>
> On Wed, Nov 6, 2024 at 4:48 PM Viktor Somogyi-Vass
> <vi...@cloudera.com.invalid> wrote:
>
>> Ok, sorry for the noise. It seems like my setup was incorrectly configured,
>> one controller had an incorrect voters config and that caused the issue.
>> Retested it and it works now.
>> But given this, I might as well just give a +1. :)
>>
>> On Wed, Nov 6, 2024 at 2:53 PM Viktor Somogyi-Vass <
>> viktor.somogyi@cloudera.com> wrote:
>>
>> > Hey All,
>> >
>> > It seems like I ran into an issue. While trying to do the migration (3.9
>> > ZK -> 3.9 KRaft), at the very last step the whole cluster (except the
>> > controller I just restarted) fails with a fatal error where a controller
>> (I
>> > think) requests a truncation that is below the HWM.
>> > I tried to capture everything but with the configs and the steps I
>> > described it can be reproduced every time.
>> >
>> > I started looking into it, but since I'm completely unfamiliar with many
>> > parts of KRaft, I'd be happy if someone else more familiar with it could
>> > look at it. Especially if I didn't follow the migration procedure well
>> (but
>> > I think I did so).
>> > All the info captured here:
>> > https://issues.apache.org/jira/browse/KAFKA-17950
>> >
>> > Viktor
>> >
>> >
>> > On Fri, Nov 1, 2024 at 11:00 PM Colin McCabe <cm...@apache.org> wrote:
>> >
>> >> Argh. That should read "5 +1 votes". :)
>> >>
>> >> Thanks, everyone.
>> >> Colin
>> >>
>> >>
>> >> On Fri, Nov 1, 2024, at 14:58, Colin McCabe wrote:
>> >> > Hi all,
>> >> >
>> >> > This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes.
>> >> >
>> >> > +1 votes
>> >> > PMC Members:
>> >> > * Justine Olshan
>> >> > * Bill Bejeck
>> >> > * David Arthur
>> >> > * Luke Chen
>> >> >
>> >> > Community:
>> >> > * Federico Valeri
>> >> >
>> >> > 0 votes
>> >> > * No votes
>> >> >
>> >> > -1 votes
>> >> > * No votes
>> >> >
>> >> > Vote thread:
>> >> > https://lists.apache.org/thread/og8l24z8oj87x0zsoqddfxo54qsj4d1o
>> >> >
>> >> > I'll continue with the release process and the release announcement
>> >> > will follow in the next few days.
>> >> >
>> >> > best,
>> >> > Colin
>> >> >
>> >> >
>> >> > On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote:
>> >> >> Hi Colin,
>> >> >>
>> >> >> Thanks for the update.
>> >> >> I've ran a sanity check as Mickael suggested for the maven artifacts
>> >> with
>> >> >> the one in https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/.
>> >> >> I used jarcomp <
>> >> https://activityworkshop.net/software/jarcomp/index.html>
>> >> >> to verify:
>> >> >> 1. kafka_2.13-3.9.0.jar
>> >> >> 2. kafka-clients-3.9.0.jar
>> >> >> 3. kafka-metadata-3.9.0.jar
>> >> >>
>> >> >> I confirmed all the class files in the jars have the same content.
>> >> >>
>> >> >> So +1 (binding) from me.
>> >> >>
>> >> >> Thank you.
>> >> >> Luke
>> >> >>
>> >> >> On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <
>> >> mickael.maison@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >>> Hi,
>> >> >>>
>> >> >>> Can we add sanity checks to ensure all artifacts are present when
>> >> >>> running the release?
>> >> >>> Also should we use a different folder for each RC? This would
>> >> >>> eliminate the risk of artifacts from previous RCs hiding an issue.
>> >> >>> Luke spotted the old artifacts this time but this could have been
>> >> >>> missed.
>> >> >>>
>> >> >>> Thanks,
>> >> >>> Mickael
>> >> >>>
>> >> >>> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cm...@apache.org>
>> >> wrote:
>> >> >>> >
>> >> >>> > Hi David,
>> >> >>> >
>> >> >>> > Thanks for checking. I took a look and yes, I did forget to upload
>> >> the
>> >> >>> Scala 2.12 artifacts previously. The script runs gradlewAll, but
>> when
>> >> I was
>> >> >>> trying to do the steps manually, I missed that and just ran gradlew,
>> >> which
>> >> >>> only did Scala 2.13.
>> >> >>> >
>> >> >>> > This should be fixed now!
>> >> >>> >
>> >> >>> > The scala 2.12 artifacts are here:
>> >> >>> >
>> >> >>> >
>> >> https://repository.apache.org/content/repositories/orgapachekafka-1827/
>> >> >>> >
>> >> >>> > and also here:
>> >> >>> >
>> >> >>> >
>> >> >>>
>> >>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/
>> >> >>> >
>> >> >>> > I wonder if we should change the release.py script to just run the
>> >> >>> gradlew steps manually. The gradlewAll script seems a bit confusing
>> >> (or
>> >> >>> maybe that's just me?)
>> >> >>> >
>> >> >>> > best,
>> >> >>> > Colin
>> >> >>> >
>> >> >>> >
>> >> >>> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote:
>> >> >>> > > Should there be Scala 2.12 artifacts in the Maven staging?
>> >> >>> > >
>> >> >>> > > I see these for 2.13:
>> >> >>> > > *
>> >> >>> > >
>> >> >>>
>> >>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/
>> >> >>> > > *
>> >> >>> > >
>> >> >>>
>> >>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/
>> >> >>> > >
>> >> >>> > > but I don't see 2.12 for 3.9.0 release.
>> >> >>> > >
>> >> >>> > > I can confirm that the other artifacts (such as kafka-clients)
>> do
>> >> have
>> >> >>> > > today's timestamp for the 3.9.0 artifact.
>> >> >>> > >
>> >> >>> > > -David
>> >> >>> > >
>> >> >>> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <
>> cmccabe@apache.org
>> >> >
>> >> >>> wrote:
>> >> >>> > >
>> >> >>> > >> Thanks for checking, Luke. I did actually visit
>> >> >>> > >> https://repository.apache.org/#stagingRepositories and hit
>> >> close on
>> >> >>> the
>> >> >>> > >> artifacts as the release.py script says to do. I don't know why
>> >> they
>> >> >>> ended
>> >> >>> > >> up not showing up.
>> >> >>> > >>
>> >> >>> > >> Today I dropped all the maven artifacts for previous 3.9 RCs,
>> >> >>> switched to
>> >> >>> > >> java 8, and re-ran "gradlew publish" and "mvn deploy
>> >> -Pgpg-signing".
>> >> >>> Then I
>> >> >>> > >> closed the two artifacts that were uploaded. You can see them
>> >> here:
>> >> >>> > >>
>> >> >>> > >>
>> >> >>>
>> >> https://repository.apache.org/content/repositories/orgapachekafka-1825/
>> >> >>> > >>
>> >> >>>
>> >> https://repository.apache.org/content/repositories/orgapachekafka-1826/
>> >> >>> > >>
>> >> >>> > >> Hopefully these should now show up in Maven as expected.
>> >> >>> > >>
>> >> >>> > >> best,
>> >> >>> > >> Colin
>> >> >>> > >>
>> >> >>> > >>
>> >> >>> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote:
>> >> >>> > >> > Hi Colin,
>> >> >>> > >> >
>> >> >>> > >> > I was trying to test the RC, but I found the artifacts in
>> Maven
>> >> >>> artifacts
>> >> >>> > >> > to be voted upon are not up-to-date.
>> >> >>> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are
>> on
>> >> >>> Oct. 10,
>> >> >>> > >> > but also the source code in the artifacts didn't include the
>> >> latest
>> >> >>> > >> commits
>> >> >>> > >> > in RC5 here <
>> >> https://github.com/apache/kafka/commits/3.9.0-rc5/>.
>> >> >>> > >> >
>> >> >>> > >> > Could you help verify it?
>> >> >>> > >> >
>> >> >>> > >> > But I confirmed the release artifacts in "
>> >> >>> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/" is
>> >> >>> up-to-date.
>> >> >>> > >> > I did:
>> >> >>> > >> > - checked the checksums and signatures
>> >> >>> > >> > - ran upgrade from v3.8.1 to v3.9.0
>> >> >>> > >> > - ran quickstart with the 2.13 binaries
>> >> >>> > >> > - verify tiered storage and KIP-950 (dynamically disablement)
>> >> >>> > >> >
>> >> >>> > >> > I'll be +1 after the maven artifacts are updated.
>> >> >>> > >> >
>> >> >>> > >> > Thank you for running the release.
>> >> >>> > >> > Luke
>> >> >>> > >> >
>> >> >>> > >> >
>> >> >>> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan
>> >> >>> > >> <jo...@confluent.io.invalid>
>> >> >>> > >> > wrote:
>> >> >>> > >> >
>> >> >>> > >> >> Hey Colin,
>> >> >>> > >> >>
>> >> >>> > >> >> Thanks for the new RC. I've tested the --feature
>> functionality
>> >> >>> that was
>> >> >>> > >> >> missing before. Seems to be working as expected now.
>> >> >>> > >> >> As a side note, I was looking into how to configure kip-853
>> >> and it
>> >> >>> took
>> >> >>> > >> me
>> >> >>> > >> >> a moment to find the instructions in the documentation, I
>> >> wonder
>> >> >>> if it
>> >> >>> > >> >> could be included in the notable changes section of the
>> >> upgrade
>> >> >>> notes.
>> >> >>> > >> Not
>> >> >>> > >> >> a blocker as I believe we can still update documentation.
>> >> >>> > >> >>
>> >> >>> > >> >> I did some other spot checks on the rest of the release. +1
>> >> >>> (binding)
>> >> >>> > >> from
>> >> >>> > >> >> me
>> >> >>> > >> >>
>> >> >>> > >> >> Justine
>> >> >>> > >> >>
>> >> >>> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <
>> >> cmccabe@apache.org>
>> >> >>> > >> wrote:
>> >> >>> > >> >>
>> >> >>> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look
>> >> at
>> >> >>> how we
>> >> >>> > >> can
>> >> >>> > >> >> > improve the docs here...
>> >> >>> > >> >> >
>> >> >>> > >> >> > best,
>> >> >>> > >> >> > Colin
>> >> >>> > >> >> >
>> >> >>> > >> >> >
>> >> >>> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote:
>> >> >>> > >> >> > > Hi Chia-Ping,
>> >> >>> > >> >> > >
>> >> >>> > >> >> > > Thanks for pointing those two fields out. I retract my
>> -1.
>> >> >>> > >> >> > >
>> >> >>> > >> >> > > Cheers,
>> >> >>> > >> >> > > Anton
>> >> >>> > >> >> > >
>> >> >>> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai <
>> >> >>> > >> >> chia7712@apache.org
>> >> >>> > >> >> > >:
>> >> >>> > >> >> > >
>> >> >>> > >> >> > >> hi Anton
>> >> >>> > >> >> > >>
>> >> >>> > >> >> > >> Thanks for sharing your insights on Kafka
>> >> serialization—it’s
>> >> >>> really
>> >> >>> > >> >> cool
>> >> >>> > >> >> > >> and interesting to me. Additionally, you inspired me to
>> >> file
>> >> >>> a JIRA
>> >> >>> > >> >> > issue
>> >> >>> > >> >> > >> (KAFKA-17882) to improve the documentation.
>> >> >>> > >> >> > >>
>> >> >>> > >> >> > >> The most important aspect of Kafka is compatibility,
>> and
>> >> the
>> >> >>> > >> >> > undocumented
>> >> >>> > >> >> > >> behavior has been in place for some time [0][1]. This
>> >> means
>> >> >>> > >> there’s no
>> >> >>> > >> >> > need
>> >> >>> > >> >> > >> to rush your improvement for 3.9, as we’ll need to
>> >> explicitly
>> >> >>> add
>> >> >>> > >> >> > default
>> >> >>> > >> >> > >> values after applying your patch to ensure we generate
>> >> the
>> >> >>> same
>> >> >>> > >> binary
>> >> >>> > >> >> > data.
>> >> >>> > >> >> > >>
>> >> >>> > >> >> > >> In short, we can improve the documentation first. In
>> the
>> >> >>> meantime,
>> >> >>> > >> we
>> >> >>> > >> >> > can
>> >> >>> > >> >> > >> continue discussing behavior clarification for 4.0, and
>> >> RM
>> >> >>> can keep
>> >> >>> > >> >> > running
>> >> >>> > >> >> > >> the RC for 3.9. Everything is on track.
>> >> >>> > >> >> > >>
>> >> >>> > >> >> > >> Best,
>> >> >>> > >> >> > >> Chia-Ping
>> >> >>> > >> >> > >>
>> >> >>> > >> >> > >> [0]
>> >> >>> > >> >> > >>
>> >> >>> > >> >> >
>> >> >>> > >> >>
>> >> >>> > >>
>> >> >>>
>> >>
>> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43
>> >> >>> > >> >> > >> [1]
>> >> >>> > >> >> > >>
>> >> >>> > >> >> >
>> >> >>> > >> >>
>> >> >>> > >>
>> >> >>>
>> >>
>> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39
>> >> >>> > >> >> > >>
>> >> >>> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote:
>> >> >>> > >> >> > >> > -1, refer to comments on the RC 2 thread.
>> >> >>> > >> >> > >> >
>> >> >>> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe <
>> >> >>> > >> >> cmccabe@apache.org
>> >> >>> > >> >> > >:
>> >> >>> > >> >> > >> >
>> >> >>> > >> >> > >> > > This is the RC5 candidate for the release of Apache
>> >> Kafka
>> >> >>> > >> 3.9.0.
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > - This is a major release, the final one in the 3.x
>> >> line.
>> >> >>> > >> (There
>> >> >>> > >> >> > may of
>> >> >>> > >> >> > >> > > course be other minor releases in this line, such
>> as
>> >> >>> 3.9.1.)
>> >> >>> > >> >> > >> > > - Tiered storage will be considered
>> production-ready
>> >> in
>> >> >>> this
>> >> >>> > >> >> > release.
>> >> >>> > >> >> > >> > > - This will be the final major release to feature
>> the
>> >> >>> > >> deprecated
>> >> >>> > >> >> > >> ZooKeeper
>> >> >>> > >> >> > >> > > mode.
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > This release includes the following KIPs:
>> >> >>> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft
>> >> controller
>> >> >>> > >> >> membership
>> >> >>> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the
>> dump
>> >> log
>> >> >>> tool
>> >> >>> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to
>> >> Kafka
>> >> >>> > >> Streams
>> >> >>> > >> >> > >> > > - KIP-1040: Improve handling of nullable values in
>> >> >>> InsertField,
>> >> >>> > >> >> > >> > > ExtractField, and other transformations
>> >> >>> > >> >> > >> > > - KIP-1031: Control offset translation in
>> >> >>> MirrorSourceConnector
>> >> >>> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for
>> >> >>> exceptions
>> >> >>> > >> >> > >> occurring
>> >> >>> > >> >> > >> > > during processing
>> >> >>> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect
>> >> >>> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and
>> >> >>> clientSecret in
>> >> >>> > >> >> > >> > > authorization header
>> >> >>> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and
>> >> TieredOffset
>> >> >>> > >> >> > >> > > - KIP-950: Tiered Storage Disablement
>> >> >>> > >> >> > >> > > - KIP-956: Tiered Storage Quotas
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > Release notes for the 3.9.0 release:
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >>
>> >> >>> > >> >> >
>> >> >>> > >> >>
>> >> >>> > >>
>> >> >>>
>> >>
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > *** Please download, test and vote by October 30,
>> >> 2024.
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to
>> sign
>> >> the
>> >> >>> > >> release:
>> >> >>> > >> >> > >> > > https://kafka.apache.org/KEYS
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > * Release artifacts to be voted upon (source and
>> >> binary):
>> >> >>> > >> >> > >> > >
>> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > * Docker release artifacts to be voted upon:
>> >> >>> > >> >> > >> > > apache/kafka:3.9.0-rc5
>> >> >>> > >> >> > >> > > apache/kafka-native:3.9.0-rc5
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > * Maven artifacts to be voted upon:
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> >
>> >> >>> > >>
>> >> >>>
>> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > * Javadoc:
>> >> >>> > >> >> > >> > >
>> >> >>> > >>
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > * Documentation:
>> >> >>> > >> >> > >> > > https://kafka.apache.org/39/documentation.html
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > * Protocol:
>> >> >>> > >> >> > >> > > https://kafka.apache.org/39/protocol.html
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the
>> >> 3.9.0-rc5
>> >> >>> tag:
>> >> >>> > >> >> > >> > >
>> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc5
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > * Successful Docker Image Github Actions Pipeline
>> >> for 3.9
>> >> >>> > >> branch:
>> >> >>> > >> >> > >> > > Docker Build Test Pipeline (JVM):
>> >> >>> > >> >> > >> > >
>> >> https://github.com/apache/kafka/actions/runs/11535300463
>> >> >>> > >> >> > >> > > Docker Build Test Pipeline (Native):
>> >> >>> > >> >> > >> > >
>> >> https://github.com/apache/kafka/actions/runs/11535328957
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > Thanks to everyone who helped with this release
>> >> candidate,
>> >> >>> > >> either
>> >> >>> > >> >> by
>> >> >>> > >> >> > >> > > contributing code, testing, or documentation.
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> > > Regards,
>> >> >>> > >> >> > >> > > Colin
>> >> >>> > >> >> > >> > >
>> >> >>> > >> >> > >> >
>> >> >>> > >> >> > >>
>> >> >>> > >> >> >
>> >> >>> > >> >>
>> >> >>> > >>
>> >> >>> > >
>> >> >>> > >
>> >> >>> > > --
>> >> >>> > > David Arthur
>> >> >>>
>> >>
>> >
>>
>
>
> -- 
> [image: Aiven] <https://www.aiven.io>
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.prat@aiven.io   |   +491715557497
> aiven.io <https://www.aiven.io>   |   <https://www.facebook.com/aivencloud>
>   <https://www.linkedin.com/company/aiven/>   <https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> Anna Richardson, Kenneth Chen
> Amtsgericht Charlottenburg, HRB 209739 B

Re: [VOTE] 3.9.0 RC5

Posted by Josep Prat <jo...@aiven.io.INVALID>.

Hi Colin,
I see that you made 3.9 the default docs version in Kafka-Site. However, as
your doc changes were pushed before the 3.8.1 release, now the upgrade.html
page doesn't mention 3.8.1 at all. I'll update the site.

On Wed, Nov 6, 2024 at 4:48 PM Viktor Somogyi-Vass
<vi...@cloudera.com.invalid> wrote:

> Ok, sorry for the noise. It seems like my setup was incorrectly configured,
> one controller had an incorrect voters config and that caused the issue.
> Retested it and it works now.
> But given this, I might as well just give a +1. :)
>
> On Wed, Nov 6, 2024 at 2:53 PM Viktor Somogyi-Vass <
> viktor.somogyi@cloudera.com> wrote:
>
> > Hey All,
> >
> > It seems like I ran into an issue. While trying to do the migration (3.9
> > ZK -> 3.9 KRaft), at the very last step the whole cluster (except the
> > controller I just restarted) fails with a fatal error where a controller
> (I
> > think) requests a truncation that is below the HWM.
> > I tried to capture everything but with the configs and the steps I
> > described it can be reproduced every time.
> >
> > I started looking into it, but since I'm completely unfamiliar with many
> > parts of KRaft, I'd be happy if someone else more familiar with it could
> > look at it. Especially if I didn't follow the migration procedure well
> (but
> > I think I did so).
> > All the info captured here:
> > https://issues.apache.org/jira/browse/KAFKA-17950
> >
> > Viktor
> >
> >
> > On Fri, Nov 1, 2024 at 11:00 PM Colin McCabe <cm...@apache.org> wrote:
> >
> >> Argh. That should read "5 +1 votes". :)
> >>
> >> Thanks, everyone.
> >> Colin
> >>
> >>
> >> On Fri, Nov 1, 2024, at 14:58, Colin McCabe wrote:
> >> > Hi all,
> >> >
> >> > This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes.
> >> >
> >> > +1 votes
> >> > PMC Members:
> >> > * Justine Olshan
> >> > * Bill Bejeck
> >> > * David Arthur
> >> > * Luke Chen
> >> >
> >> > Community:
> >> > * Federico Valeri
> >> >
> >> > 0 votes
> >> > * No votes
> >> >
> >> > -1 votes
> >> > * No votes
> >> >
> >> > Vote thread:
> >> > https://lists.apache.org/thread/og8l24z8oj87x0zsoqddfxo54qsj4d1o
> >> >
> >> > I'll continue with the release process and the release announcement
> >> > will follow in the next few days.
> >> >
> >> > best,
> >> > Colin
> >> >
> >> >
> >> > On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote:
> >> >> Hi Colin,
> >> >>
> >> >> Thanks for the update.
> >> >> I've ran a sanity check as Mickael suggested for the maven artifacts
> >> with
> >> >> the one in https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/.
> >> >> I used jarcomp <
> >> https://activityworkshop.net/software/jarcomp/index.html>
> >> >> to verify:
> >> >> 1. kafka_2.13-3.9.0.jar
> >> >> 2. kafka-clients-3.9.0.jar
> >> >> 3. kafka-metadata-3.9.0.jar
> >> >>
> >> >> I confirmed all the class files in the jars have the same content.
> >> >>
> >> >> So +1 (binding) from me.
> >> >>
> >> >> Thank you.
> >> >> Luke
> >> >>
> >> >> On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <
> >> mickael.maison@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Hi,
> >> >>>
> >> >>> Can we add sanity checks to ensure all artifacts are present when
> >> >>> running the release?
> >> >>> Also should we use a different folder for each RC? This would
> >> >>> eliminate the risk of artifacts from previous RCs hiding an issue.
> >> >>> Luke spotted the old artifacts this time but this could have been
> >> >>> missed.
> >> >>>
> >> >>> Thanks,
> >> >>> Mickael
> >> >>>
> >> >>> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cm...@apache.org>
> >> wrote:
> >> >>> >
> >> >>> > Hi David,
> >> >>> >
> >> >>> > Thanks for checking. I took a look and yes, I did forget to upload
> >> the
> >> >>> Scala 2.12 artifacts previously. The script runs gradlewAll, but
> when
> >> I was
> >> >>> trying to do the steps manually, I missed that and just ran gradlew,
> >> which
> >> >>> only did Scala 2.13.
> >> >>> >
> >> >>> > This should be fixed now!
> >> >>> >
> >> >>> > The scala 2.12 artifacts are here:
> >> >>> >
> >> >>> >
> >> https://repository.apache.org/content/repositories/orgapachekafka-1827/
> >> >>> >
> >> >>> > and also here:
> >> >>> >
> >> >>> >
> >> >>>
> >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/
> >> >>> >
> >> >>> > I wonder if we should change the release.py script to just run the
> >> >>> gradlew steps manually. The gradlewAll script seems a bit confusing
> >> (or
> >> >>> maybe that's just me?)
> >> >>> >
> >> >>> > best,
> >> >>> > Colin
> >> >>> >
> >> >>> >
> >> >>> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote:
> >> >>> > > Should there be Scala 2.12 artifacts in the Maven staging?
> >> >>> > >
> >> >>> > > I see these for 2.13:
> >> >>> > > *
> >> >>> > >
> >> >>>
> >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/
> >> >>> > > *
> >> >>> > >
> >> >>>
> >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/
> >> >>> > >
> >> >>> > > but I don't see 2.12 for 3.9.0 release.
> >> >>> > >
> >> >>> > > I can confirm that the other artifacts (such as kafka-clients)
> do
> >> have
> >> >>> > > today's timestamp for the 3.9.0 artifact.
> >> >>> > >
> >> >>> > > -David
> >> >>> > >
> >> >>> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <
> cmccabe@apache.org
> >> >
> >> >>> wrote:
> >> >>> > >
> >> >>> > >> Thanks for checking, Luke. I did actually visit
> >> >>> > >> https://repository.apache.org/#stagingRepositories and hit
> >> close on
> >> >>> the
> >> >>> > >> artifacts as the release.py script says to do. I don't know why
> >> they
> >> >>> ended
> >> >>> > >> up not showing up.
> >> >>> > >>
> >> >>> > >> Today I dropped all the maven artifacts for previous 3.9 RCs,
> >> >>> switched to
> >> >>> > >> java 8, and re-ran "gradlew publish" and "mvn deploy
> >> -Pgpg-signing".
> >> >>> Then I
> >> >>> > >> closed the two artifacts that were uploaded. You can see them
> >> here:
> >> >>> > >>
> >> >>> > >>
> >> >>>
> >> https://repository.apache.org/content/repositories/orgapachekafka-1825/
> >> >>> > >>
> >> >>>
> >> https://repository.apache.org/content/repositories/orgapachekafka-1826/
> >> >>> > >>
> >> >>> > >> Hopefully these should now show up in Maven as expected.
> >> >>> > >>
> >> >>> > >> best,
> >> >>> > >> Colin
> >> >>> > >>
> >> >>> > >>
> >> >>> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote:
> >> >>> > >> > Hi Colin,
> >> >>> > >> >
> >> >>> > >> > I was trying to test the RC, but I found the artifacts in
> Maven
> >> >>> artifacts
> >> >>> > >> > to be voted upon are not up-to-date.
> >> >>> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are
> on
> >> >>> Oct. 10,
> >> >>> > >> > but also the source code in the artifacts didn't include the
> >> latest
> >> >>> > >> commits
> >> >>> > >> > in RC5 here <
> >> https://github.com/apache/kafka/commits/3.9.0-rc5/>.
> >> >>> > >> >
> >> >>> > >> > Could you help verify it?
> >> >>> > >> >
> >> >>> > >> > But I confirmed the release artifacts in "
> >> >>> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/" is
> >> >>> up-to-date.
> >> >>> > >> > I did:
> >> >>> > >> > - checked the checksums and signatures
> >> >>> > >> > - ran upgrade from v3.8.1 to v3.9.0
> >> >>> > >> > - ran quickstart with the 2.13 binaries
> >> >>> > >> > - verify tiered storage and KIP-950 (dynamically disablement)
> >> >>> > >> >
> >> >>> > >> > I'll be +1 after the maven artifacts are updated.
> >> >>> > >> >
> >> >>> > >> > Thank you for running the release.
> >> >>> > >> > Luke
> >> >>> > >> >
> >> >>> > >> >
> >> >>> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan
> >> >>> > >> <jo...@confluent.io.invalid>
> >> >>> > >> > wrote:
> >> >>> > >> >
> >> >>> > >> >> Hey Colin,
> >> >>> > >> >>
> >> >>> > >> >> Thanks for the new RC. I've tested the --feature
> functionality
> >> >>> that was
> >> >>> > >> >> missing before. Seems to be working as expected now.
> >> >>> > >> >> As a side note, I was looking into how to configure kip-853
> >> and it
> >> >>> took
> >> >>> > >> me
> >> >>> > >> >> a moment to find the instructions in the documentation, I
> >> wonder
> >> >>> if it
> >> >>> > >> >> could be included in the notable changes section of the
> >> upgrade
> >> >>> notes.
> >> >>> > >> Not
> >> >>> > >> >> a blocker as I believe we can still update documentation.
> >> >>> > >> >>
> >> >>> > >> >> I did some other spot checks on the rest of the release. +1
> >> >>> (binding)
> >> >>> > >> from
> >> >>> > >> >> me
> >> >>> > >> >>
> >> >>> > >> >> Justine
> >> >>> > >> >>
> >> >>> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <
> >> cmccabe@apache.org>
> >> >>> > >> wrote:
> >> >>> > >> >>
> >> >>> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look
> >> at
> >> >>> how we
> >> >>> > >> can
> >> >>> > >> >> > improve the docs here...
> >> >>> > >> >> >
> >> >>> > >> >> > best,
> >> >>> > >> >> > Colin
> >> >>> > >> >> >
> >> >>> > >> >> >
> >> >>> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote:
> >> >>> > >> >> > > Hi Chia-Ping,
> >> >>> > >> >> > >
> >> >>> > >> >> > > Thanks for pointing those two fields out. I retract my
> -1.
> >> >>> > >> >> > >
> >> >>> > >> >> > > Cheers,
> >> >>> > >> >> > > Anton
> >> >>> > >> >> > >
> >> >>> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai <
> >> >>> > >> >> chia7712@apache.org
> >> >>> > >> >> > >:
> >> >>> > >> >> > >
> >> >>> > >> >> > >> hi Anton
> >> >>> > >> >> > >>
> >> >>> > >> >> > >> Thanks for sharing your insights on Kafka
> >> serialization—it’s
> >> >>> really
> >> >>> > >> >> cool
> >> >>> > >> >> > >> and interesting to me. Additionally, you inspired me to
> >> file
> >> >>> a JIRA
> >> >>> > >> >> > issue
> >> >>> > >> >> > >> (KAFKA-17882) to improve the documentation.
> >> >>> > >> >> > >>
> >> >>> > >> >> > >> The most important aspect of Kafka is compatibility,
> and
> >> the
> >> >>> > >> >> > undocumented
> >> >>> > >> >> > >> behavior has been in place for some time [0][1]. This
> >> means
> >> >>> > >> there’s no
> >> >>> > >> >> > need
> >> >>> > >> >> > >> to rush your improvement for 3.9, as we’ll need to
> >> explicitly
> >> >>> add
> >> >>> > >> >> > default
> >> >>> > >> >> > >> values after applying your patch to ensure we generate
> >> the
> >> >>> same
> >> >>> > >> binary
> >> >>> > >> >> > data.
> >> >>> > >> >> > >>
> >> >>> > >> >> > >> In short, we can improve the documentation first. In
> the
> >> >>> meantime,
> >> >>> > >> we
> >> >>> > >> >> > can
> >> >>> > >> >> > >> continue discussing behavior clarification for 4.0, and
> >> RM
> >> >>> can keep
> >> >>> > >> >> > running
> >> >>> > >> >> > >> the RC for 3.9. Everything is on track.
> >> >>> > >> >> > >>
> >> >>> > >> >> > >> Best,
> >> >>> > >> >> > >> Chia-Ping
> >> >>> > >> >> > >>
> >> >>> > >> >> > >> [0]
> >> >>> > >> >> > >>
> >> >>> > >> >> >
> >> >>> > >> >>
> >> >>> > >>
> >> >>>
> >>
> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43
> >> >>> > >> >> > >> [1]
> >> >>> > >> >> > >>
> >> >>> > >> >> >
> >> >>> > >> >>
> >> >>> > >>
> >> >>>
> >>
> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39
> >> >>> > >> >> > >>
> >> >>> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote:
> >> >>> > >> >> > >> > -1, refer to comments on the RC 2 thread.
> >> >>> > >> >> > >> >
> >> >>> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe <
> >> >>> > >> >> cmccabe@apache.org
> >> >>> > >> >> > >:
> >> >>> > >> >> > >> >
> >> >>> > >> >> > >> > > This is the RC5 candidate for the release of Apache
> >> Kafka
> >> >>> > >> 3.9.0.
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > - This is a major release, the final one in the 3.x
> >> line.
> >> >>> > >> (There
> >> >>> > >> >> > may of
> >> >>> > >> >> > >> > > course be other minor releases in this line, such
> as
> >> >>> 3.9.1.)
> >> >>> > >> >> > >> > > - Tiered storage will be considered
> production-ready
> >> in
> >> >>> this
> >> >>> > >> >> > release.
> >> >>> > >> >> > >> > > - This will be the final major release to feature
> the
> >> >>> > >> deprecated
> >> >>> > >> >> > >> ZooKeeper
> >> >>> > >> >> > >> > > mode.
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > This release includes the following KIPs:
> >> >>> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft
> >> controller
> >> >>> > >> >> membership
> >> >>> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the
> dump
> >> log
> >> >>> tool
> >> >>> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to
> >> Kafka
> >> >>> > >> Streams
> >> >>> > >> >> > >> > > - KIP-1040: Improve handling of nullable values in
> >> >>> InsertField,
> >> >>> > >> >> > >> > > ExtractField, and other transformations
> >> >>> > >> >> > >> > > - KIP-1031: Control offset translation in
> >> >>> MirrorSourceConnector
> >> >>> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for
> >> >>> exceptions
> >> >>> > >> >> > >> occurring
> >> >>> > >> >> > >> > > during processing
> >> >>> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect
> >> >>> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and
> >> >>> clientSecret in
> >> >>> > >> >> > >> > > authorization header
> >> >>> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and
> >> TieredOffset
> >> >>> > >> >> > >> > > - KIP-950: Tiered Storage Disablement
> >> >>> > >> >> > >> > > - KIP-956: Tiered Storage Quotas
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > Release notes for the 3.9.0 release:
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >>
> >> >>> > >> >> >
> >> >>> > >> >>
> >> >>> > >>
> >> >>>
> >>
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > *** Please download, test and vote by October 30,
> >> 2024.
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to
> sign
> >> the
> >> >>> > >> release:
> >> >>> > >> >> > >> > > https://kafka.apache.org/KEYS
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > * Release artifacts to be voted upon (source and
> >> binary):
> >> >>> > >> >> > >> > >
> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > * Docker release artifacts to be voted upon:
> >> >>> > >> >> > >> > > apache/kafka:3.9.0-rc5
> >> >>> > >> >> > >> > > apache/kafka-native:3.9.0-rc5
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > * Maven artifacts to be voted upon:
> >> >>> > >> >> > >> > >
> >> >>> > >> >> >
> >> >>> > >>
> >> >>>
> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > * Javadoc:
> >> >>> > >> >> > >> > >
> >> >>> > >>
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > * Documentation:
> >> >>> > >> >> > >> > > https://kafka.apache.org/39/documentation.html
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > * Protocol:
> >> >>> > >> >> > >> > > https://kafka.apache.org/39/protocol.html
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the
> >> 3.9.0-rc5
> >> >>> tag:
> >> >>> > >> >> > >> > >
> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc5
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > * Successful Docker Image Github Actions Pipeline
> >> for 3.9
> >> >>> > >> branch:
> >> >>> > >> >> > >> > > Docker Build Test Pipeline (JVM):
> >> >>> > >> >> > >> > >
> >> https://github.com/apache/kafka/actions/runs/11535300463
> >> >>> > >> >> > >> > > Docker Build Test Pipeline (Native):
> >> >>> > >> >> > >> > >
> >> https://github.com/apache/kafka/actions/runs/11535328957
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > Thanks to everyone who helped with this release
> >> candidate,
> >> >>> > >> either
> >> >>> > >> >> by
> >> >>> > >> >> > >> > > contributing code, testing, or documentation.
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> > > Regards,
> >> >>> > >> >> > >> > > Colin
> >> >>> > >> >> > >> > >
> >> >>> > >> >> > >> >
> >> >>> > >> >> > >>
> >> >>> > >> >> >
> >> >>> > >> >>
> >> >>> > >>
> >> >>> > >
> >> >>> > >
> >> >>> > > --
> >> >>> > > David Arthur
> >> >>>
> >>
> >
>


-- 
[image: Aiven] <https://www.aiven.io>

*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.prat@aiven.io   |   +491715557497
aiven.io <https://www.aiven.io>   |   <https://www.facebook.com/aivencloud>
  <https://www.linkedin.com/company/aiven/>   <https://twitter.com/aiven_io>
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
Anna Richardson, Kenneth Chen
Amtsgericht Charlottenburg, HRB 209739 B

Re: [VOTE] 3.9.0 RC5

Posted by Viktor Somogyi-Vass <vi...@cloudera.com.INVALID>.

Ok, sorry for the noise. It seems like my setup was incorrectly configured,
one controller had an incorrect voters config and that caused the issue.
Retested it and it works now.
But given this, I might as well just give a +1. :)

On Wed, Nov 6, 2024 at 2:53 PM Viktor Somogyi-Vass <
viktor.somogyi@cloudera.com> wrote:

> Hey All,
>
> It seems like I ran into an issue. While trying to do the migration (3.9
> ZK -> 3.9 KRaft), at the very last step the whole cluster (except the
> controller I just restarted) fails with a fatal error where a controller (I
> think) requests a truncation that is below the HWM.
> I tried to capture everything but with the configs and the steps I
> described it can be reproduced every time.
>
> I started looking into it, but since I'm completely unfamiliar with many
> parts of KRaft, I'd be happy if someone else more familiar with it could
> look at it. Especially if I didn't follow the migration procedure well (but
> I think I did so).
> All the info captured here:
> https://issues.apache.org/jira/browse/KAFKA-17950
>
> Viktor
>
>
> On Fri, Nov 1, 2024 at 11:00 PM Colin McCabe <cm...@apache.org> wrote:
>
>> Argh. That should read "5 +1 votes". :)
>>
>> Thanks, everyone.
>> Colin
>>
>>
>> On Fri, Nov 1, 2024, at 14:58, Colin McCabe wrote:
>> > Hi all,
>> >
>> > This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes.
>> >
>> > +1 votes
>> > PMC Members:
>> > * Justine Olshan
>> > * Bill Bejeck
>> > * David Arthur
>> > * Luke Chen
>> >
>> > Community:
>> > * Federico Valeri
>> >
>> > 0 votes
>> > * No votes
>> >
>> > -1 votes
>> > * No votes
>> >
>> > Vote thread:
>> > https://lists.apache.org/thread/og8l24z8oj87x0zsoqddfxo54qsj4d1o
>> >
>> > I'll continue with the release process and the release announcement
>> > will follow in the next few days.
>> >
>> > best,
>> > Colin
>> >
>> >
>> > On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote:
>> >> Hi Colin,
>> >>
>> >> Thanks for the update.
>> >> I've ran a sanity check as Mickael suggested for the maven artifacts
>> with
>> >> the one in https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/.
>> >> I used jarcomp <
>> https://activityworkshop.net/software/jarcomp/index.html>
>> >> to verify:
>> >> 1. kafka_2.13-3.9.0.jar
>> >> 2. kafka-clients-3.9.0.jar
>> >> 3. kafka-metadata-3.9.0.jar
>> >>
>> >> I confirmed all the class files in the jars have the same content.
>> >>
>> >> So +1 (binding) from me.
>> >>
>> >> Thank you.
>> >> Luke
>> >>
>> >> On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <
>> mickael.maison@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> Can we add sanity checks to ensure all artifacts are present when
>> >>> running the release?
>> >>> Also should we use a different folder for each RC? This would
>> >>> eliminate the risk of artifacts from previous RCs hiding an issue.
>> >>> Luke spotted the old artifacts this time but this could have been
>> >>> missed.
>> >>>
>> >>> Thanks,
>> >>> Mickael
>> >>>
>> >>> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cm...@apache.org>
>> wrote:
>> >>> >
>> >>> > Hi David,
>> >>> >
>> >>> > Thanks for checking. I took a look and yes, I did forget to upload
>> the
>> >>> Scala 2.12 artifacts previously. The script runs gradlewAll, but when
>> I was
>> >>> trying to do the steps manually, I missed that and just ran gradlew,
>> which
>> >>> only did Scala 2.13.
>> >>> >
>> >>> > This should be fixed now!
>> >>> >
>> >>> > The scala 2.12 artifacts are here:
>> >>> >
>> >>> >
>> https://repository.apache.org/content/repositories/orgapachekafka-1827/
>> >>> >
>> >>> > and also here:
>> >>> >
>> >>> >
>> >>>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/
>> >>> >
>> >>> > I wonder if we should change the release.py script to just run the
>> >>> gradlew steps manually. The gradlewAll script seems a bit confusing
>> (or
>> >>> maybe that's just me?)
>> >>> >
>> >>> > best,
>> >>> > Colin
>> >>> >
>> >>> >
>> >>> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote:
>> >>> > > Should there be Scala 2.12 artifacts in the Maven staging?
>> >>> > >
>> >>> > > I see these for 2.13:
>> >>> > > *
>> >>> > >
>> >>>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/
>> >>> > > *
>> >>> > >
>> >>>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/
>> >>> > >
>> >>> > > but I don't see 2.12 for 3.9.0 release.
>> >>> > >
>> >>> > > I can confirm that the other artifacts (such as kafka-clients) do
>> have
>> >>> > > today's timestamp for the 3.9.0 artifact.
>> >>> > >
>> >>> > > -David
>> >>> > >
>> >>> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <cmccabe@apache.org
>> >
>> >>> wrote:
>> >>> > >
>> >>> > >> Thanks for checking, Luke. I did actually visit
>> >>> > >> https://repository.apache.org/#stagingRepositories and hit
>> close on
>> >>> the
>> >>> > >> artifacts as the release.py script says to do. I don't know why
>> they
>> >>> ended
>> >>> > >> up not showing up.
>> >>> > >>
>> >>> > >> Today I dropped all the maven artifacts for previous 3.9 RCs,
>> >>> switched to
>> >>> > >> java 8, and re-ran "gradlew publish" and "mvn deploy
>> -Pgpg-signing".
>> >>> Then I
>> >>> > >> closed the two artifacts that were uploaded. You can see them
>> here:
>> >>> > >>
>> >>> > >>
>> >>>
>> https://repository.apache.org/content/repositories/orgapachekafka-1825/
>> >>> > >>
>> >>>
>> https://repository.apache.org/content/repositories/orgapachekafka-1826/
>> >>> > >>
>> >>> > >> Hopefully these should now show up in Maven as expected.
>> >>> > >>
>> >>> > >> best,
>> >>> > >> Colin
>> >>> > >>
>> >>> > >>
>> >>> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote:
>> >>> > >> > Hi Colin,
>> >>> > >> >
>> >>> > >> > I was trying to test the RC, but I found the artifacts in Maven
>> >>> artifacts
>> >>> > >> > to be voted upon are not up-to-date.
>> >>> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are on
>> >>> Oct. 10,
>> >>> > >> > but also the source code in the artifacts didn't include the
>> latest
>> >>> > >> commits
>> >>> > >> > in RC5 here <
>> https://github.com/apache/kafka/commits/3.9.0-rc5/>.
>> >>> > >> >
>> >>> > >> > Could you help verify it?
>> >>> > >> >
>> >>> > >> > But I confirmed the release artifacts in "
>> >>> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/" is
>> >>> up-to-date.
>> >>> > >> > I did:
>> >>> > >> > - checked the checksums and signatures
>> >>> > >> > - ran upgrade from v3.8.1 to v3.9.0
>> >>> > >> > - ran quickstart with the 2.13 binaries
>> >>> > >> > - verify tiered storage and KIP-950 (dynamically disablement)
>> >>> > >> >
>> >>> > >> > I'll be +1 after the maven artifacts are updated.
>> >>> > >> >
>> >>> > >> > Thank you for running the release.
>> >>> > >> > Luke
>> >>> > >> >
>> >>> > >> >
>> >>> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan
>> >>> > >> <jo...@confluent.io.invalid>
>> >>> > >> > wrote:
>> >>> > >> >
>> >>> > >> >> Hey Colin,
>> >>> > >> >>
>> >>> > >> >> Thanks for the new RC. I've tested the --feature functionality
>> >>> that was
>> >>> > >> >> missing before. Seems to be working as expected now.
>> >>> > >> >> As a side note, I was looking into how to configure kip-853
>> and it
>> >>> took
>> >>> > >> me
>> >>> > >> >> a moment to find the instructions in the documentation, I
>> wonder
>> >>> if it
>> >>> > >> >> could be included in the notable changes section of the
>> upgrade
>> >>> notes.
>> >>> > >> Not
>> >>> > >> >> a blocker as I believe we can still update documentation.
>> >>> > >> >>
>> >>> > >> >> I did some other spot checks on the rest of the release. +1
>> >>> (binding)
>> >>> > >> from
>> >>> > >> >> me
>> >>> > >> >>
>> >>> > >> >> Justine
>> >>> > >> >>
>> >>> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <
>> cmccabe@apache.org>
>> >>> > >> wrote:
>> >>> > >> >>
>> >>> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look
>> at
>> >>> how we
>> >>> > >> can
>> >>> > >> >> > improve the docs here...
>> >>> > >> >> >
>> >>> > >> >> > best,
>> >>> > >> >> > Colin
>> >>> > >> >> >
>> >>> > >> >> >
>> >>> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote:
>> >>> > >> >> > > Hi Chia-Ping,
>> >>> > >> >> > >
>> >>> > >> >> > > Thanks for pointing those two fields out. I retract my -1.
>> >>> > >> >> > >
>> >>> > >> >> > > Cheers,
>> >>> > >> >> > > Anton
>> >>> > >> >> > >
>> >>> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai <
>> >>> > >> >> chia7712@apache.org
>> >>> > >> >> > >:
>> >>> > >> >> > >
>> >>> > >> >> > >> hi Anton
>> >>> > >> >> > >>
>> >>> > >> >> > >> Thanks for sharing your insights on Kafka
>> serialization—it’s
>> >>> really
>> >>> > >> >> cool
>> >>> > >> >> > >> and interesting to me. Additionally, you inspired me to
>> file
>> >>> a JIRA
>> >>> > >> >> > issue
>> >>> > >> >> > >> (KAFKA-17882) to improve the documentation.
>> >>> > >> >> > >>
>> >>> > >> >> > >> The most important aspect of Kafka is compatibility, and
>> the
>> >>> > >> >> > undocumented
>> >>> > >> >> > >> behavior has been in place for some time [0][1]. This
>> means
>> >>> > >> there’s no
>> >>> > >> >> > need
>> >>> > >> >> > >> to rush your improvement for 3.9, as we’ll need to
>> explicitly
>> >>> add
>> >>> > >> >> > default
>> >>> > >> >> > >> values after applying your patch to ensure we generate
>> the
>> >>> same
>> >>> > >> binary
>> >>> > >> >> > data.
>> >>> > >> >> > >>
>> >>> > >> >> > >> In short, we can improve the documentation first. In the
>> >>> meantime,
>> >>> > >> we
>> >>> > >> >> > can
>> >>> > >> >> > >> continue discussing behavior clarification for 4.0, and
>> RM
>> >>> can keep
>> >>> > >> >> > running
>> >>> > >> >> > >> the RC for 3.9. Everything is on track.
>> >>> > >> >> > >>
>> >>> > >> >> > >> Best,
>> >>> > >> >> > >> Chia-Ping
>> >>> > >> >> > >>
>> >>> > >> >> > >> [0]
>> >>> > >> >> > >>
>> >>> > >> >> >
>> >>> > >> >>
>> >>> > >>
>> >>>
>> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43
>> >>> > >> >> > >> [1]
>> >>> > >> >> > >>
>> >>> > >> >> >
>> >>> > >> >>
>> >>> > >>
>> >>>
>> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39
>> >>> > >> >> > >>
>> >>> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote:
>> >>> > >> >> > >> > -1, refer to comments on the RC 2 thread.
>> >>> > >> >> > >> >
>> >>> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe <
>> >>> > >> >> cmccabe@apache.org
>> >>> > >> >> > >:
>> >>> > >> >> > >> >
>> >>> > >> >> > >> > > This is the RC5 candidate for the release of Apache
>> Kafka
>> >>> > >> 3.9.0.
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > - This is a major release, the final one in the 3.x
>> line.
>> >>> > >> (There
>> >>> > >> >> > may of
>> >>> > >> >> > >> > > course be other minor releases in this line, such as
>> >>> 3.9.1.)
>> >>> > >> >> > >> > > - Tiered storage will be considered production-ready
>> in
>> >>> this
>> >>> > >> >> > release.
>> >>> > >> >> > >> > > - This will be the final major release to feature the
>> >>> > >> deprecated
>> >>> > >> >> > >> ZooKeeper
>> >>> > >> >> > >> > > mode.
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > This release includes the following KIPs:
>> >>> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft
>> controller
>> >>> > >> >> membership
>> >>> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the dump
>> log
>> >>> tool
>> >>> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to
>> Kafka
>> >>> > >> Streams
>> >>> > >> >> > >> > > - KIP-1040: Improve handling of nullable values in
>> >>> InsertField,
>> >>> > >> >> > >> > > ExtractField, and other transformations
>> >>> > >> >> > >> > > - KIP-1031: Control offset translation in
>> >>> MirrorSourceConnector
>> >>> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for
>> >>> exceptions
>> >>> > >> >> > >> occurring
>> >>> > >> >> > >> > > during processing
>> >>> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect
>> >>> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and
>> >>> clientSecret in
>> >>> > >> >> > >> > > authorization header
>> >>> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and
>> TieredOffset
>> >>> > >> >> > >> > > - KIP-950: Tiered Storage Disablement
>> >>> > >> >> > >> > > - KIP-956: Tiered Storage Quotas
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > Release notes for the 3.9.0 release:
>> >>> > >> >> > >> > >
>> >>> > >> >> > >>
>> >>> > >> >> >
>> >>> > >> >>
>> >>> > >>
>> >>>
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > *** Please download, test and vote by October 30,
>> 2024.
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to sign
>> the
>> >>> > >> release:
>> >>> > >> >> > >> > > https://kafka.apache.org/KEYS
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > * Release artifacts to be voted upon (source and
>> binary):
>> >>> > >> >> > >> > >
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > * Docker release artifacts to be voted upon:
>> >>> > >> >> > >> > > apache/kafka:3.9.0-rc5
>> >>> > >> >> > >> > > apache/kafka-native:3.9.0-rc5
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > * Maven artifacts to be voted upon:
>> >>> > >> >> > >> > >
>> >>> > >> >> >
>> >>> > >>
>> >>>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > * Javadoc:
>> >>> > >> >> > >> > >
>> >>> > >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > * Documentation:
>> >>> > >> >> > >> > > https://kafka.apache.org/39/documentation.html
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > * Protocol:
>> >>> > >> >> > >> > > https://kafka.apache.org/39/protocol.html
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the
>> 3.9.0-rc5
>> >>> tag:
>> >>> > >> >> > >> > >
>> https://github.com/apache/kafka/releases/tag/3.9.0-rc5
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > * Successful Docker Image Github Actions Pipeline
>> for 3.9
>> >>> > >> branch:
>> >>> > >> >> > >> > > Docker Build Test Pipeline (JVM):
>> >>> > >> >> > >> > >
>> https://github.com/apache/kafka/actions/runs/11535300463
>> >>> > >> >> > >> > > Docker Build Test Pipeline (Native):
>> >>> > >> >> > >> > >
>> https://github.com/apache/kafka/actions/runs/11535328957
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > Thanks to everyone who helped with this release
>> candidate,
>> >>> > >> either
>> >>> > >> >> by
>> >>> > >> >> > >> > > contributing code, testing, or documentation.
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> > > Regards,
>> >>> > >> >> > >> > > Colin
>> >>> > >> >> > >> > >
>> >>> > >> >> > >> >
>> >>> > >> >> > >>
>> >>> > >> >> >
>> >>> > >> >>
>> >>> > >>
>> >>> > >
>> >>> > >
>> >>> > > --
>> >>> > > David Arthur
>> >>>
>>
>

Re: [VOTE] 3.9.0 RC5

Posted by Viktor Somogyi-Vass <vi...@cloudera.com.INVALID>.

Hey All,

It seems like I ran into an issue. While trying to do the migration (3.9 ZK
-> 3.9 KRaft), at the very last step the whole cluster (except the
controller I just restarted) fails with a fatal error where a controller (I
think) requests a truncation that is below the HWM.
I tried to capture everything but with the configs and the steps I
described it can be reproduced every time.

I started looking into it, but since I'm completely unfamiliar with many
parts of KRaft, I'd be happy if someone else more familiar with it could
look at it. Especially if I didn't follow the migration procedure well (but
I think I did so).
All the info captured here:
https://issues.apache.org/jira/browse/KAFKA-17950

Viktor


On Fri, Nov 1, 2024 at 11:00 PM Colin McCabe <cm...@apache.org> wrote:

> Argh. That should read "5 +1 votes". :)
>
> Thanks, everyone.
> Colin
>
>
> On Fri, Nov 1, 2024, at 14:58, Colin McCabe wrote:
> > Hi all,
> >
> > This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes.
> >
> > +1 votes
> > PMC Members:
> > * Justine Olshan
> > * Bill Bejeck
> > * David Arthur
> > * Luke Chen
> >
> > Community:
> > * Federico Valeri
> >
> > 0 votes
> > * No votes
> >
> > -1 votes
> > * No votes
> >
> > Vote thread:
> > https://lists.apache.org/thread/og8l24z8oj87x0zsoqddfxo54qsj4d1o
> >
> > I'll continue with the release process and the release announcement
> > will follow in the next few days.
> >
> > best,
> > Colin
> >
> >
> > On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote:
> >> Hi Colin,
> >>
> >> Thanks for the update.
> >> I've ran a sanity check as Mickael suggested for the maven artifacts
> with
> >> the one in https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/.
> >> I used jarcomp <
> https://activityworkshop.net/software/jarcomp/index.html>
> >> to verify:
> >> 1. kafka_2.13-3.9.0.jar
> >> 2. kafka-clients-3.9.0.jar
> >> 3. kafka-metadata-3.9.0.jar
> >>
> >> I confirmed all the class files in the jars have the same content.
> >>
> >> So +1 (binding) from me.
> >>
> >> Thank you.
> >> Luke
> >>
> >> On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <mickael.maison@gmail.com
> >
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> Can we add sanity checks to ensure all artifacts are present when
> >>> running the release?
> >>> Also should we use a different folder for each RC? This would
> >>> eliminate the risk of artifacts from previous RCs hiding an issue.
> >>> Luke spotted the old artifacts this time but this could have been
> >>> missed.
> >>>
> >>> Thanks,
> >>> Mickael
> >>>
> >>> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cm...@apache.org>
> wrote:
> >>> >
> >>> > Hi David,
> >>> >
> >>> > Thanks for checking. I took a look and yes, I did forget to upload
> the
> >>> Scala 2.12 artifacts previously. The script runs gradlewAll, but when
> I was
> >>> trying to do the steps manually, I missed that and just ran gradlew,
> which
> >>> only did Scala 2.13.
> >>> >
> >>> > This should be fixed now!
> >>> >
> >>> > The scala 2.12 artifacts are here:
> >>> >
> >>> >
> https://repository.apache.org/content/repositories/orgapachekafka-1827/
> >>> >
> >>> > and also here:
> >>> >
> >>> >
> >>>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/
> >>> >
> >>> > I wonder if we should change the release.py script to just run the
> >>> gradlew steps manually. The gradlewAll script seems a bit confusing (or
> >>> maybe that's just me?)
> >>> >
> >>> > best,
> >>> > Colin
> >>> >
> >>> >
> >>> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote:
> >>> > > Should there be Scala 2.12 artifacts in the Maven staging?
> >>> > >
> >>> > > I see these for 2.13:
> >>> > > *
> >>> > >
> >>>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/
> >>> > > *
> >>> > >
> >>>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/
> >>> > >
> >>> > > but I don't see 2.12 for 3.9.0 release.
> >>> > >
> >>> > > I can confirm that the other artifacts (such as kafka-clients) do
> have
> >>> > > today's timestamp for the 3.9.0 artifact.
> >>> > >
> >>> > > -David
> >>> > >
> >>> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <cm...@apache.org>
> >>> wrote:
> >>> > >
> >>> > >> Thanks for checking, Luke. I did actually visit
> >>> > >> https://repository.apache.org/#stagingRepositories and hit close
> on
> >>> the
> >>> > >> artifacts as the release.py script says to do. I don't know why
> they
> >>> ended
> >>> > >> up not showing up.
> >>> > >>
> >>> > >> Today I dropped all the maven artifacts for previous 3.9 RCs,
> >>> switched to
> >>> > >> java 8, and re-ran "gradlew publish" and "mvn deploy
> -Pgpg-signing".
> >>> Then I
> >>> > >> closed the two artifacts that were uploaded. You can see them
> here:
> >>> > >>
> >>> > >>
> >>>
> https://repository.apache.org/content/repositories/orgapachekafka-1825/
> >>> > >>
> >>>
> https://repository.apache.org/content/repositories/orgapachekafka-1826/
> >>> > >>
> >>> > >> Hopefully these should now show up in Maven as expected.
> >>> > >>
> >>> > >> best,
> >>> > >> Colin
> >>> > >>
> >>> > >>
> >>> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote:
> >>> > >> > Hi Colin,
> >>> > >> >
> >>> > >> > I was trying to test the RC, but I found the artifacts in Maven
> >>> artifacts
> >>> > >> > to be voted upon are not up-to-date.
> >>> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are on
> >>> Oct. 10,
> >>> > >> > but also the source code in the artifacts didn't include the
> latest
> >>> > >> commits
> >>> > >> > in RC5 here <https://github.com/apache/kafka/commits/3.9.0-rc5/
> >.
> >>> > >> >
> >>> > >> > Could you help verify it?
> >>> > >> >
> >>> > >> > But I confirmed the release artifacts in "
> >>> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/" is
> >>> up-to-date.
> >>> > >> > I did:
> >>> > >> > - checked the checksums and signatures
> >>> > >> > - ran upgrade from v3.8.1 to v3.9.0
> >>> > >> > - ran quickstart with the 2.13 binaries
> >>> > >> > - verify tiered storage and KIP-950 (dynamically disablement)
> >>> > >> >
> >>> > >> > I'll be +1 after the maven artifacts are updated.
> >>> > >> >
> >>> > >> > Thank you for running the release.
> >>> > >> > Luke
> >>> > >> >
> >>> > >> >
> >>> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan
> >>> > >> <jo...@confluent.io.invalid>
> >>> > >> > wrote:
> >>> > >> >
> >>> > >> >> Hey Colin,
> >>> > >> >>
> >>> > >> >> Thanks for the new RC. I've tested the --feature functionality
> >>> that was
> >>> > >> >> missing before. Seems to be working as expected now.
> >>> > >> >> As a side note, I was looking into how to configure kip-853
> and it
> >>> took
> >>> > >> me
> >>> > >> >> a moment to find the instructions in the documentation, I
> wonder
> >>> if it
> >>> > >> >> could be included in the notable changes section of the upgrade
> >>> notes.
> >>> > >> Not
> >>> > >> >> a blocker as I believe we can still update documentation.
> >>> > >> >>
> >>> > >> >> I did some other spot checks on the rest of the release. +1
> >>> (binding)
> >>> > >> from
> >>> > >> >> me
> >>> > >> >>
> >>> > >> >> Justine
> >>> > >> >>
> >>> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <
> cmccabe@apache.org>
> >>> > >> wrote:
> >>> > >> >>
> >>> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look at
> >>> how we
> >>> > >> can
> >>> > >> >> > improve the docs here...
> >>> > >> >> >
> >>> > >> >> > best,
> >>> > >> >> > Colin
> >>> > >> >> >
> >>> > >> >> >
> >>> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote:
> >>> > >> >> > > Hi Chia-Ping,
> >>> > >> >> > >
> >>> > >> >> > > Thanks for pointing those two fields out. I retract my -1.
> >>> > >> >> > >
> >>> > >> >> > > Cheers,
> >>> > >> >> > > Anton
> >>> > >> >> > >
> >>> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai <
> >>> > >> >> chia7712@apache.org
> >>> > >> >> > >:
> >>> > >> >> > >
> >>> > >> >> > >> hi Anton
> >>> > >> >> > >>
> >>> > >> >> > >> Thanks for sharing your insights on Kafka
> serialization—it’s
> >>> really
> >>> > >> >> cool
> >>> > >> >> > >> and interesting to me. Additionally, you inspired me to
> file
> >>> a JIRA
> >>> > >> >> > issue
> >>> > >> >> > >> (KAFKA-17882) to improve the documentation.
> >>> > >> >> > >>
> >>> > >> >> > >> The most important aspect of Kafka is compatibility, and
> the
> >>> > >> >> > undocumented
> >>> > >> >> > >> behavior has been in place for some time [0][1]. This
> means
> >>> > >> there’s no
> >>> > >> >> > need
> >>> > >> >> > >> to rush your improvement for 3.9, as we’ll need to
> explicitly
> >>> add
> >>> > >> >> > default
> >>> > >> >> > >> values after applying your patch to ensure we generate the
> >>> same
> >>> > >> binary
> >>> > >> >> > data.
> >>> > >> >> > >>
> >>> > >> >> > >> In short, we can improve the documentation first. In the
> >>> meantime,
> >>> > >> we
> >>> > >> >> > can
> >>> > >> >> > >> continue discussing behavior clarification for 4.0, and RM
> >>> can keep
> >>> > >> >> > running
> >>> > >> >> > >> the RC for 3.9. Everything is on track.
> >>> > >> >> > >>
> >>> > >> >> > >> Best,
> >>> > >> >> > >> Chia-Ping
> >>> > >> >> > >>
> >>> > >> >> > >> [0]
> >>> > >> >> > >>
> >>> > >> >> >
> >>> > >> >>
> >>> > >>
> >>>
> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43
> >>> > >> >> > >> [1]
> >>> > >> >> > >>
> >>> > >> >> >
> >>> > >> >>
> >>> > >>
> >>>
> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39
> >>> > >> >> > >>
> >>> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote:
> >>> > >> >> > >> > -1, refer to comments on the RC 2 thread.
> >>> > >> >> > >> >
> >>> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe <
> >>> > >> >> cmccabe@apache.org
> >>> > >> >> > >:
> >>> > >> >> > >> >
> >>> > >> >> > >> > > This is the RC5 candidate for the release of Apache
> Kafka
> >>> > >> 3.9.0.
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > - This is a major release, the final one in the 3.x
> line.
> >>> > >> (There
> >>> > >> >> > may of
> >>> > >> >> > >> > > course be other minor releases in this line, such as
> >>> 3.9.1.)
> >>> > >> >> > >> > > - Tiered storage will be considered production-ready
> in
> >>> this
> >>> > >> >> > release.
> >>> > >> >> > >> > > - This will be the final major release to feature the
> >>> > >> deprecated
> >>> > >> >> > >> ZooKeeper
> >>> > >> >> > >> > > mode.
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > This release includes the following KIPs:
> >>> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft
> controller
> >>> > >> >> membership
> >>> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the dump
> log
> >>> tool
> >>> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to
> Kafka
> >>> > >> Streams
> >>> > >> >> > >> > > - KIP-1040: Improve handling of nullable values in
> >>> InsertField,
> >>> > >> >> > >> > > ExtractField, and other transformations
> >>> > >> >> > >> > > - KIP-1031: Control offset translation in
> >>> MirrorSourceConnector
> >>> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for
> >>> exceptions
> >>> > >> >> > >> occurring
> >>> > >> >> > >> > > during processing
> >>> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect
> >>> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and
> >>> clientSecret in
> >>> > >> >> > >> > > authorization header
> >>> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and
> TieredOffset
> >>> > >> >> > >> > > - KIP-950: Tiered Storage Disablement
> >>> > >> >> > >> > > - KIP-956: Tiered Storage Quotas
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > Release notes for the 3.9.0 release:
> >>> > >> >> > >> > >
> >>> > >> >> > >>
> >>> > >> >> >
> >>> > >> >>
> >>> > >>
> >>>
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > *** Please download, test and vote by October 30,
> 2024.
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to sign
> the
> >>> > >> release:
> >>> > >> >> > >> > > https://kafka.apache.org/KEYS
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > * Release artifacts to be voted upon (source and
> binary):
> >>> > >> >> > >> > >
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > * Docker release artifacts to be voted upon:
> >>> > >> >> > >> > > apache/kafka:3.9.0-rc5
> >>> > >> >> > >> > > apache/kafka-native:3.9.0-rc5
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > * Maven artifacts to be voted upon:
> >>> > >> >> > >> > >
> >>> > >> >> >
> >>> > >>
> >>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > * Javadoc:
> >>> > >> >> > >> > >
> >>> > >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > * Documentation:
> >>> > >> >> > >> > > https://kafka.apache.org/39/documentation.html
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > * Protocol:
> >>> > >> >> > >> > > https://kafka.apache.org/39/protocol.html
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the
> 3.9.0-rc5
> >>> tag:
> >>> > >> >> > >> > >
> https://github.com/apache/kafka/releases/tag/3.9.0-rc5
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > * Successful Docker Image Github Actions Pipeline for
> 3.9
> >>> > >> branch:
> >>> > >> >> > >> > > Docker Build Test Pipeline (JVM):
> >>> > >> >> > >> > >
> https://github.com/apache/kafka/actions/runs/11535300463
> >>> > >> >> > >> > > Docker Build Test Pipeline (Native):
> >>> > >> >> > >> > >
> https://github.com/apache/kafka/actions/runs/11535328957
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > Thanks to everyone who helped with this release
> candidate,
> >>> > >> either
> >>> > >> >> by
> >>> > >> >> > >> > > contributing code, testing, or documentation.
> >>> > >> >> > >> > >
> >>> > >> >> > >> > > Regards,
> >>> > >> >> > >> > > Colin
> >>> > >> >> > >> > >
> >>> > >> >> > >> >
> >>> > >> >> > >>
> >>> > >> >> >
> >>> > >> >>
> >>> > >>
> >>> > >
> >>> > >
> >>> > > --
> >>> > > David Arthur
> >>>
>

Re: [VOTE] 3.9.0 RC5

Posted by Colin McCabe <cm...@apache.org>.

Argh. That should read "5 +1 votes". :)

Thanks, everyone.
Colin


On Fri, Nov 1, 2024, at 14:58, Colin McCabe wrote:
> Hi all,
>
> This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes.
>
> +1 votes
> PMC Members:
> * Justine Olshan
> * Bill Bejeck
> * David Arthur
> * Luke Chen
>
> Community:
> * Federico Valeri
>
> 0 votes
> * No votes
>
> -1 votes
> * No votes
>
> Vote thread:
> https://lists.apache.org/thread/og8l24z8oj87x0zsoqddfxo54qsj4d1o
>
> I'll continue with the release process and the release announcement 
> will follow in the next few days.
>
> best,
> Colin
>
>
> On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote:
>> Hi Colin,
>>
>> Thanks for the update.
>> I've ran a sanity check as Mickael suggested for the maven artifacts with
>> the one in https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/.
>> I used jarcomp <https://activityworkshop.net/software/jarcomp/index.html>
>> to verify:
>> 1. kafka_2.13-3.9.0.jar
>> 2. kafka-clients-3.9.0.jar
>> 3. kafka-metadata-3.9.0.jar
>>
>> I confirmed all the class files in the jars have the same content.
>>
>> So +1 (binding) from me.
>>
>> Thank you.
>> Luke
>>
>> On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <mi...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Can we add sanity checks to ensure all artifacts are present when
>>> running the release?
>>> Also should we use a different folder for each RC? This would
>>> eliminate the risk of artifacts from previous RCs hiding an issue.
>>> Luke spotted the old artifacts this time but this could have been
>>> missed.
>>>
>>> Thanks,
>>> Mickael
>>>
>>> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cm...@apache.org> wrote:
>>> >
>>> > Hi David,
>>> >
>>> > Thanks for checking. I took a look and yes, I did forget to upload the
>>> Scala 2.12 artifacts previously. The script runs gradlewAll, but when I was
>>> trying to do the steps manually, I missed that and just ran gradlew, which
>>> only did Scala 2.13.
>>> >
>>> > This should be fixed now!
>>> >
>>> > The scala 2.12 artifacts are here:
>>> >
>>> > https://repository.apache.org/content/repositories/orgapachekafka-1827/
>>> >
>>> > and also here:
>>> >
>>> >
>>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/
>>> >
>>> > I wonder if we should change the release.py script to just run the
>>> gradlew steps manually. The gradlewAll script seems a bit confusing (or
>>> maybe that's just me?)
>>> >
>>> > best,
>>> > Colin
>>> >
>>> >
>>> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote:
>>> > > Should there be Scala 2.12 artifacts in the Maven staging?
>>> > >
>>> > > I see these for 2.13:
>>> > > *
>>> > >
>>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/
>>> > > *
>>> > >
>>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/
>>> > >
>>> > > but I don't see 2.12 for 3.9.0 release.
>>> > >
>>> > > I can confirm that the other artifacts (such as kafka-clients) do have
>>> > > today's timestamp for the 3.9.0 artifact.
>>> > >
>>> > > -David
>>> > >
>>> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <cm...@apache.org>
>>> wrote:
>>> > >
>>> > >> Thanks for checking, Luke. I did actually visit
>>> > >> https://repository.apache.org/#stagingRepositories and hit close on
>>> the
>>> > >> artifacts as the release.py script says to do. I don't know why they
>>> ended
>>> > >> up not showing up.
>>> > >>
>>> > >> Today I dropped all the maven artifacts for previous 3.9 RCs,
>>> switched to
>>> > >> java 8, and re-ran "gradlew publish" and "mvn deploy -Pgpg-signing".
>>> Then I
>>> > >> closed the two artifacts that were uploaded. You can see them here:
>>> > >>
>>> > >>
>>> https://repository.apache.org/content/repositories/orgapachekafka-1825/
>>> > >>
>>> https://repository.apache.org/content/repositories/orgapachekafka-1826/
>>> > >>
>>> > >> Hopefully these should now show up in Maven as expected.
>>> > >>
>>> > >> best,
>>> > >> Colin
>>> > >>
>>> > >>
>>> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote:
>>> > >> > Hi Colin,
>>> > >> >
>>> > >> > I was trying to test the RC, but I found the artifacts in Maven
>>> artifacts
>>> > >> > to be voted upon are not up-to-date.
>>> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are on
>>> Oct. 10,
>>> > >> > but also the source code in the artifacts didn't include the latest
>>> > >> commits
>>> > >> > in RC5 here <https://github.com/apache/kafka/commits/3.9.0-rc5/>.
>>> > >> >
>>> > >> > Could you help verify it?
>>> > >> >
>>> > >> > But I confirmed the release artifacts in "
>>> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/" is
>>> up-to-date.
>>> > >> > I did:
>>> > >> > - checked the checksums and signatures
>>> > >> > - ran upgrade from v3.8.1 to v3.9.0
>>> > >> > - ran quickstart with the 2.13 binaries
>>> > >> > - verify tiered storage and KIP-950 (dynamically disablement)
>>> > >> >
>>> > >> > I'll be +1 after the maven artifacts are updated.
>>> > >> >
>>> > >> > Thank you for running the release.
>>> > >> > Luke
>>> > >> >
>>> > >> >
>>> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan
>>> > >> <jo...@confluent.io.invalid>
>>> > >> > wrote:
>>> > >> >
>>> > >> >> Hey Colin,
>>> > >> >>
>>> > >> >> Thanks for the new RC. I've tested the --feature functionality
>>> that was
>>> > >> >> missing before. Seems to be working as expected now.
>>> > >> >> As a side note, I was looking into how to configure kip-853 and it
>>> took
>>> > >> me
>>> > >> >> a moment to find the instructions in the documentation, I wonder
>>> if it
>>> > >> >> could be included in the notable changes section of the upgrade
>>> notes.
>>> > >> Not
>>> > >> >> a blocker as I believe we can still update documentation.
>>> > >> >>
>>> > >> >> I did some other spot checks on the rest of the release. +1
>>> (binding)
>>> > >> from
>>> > >> >> me
>>> > >> >>
>>> > >> >> Justine
>>> > >> >>
>>> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <cm...@apache.org>
>>> > >> wrote:
>>> > >> >>
>>> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look at
>>> how we
>>> > >> can
>>> > >> >> > improve the docs here...
>>> > >> >> >
>>> > >> >> > best,
>>> > >> >> > Colin
>>> > >> >> >
>>> > >> >> >
>>> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote:
>>> > >> >> > > Hi Chia-Ping,
>>> > >> >> > >
>>> > >> >> > > Thanks for pointing those two fields out. I retract my -1.
>>> > >> >> > >
>>> > >> >> > > Cheers,
>>> > >> >> > > Anton
>>> > >> >> > >
>>> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai <
>>> > >> >> chia7712@apache.org
>>> > >> >> > >:
>>> > >> >> > >
>>> > >> >> > >> hi Anton
>>> > >> >> > >>
>>> > >> >> > >> Thanks for sharing your insights on Kafka serialization—it’s
>>> really
>>> > >> >> cool
>>> > >> >> > >> and interesting to me. Additionally, you inspired me to file
>>> a JIRA
>>> > >> >> > issue
>>> > >> >> > >> (KAFKA-17882) to improve the documentation.
>>> > >> >> > >>
>>> > >> >> > >> The most important aspect of Kafka is compatibility, and the
>>> > >> >> > undocumented
>>> > >> >> > >> behavior has been in place for some time [0][1]. This means
>>> > >> there’s no
>>> > >> >> > need
>>> > >> >> > >> to rush your improvement for 3.9, as we’ll need to explicitly
>>> add
>>> > >> >> > default
>>> > >> >> > >> values after applying your patch to ensure we generate the
>>> same
>>> > >> binary
>>> > >> >> > data.
>>> > >> >> > >>
>>> > >> >> > >> In short, we can improve the documentation first. In the
>>> meantime,
>>> > >> we
>>> > >> >> > can
>>> > >> >> > >> continue discussing behavior clarification for 4.0, and RM
>>> can keep
>>> > >> >> > running
>>> > >> >> > >> the RC for 3.9. Everything is on track.
>>> > >> >> > >>
>>> > >> >> > >> Best,
>>> > >> >> > >> Chia-Ping
>>> > >> >> > >>
>>> > >> >> > >> [0]
>>> > >> >> > >>
>>> > >> >> >
>>> > >> >>
>>> > >>
>>> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43
>>> > >> >> > >> [1]
>>> > >> >> > >>
>>> > >> >> >
>>> > >> >>
>>> > >>
>>> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39
>>> > >> >> > >>
>>> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote:
>>> > >> >> > >> > -1, refer to comments on the RC 2 thread.
>>> > >> >> > >> >
>>> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe <
>>> > >> >> cmccabe@apache.org
>>> > >> >> > >:
>>> > >> >> > >> >
>>> > >> >> > >> > > This is the RC5 candidate for the release of Apache Kafka
>>> > >> 3.9.0.
>>> > >> >> > >> > >
>>> > >> >> > >> > > - This is a major release, the final one in the 3.x line.
>>> > >> (There
>>> > >> >> > may of
>>> > >> >> > >> > > course be other minor releases in this line, such as
>>> 3.9.1.)
>>> > >> >> > >> > > - Tiered storage will be considered production-ready in
>>> this
>>> > >> >> > release.
>>> > >> >> > >> > > - This will be the final major release to feature the
>>> > >> deprecated
>>> > >> >> > >> ZooKeeper
>>> > >> >> > >> > > mode.
>>> > >> >> > >> > >
>>> > >> >> > >> > > This release includes the following KIPs:
>>> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft controller
>>> > >> >> membership
>>> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the dump log
>>> tool
>>> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to Kafka
>>> > >> Streams
>>> > >> >> > >> > > - KIP-1040: Improve handling of nullable values in
>>> InsertField,
>>> > >> >> > >> > > ExtractField, and other transformations
>>> > >> >> > >> > > - KIP-1031: Control offset translation in
>>> MirrorSourceConnector
>>> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for
>>> exceptions
>>> > >> >> > >> occurring
>>> > >> >> > >> > > during processing
>>> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect
>>> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and
>>> clientSecret in
>>> > >> >> > >> > > authorization header
>>> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and TieredOffset
>>> > >> >> > >> > > - KIP-950: Tiered Storage Disablement
>>> > >> >> > >> > > - KIP-956: Tiered Storage Quotas
>>> > >> >> > >> > >
>>> > >> >> > >> > > Release notes for the 3.9.0 release:
>>> > >> >> > >> > >
>>> > >> >> > >>
>>> > >> >> >
>>> > >> >>
>>> > >>
>>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html
>>> > >> >> > >> > >
>>> > >> >> > >> > > *** Please download, test and vote by October 30, 2024.
>>> > >> >> > >> > >
>>> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to sign the
>>> > >> release:
>>> > >> >> > >> > > https://kafka.apache.org/KEYS
>>> > >> >> > >> > >
>>> > >> >> > >> > > * Release artifacts to be voted upon (source and binary):
>>> > >> >> > >> > > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/
>>> > >> >> > >> > >
>>> > >> >> > >> > > * Docker release artifacts to be voted upon:
>>> > >> >> > >> > > apache/kafka:3.9.0-rc5
>>> > >> >> > >> > > apache/kafka-native:3.9.0-rc5
>>> > >> >> > >> > >
>>> > >> >> > >> > > * Maven artifacts to be voted upon:
>>> > >> >> > >> > >
>>> > >> >> >
>>> > >>
>>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>> > >> >> > >> > >
>>> > >> >> > >> > > * Javadoc:
>>> > >> >> > >> > >
>>> > >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/
>>> > >> >> > >> > >
>>> > >> >> > >> > > * Documentation:
>>> > >> >> > >> > > https://kafka.apache.org/39/documentation.html
>>> > >> >> > >> > >
>>> > >> >> > >> > > * Protocol:
>>> > >> >> > >> > > https://kafka.apache.org/39/protocol.html
>>> > >> >> > >> > >
>>> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc5
>>> tag:
>>> > >> >> > >> > > https://github.com/apache/kafka/releases/tag/3.9.0-rc5
>>> > >> >> > >> > >
>>> > >> >> > >> > > * Successful Docker Image Github Actions Pipeline for 3.9
>>> > >> branch:
>>> > >> >> > >> > > Docker Build Test Pipeline (JVM):
>>> > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535300463
>>> > >> >> > >> > > Docker Build Test Pipeline (Native):
>>> > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535328957
>>> > >> >> > >> > >
>>> > >> >> > >> > > Thanks to everyone who helped with this release candidate,
>>> > >> either
>>> > >> >> by
>>> > >> >> > >> > > contributing code, testing, or documentation.
>>> > >> >> > >> > >
>>> > >> >> > >> > > Regards,
>>> > >> >> > >> > > Colin
>>> > >> >> > >> > >
>>> > >> >> > >> >
>>> > >> >> > >>
>>> > >> >> >
>>> > >> >>
>>> > >>
>>> > >
>>> > >
>>> > > --
>>> > > David Arthur
>>>

Re: [VOTE] 3.9.0 RC5

Posted by Colin McCabe <cm...@apache.org>.

Hi all,

This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes.

+1 votes
PMC Members:
* Justine Olshan
* Bill Bejeck
* David Arthur
* Luke Chen

Community:
* Federico Valeri

0 votes
* No votes

-1 votes
* No votes

Vote thread:
https://lists.apache.org/thread/og8l24z8oj87x0zsoqddfxo54qsj4d1o

I'll continue with the release process and the release announcement will follow in the next few days.

best,
Colin


On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote:
> Hi Colin,
>
> Thanks for the update.
> I've ran a sanity check as Mickael suggested for the maven artifacts with
> the one in https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/.
> I used jarcomp <https://activityworkshop.net/software/jarcomp/index.html>
> to verify:
> 1. kafka_2.13-3.9.0.jar
> 2. kafka-clients-3.9.0.jar
> 3. kafka-metadata-3.9.0.jar
>
> I confirmed all the class files in the jars have the same content.
>
> So +1 (binding) from me.
>
> Thank you.
> Luke
>
> On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <mi...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Can we add sanity checks to ensure all artifacts are present when
>> running the release?
>> Also should we use a different folder for each RC? This would
>> eliminate the risk of artifacts from previous RCs hiding an issue.
>> Luke spotted the old artifacts this time but this could have been
>> missed.
>>
>> Thanks,
>> Mickael
>>
>> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cm...@apache.org> wrote:
>> >
>> > Hi David,
>> >
>> > Thanks for checking. I took a look and yes, I did forget to upload the
>> Scala 2.12 artifacts previously. The script runs gradlewAll, but when I was
>> trying to do the steps manually, I missed that and just ran gradlew, which
>> only did Scala 2.13.
>> >
>> > This should be fixed now!
>> >
>> > The scala 2.12 artifacts are here:
>> >
>> > https://repository.apache.org/content/repositories/orgapachekafka-1827/
>> >
>> > and also here:
>> >
>> >
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/
>> >
>> > I wonder if we should change the release.py script to just run the
>> gradlew steps manually. The gradlewAll script seems a bit confusing (or
>> maybe that's just me?)
>> >
>> > best,
>> > Colin
>> >
>> >
>> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote:
>> > > Should there be Scala 2.12 artifacts in the Maven staging?
>> > >
>> > > I see these for 2.13:
>> > > *
>> > >
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/
>> > > *
>> > >
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/
>> > >
>> > > but I don't see 2.12 for 3.9.0 release.
>> > >
>> > > I can confirm that the other artifacts (such as kafka-clients) do have
>> > > today's timestamp for the 3.9.0 artifact.
>> > >
>> > > -David
>> > >
>> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <cm...@apache.org>
>> wrote:
>> > >
>> > >> Thanks for checking, Luke. I did actually visit
>> > >> https://repository.apache.org/#stagingRepositories and hit close on
>> the
>> > >> artifacts as the release.py script says to do. I don't know why they
>> ended
>> > >> up not showing up.
>> > >>
>> > >> Today I dropped all the maven artifacts for previous 3.9 RCs,
>> switched to
>> > >> java 8, and re-ran "gradlew publish" and "mvn deploy -Pgpg-signing".
>> Then I
>> > >> closed the two artifacts that were uploaded. You can see them here:
>> > >>
>> > >>
>> https://repository.apache.org/content/repositories/orgapachekafka-1825/
>> > >>
>> https://repository.apache.org/content/repositories/orgapachekafka-1826/
>> > >>
>> > >> Hopefully these should now show up in Maven as expected.
>> > >>
>> > >> best,
>> > >> Colin
>> > >>
>> > >>
>> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote:
>> > >> > Hi Colin,
>> > >> >
>> > >> > I was trying to test the RC, but I found the artifacts in Maven
>> artifacts
>> > >> > to be voted upon are not up-to-date.
>> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are on
>> Oct. 10,
>> > >> > but also the source code in the artifacts didn't include the latest
>> > >> commits
>> > >> > in RC5 here <https://github.com/apache/kafka/commits/3.9.0-rc5/>.
>> > >> >
>> > >> > Could you help verify it?
>> > >> >
>> > >> > But I confirmed the release artifacts in "
>> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/" is
>> up-to-date.
>> > >> > I did:
>> > >> > - checked the checksums and signatures
>> > >> > - ran upgrade from v3.8.1 to v3.9.0
>> > >> > - ran quickstart with the 2.13 binaries
>> > >> > - verify tiered storage and KIP-950 (dynamically disablement)
>> > >> >
>> > >> > I'll be +1 after the maven artifacts are updated.
>> > >> >
>> > >> > Thank you for running the release.
>> > >> > Luke
>> > >> >
>> > >> >
>> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan
>> > >> <jo...@confluent.io.invalid>
>> > >> > wrote:
>> > >> >
>> > >> >> Hey Colin,
>> > >> >>
>> > >> >> Thanks for the new RC. I've tested the --feature functionality
>> that was
>> > >> >> missing before. Seems to be working as expected now.
>> > >> >> As a side note, I was looking into how to configure kip-853 and it
>> took
>> > >> me
>> > >> >> a moment to find the instructions in the documentation, I wonder
>> if it
>> > >> >> could be included in the notable changes section of the upgrade
>> notes.
>> > >> Not
>> > >> >> a blocker as I believe we can still update documentation.
>> > >> >>
>> > >> >> I did some other spot checks on the rest of the release. +1
>> (binding)
>> > >> from
>> > >> >> me
>> > >> >>
>> > >> >> Justine
>> > >> >>
>> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <cm...@apache.org>
>> > >> wrote:
>> > >> >>
>> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look at
>> how we
>> > >> can
>> > >> >> > improve the docs here...
>> > >> >> >
>> > >> >> > best,
>> > >> >> > Colin
>> > >> >> >
>> > >> >> >
>> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote:
>> > >> >> > > Hi Chia-Ping,
>> > >> >> > >
>> > >> >> > > Thanks for pointing those two fields out. I retract my -1.
>> > >> >> > >
>> > >> >> > > Cheers,
>> > >> >> > > Anton
>> > >> >> > >
>> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai <
>> > >> >> chia7712@apache.org
>> > >> >> > >:
>> > >> >> > >
>> > >> >> > >> hi Anton
>> > >> >> > >>
>> > >> >> > >> Thanks for sharing your insights on Kafka serialization—it’s
>> really
>> > >> >> cool
>> > >> >> > >> and interesting to me. Additionally, you inspired me to file
>> a JIRA
>> > >> >> > issue
>> > >> >> > >> (KAFKA-17882) to improve the documentation.
>> > >> >> > >>
>> > >> >> > >> The most important aspect of Kafka is compatibility, and the
>> > >> >> > undocumented
>> > >> >> > >> behavior has been in place for some time [0][1]. This means
>> > >> there’s no
>> > >> >> > need
>> > >> >> > >> to rush your improvement for 3.9, as we’ll need to explicitly
>> add
>> > >> >> > default
>> > >> >> > >> values after applying your patch to ensure we generate the
>> same
>> > >> binary
>> > >> >> > data.
>> > >> >> > >>
>> > >> >> > >> In short, we can improve the documentation first. In the
>> meantime,
>> > >> we
>> > >> >> > can
>> > >> >> > >> continue discussing behavior clarification for 4.0, and RM
>> can keep
>> > >> >> > running
>> > >> >> > >> the RC for 3.9. Everything is on track.
>> > >> >> > >>
>> > >> >> > >> Best,
>> > >> >> > >> Chia-Ping
>> > >> >> > >>
>> > >> >> > >> [0]
>> > >> >> > >>
>> > >> >> >
>> > >> >>
>> > >>
>> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43
>> > >> >> > >> [1]
>> > >> >> > >>
>> > >> >> >
>> > >> >>
>> > >>
>> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39
>> > >> >> > >>
>> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote:
>> > >> >> > >> > -1, refer to comments on the RC 2 thread.
>> > >> >> > >> >
>> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe <
>> > >> >> cmccabe@apache.org
>> > >> >> > >:
>> > >> >> > >> >
>> > >> >> > >> > > This is the RC5 candidate for the release of Apache Kafka
>> > >> 3.9.0.
>> > >> >> > >> > >
>> > >> >> > >> > > - This is a major release, the final one in the 3.x line.
>> > >> (There
>> > >> >> > may of
>> > >> >> > >> > > course be other minor releases in this line, such as
>> 3.9.1.)
>> > >> >> > >> > > - Tiered storage will be considered production-ready in
>> this
>> > >> >> > release.
>> > >> >> > >> > > - This will be the final major release to feature the
>> > >> deprecated
>> > >> >> > >> ZooKeeper
>> > >> >> > >> > > mode.
>> > >> >> > >> > >
>> > >> >> > >> > > This release includes the following KIPs:
>> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft controller
>> > >> >> membership
>> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the dump log
>> tool
>> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to Kafka
>> > >> Streams
>> > >> >> > >> > > - KIP-1040: Improve handling of nullable values in
>> InsertField,
>> > >> >> > >> > > ExtractField, and other transformations
>> > >> >> > >> > > - KIP-1031: Control offset translation in
>> MirrorSourceConnector
>> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for
>> exceptions
>> > >> >> > >> occurring
>> > >> >> > >> > > during processing
>> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect
>> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and
>> clientSecret in
>> > >> >> > >> > > authorization header
>> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and TieredOffset
>> > >> >> > >> > > - KIP-950: Tiered Storage Disablement
>> > >> >> > >> > > - KIP-956: Tiered Storage Quotas
>> > >> >> > >> > >
>> > >> >> > >> > > Release notes for the 3.9.0 release:
>> > >> >> > >> > >
>> > >> >> > >>
>> > >> >> >
>> > >> >>
>> > >>
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html
>> > >> >> > >> > >
>> > >> >> > >> > > *** Please download, test and vote by October 30, 2024.
>> > >> >> > >> > >
>> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to sign the
>> > >> release:
>> > >> >> > >> > > https://kafka.apache.org/KEYS
>> > >> >> > >> > >
>> > >> >> > >> > > * Release artifacts to be voted upon (source and binary):
>> > >> >> > >> > > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/
>> > >> >> > >> > >
>> > >> >> > >> > > * Docker release artifacts to be voted upon:
>> > >> >> > >> > > apache/kafka:3.9.0-rc5
>> > >> >> > >> > > apache/kafka-native:3.9.0-rc5
>> > >> >> > >> > >
>> > >> >> > >> > > * Maven artifacts to be voted upon:
>> > >> >> > >> > >
>> > >> >> >
>> > >>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> > >> >> > >> > >
>> > >> >> > >> > > * Javadoc:
>> > >> >> > >> > >
>> > >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/
>> > >> >> > >> > >
>> > >> >> > >> > > * Documentation:
>> > >> >> > >> > > https://kafka.apache.org/39/documentation.html
>> > >> >> > >> > >
>> > >> >> > >> > > * Protocol:
>> > >> >> > >> > > https://kafka.apache.org/39/protocol.html
>> > >> >> > >> > >
>> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc5
>> tag:
>> > >> >> > >> > > https://github.com/apache/kafka/releases/tag/3.9.0-rc5
>> > >> >> > >> > >
>> > >> >> > >> > > * Successful Docker Image Github Actions Pipeline for 3.9
>> > >> branch:
>> > >> >> > >> > > Docker Build Test Pipeline (JVM):
>> > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535300463
>> > >> >> > >> > > Docker Build Test Pipeline (Native):
>> > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535328957
>> > >> >> > >> > >
>> > >> >> > >> > > Thanks to everyone who helped with this release candidate,
>> > >> either
>> > >> >> by
>> > >> >> > >> > > contributing code, testing, or documentation.
>> > >> >> > >> > >
>> > >> >> > >> > > Regards,
>> > >> >> > >> > > Colin
>> > >> >> > >> > >
>> > >> >> > >> >
>> > >> >> > >>
>> > >> >> >
>> > >> >>
>> > >>
>> > >
>> > >
>> > > --
>> > > David Arthur
>>