You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Manikumar <ma...@gmail.com> on 2019/11/20 15:37:17 UTC
[VOTE] 2.4.0 RC1
Hello Kafka users, developers and client-developers,
This is the second candidate for release of Apache Kafka 2.4.0.
This release includes many new features, including:
- Allow consumers to fetch from closest replica
- Support for incremental cooperative rebalancing to the consumer rebalance
protocol
- MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter replication
engine
- New Java authorizer Interface
- Support for non-key joining in KTable
- Administrative API for replica reassignment
- Sticky partitioner
- Return topic metadata and configs in CreateTopics response
- Securing Internal connect REST endpoints
- API to delete consumer offsets and expose it via the AdminClient.
Release notes for the 2.4.0 release:
https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
** Please download, test and vote by Tuesday, November 26, 9am PT **
Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS
* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/
* Javadoc:
https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
* Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
https://github.com/apache/kafka/releases/tag/2.4.0-rc1
* Documentation:
https://kafka.apache.org/24/documentation.html
* Protocol:
https://kafka.apache.org/24/protocol.html
Thanks,
Manikumar
Re: [VOTE] 2.4.0 RC1
Posted by Manikumar <ma...@gmail.com>.
Hi All,
We will consider KAFKA-9244
<https://issues.apache.org/jira/browse/KAFKA-9244> as blocker and include
the fix in 2.4 release.
I am canceling this VOTE and will create third release candidate.
Thank you all for testing.
On Fri, Nov 29, 2019 at 10:52 AM Matthias J. Sax <ma...@confluent.io>
wrote:
> I did not find the bug -- it was reported by Kin Sui
> (https://issues.apache.org/jira/browse/KAFKA-9244)
>
> If the bug is a blocker is a judgment call thought, because it's
> technically not a regression. However, if we don't include the fix in
> 2.4.0, as Adam pointed out, the new foreign-key join would compute
> incorrect results, and thus, it's at least a critical issue.
>
>
> -Matthias
>
>
>
> On 11/28/19 11:48 AM, Adam Bellemare wrote:
> > mjsax found an important issue for the foreign-key joiner, which I think
> > should be a blocker (if it isn't already) since it is functionally
> > incorrect without the fix:
> >
> > https://github.com/apache/kafka/pull/7758
> >
> >
> >
> > On Tue, Nov 26, 2019 at 6:26 PM Sean Glover <se...@lightbend.com>
> > wrote:
> >
> >> Hi,
> >>
> >> I also used Eric's test script. I had a few issues running it that I
> >> address below[0][1], otherwise looks good.
> >>
> >> - Signing keys all good
> >> - All md5, sha1sums and sha512sums are good
> >> - A couple transient test failures that passed on a second run
> >> (ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker,
> >> SaslScramSslEndToEndAuthorizationTest.
> >> testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl)
> >> - Passes our own test suite for Alpakka Kafka (
> >> https://travis-ci.org/akka/alpakka-kafka/builds/616861540,
> >> https://github.com/akka/alpakka-kafka/pull/971)
> >>
> >> +1 (non-binding)
> >>
> >> ..
> >>
> >> Issues while running test script:
> >>
> >> [0] Error with Eric test script. I had an issue running the script
> with my
> >> version of bash (TMPDIR was unassigned), which I provided a PR for (
> >> https://github.com/elalonde/kafka/pull/1)
> >> [1] Gradle incompatibility. I ran into difficulty running the gradle
> build
> >> with the latest version of gradle (6.0.1). I had to revert to the last
> >> patch of version 5 (5.6.4):
> >>
> >> ✘ seglo@slice
> /tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src
> >> gradle wrapper --warning-mode all
> >>
> >>> Configure project :
> >> The maven plugin has been deprecated. This is scheduled to be removed in
> >> Gradle 7.0. Please use the maven-publish plugin instead.
> >> at
> >>
> >>
> build_c0129pbfzzxjolwxmds3lsevz$_run_closure5.doCall(/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle:160)
> >> (Run with --stacktrace to get the full stack trace of this
> >> deprecation warning.)
> >>
> >> FAILURE: Build failed with an exception.
> >>
> >> * Where:
> >> Build file
> >> '/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle'
> line:
> >> 472
> >>
> >> * What went wrong:
> >> A problem occurred evaluating root project 'kafka-2.4.0-src'.
> >>> Could not create task ':clients:spotbugsMain'.
> >> > Could not create task of type 'SpotBugsTask'.
> >> > Could not create an instance of type
> >> com.github.spotbugs.internal.SpotBugsReportsImpl.
> >> >
> >>
> >>
> org.gradle.api.reporting.internal.TaskReportContainer.<init>(Ljava/lang/Class;Lorg/gradle/api/Task;)V
> >>
> >> * Try:
> >> Run with --stacktrace option to get the stack trace. Run with --info or
> >> --debug option to get more log output. Run with --scan to get full
> >> insights.
> >>
> >> * Get more help at https://help.gradle.org
> >>
> >> BUILD FAILED in 699ms
> >>
> >> On Tue, Nov 26, 2019 at 1:31 PM Manikumar <ma...@gmail.com>
> >> wrote:
> >>
> >>> Hi All,
> >>>
> >>> Please download, test and vote the RC1 in order to provide quality
> >>> assurance for the forthcoming 2.4 release.
> >>>
> >>> Thanks.
> >>>
> >>> On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <
> adam.bellemare@gmail.com
> >>>
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> Ran Eric's test script:
> >>>> $ git clone https://github.com/elalonde/kafka
> >>>> $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
> >>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1
> >>>> <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
> >>>>
> >>>> - All PGP signatures are good
> >>>> - All md5, sha1sums and sha512sums pass
> >>>> - Had a few intermittent failures in tests that passed upon rerunning.
> >>>>
> >>>> +1 (non-binding) from me.
> >>>>
> >>>> Adam
> >>>>
> >>>> On Wed, Nov 20, 2019 at 10:37 AM Manikumar <manikumar.reddy@gmail.com
> >
> >>>> wrote:
> >>>>
> >>>>> Hello Kafka users, developers and client-developers,
> >>>>>
> >>>>> This is the second candidate for release of Apache Kafka 2.4.0.
> >>>>>
> >>>>> This release includes many new features, including:
> >>>>> - Allow consumers to fetch from closest replica
> >>>>> - Support for incremental cooperative rebalancing to the consumer
> >>>> rebalance
> >>>>> protocol
> >>>>> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
> >>>> replication
> >>>>> engine
> >>>>> - New Java authorizer Interface
> >>>>> - Support for non-key joining in KTable
> >>>>> - Administrative API for replica reassignment
> >>>>> - Sticky partitioner
> >>>>> - Return topic metadata and configs in CreateTopics response
> >>>>> - Securing Internal connect REST endpoints
> >>>>> - API to delete consumer offsets and expose it via the AdminClient.
> >>>>>
> >>>>> Release notes for the 2.4.0 release:
> >>>>>
> >> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
> >>>>>
> >>>>> ** Please download, test and vote by Tuesday, November 26, 9am PT **
> >>>>>
> >>>>> Kafka's KEYS file containing PGP keys we use to sign the release:
> >>>>> https://kafka.apache.org/KEYS
> >>>>>
> >>>>> * Release artifacts to be voted upon (source and binary):
> >>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
> >>>>>
> >>>>> * Maven artifacts to be voted upon:
> >>>>>
> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >>>>>
> >>>>> * Javadoc:
> >>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
> >>>>>
> >>>>> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> >>>>> https://github.com/apache/kafka/releases/tag/2.4.0-rc1
> >>>>>
> >>>>> * Documentation:
> >>>>> https://kafka.apache.org/24/documentation.html
> >>>>>
> >>>>> * Protocol:
> >>>>> https://kafka.apache.org/24/protocol.html
> >>>>>
> >>>>> Thanks,
> >>>>> Manikumar
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: [VOTE] 2.4.0 RC1
Posted by Manikumar <ma...@gmail.com>.
Hi All,
We will consider KAFKA-9244
<https://issues.apache.org/jira/browse/KAFKA-9244> as blocker and include
the fix in 2.4 release.
I am canceling this VOTE and will create third release candidate.
Thank you all for testing.
On Fri, Nov 29, 2019 at 10:52 AM Matthias J. Sax <ma...@confluent.io>
wrote:
> I did not find the bug -- it was reported by Kin Sui
> (https://issues.apache.org/jira/browse/KAFKA-9244)
>
> If the bug is a blocker is a judgment call thought, because it's
> technically not a regression. However, if we don't include the fix in
> 2.4.0, as Adam pointed out, the new foreign-key join would compute
> incorrect results, and thus, it's at least a critical issue.
>
>
> -Matthias
>
>
>
> On 11/28/19 11:48 AM, Adam Bellemare wrote:
> > mjsax found an important issue for the foreign-key joiner, which I think
> > should be a blocker (if it isn't already) since it is functionally
> > incorrect without the fix:
> >
> > https://github.com/apache/kafka/pull/7758
> >
> >
> >
> > On Tue, Nov 26, 2019 at 6:26 PM Sean Glover <se...@lightbend.com>
> > wrote:
> >
> >> Hi,
> >>
> >> I also used Eric's test script. I had a few issues running it that I
> >> address below[0][1], otherwise looks good.
> >>
> >> - Signing keys all good
> >> - All md5, sha1sums and sha512sums are good
> >> - A couple transient test failures that passed on a second run
> >> (ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker,
> >> SaslScramSslEndToEndAuthorizationTest.
> >> testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl)
> >> - Passes our own test suite for Alpakka Kafka (
> >> https://travis-ci.org/akka/alpakka-kafka/builds/616861540,
> >> https://github.com/akka/alpakka-kafka/pull/971)
> >>
> >> +1 (non-binding)
> >>
> >> ..
> >>
> >> Issues while running test script:
> >>
> >> [0] Error with Eric test script. I had an issue running the script
> with my
> >> version of bash (TMPDIR was unassigned), which I provided a PR for (
> >> https://github.com/elalonde/kafka/pull/1)
> >> [1] Gradle incompatibility. I ran into difficulty running the gradle
> build
> >> with the latest version of gradle (6.0.1). I had to revert to the last
> >> patch of version 5 (5.6.4):
> >>
> >> ✘ seglo@slice
> /tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src
> >> gradle wrapper --warning-mode all
> >>
> >>> Configure project :
> >> The maven plugin has been deprecated. This is scheduled to be removed in
> >> Gradle 7.0. Please use the maven-publish plugin instead.
> >> at
> >>
> >>
> build_c0129pbfzzxjolwxmds3lsevz$_run_closure5.doCall(/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle:160)
> >> (Run with --stacktrace to get the full stack trace of this
> >> deprecation warning.)
> >>
> >> FAILURE: Build failed with an exception.
> >>
> >> * Where:
> >> Build file
> >> '/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle'
> line:
> >> 472
> >>
> >> * What went wrong:
> >> A problem occurred evaluating root project 'kafka-2.4.0-src'.
> >>> Could not create task ':clients:spotbugsMain'.
> >> > Could not create task of type 'SpotBugsTask'.
> >> > Could not create an instance of type
> >> com.github.spotbugs.internal.SpotBugsReportsImpl.
> >> >
> >>
> >>
> org.gradle.api.reporting.internal.TaskReportContainer.<init>(Ljava/lang/Class;Lorg/gradle/api/Task;)V
> >>
> >> * Try:
> >> Run with --stacktrace option to get the stack trace. Run with --info or
> >> --debug option to get more log output. Run with --scan to get full
> >> insights.
> >>
> >> * Get more help at https://help.gradle.org
> >>
> >> BUILD FAILED in 699ms
> >>
> >> On Tue, Nov 26, 2019 at 1:31 PM Manikumar <ma...@gmail.com>
> >> wrote:
> >>
> >>> Hi All,
> >>>
> >>> Please download, test and vote the RC1 in order to provide quality
> >>> assurance for the forthcoming 2.4 release.
> >>>
> >>> Thanks.
> >>>
> >>> On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <
> adam.bellemare@gmail.com
> >>>
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> Ran Eric's test script:
> >>>> $ git clone https://github.com/elalonde/kafka
> >>>> $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
> >>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1
> >>>> <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
> >>>>
> >>>> - All PGP signatures are good
> >>>> - All md5, sha1sums and sha512sums pass
> >>>> - Had a few intermittent failures in tests that passed upon rerunning.
> >>>>
> >>>> +1 (non-binding) from me.
> >>>>
> >>>> Adam
> >>>>
> >>>> On Wed, Nov 20, 2019 at 10:37 AM Manikumar <manikumar.reddy@gmail.com
> >
> >>>> wrote:
> >>>>
> >>>>> Hello Kafka users, developers and client-developers,
> >>>>>
> >>>>> This is the second candidate for release of Apache Kafka 2.4.0.
> >>>>>
> >>>>> This release includes many new features, including:
> >>>>> - Allow consumers to fetch from closest replica
> >>>>> - Support for incremental cooperative rebalancing to the consumer
> >>>> rebalance
> >>>>> protocol
> >>>>> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
> >>>> replication
> >>>>> engine
> >>>>> - New Java authorizer Interface
> >>>>> - Support for non-key joining in KTable
> >>>>> - Administrative API for replica reassignment
> >>>>> - Sticky partitioner
> >>>>> - Return topic metadata and configs in CreateTopics response
> >>>>> - Securing Internal connect REST endpoints
> >>>>> - API to delete consumer offsets and expose it via the AdminClient.
> >>>>>
> >>>>> Release notes for the 2.4.0 release:
> >>>>>
> >> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
> >>>>>
> >>>>> ** Please download, test and vote by Tuesday, November 26, 9am PT **
> >>>>>
> >>>>> Kafka's KEYS file containing PGP keys we use to sign the release:
> >>>>> https://kafka.apache.org/KEYS
> >>>>>
> >>>>> * Release artifacts to be voted upon (source and binary):
> >>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
> >>>>>
> >>>>> * Maven artifacts to be voted upon:
> >>>>>
> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >>>>>
> >>>>> * Javadoc:
> >>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
> >>>>>
> >>>>> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> >>>>> https://github.com/apache/kafka/releases/tag/2.4.0-rc1
> >>>>>
> >>>>> * Documentation:
> >>>>> https://kafka.apache.org/24/documentation.html
> >>>>>
> >>>>> * Protocol:
> >>>>> https://kafka.apache.org/24/protocol.html
> >>>>>
> >>>>> Thanks,
> >>>>> Manikumar
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: [VOTE] 2.4.0 RC1
Posted by "Matthias J. Sax" <ma...@confluent.io>.
I did not find the bug -- it was reported by Kin Sui
(https://issues.apache.org/jira/browse/KAFKA-9244)
If the bug is a blocker is a judgment call thought, because it's
technically not a regression. However, if we don't include the fix in
2.4.0, as Adam pointed out, the new foreign-key join would compute
incorrect results, and thus, it's at least a critical issue.
-Matthias
On 11/28/19 11:48 AM, Adam Bellemare wrote:
> mjsax found an important issue for the foreign-key joiner, which I think
> should be a blocker (if it isn't already) since it is functionally
> incorrect without the fix:
>
> https://github.com/apache/kafka/pull/7758
>
>
>
> On Tue, Nov 26, 2019 at 6:26 PM Sean Glover <se...@lightbend.com>
> wrote:
>
>> Hi,
>>
>> I also used Eric's test script. I had a few issues running it that I
>> address below[0][1], otherwise looks good.
>>
>> - Signing keys all good
>> - All md5, sha1sums and sha512sums are good
>> - A couple transient test failures that passed on a second run
>> (ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker,
>> SaslScramSslEndToEndAuthorizationTest.
>> testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl)
>> - Passes our own test suite for Alpakka Kafka (
>> https://travis-ci.org/akka/alpakka-kafka/builds/616861540,
>> https://github.com/akka/alpakka-kafka/pull/971)
>>
>> +1 (non-binding)
>>
>> ..
>>
>> Issues while running test script:
>>
>> [0] Error with Eric test script. I had an issue running the script with my
>> version of bash (TMPDIR was unassigned), which I provided a PR for (
>> https://github.com/elalonde/kafka/pull/1)
>> [1] Gradle incompatibility. I ran into difficulty running the gradle build
>> with the latest version of gradle (6.0.1). I had to revert to the last
>> patch of version 5 (5.6.4):
>>
>> ✘ seglo@slice /tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src
>> gradle wrapper --warning-mode all
>>
>>> Configure project :
>> The maven plugin has been deprecated. This is scheduled to be removed in
>> Gradle 7.0. Please use the maven-publish plugin instead.
>> at
>>
>> build_c0129pbfzzxjolwxmds3lsevz$_run_closure5.doCall(/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle:160)
>> (Run with --stacktrace to get the full stack trace of this
>> deprecation warning.)
>>
>> FAILURE: Build failed with an exception.
>>
>> * Where:
>> Build file
>> '/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle' line:
>> 472
>>
>> * What went wrong:
>> A problem occurred evaluating root project 'kafka-2.4.0-src'.
>>> Could not create task ':clients:spotbugsMain'.
>> > Could not create task of type 'SpotBugsTask'.
>> > Could not create an instance of type
>> com.github.spotbugs.internal.SpotBugsReportsImpl.
>> >
>>
>> org.gradle.api.reporting.internal.TaskReportContainer.<init>(Ljava/lang/Class;Lorg/gradle/api/Task;)V
>>
>> * Try:
>> Run with --stacktrace option to get the stack trace. Run with --info or
>> --debug option to get more log output. Run with --scan to get full
>> insights.
>>
>> * Get more help at https://help.gradle.org
>>
>> BUILD FAILED in 699ms
>>
>> On Tue, Nov 26, 2019 at 1:31 PM Manikumar <ma...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> Please download, test and vote the RC1 in order to provide quality
>>> assurance for the forthcoming 2.4 release.
>>>
>>> Thanks.
>>>
>>> On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <adam.bellemare@gmail.com
>>>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> Ran Eric's test script:
>>>> $ git clone https://github.com/elalonde/kafka
>>>> $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1
>>>> <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
>>>>
>>>> - All PGP signatures are good
>>>> - All md5, sha1sums and sha512sums pass
>>>> - Had a few intermittent failures in tests that passed upon rerunning.
>>>>
>>>> +1 (non-binding) from me.
>>>>
>>>> Adam
>>>>
>>>> On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Kafka users, developers and client-developers,
>>>>>
>>>>> This is the second candidate for release of Apache Kafka 2.4.0.
>>>>>
>>>>> This release includes many new features, including:
>>>>> - Allow consumers to fetch from closest replica
>>>>> - Support for incremental cooperative rebalancing to the consumer
>>>> rebalance
>>>>> protocol
>>>>> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
>>>> replication
>>>>> engine
>>>>> - New Java authorizer Interface
>>>>> - Support for non-key joining in KTable
>>>>> - Administrative API for replica reassignment
>>>>> - Sticky partitioner
>>>>> - Return topic metadata and configs in CreateTopics response
>>>>> - Securing Internal connect REST endpoints
>>>>> - API to delete consumer offsets and expose it via the AdminClient.
>>>>>
>>>>> Release notes for the 2.4.0 release:
>>>>>
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
>>>>>
>>>>> ** Please download, test and vote by Tuesday, November 26, 9am PT **
>>>>>
>>>>> Kafka's KEYS file containing PGP keys we use to sign the release:
>>>>> https://kafka.apache.org/KEYS
>>>>>
>>>>> * Release artifacts to be voted upon (source and binary):
>>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
>>>>>
>>>>> * Maven artifacts to be voted upon:
>>>>>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>>>>
>>>>> * Javadoc:
>>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
>>>>>
>>>>> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
>>>>> https://github.com/apache/kafka/releases/tag/2.4.0-rc1
>>>>>
>>>>> * Documentation:
>>>>> https://kafka.apache.org/24/documentation.html
>>>>>
>>>>> * Protocol:
>>>>> https://kafka.apache.org/24/protocol.html
>>>>>
>>>>> Thanks,
>>>>> Manikumar
>>>>>
>>>>
>>>
>>
>
Re: [VOTE] 2.4.0 RC1
Posted by "Matthias J. Sax" <ma...@confluent.io>.
I did not find the bug -- it was reported by Kin Sui
(https://issues.apache.org/jira/browse/KAFKA-9244)
If the bug is a blocker is a judgment call thought, because it's
technically not a regression. However, if we don't include the fix in
2.4.0, as Adam pointed out, the new foreign-key join would compute
incorrect results, and thus, it's at least a critical issue.
-Matthias
On 11/28/19 11:48 AM, Adam Bellemare wrote:
> mjsax found an important issue for the foreign-key joiner, which I think
> should be a blocker (if it isn't already) since it is functionally
> incorrect without the fix:
>
> https://github.com/apache/kafka/pull/7758
>
>
>
> On Tue, Nov 26, 2019 at 6:26 PM Sean Glover <se...@lightbend.com>
> wrote:
>
>> Hi,
>>
>> I also used Eric's test script. I had a few issues running it that I
>> address below[0][1], otherwise looks good.
>>
>> - Signing keys all good
>> - All md5, sha1sums and sha512sums are good
>> - A couple transient test failures that passed on a second run
>> (ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker,
>> SaslScramSslEndToEndAuthorizationTest.
>> testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl)
>> - Passes our own test suite for Alpakka Kafka (
>> https://travis-ci.org/akka/alpakka-kafka/builds/616861540,
>> https://github.com/akka/alpakka-kafka/pull/971)
>>
>> +1 (non-binding)
>>
>> ..
>>
>> Issues while running test script:
>>
>> [0] Error with Eric test script. I had an issue running the script with my
>> version of bash (TMPDIR was unassigned), which I provided a PR for (
>> https://github.com/elalonde/kafka/pull/1)
>> [1] Gradle incompatibility. I ran into difficulty running the gradle build
>> with the latest version of gradle (6.0.1). I had to revert to the last
>> patch of version 5 (5.6.4):
>>
>> ✘ seglo@slice /tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src
>> gradle wrapper --warning-mode all
>>
>>> Configure project :
>> The maven plugin has been deprecated. This is scheduled to be removed in
>> Gradle 7.0. Please use the maven-publish plugin instead.
>> at
>>
>> build_c0129pbfzzxjolwxmds3lsevz$_run_closure5.doCall(/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle:160)
>> (Run with --stacktrace to get the full stack trace of this
>> deprecation warning.)
>>
>> FAILURE: Build failed with an exception.
>>
>> * Where:
>> Build file
>> '/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle' line:
>> 472
>>
>> * What went wrong:
>> A problem occurred evaluating root project 'kafka-2.4.0-src'.
>>> Could not create task ':clients:spotbugsMain'.
>> > Could not create task of type 'SpotBugsTask'.
>> > Could not create an instance of type
>> com.github.spotbugs.internal.SpotBugsReportsImpl.
>> >
>>
>> org.gradle.api.reporting.internal.TaskReportContainer.<init>(Ljava/lang/Class;Lorg/gradle/api/Task;)V
>>
>> * Try:
>> Run with --stacktrace option to get the stack trace. Run with --info or
>> --debug option to get more log output. Run with --scan to get full
>> insights.
>>
>> * Get more help at https://help.gradle.org
>>
>> BUILD FAILED in 699ms
>>
>> On Tue, Nov 26, 2019 at 1:31 PM Manikumar <ma...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> Please download, test and vote the RC1 in order to provide quality
>>> assurance for the forthcoming 2.4 release.
>>>
>>> Thanks.
>>>
>>> On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <adam.bellemare@gmail.com
>>>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> Ran Eric's test script:
>>>> $ git clone https://github.com/elalonde/kafka
>>>> $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1
>>>> <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
>>>>
>>>> - All PGP signatures are good
>>>> - All md5, sha1sums and sha512sums pass
>>>> - Had a few intermittent failures in tests that passed upon rerunning.
>>>>
>>>> +1 (non-binding) from me.
>>>>
>>>> Adam
>>>>
>>>> On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Kafka users, developers and client-developers,
>>>>>
>>>>> This is the second candidate for release of Apache Kafka 2.4.0.
>>>>>
>>>>> This release includes many new features, including:
>>>>> - Allow consumers to fetch from closest replica
>>>>> - Support for incremental cooperative rebalancing to the consumer
>>>> rebalance
>>>>> protocol
>>>>> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
>>>> replication
>>>>> engine
>>>>> - New Java authorizer Interface
>>>>> - Support for non-key joining in KTable
>>>>> - Administrative API for replica reassignment
>>>>> - Sticky partitioner
>>>>> - Return topic metadata and configs in CreateTopics response
>>>>> - Securing Internal connect REST endpoints
>>>>> - API to delete consumer offsets and expose it via the AdminClient.
>>>>>
>>>>> Release notes for the 2.4.0 release:
>>>>>
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
>>>>>
>>>>> ** Please download, test and vote by Tuesday, November 26, 9am PT **
>>>>>
>>>>> Kafka's KEYS file containing PGP keys we use to sign the release:
>>>>> https://kafka.apache.org/KEYS
>>>>>
>>>>> * Release artifacts to be voted upon (source and binary):
>>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
>>>>>
>>>>> * Maven artifacts to be voted upon:
>>>>>
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>>>>
>>>>> * Javadoc:
>>>>> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
>>>>>
>>>>> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
>>>>> https://github.com/apache/kafka/releases/tag/2.4.0-rc1
>>>>>
>>>>> * Documentation:
>>>>> https://kafka.apache.org/24/documentation.html
>>>>>
>>>>> * Protocol:
>>>>> https://kafka.apache.org/24/protocol.html
>>>>>
>>>>> Thanks,
>>>>> Manikumar
>>>>>
>>>>
>>>
>>
>
Re: [VOTE] 2.4.0 RC1
Posted by Adam Bellemare <ad...@gmail.com>.
mjsax found an important issue for the foreign-key joiner, which I think
should be a blocker (if it isn't already) since it is functionally
incorrect without the fix:
https://github.com/apache/kafka/pull/7758
On Tue, Nov 26, 2019 at 6:26 PM Sean Glover <se...@lightbend.com>
wrote:
> Hi,
>
> I also used Eric's test script. I had a few issues running it that I
> address below[0][1], otherwise looks good.
>
> - Signing keys all good
> - All md5, sha1sums and sha512sums are good
> - A couple transient test failures that passed on a second run
> (ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker,
> SaslScramSslEndToEndAuthorizationTest.
> testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl)
> - Passes our own test suite for Alpakka Kafka (
> https://travis-ci.org/akka/alpakka-kafka/builds/616861540,
> https://github.com/akka/alpakka-kafka/pull/971)
>
> +1 (non-binding)
>
> ..
>
> Issues while running test script:
>
> [0] Error with Eric test script. I had an issue running the script with my
> version of bash (TMPDIR was unassigned), which I provided a PR for (
> https://github.com/elalonde/kafka/pull/1)
> [1] Gradle incompatibility. I ran into difficulty running the gradle build
> with the latest version of gradle (6.0.1). I had to revert to the last
> patch of version 5 (5.6.4):
>
> ✘ seglo@slice /tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src
> gradle wrapper --warning-mode all
>
> > Configure project :
> The maven plugin has been deprecated. This is scheduled to be removed in
> Gradle 7.0. Please use the maven-publish plugin instead.
> at
>
> build_c0129pbfzzxjolwxmds3lsevz$_run_closure5.doCall(/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle:160)
> (Run with --stacktrace to get the full stack trace of this
> deprecation warning.)
>
> FAILURE: Build failed with an exception.
>
> * Where:
> Build file
> '/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle' line:
> 472
>
> * What went wrong:
> A problem occurred evaluating root project 'kafka-2.4.0-src'.
> > Could not create task ':clients:spotbugsMain'.
> > Could not create task of type 'SpotBugsTask'.
> > Could not create an instance of type
> com.github.spotbugs.internal.SpotBugsReportsImpl.
> >
>
> org.gradle.api.reporting.internal.TaskReportContainer.<init>(Ljava/lang/Class;Lorg/gradle/api/Task;)V
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full
> insights.
>
> * Get more help at https://help.gradle.org
>
> BUILD FAILED in 699ms
>
> On Tue, Nov 26, 2019 at 1:31 PM Manikumar <ma...@gmail.com>
> wrote:
>
> > Hi All,
> >
> > Please download, test and vote the RC1 in order to provide quality
> > assurance for the forthcoming 2.4 release.
> >
> > Thanks.
> >
> > On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <adam.bellemare@gmail.com
> >
> > wrote:
> >
> > > Hello,
> > >
> > > Ran Eric's test script:
> > > $ git clone https://github.com/elalonde/kafka
> > > $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
> > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1
> > > <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
> > >
> > > - All PGP signatures are good
> > > - All md5, sha1sums and sha512sums pass
> > > - Had a few intermittent failures in tests that passed upon rerunning.
> > >
> > > +1 (non-binding) from me.
> > >
> > > Adam
> > >
> > > On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
> > > wrote:
> > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the second candidate for release of Apache Kafka 2.4.0.
> > > >
> > > > This release includes many new features, including:
> > > > - Allow consumers to fetch from closest replica
> > > > - Support for incremental cooperative rebalancing to the consumer
> > > rebalance
> > > > protocol
> > > > - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
> > > replication
> > > > engine
> > > > - New Java authorizer Interface
> > > > - Support for non-key joining in KTable
> > > > - Administrative API for replica reassignment
> > > > - Sticky partitioner
> > > > - Return topic metadata and configs in CreateTopics response
> > > > - Securing Internal connect REST endpoints
> > > > - API to delete consumer offsets and expose it via the AdminClient.
> > > >
> > > > Release notes for the 2.4.0 release:
> > > >
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
> > > >
> > > > ** Please download, test and vote by Tuesday, November 26, 9am PT **
> > > >
> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > https://kafka.apache.org/KEYS
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
> > > >
> > > > * Maven artifacts to be voted upon:
> > > >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >
> > > > * Javadoc:
> > > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
> > > >
> > > > * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> > > > https://github.com/apache/kafka/releases/tag/2.4.0-rc1
> > > >
> > > > * Documentation:
> > > > https://kafka.apache.org/24/documentation.html
> > > >
> > > > * Protocol:
> > > > https://kafka.apache.org/24/protocol.html
> > > >
> > > > Thanks,
> > > > Manikumar
> > > >
> > >
> >
>
Re: [VOTE] 2.4.0 RC1
Posted by Adam Bellemare <ad...@gmail.com>.
mjsax found an important issue for the foreign-key joiner, which I think
should be a blocker (if it isn't already) since it is functionally
incorrect without the fix:
https://github.com/apache/kafka/pull/7758
On Tue, Nov 26, 2019 at 6:26 PM Sean Glover <se...@lightbend.com>
wrote:
> Hi,
>
> I also used Eric's test script. I had a few issues running it that I
> address below[0][1], otherwise looks good.
>
> - Signing keys all good
> - All md5, sha1sums and sha512sums are good
> - A couple transient test failures that passed on a second run
> (ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker,
> SaslScramSslEndToEndAuthorizationTest.
> testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl)
> - Passes our own test suite for Alpakka Kafka (
> https://travis-ci.org/akka/alpakka-kafka/builds/616861540,
> https://github.com/akka/alpakka-kafka/pull/971)
>
> +1 (non-binding)
>
> ..
>
> Issues while running test script:
>
> [0] Error with Eric test script. I had an issue running the script with my
> version of bash (TMPDIR was unassigned), which I provided a PR for (
> https://github.com/elalonde/kafka/pull/1)
> [1] Gradle incompatibility. I ran into difficulty running the gradle build
> with the latest version of gradle (6.0.1). I had to revert to the last
> patch of version 5 (5.6.4):
>
> ✘ seglo@slice /tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src
> gradle wrapper --warning-mode all
>
> > Configure project :
> The maven plugin has been deprecated. This is scheduled to be removed in
> Gradle 7.0. Please use the maven-publish plugin instead.
> at
>
> build_c0129pbfzzxjolwxmds3lsevz$_run_closure5.doCall(/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle:160)
> (Run with --stacktrace to get the full stack trace of this
> deprecation warning.)
>
> FAILURE: Build failed with an exception.
>
> * Where:
> Build file
> '/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle' line:
> 472
>
> * What went wrong:
> A problem occurred evaluating root project 'kafka-2.4.0-src'.
> > Could not create task ':clients:spotbugsMain'.
> > Could not create task of type 'SpotBugsTask'.
> > Could not create an instance of type
> com.github.spotbugs.internal.SpotBugsReportsImpl.
> >
>
> org.gradle.api.reporting.internal.TaskReportContainer.<init>(Ljava/lang/Class;Lorg/gradle/api/Task;)V
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full
> insights.
>
> * Get more help at https://help.gradle.org
>
> BUILD FAILED in 699ms
>
> On Tue, Nov 26, 2019 at 1:31 PM Manikumar <ma...@gmail.com>
> wrote:
>
> > Hi All,
> >
> > Please download, test and vote the RC1 in order to provide quality
> > assurance for the forthcoming 2.4 release.
> >
> > Thanks.
> >
> > On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <adam.bellemare@gmail.com
> >
> > wrote:
> >
> > > Hello,
> > >
> > > Ran Eric's test script:
> > > $ git clone https://github.com/elalonde/kafka
> > > $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
> > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1
> > > <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
> > >
> > > - All PGP signatures are good
> > > - All md5, sha1sums and sha512sums pass
> > > - Had a few intermittent failures in tests that passed upon rerunning.
> > >
> > > +1 (non-binding) from me.
> > >
> > > Adam
> > >
> > > On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
> > > wrote:
> > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the second candidate for release of Apache Kafka 2.4.0.
> > > >
> > > > This release includes many new features, including:
> > > > - Allow consumers to fetch from closest replica
> > > > - Support for incremental cooperative rebalancing to the consumer
> > > rebalance
> > > > protocol
> > > > - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
> > > replication
> > > > engine
> > > > - New Java authorizer Interface
> > > > - Support for non-key joining in KTable
> > > > - Administrative API for replica reassignment
> > > > - Sticky partitioner
> > > > - Return topic metadata and configs in CreateTopics response
> > > > - Securing Internal connect REST endpoints
> > > > - API to delete consumer offsets and expose it via the AdminClient.
> > > >
> > > > Release notes for the 2.4.0 release:
> > > >
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
> > > >
> > > > ** Please download, test and vote by Tuesday, November 26, 9am PT **
> > > >
> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > https://kafka.apache.org/KEYS
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
> > > >
> > > > * Maven artifacts to be voted upon:
> > > >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >
> > > > * Javadoc:
> > > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
> > > >
> > > > * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> > > > https://github.com/apache/kafka/releases/tag/2.4.0-rc1
> > > >
> > > > * Documentation:
> > > > https://kafka.apache.org/24/documentation.html
> > > >
> > > > * Protocol:
> > > > https://kafka.apache.org/24/protocol.html
> > > >
> > > > Thanks,
> > > > Manikumar
> > > >
> > >
> >
>
Re: [VOTE] 2.4.0 RC1
Posted by Sean Glover <se...@lightbend.com>.
Hi,
I also used Eric's test script. I had a few issues running it that I
address below[0][1], otherwise looks good.
- Signing keys all good
- All md5, sha1sums and sha512sums are good
- A couple transient test failures that passed on a second run
(ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker,
SaslScramSslEndToEndAuthorizationTest.
testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl)
- Passes our own test suite for Alpakka Kafka (
https://travis-ci.org/akka/alpakka-kafka/builds/616861540,
https://github.com/akka/alpakka-kafka/pull/971)
+1 (non-binding)
..
Issues while running test script:
[0] Error with Eric test script. I had an issue running the script with my
version of bash (TMPDIR was unassigned), which I provided a PR for (
https://github.com/elalonde/kafka/pull/1)
[1] Gradle incompatibility. I ran into difficulty running the gradle build
with the latest version of gradle (6.0.1). I had to revert to the last
patch of version 5 (5.6.4):
✘ seglo@slice /tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src
gradle wrapper --warning-mode all
> Configure project :
The maven plugin has been deprecated. This is scheduled to be removed in
Gradle 7.0. Please use the maven-publish plugin instead.
at
build_c0129pbfzzxjolwxmds3lsevz$_run_closure5.doCall(/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle:160)
(Run with --stacktrace to get the full stack trace of this
deprecation warning.)
FAILURE: Build failed with an exception.
* Where:
Build file
'/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle' line:
472
* What went wrong:
A problem occurred evaluating root project 'kafka-2.4.0-src'.
> Could not create task ':clients:spotbugsMain'.
> Could not create task of type 'SpotBugsTask'.
> Could not create an instance of type
com.github.spotbugs.internal.SpotBugsReportsImpl.
>
org.gradle.api.reporting.internal.TaskReportContainer.<init>(Ljava/lang/Class;Lorg/gradle/api/Task;)V
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or
--debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 699ms
On Tue, Nov 26, 2019 at 1:31 PM Manikumar <ma...@gmail.com> wrote:
> Hi All,
>
> Please download, test and vote the RC1 in order to provide quality
> assurance for the forthcoming 2.4 release.
>
> Thanks.
>
> On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <ad...@gmail.com>
> wrote:
>
> > Hello,
> >
> > Ran Eric's test script:
> > $ git clone https://github.com/elalonde/kafka
> > $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
> > https://home.apache.org/~manikumar/kafka-2.4.0-rc1
> > <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
> >
> > - All PGP signatures are good
> > - All md5, sha1sums and sha512sums pass
> > - Had a few intermittent failures in tests that passed upon rerunning.
> >
> > +1 (non-binding) from me.
> >
> > Adam
> >
> > On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the second candidate for release of Apache Kafka 2.4.0.
> > >
> > > This release includes many new features, including:
> > > - Allow consumers to fetch from closest replica
> > > - Support for incremental cooperative rebalancing to the consumer
> > rebalance
> > > protocol
> > > - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
> > replication
> > > engine
> > > - New Java authorizer Interface
> > > - Support for non-key joining in KTable
> > > - Administrative API for replica reassignment
> > > - Sticky partitioner
> > > - Return topic metadata and configs in CreateTopics response
> > > - Securing Internal connect REST endpoints
> > > - API to delete consumer offsets and expose it via the AdminClient.
> > >
> > > Release notes for the 2.4.0 release:
> > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
> > >
> > > ** Please download, test and vote by Tuesday, November 26, 9am PT **
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > https://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >
> > > * Javadoc:
> > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
> > >
> > > * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> > > https://github.com/apache/kafka/releases/tag/2.4.0-rc1
> > >
> > > * Documentation:
> > > https://kafka.apache.org/24/documentation.html
> > >
> > > * Protocol:
> > > https://kafka.apache.org/24/protocol.html
> > >
> > > Thanks,
> > > Manikumar
> > >
> >
>
Re: [VOTE] 2.4.0 RC1
Posted by Sean Glover <se...@lightbend.com>.
Hi,
I also used Eric's test script. I had a few issues running it that I
address below[0][1], otherwise looks good.
- Signing keys all good
- All md5, sha1sums and sha512sums are good
- A couple transient test failures that passed on a second run
(ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker,
SaslScramSslEndToEndAuthorizationTest.
testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl)
- Passes our own test suite for Alpakka Kafka (
https://travis-ci.org/akka/alpakka-kafka/builds/616861540,
https://github.com/akka/alpakka-kafka/pull/971)
+1 (non-binding)
..
Issues while running test script:
[0] Error with Eric test script. I had an issue running the script with my
version of bash (TMPDIR was unassigned), which I provided a PR for (
https://github.com/elalonde/kafka/pull/1)
[1] Gradle incompatibility. I ran into difficulty running the gradle build
with the latest version of gradle (6.0.1). I had to revert to the last
patch of version 5 (5.6.4):
✘ seglo@slice /tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src
gradle wrapper --warning-mode all
> Configure project :
The maven plugin has been deprecated. This is scheduled to be removed in
Gradle 7.0. Please use the maven-publish plugin instead.
at
build_c0129pbfzzxjolwxmds3lsevz$_run_closure5.doCall(/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle:160)
(Run with --stacktrace to get the full stack trace of this
deprecation warning.)
FAILURE: Build failed with an exception.
* Where:
Build file
'/tmp/verify-kafka-SP06GE1GpP/10169.out/kafka-2.4.0-src/build.gradle' line:
472
* What went wrong:
A problem occurred evaluating root project 'kafka-2.4.0-src'.
> Could not create task ':clients:spotbugsMain'.
> Could not create task of type 'SpotBugsTask'.
> Could not create an instance of type
com.github.spotbugs.internal.SpotBugsReportsImpl.
>
org.gradle.api.reporting.internal.TaskReportContainer.<init>(Ljava/lang/Class;Lorg/gradle/api/Task;)V
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or
--debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 699ms
On Tue, Nov 26, 2019 at 1:31 PM Manikumar <ma...@gmail.com> wrote:
> Hi All,
>
> Please download, test and vote the RC1 in order to provide quality
> assurance for the forthcoming 2.4 release.
>
> Thanks.
>
> On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <ad...@gmail.com>
> wrote:
>
> > Hello,
> >
> > Ran Eric's test script:
> > $ git clone https://github.com/elalonde/kafka
> > $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
> > https://home.apache.org/~manikumar/kafka-2.4.0-rc1
> > <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
> >
> > - All PGP signatures are good
> > - All md5, sha1sums and sha512sums pass
> > - Had a few intermittent failures in tests that passed upon rerunning.
> >
> > +1 (non-binding) from me.
> >
> > Adam
> >
> > On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the second candidate for release of Apache Kafka 2.4.0.
> > >
> > > This release includes many new features, including:
> > > - Allow consumers to fetch from closest replica
> > > - Support for incremental cooperative rebalancing to the consumer
> > rebalance
> > > protocol
> > > - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
> > replication
> > > engine
> > > - New Java authorizer Interface
> > > - Support for non-key joining in KTable
> > > - Administrative API for replica reassignment
> > > - Sticky partitioner
> > > - Return topic metadata and configs in CreateTopics response
> > > - Securing Internal connect REST endpoints
> > > - API to delete consumer offsets and expose it via the AdminClient.
> > >
> > > Release notes for the 2.4.0 release:
> > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
> > >
> > > ** Please download, test and vote by Tuesday, November 26, 9am PT **
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > https://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >
> > > * Javadoc:
> > > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
> > >
> > > * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> > > https://github.com/apache/kafka/releases/tag/2.4.0-rc1
> > >
> > > * Documentation:
> > > https://kafka.apache.org/24/documentation.html
> > >
> > > * Protocol:
> > > https://kafka.apache.org/24/protocol.html
> > >
> > > Thanks,
> > > Manikumar
> > >
> >
>
Re: [VOTE] 2.4.0 RC1
Posted by Manikumar <ma...@gmail.com>.
Hi All,
Please download, test and vote the RC1 in order to provide quality
assurance for the forthcoming 2.4 release.
Thanks.
On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <ad...@gmail.com>
wrote:
> Hello,
>
> Ran Eric's test script:
> $ git clone https://github.com/elalonde/kafka
> $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1
> <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
>
> - All PGP signatures are good
> - All md5, sha1sums and sha512sums pass
> - Had a few intermittent failures in tests that passed upon rerunning.
>
> +1 (non-binding) from me.
>
> Adam
>
> On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
> wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the second candidate for release of Apache Kafka 2.4.0.
> >
> > This release includes many new features, including:
> > - Allow consumers to fetch from closest replica
> > - Support for incremental cooperative rebalancing to the consumer
> rebalance
> > protocol
> > - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
> replication
> > engine
> > - New Java authorizer Interface
> > - Support for non-key joining in KTable
> > - Administrative API for replica reassignment
> > - Sticky partitioner
> > - Return topic metadata and configs in CreateTopics response
> > - Securing Internal connect REST endpoints
> > - API to delete consumer offsets and expose it via the AdminClient.
> >
> > Release notes for the 2.4.0 release:
> > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
> >
> > ** Please download, test and vote by Tuesday, November 26, 9am PT **
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
> >
> > * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> > https://github.com/apache/kafka/releases/tag/2.4.0-rc1
> >
> > * Documentation:
> > https://kafka.apache.org/24/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/24/protocol.html
> >
> > Thanks,
> > Manikumar
> >
>
Re: [VOTE] 2.4.0 RC1
Posted by Manikumar <ma...@gmail.com>.
Hi All,
Please download, test and vote the RC1 in order to provide quality
assurance for the forthcoming 2.4 release.
Thanks.
On Tue, Nov 26, 2019 at 8:11 PM Adam Bellemare <ad...@gmail.com>
wrote:
> Hello,
>
> Ran Eric's test script:
> $ git clone https://github.com/elalonde/kafka
> $ ./kafka/bin/verify-kafka-rc.sh 2.4.0
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1
> <https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
>
> - All PGP signatures are good
> - All md5, sha1sums and sha512sums pass
> - Had a few intermittent failures in tests that passed upon rerunning.
>
> +1 (non-binding) from me.
>
> Adam
>
> On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
> wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the second candidate for release of Apache Kafka 2.4.0.
> >
> > This release includes many new features, including:
> > - Allow consumers to fetch from closest replica
> > - Support for incremental cooperative rebalancing to the consumer
> rebalance
> > protocol
> > - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
> replication
> > engine
> > - New Java authorizer Interface
> > - Support for non-key joining in KTable
> > - Administrative API for replica reassignment
> > - Sticky partitioner
> > - Return topic metadata and configs in CreateTopics response
> > - Securing Internal connect REST endpoints
> > - API to delete consumer offsets and expose it via the AdminClient.
> >
> > Release notes for the 2.4.0 release:
> > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
> >
> > ** Please download, test and vote by Tuesday, November 26, 9am PT **
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
> >
> > * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> > https://github.com/apache/kafka/releases/tag/2.4.0-rc1
> >
> > * Documentation:
> > https://kafka.apache.org/24/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/24/protocol.html
> >
> > Thanks,
> > Manikumar
> >
>
Re: [VOTE] 2.4.0 RC1
Posted by Adam Bellemare <ad...@gmail.com>.
Hello,
Ran Eric's test script:
$ git clone https://github.com/elalonde/kafka
$ ./kafka/bin/verify-kafka-rc.sh 2.4.0
https://home.apache.org/~manikumar/kafka-2.4.0-rc1
<https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
- All PGP signatures are good
- All md5, sha1sums and sha512sums pass
- Had a few intermittent failures in tests that passed upon rerunning.
+1 (non-binding) from me.
Adam
On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the second candidate for release of Apache Kafka 2.4.0.
>
> This release includes many new features, including:
> - Allow consumers to fetch from closest replica
> - Support for incremental cooperative rebalancing to the consumer rebalance
> protocol
> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter replication
> engine
> - New Java authorizer Interface
> - Support for non-key joining in KTable
> - Administrative API for replica reassignment
> - Sticky partitioner
> - Return topic metadata and configs in CreateTopics response
> - Securing Internal connect REST endpoints
> - API to delete consumer offsets and expose it via the AdminClient.
>
> Release notes for the 2.4.0 release:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
>
> ** Please download, test and vote by Tuesday, November 26, 9am PT **
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
>
> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> https://github.com/apache/kafka/releases/tag/2.4.0-rc1
>
> * Documentation:
> https://kafka.apache.org/24/documentation.html
>
> * Protocol:
> https://kafka.apache.org/24/protocol.html
>
> Thanks,
> Manikumar
>
Re: [VOTE] 2.4.0 RC1
Posted by Adam Bellemare <ad...@gmail.com>.
Hello,
Ran Eric's test script:
$ git clone https://github.com/elalonde/kafka
$ ./kafka/bin/verify-kafka-rc.sh 2.4.0
https://home.apache.org/~manikumar/kafka-2.4.0-rc1
<https://home.apache.org/~manikumar/kafka-2.4.0-rc0>
- All PGP signatures are good
- All md5, sha1sums and sha512sums pass
- Had a few intermittent failures in tests that passed upon rerunning.
+1 (non-binding) from me.
Adam
On Wed, Nov 20, 2019 at 10:37 AM Manikumar <ma...@gmail.com>
wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the second candidate for release of Apache Kafka 2.4.0.
>
> This release includes many new features, including:
> - Allow consumers to fetch from closest replica
> - Support for incremental cooperative rebalancing to the consumer rebalance
> protocol
> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter replication
> engine
> - New Java authorizer Interface
> - Support for non-key joining in KTable
> - Administrative API for replica reassignment
> - Sticky partitioner
> - Return topic metadata and configs in CreateTopics response
> - Securing Internal connect REST endpoints
> - API to delete consumer offsets and expose it via the AdminClient.
>
> Release notes for the 2.4.0 release:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html
>
> ** Please download, test and vote by Tuesday, November 26, 9am PT **
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc1/javadoc/
>
> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> https://github.com/apache/kafka/releases/tag/2.4.0-rc1
>
> * Documentation:
> https://kafka.apache.org/24/documentation.html
>
> * Protocol:
> https://kafka.apache.org/24/protocol.html
>
> Thanks,
> Manikumar
>