You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Israel Ekpo <is...@gmail.com> on 2019/12/06 21:49:17 UTC
Re: [kafka-clients] [VOTE] 2.4.0 RC3
Ran the tests in the following environments using Eric's script available
here:
https://github.com/elalonde/kafka/blob/master/bin/verify-kafka-rc.sh
OS: Ubuntu 18.04.3 LTS
Java Version: OpenJDK 11.0.4
Scala Versions: 2.12.10, 12.13.0, 12.13.1
Gradle Version: 5.6.2
I have made one observation in the release artifacts here:
https://home.apache.org/~manikumar/kafka-2.4.0-rc3/kafka-2.4.0-src.tgz
The latest available release for Scala is 12.13.1
It looks like the artifacts were built in an environment where the scala
version in the CLI is set to 2.12.10 so this is the value in
gradle.properties for the source artifact here:
scalaVersion=2.12.10
Not sure if that needs to change but it seems in the previous releases this
is usually set to the highest available scala version at the time of
release.
Also, the test
kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker
failed the first time I ran all the tests but it passed the second and
third time I ran the group kafka.admin.ReassignPartitionsClusterTest and
just the single method
(kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker).
I will re-run the tests again in Scala versions 12.13.0, 12.13.1 and share
my observations later.
So far, it looks good.
isekpo@ossvalidator:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.3 LTS
Release: 18.04
Codename: bionic
isekpo@ossvalidator:~$ java -version
openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3,
mixed mode, sharing)
isekpo@ossvalidator:~$ scala -version
Scala code runner version 2.12.10 -- Copyright 2002-2019, LAMP/EPFL and
Lightbend, Inc.
isekpo@ossvalidator:~$
isekpo@ossvalidator:~$ gradle -version
------------------------------------------------------------
Gradle 5.6.2
------------------------------------------------------------
Build time: 2019-09-05 16:13:54 UTC
Revision: 55a5e53d855db8fc7b0e494412fc624051a8e781
Kotlin: 1.3.41
Groovy: 2.5.4
Ant: Apache Ant(TM) version 1.9.14 compiled on March 12 2019
JVM: 11.0.4 (Ubuntu 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
OS: Linux 5.0.0-1027-azure amd64
1309 tests completed, 1 failed, 17 skipped
> Task :core:integrationTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':core:integrationTest'.
> There were failing tests. See the report at:
file:///home/isekpo/scratchpad/14891.out/kafka-2.4.0-src/core/build/reports/tests/integrationTest/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or
--debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible
with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 53m 52s
14 actionable tasks: 3 executed, 11 up-to-date
Details for failed test:
shouldMoveSinglePartitionWithinBroker
org.scalatest.exceptions.TestFailedException: Partition should have been
moved to the expected log directory
at
org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:530)
at
org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:529)
at
org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1389)
at org.scalatest.Assertions.fail(Assertions.scala:1091)
at org.scalatest.Assertions.fail$(Assertions.scala:1087)
at org.scalatest.Assertions$.fail(Assertions.scala:1389)
at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:842)
at
kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker(ReassignPartitionsClusterTest.scala:177)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:65)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
at org.junit.runners.ParentRunner.run(ParentRunner.java:412)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
at
org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
at
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
at
org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
at
org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.base/java.lang.Thread.run(Thread.java:834)
On Thu, Dec 5, 2019 at 12:10 AM Manikumar <ma...@gmail.com> wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the fourth candidate for release of Apache Kafka 2.4.0.
>
> This release includes many new features, including:
> - Allow consumers to fetch from closest replica
> - Support for incremental cooperative rebalancing to the consumer
> rebalance protocol
> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter replication
> engine
> - New Java authorizer Interface
> - Support for non-key joining in KTable
> - Administrative API for replica reassignment
> - Sticky partitioner
> - Return topic metadata and configs in CreateTopics response
> - Securing Internal connect REST endpoints
> - API to delete consumer offsets and expose it via the AdminClient.
>
> Release notes for the 2.4.0 release:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/RELEASE_NOTES.html
>
> *** Please download, test and vote by Monday, December 9th, 9 pm PT
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/javadoc/
>
> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
> https://github.com/apache/kafka/releases/tag/2.4.0-rc3
>
> * Documentation:
> https://kafka.apache.org/24/documentation.html
>
> * Protocol:
> https://kafka.apache.org/24/protocol.html
>
> Thanks,
> Manikumar
>
> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
Re: [kafka-clients] [VOTE] 2.4.0 RC3
Posted by Manikumar <ma...@gmail.com>.
Hi all,
We have merged the PR for KAFKA-9212. Thanks to Jason for the fixing the
issue.
Thanks to Yannick for reporting the issue and Michael Jaschob for providing
extra details.
I am canceling this vote and will create new RC shortly.
Thanks,
Manikumar
On Sat, Dec 7, 2019 at 3:19 AM Israel Ekpo <is...@gmail.com> wrote:
>
> Ran the tests in the following environments using Eric's script available
> here:
>
> https://github.com/elalonde/kafka/blob/master/bin/verify-kafka-rc.sh
>
> OS: Ubuntu 18.04.3 LTS
> Java Version: OpenJDK 11.0.4
> Scala Versions: 2.12.10, 12.13.0, 12.13.1
> Gradle Version: 5.6.2
>
> I have made one observation in the release artifacts here:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/kafka-2.4.0-src.tgz
>
> The latest available release for Scala is 12.13.1
>
> It looks like the artifacts were built in an environment where the scala
> version in the CLI is set to 2.12.10 so this is the value in
> gradle.properties for the source artifact here:
>
> scalaVersion=2.12.10
>
> Not sure if that needs to change but it seems in the previous releases
> this is usually set to the highest available scala version at the time of
> release.
>
> Also, the test
> kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker
> failed the first time I ran all the tests but it passed the second and
> third time I ran the group kafka.admin.ReassignPartitionsClusterTest and
> just the single method
> (kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker).
>
>
> I will re-run the tests again in Scala versions 12.13.0, 12.13.1 and share
> my observations later.
>
> So far, it looks good.
>
> isekpo@ossvalidator:~$ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description: Ubuntu 18.04.3 LTS
> Release: 18.04
> Codename: bionic
>
> isekpo@ossvalidator:~$ java -version
> openjdk version "11.0.4" 2019-07-16
> OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
> OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3,
> mixed mode, sharing)
>
> isekpo@ossvalidator:~$ scala -version
> Scala code runner version 2.12.10 -- Copyright 2002-2019, LAMP/EPFL and
> Lightbend, Inc.
> isekpo@ossvalidator:~$
>
> isekpo@ossvalidator:~$ gradle -version
>
> ------------------------------------------------------------
> Gradle 5.6.2
> ------------------------------------------------------------
>
> Build time: 2019-09-05 16:13:54 UTC
> Revision: 55a5e53d855db8fc7b0e494412fc624051a8e781
>
> Kotlin: 1.3.41
> Groovy: 2.5.4
> Ant: Apache Ant(TM) version 1.9.14 compiled on March 12 2019
> JVM: 11.0.4 (Ubuntu 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
> OS: Linux 5.0.0-1027-azure amd64
>
> 1309 tests completed, 1 failed, 17 skipped
>
> > Task :core:integrationTest FAILED
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
> Execution failed for task ':core:integrationTest'.
> > There were failing tests. See the report at:
> file:///home/isekpo/scratchpad/14891.out/kafka-2.4.0-src/core/build/reports/tests/integrationTest/index.html
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full insights.
>
> * Get more help at https://help.gradle.org
>
> Deprecated Gradle features were used in this build, making it incompatible
> with Gradle 6.0.
> Use '--warning-mode all' to show the individual deprecation warnings.
> See
> https://docs.gradle.org/5.6.2/userguide/command_line_interface.html#sec:command_line_warnings
>
> BUILD FAILED in 53m 52s
> 14 actionable tasks: 3 executed, 11 up-to-date
>
>
> Details for failed test:
>
> shouldMoveSinglePartitionWithinBroker
> org.scalatest.exceptions.TestFailedException: Partition should have been
> moved to the expected log directory
> at
> org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:530)
> at
> org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:529)
> at
> org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1389)
> at org.scalatest.Assertions.fail(Assertions.scala:1091)
> at org.scalatest.Assertions.fail$(Assertions.scala:1087)
> at org.scalatest.Assertions$.fail(Assertions.scala:1389)
> at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:842)
> at
> kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker(ReassignPartitionsClusterTest.scala:177)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:65)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:412)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
> at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
> at
> org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
> at
> org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
> at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
> at
> org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
> at java.base/java.lang.Thread.run(Thread.java:834)
>
>
>
> On Thu, Dec 5, 2019 at 12:10 AM Manikumar <ma...@gmail.com>
> wrote:
>
>> Hello Kafka users, developers and client-developers,
>>
>> This is the fourth candidate for release of Apache Kafka 2.4.0.
>>
>> This release includes many new features, including:
>> - Allow consumers to fetch from closest replica
>> - Support for incremental cooperative rebalancing to the consumer
>> rebalance protocol
>> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
>> replication engine
>> - New Java authorizer Interface
>> - Support for non-key joining in KTable
>> - Administrative API for replica reassignment
>> - Sticky partitioner
>> - Return topic metadata and configs in CreateTopics response
>> - Securing Internal connect REST endpoints
>> - API to delete consumer offsets and expose it via the AdminClient.
>>
>> Release notes for the 2.4.0 release:
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by Monday, December 9th, 9 pm PT
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> https://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>
>> * Javadoc:
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/javadoc/
>>
>> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
>> https://github.com/apache/kafka/releases/tag/2.4.0-rc3
>>
>> * Documentation:
>> https://kafka.apache.org/24/documentation.html
>>
>> * Protocol:
>> https://kafka.apache.org/24/protocol.html
>>
>> Thanks,
>> Manikumar
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "kafka-clients" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kafka-clients+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com
>> <https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>
Re: [kafka-clients] [VOTE] 2.4.0 RC3
Posted by Israel Ekpo <is...@gmail.com>.
This particular test keeps failing for me
./gradlew :core:integrationTest --tests
kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker
I have created an issue to document my observations
https://issues.apache.org/jira/browse/KAFKA-9283
It fails sometimes when the environment is scala 2.12.10 but fails
consistently in 2.13.0 and 2.13.1
All the other tests do pass without any issues but just this one
My environment details are in the issue
On Fri, Dec 6, 2019 at 4:49 PM Israel Ekpo <is...@gmail.com> wrote:
>
> Ran the tests in the following environments using Eric's script available
> here:
>
> https://github.com/elalonde/kafka/blob/master/bin/verify-kafka-rc.sh
>
> OS: Ubuntu 18.04.3 LTS
> Java Version: OpenJDK 11.0.4
> Scala Versions: 2.12.10, 12.13.0, 12.13.1
> Gradle Version: 5.6.2
>
> I have made one observation in the release artifacts here:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/kafka-2.4.0-src.tgz
>
> The latest available release for Scala is 12.13.1
>
> It looks like the artifacts were built in an environment where the scala
> version in the CLI is set to 2.12.10 so this is the value in
> gradle.properties for the source artifact here:
>
> scalaVersion=2.12.10
>
> Not sure if that needs to change but it seems in the previous releases
> this is usually set to the highest available scala version at the time of
> release.
>
> Also, the test
> kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker
> failed the first time I ran all the tests but it passed the second and
> third time I ran the group kafka.admin.ReassignPartitionsClusterTest and
> just the single method
> (kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker).
>
>
> I will re-run the tests again in Scala versions 12.13.0, 12.13.1 and share
> my observations later.
>
> So far, it looks good.
>
> isekpo@ossvalidator:~$ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description: Ubuntu 18.04.3 LTS
> Release: 18.04
> Codename: bionic
>
> isekpo@ossvalidator:~$ java -version
> openjdk version "11.0.4" 2019-07-16
> OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
> OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3,
> mixed mode, sharing)
>
> isekpo@ossvalidator:~$ scala -version
> Scala code runner version 2.12.10 -- Copyright 2002-2019, LAMP/EPFL and
> Lightbend, Inc.
> isekpo@ossvalidator:~$
>
> isekpo@ossvalidator:~$ gradle -version
>
> ------------------------------------------------------------
> Gradle 5.6.2
> ------------------------------------------------------------
>
> Build time: 2019-09-05 16:13:54 UTC
> Revision: 55a5e53d855db8fc7b0e494412fc624051a8e781
>
> Kotlin: 1.3.41
> Groovy: 2.5.4
> Ant: Apache Ant(TM) version 1.9.14 compiled on March 12 2019
> JVM: 11.0.4 (Ubuntu 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
> OS: Linux 5.0.0-1027-azure amd64
>
> 1309 tests completed, 1 failed, 17 skipped
>
> > Task :core:integrationTest FAILED
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
> Execution failed for task ':core:integrationTest'.
> > There were failing tests. See the report at:
> file:///home/isekpo/scratchpad/14891.out/kafka-2.4.0-src/core/build/reports/tests/integrationTest/index.html
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full insights.
>
> * Get more help at https://help.gradle.org
>
> Deprecated Gradle features were used in this build, making it incompatible
> with Gradle 6.0.
> Use '--warning-mode all' to show the individual deprecation warnings.
> See
> https://docs.gradle.org/5.6.2/userguide/command_line_interface.html#sec:command_line_warnings
>
> BUILD FAILED in 53m 52s
> 14 actionable tasks: 3 executed, 11 up-to-date
>
>
> Details for failed test:
>
> shouldMoveSinglePartitionWithinBroker
> org.scalatest.exceptions.TestFailedException: Partition should have been
> moved to the expected log directory
> at
> org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:530)
> at
> org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:529)
> at
> org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1389)
> at org.scalatest.Assertions.fail(Assertions.scala:1091)
> at org.scalatest.Assertions.fail$(Assertions.scala:1087)
> at org.scalatest.Assertions$.fail(Assertions.scala:1389)
> at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:842)
> at
> kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker(ReassignPartitionsClusterTest.scala:177)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:65)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:412)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
> at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
> at
> org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
> at
> org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
> at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
> at
> org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
> at java.base/java.lang.Thread.run(Thread.java:834)
>
>
>
> On Thu, Dec 5, 2019 at 12:10 AM Manikumar <ma...@gmail.com>
> wrote:
>
>> Hello Kafka users, developers and client-developers,
>>
>> This is the fourth candidate for release of Apache Kafka 2.4.0.
>>
>> This release includes many new features, including:
>> - Allow consumers to fetch from closest replica
>> - Support for incremental cooperative rebalancing to the consumer
>> rebalance protocol
>> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
>> replication engine
>> - New Java authorizer Interface
>> - Support for non-key joining in KTable
>> - Administrative API for replica reassignment
>> - Sticky partitioner
>> - Return topic metadata and configs in CreateTopics response
>> - Securing Internal connect REST endpoints
>> - API to delete consumer offsets and expose it via the AdminClient.
>>
>> Release notes for the 2.4.0 release:
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by Monday, December 9th, 9 pm PT
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> https://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>
>> * Javadoc:
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/javadoc/
>>
>> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
>> https://github.com/apache/kafka/releases/tag/2.4.0-rc3
>>
>> * Documentation:
>> https://kafka.apache.org/24/documentation.html
>>
>> * Protocol:
>> https://kafka.apache.org/24/protocol.html
>>
>> Thanks,
>> Manikumar
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "kafka-clients" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kafka-clients+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com
>> <https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>
Re: [kafka-clients] [VOTE] 2.4.0 RC3
Posted by Manikumar <ma...@gmail.com>.
Hi all,
We have merged the PR for KAFKA-9212. Thanks to Jason for the fixing the
issue.
Thanks to Yannick for reporting the issue and Michael Jaschob for providing
extra details.
I am canceling this vote and will create new RC shortly.
Thanks,
Manikumar
On Sat, Dec 7, 2019 at 3:19 AM Israel Ekpo <is...@gmail.com> wrote:
>
> Ran the tests in the following environments using Eric's script available
> here:
>
> https://github.com/elalonde/kafka/blob/master/bin/verify-kafka-rc.sh
>
> OS: Ubuntu 18.04.3 LTS
> Java Version: OpenJDK 11.0.4
> Scala Versions: 2.12.10, 12.13.0, 12.13.1
> Gradle Version: 5.6.2
>
> I have made one observation in the release artifacts here:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/kafka-2.4.0-src.tgz
>
> The latest available release for Scala is 12.13.1
>
> It looks like the artifacts were built in an environment where the scala
> version in the CLI is set to 2.12.10 so this is the value in
> gradle.properties for the source artifact here:
>
> scalaVersion=2.12.10
>
> Not sure if that needs to change but it seems in the previous releases
> this is usually set to the highest available scala version at the time of
> release.
>
> Also, the test
> kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker
> failed the first time I ran all the tests but it passed the second and
> third time I ran the group kafka.admin.ReassignPartitionsClusterTest and
> just the single method
> (kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker).
>
>
> I will re-run the tests again in Scala versions 12.13.0, 12.13.1 and share
> my observations later.
>
> So far, it looks good.
>
> isekpo@ossvalidator:~$ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description: Ubuntu 18.04.3 LTS
> Release: 18.04
> Codename: bionic
>
> isekpo@ossvalidator:~$ java -version
> openjdk version "11.0.4" 2019-07-16
> OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
> OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3,
> mixed mode, sharing)
>
> isekpo@ossvalidator:~$ scala -version
> Scala code runner version 2.12.10 -- Copyright 2002-2019, LAMP/EPFL and
> Lightbend, Inc.
> isekpo@ossvalidator:~$
>
> isekpo@ossvalidator:~$ gradle -version
>
> ------------------------------------------------------------
> Gradle 5.6.2
> ------------------------------------------------------------
>
> Build time: 2019-09-05 16:13:54 UTC
> Revision: 55a5e53d855db8fc7b0e494412fc624051a8e781
>
> Kotlin: 1.3.41
> Groovy: 2.5.4
> Ant: Apache Ant(TM) version 1.9.14 compiled on March 12 2019
> JVM: 11.0.4 (Ubuntu 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
> OS: Linux 5.0.0-1027-azure amd64
>
> 1309 tests completed, 1 failed, 17 skipped
>
> > Task :core:integrationTest FAILED
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
> Execution failed for task ':core:integrationTest'.
> > There were failing tests. See the report at:
> file:///home/isekpo/scratchpad/14891.out/kafka-2.4.0-src/core/build/reports/tests/integrationTest/index.html
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full insights.
>
> * Get more help at https://help.gradle.org
>
> Deprecated Gradle features were used in this build, making it incompatible
> with Gradle 6.0.
> Use '--warning-mode all' to show the individual deprecation warnings.
> See
> https://docs.gradle.org/5.6.2/userguide/command_line_interface.html#sec:command_line_warnings
>
> BUILD FAILED in 53m 52s
> 14 actionable tasks: 3 executed, 11 up-to-date
>
>
> Details for failed test:
>
> shouldMoveSinglePartitionWithinBroker
> org.scalatest.exceptions.TestFailedException: Partition should have been
> moved to the expected log directory
> at
> org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:530)
> at
> org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:529)
> at
> org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1389)
> at org.scalatest.Assertions.fail(Assertions.scala:1091)
> at org.scalatest.Assertions.fail$(Assertions.scala:1087)
> at org.scalatest.Assertions$.fail(Assertions.scala:1389)
> at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:842)
> at
> kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker(ReassignPartitionsClusterTest.scala:177)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:65)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:412)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
> at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
> at
> org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
> at
> org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
> at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
> at
> org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
> at java.base/java.lang.Thread.run(Thread.java:834)
>
>
>
> On Thu, Dec 5, 2019 at 12:10 AM Manikumar <ma...@gmail.com>
> wrote:
>
>> Hello Kafka users, developers and client-developers,
>>
>> This is the fourth candidate for release of Apache Kafka 2.4.0.
>>
>> This release includes many new features, including:
>> - Allow consumers to fetch from closest replica
>> - Support for incremental cooperative rebalancing to the consumer
>> rebalance protocol
>> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
>> replication engine
>> - New Java authorizer Interface
>> - Support for non-key joining in KTable
>> - Administrative API for replica reassignment
>> - Sticky partitioner
>> - Return topic metadata and configs in CreateTopics response
>> - Securing Internal connect REST endpoints
>> - API to delete consumer offsets and expose it via the AdminClient.
>>
>> Release notes for the 2.4.0 release:
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by Monday, December 9th, 9 pm PT
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> https://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>
>> * Javadoc:
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/javadoc/
>>
>> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
>> https://github.com/apache/kafka/releases/tag/2.4.0-rc3
>>
>> * Documentation:
>> https://kafka.apache.org/24/documentation.html
>>
>> * Protocol:
>> https://kafka.apache.org/24/protocol.html
>>
>> Thanks,
>> Manikumar
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "kafka-clients" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kafka-clients+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com
>> <https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>
Re: [kafka-clients] [VOTE] 2.4.0 RC3
Posted by Israel Ekpo <is...@gmail.com>.
This particular test keeps failing for me
./gradlew :core:integrationTest --tests
kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker
I have created an issue to document my observations
https://issues.apache.org/jira/browse/KAFKA-9283
It fails sometimes when the environment is scala 2.12.10 but fails
consistently in 2.13.0 and 2.13.1
All the other tests do pass without any issues but just this one
My environment details are in the issue
On Fri, Dec 6, 2019 at 4:49 PM Israel Ekpo <is...@gmail.com> wrote:
>
> Ran the tests in the following environments using Eric's script available
> here:
>
> https://github.com/elalonde/kafka/blob/master/bin/verify-kafka-rc.sh
>
> OS: Ubuntu 18.04.3 LTS
> Java Version: OpenJDK 11.0.4
> Scala Versions: 2.12.10, 12.13.0, 12.13.1
> Gradle Version: 5.6.2
>
> I have made one observation in the release artifacts here:
> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/kafka-2.4.0-src.tgz
>
> The latest available release for Scala is 12.13.1
>
> It looks like the artifacts were built in an environment where the scala
> version in the CLI is set to 2.12.10 so this is the value in
> gradle.properties for the source artifact here:
>
> scalaVersion=2.12.10
>
> Not sure if that needs to change but it seems in the previous releases
> this is usually set to the highest available scala version at the time of
> release.
>
> Also, the test
> kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker
> failed the first time I ran all the tests but it passed the second and
> third time I ran the group kafka.admin.ReassignPartitionsClusterTest and
> just the single method
> (kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker).
>
>
> I will re-run the tests again in Scala versions 12.13.0, 12.13.1 and share
> my observations later.
>
> So far, it looks good.
>
> isekpo@ossvalidator:~$ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description: Ubuntu 18.04.3 LTS
> Release: 18.04
> Codename: bionic
>
> isekpo@ossvalidator:~$ java -version
> openjdk version "11.0.4" 2019-07-16
> OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
> OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3,
> mixed mode, sharing)
>
> isekpo@ossvalidator:~$ scala -version
> Scala code runner version 2.12.10 -- Copyright 2002-2019, LAMP/EPFL and
> Lightbend, Inc.
> isekpo@ossvalidator:~$
>
> isekpo@ossvalidator:~$ gradle -version
>
> ------------------------------------------------------------
> Gradle 5.6.2
> ------------------------------------------------------------
>
> Build time: 2019-09-05 16:13:54 UTC
> Revision: 55a5e53d855db8fc7b0e494412fc624051a8e781
>
> Kotlin: 1.3.41
> Groovy: 2.5.4
> Ant: Apache Ant(TM) version 1.9.14 compiled on March 12 2019
> JVM: 11.0.4 (Ubuntu 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
> OS: Linux 5.0.0-1027-azure amd64
>
> 1309 tests completed, 1 failed, 17 skipped
>
> > Task :core:integrationTest FAILED
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
> Execution failed for task ':core:integrationTest'.
> > There were failing tests. See the report at:
> file:///home/isekpo/scratchpad/14891.out/kafka-2.4.0-src/core/build/reports/tests/integrationTest/index.html
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full insights.
>
> * Get more help at https://help.gradle.org
>
> Deprecated Gradle features were used in this build, making it incompatible
> with Gradle 6.0.
> Use '--warning-mode all' to show the individual deprecation warnings.
> See
> https://docs.gradle.org/5.6.2/userguide/command_line_interface.html#sec:command_line_warnings
>
> BUILD FAILED in 53m 52s
> 14 actionable tasks: 3 executed, 11 up-to-date
>
>
> Details for failed test:
>
> shouldMoveSinglePartitionWithinBroker
> org.scalatest.exceptions.TestFailedException: Partition should have been
> moved to the expected log directory
> at
> org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:530)
> at
> org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:529)
> at
> org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1389)
> at org.scalatest.Assertions.fail(Assertions.scala:1091)
> at org.scalatest.Assertions.fail$(Assertions.scala:1087)
> at org.scalatest.Assertions$.fail(Assertions.scala:1389)
> at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:842)
> at
> kafka.admin.ReassignPartitionsClusterTest.shouldMoveSinglePartitionWithinBroker(ReassignPartitionsClusterTest.scala:177)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:65)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:412)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
> at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
> at
> org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
> at
> org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
> at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
> at
> org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
> at java.base/java.lang.Thread.run(Thread.java:834)
>
>
>
> On Thu, Dec 5, 2019 at 12:10 AM Manikumar <ma...@gmail.com>
> wrote:
>
>> Hello Kafka users, developers and client-developers,
>>
>> This is the fourth candidate for release of Apache Kafka 2.4.0.
>>
>> This release includes many new features, including:
>> - Allow consumers to fetch from closest replica
>> - Support for incremental cooperative rebalancing to the consumer
>> rebalance protocol
>> - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter
>> replication engine
>> - New Java authorizer Interface
>> - Support for non-key joining in KTable
>> - Administrative API for replica reassignment
>> - Sticky partitioner
>> - Return topic metadata and configs in CreateTopics response
>> - Securing Internal connect REST endpoints
>> - API to delete consumer offsets and expose it via the AdminClient.
>>
>> Release notes for the 2.4.0 release:
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by Monday, December 9th, 9 pm PT
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> https://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>
>> * Javadoc:
>> https://home.apache.org/~manikumar/kafka-2.4.0-rc3/javadoc/
>>
>> * Tag to be voted upon (off 2.4 branch) is the 2.4.0 tag:
>> https://github.com/apache/kafka/releases/tag/2.4.0-rc3
>>
>> * Documentation:
>> https://kafka.apache.org/24/documentation.html
>>
>> * Protocol:
>> https://kafka.apache.org/24/protocol.html
>>
>> Thanks,
>> Manikumar
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "kafka-clients" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kafka-clients+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com
>> <https://groups.google.com/d/msgid/kafka-clients/CAMVt_Ayc17CpOqt-E-hvNHkOA-JGoJ2UgxcnVyPM8-yrQC7L%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>