You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/12/10 21:02:36 UTC
[GitHub] [kafka] zigarn opened a new pull request, #12976: Improve TestUtils for temporary properties file
zigarn opened a new pull request, #12976:
URL: https://github.com/apache/kafka/pull/12976
Follow-up of suggestion by @dajac in https://github.com/apache/kafka/pull/12175#discussion_r950139983
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] zigarn commented on pull request #12976: Improve TestUtils for temporary properties file
Posted by "zigarn (via GitHub)" <gi...@apache.org>.
zigarn commented on PR #12976:
URL: https://github.com/apache/kafka/pull/12976#issuecomment-1594798209
@divijvaidya: rebased!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] zigarn commented on pull request #12976: Improve TestUtils for temporary properties file
Posted by "zigarn (via GitHub)" <gi...@apache.org>.
zigarn commented on PR #12976:
URL: https://github.com/apache/kafka/pull/12976#issuecomment-1596990442
Thanks @divijvaidya!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #12976: Improve TestUtils for temporary properties file
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #12976:
URL: https://github.com/apache/kafka/pull/12976#issuecomment-1594744324
@zigarn can you please rebase this PR with trunk?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #12976: Improve TestUtils for temporary properties file
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #12976:
URL: https://github.com/apache/kafka/pull/12976#issuecomment-1596983653
Unrelated failures:
```
[Build / JDK 17 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testReplicateSourceDefault()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationSSLTest/Build___JDK_17_and_Scala_2_13___testReplicateSourceDefault__/)
[Build / JDK 17 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationTransactionsTest.testReplication()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationTransactionsTest/Build___JDK_17_and_Scala_2_13___testReplication__/)
[Build / JDK 17 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationTransactionsTest.testReplication()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationTransactionsTest/Build___JDK_17_and_Scala_2_13___testReplication___2/)
[Build / JDK 17 and Scala 2.13 / kafka.admin.AddPartitionsTest.testReplicaPlacementAllServers(String).quorum=zk](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/kafka.admin/AddPartitionsTest/Build___JDK_17_and_Scala_2_13___testReplicaPlacementAllServers_String__quorum_zk/)
[Build / JDK 17 and Scala 2.13 / kafka.zk.ZkMigrationIntegrationTest.[1] Type=ZK, Name=testNewAndChangedTopicsInDualWrite, MetadataVersion=3.4-IV0, Security=PLAINTEXT](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/kafka.zk/ZkMigrationIntegrationTest/Build___JDK_17_and_Scala_2_13____1__Type_ZK__Name_testNewAndChangedTopicsInDualWrite__MetadataVersion_3_4_IV0__Security_PLAINTEXT/)
[Build / JDK 17 and Scala 2.13 / org.apache.kafka.controller.QuorumControllerTest.testBalancePartitionLeaders()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.controller/QuorumControllerTest/Build___JDK_17_and_Scala_2_13___testBalancePartitionLeaders__/)
[Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationTransactionsTest.testReplication()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationTransactionsTest/Build___JDK_11_and_Scala_2_13___testReplication__/)
[Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationTransactionsTest.testSyncTopicConfigs()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationTransactionsTest/Build___JDK_11_and_Scala_2_13___testSyncTopicConfigs__/)
[Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationTransactionsTest.testReplication()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationTransactionsTest/Build___JDK_11_and_Scala_2_13___testReplication___2/)
[Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsWithCustomForwardingAdminIntegrationTest.testSyncTopicConfigs()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsWithCustomForwardingAdminIntegrationTest/Build___JDK_11_and_Scala_2_13___testSyncTopicConfigs__/)
[Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.runtime.isolation.DelegatingClassLoaderTest.testLoadingMixOfValidAndInvalidPlugins](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.runtime.isolation/DelegatingClassLoaderTest/Build___JDK_11_and_Scala_2_13___testLoadingMixOfValidAndInvalidPlugins/)
[Build / JDK 11 and Scala 2.13 / org.apache.kafka.controller.QuorumControllerTest.testBalancePartitionLeaders()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.controller/QuorumControllerTest/Build___JDK_11_and_Scala_2_13___testBalancePartitionLeaders__/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.common.network.SslTransportLayerTest.[1] tlsProtocol=TLSv1.2, useInlinePem=false](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.common.network/SslTransportLayerTest/Build___JDK_8_and_Scala_2_12____1__tlsProtocol_TLSv1_2__useInlinePem_false/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationTransactionsTest.testReplication()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationTransactionsTest/Build___JDK_8_and_Scala_2_12___testReplication__/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationTransactionsTest.testReplication()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationTransactionsTest/Build___JDK_8_and_Scala_2_12___testReplication___2/)
[Build / JDK 8 and Scala 2.12 / integration.kafka.server.FetchFromFollowerIntegrationTest.testRackAwareRangeAssignor()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/integration.kafka.server/FetchFromFollowerIntegrationTest/Build___JDK_8_and_Scala_2_12___testRackAwareRangeAssignor__/)
[Build / JDK 8 and Scala 2.12 / kafka.api.DescribeAuthorizedOperationsTest.testClusterAuthorizedOperations()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/kafka.api/DescribeAuthorizedOperationsTest/Build___JDK_8_and_Scala_2_12___testClusterAuthorizedOperations__/)
[Build / JDK 8 and Scala 2.12 / kafka.api.TransactionsTest.testBumpTransactionalEpoch(String).quorum=kraft](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/kafka.api/TransactionsTest/Build___JDK_8_and_Scala_2_12___testBumpTransactionalEpoch_String__quorum_kraft/)
[Build / JDK 8 and Scala 2.12 / kafka.api.TransactionsTest.testBumpTransactionalEpoch(String).quorum=kraft](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-12976/3/testReport/junit/kafka.api/TransactionsTest/Build___JDK_8_and_Scala_2_12___testBumpTransactionalEpoch_String__quorum_kraft_2/)
```
The MirrorConnectorsIntegrationTransactionsTest have been fixed in trunk.
FetchFromFollowerIntegrationTest.testRackAwareRangeAssignor() has an open JIRA.
kafka.api.TransactionsTest.testBumpTransactionalEpoch are known failures - https://issues.apache.org/jira/browse/KAFKA-15099
DescribeAuthorizedOperationsTest.testClusterAuthorizedOperations() failed due to infra failures
```
net.sf.ehcache.CacheException: java.lang.OutOfMemoryError: unable to create new native thread
at net.sf.ehcache.CacheManager.init(CacheManager.java:425)
at net.sf.ehcache.CacheManager.<init>(CacheManager.java:269)
at org.apache.directory.server.core.api.CacheService.initialize(CacheService.java:164)
at org.apache.directory.server.core.DefaultDirectoryService.initialize(DefaultDirectoryService.java:1795)
at org.apache.directory.server.core.DefaultDirectoryService.startup(DefaultDirectoryService.java:1251)
at kafka.security.minikdc.MiniKdc.initDirectoryService(MiniKdc.scala:181)
at kafka.security.minikdc.MiniKdc.start(MiniKdc.scala:121)
at kafka.api.SaslSetup.initializeKerberos(SaslSetup.scala:83)
at kafka.api.SaslSetup.initializeKerberos$(SaslSetup.scala:80)
at kafka.api.DescribeAuthorizedOperationsTest.initializeKerberos(DescribeAuthorizedOperationsTest.scala:74)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:719)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
at java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
at java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:597)
at net.sf.ehcache.store.disk.DiskStorageFactory.<init>(DiskStorageFactory.java:165)
at net.sf.ehcache.store.disk.DiskStore.create(DiskStore.java:154)
at net.sf.ehcache.store.disk.DiskStore.createCacheStore(DiskStore.java:182)
```
SslTransportLayerTest failed due to channel getting closed in idle state.
testBalancePartitionLeaders() has an open JIRA for being flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya merged pull request #12976: Improve TestUtils for temporary properties file
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya merged PR #12976:
URL: https://github.com/apache/kafka/pull/12976
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on a diff in pull request #12976: Improve TestUtils for temporary properties file
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on code in PR #12976:
URL: https://github.com/apache/kafka/pull/12976#discussion_r1232345504
##########
core/src/test/scala/unit/kafka/utils/TestUtils.scala:
##########
@@ -156,6 +156,21 @@ object TestUtils extends Logging {
*/
def tempFile(prefix: String, suffix: String): File = JTestUtils.tempFile(prefix, suffix)
+ /**
+ * Create a file with the given contents in the default temporary-file directory,
+ * using `kafka` as the prefix and `tmp` as the suffix to generate its name.
+ */
+ def tempFile(contents: String): File = JTestUtils.tempFile(contents)
+
+ def tempPropertiesFile(properties: Properties): File = {
+ return tempPropertiesFile(properties.asScala)
+ }
+
+ def tempPropertiesFile(properties: Map[String, String]): File = {
+ val content = properties.map{case (k, v) => k + "=" + v}.mkString("\n")
Review Comment:
please use `System.lineSeparator()` instead of `\n`. It's a much safer way to write newline characters that make the code system independent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org