You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "kamalcph (via GitHub)" <gi...@apache.org> on 2023/08/24 06:51:17 UTC
[GitHub] [kafka] kamalcph opened a new pull request, #14283: KAFKA-15400: Use readLock when removing an item from the RemoteIndexCache
kamalcph opened a new pull request, #14283:
URL: https://github.com/apache/kafka/pull/14283
- Caffeine cache is thread safe, we want to hold the writeLock only during the close.
- Fix the flaky tests
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on a diff in pull request #14283: KAFKA-15400: Use readLock when removing an item from the RemoteIndexCache
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on code in PR #14283:
URL: https://github.com/apache/kafka/pull/14283#discussion_r1304038849
##########
core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala:
##########
@@ -91,7 +91,11 @@ class RemoteIndexCacheTest {
Utils.closeQuietly(cache, "RemoteIndexCache created for unit test")
// best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
// will be deleted at the end of test.
- Utils.delete(logDir)
Review Comment:
Thank you for catching this. Dumb of me to assume that Utils.delete won't throw an error!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] kamalcph commented on a diff in pull request #14283: KAFKA-15400: Use readLock when removing an item from the RemoteIndexCache
Posted by "kamalcph (via GitHub)" <gi...@apache.org>.
kamalcph commented on code in PR #14283:
URL: https://github.com/apache/kafka/pull/14283#discussion_r1303913325
##########
core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala:
##########
@@ -91,7 +91,11 @@ class RemoteIndexCacheTest {
Utils.closeQuietly(cache, "RemoteIndexCache created for unit test")
// best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
// will be deleted at the end of test.
- Utils.delete(logDir)
Review Comment:
since `logDir` is a tempDir. It will be removed on exit anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #14283: KAFKA-15400: Use readLock when removing an item from the RemoteIndexCache
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #14283:
URL: https://github.com/apache/kafka/pull/14283#issuecomment-1691520548
Merging to 3.6 branch as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #14283: KAFKA-15400: Use readLock when removing an item from the RemoteIndexCache
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #14283:
URL: https://github.com/apache/kafka/pull/14283#issuecomment-1691518898
Unrelated test failures:
```
[Build / JDK 20 and Scala 2.13 / kafka.api.GroupEndToEndAuthorizationTest.testAuthentications(String).quorum=kraft](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/kafka.api/GroupEndToEndAuthorizationTest/Build___JDK_20_and_Scala_2_13___testAuthentications_String__quorum_kraft/)
[Build / JDK 20 and Scala 2.13 / kafka.server.DynamicBrokerReconfigurationTest.testThreadPoolResize()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/kafka.server/DynamicBrokerReconfigurationTest/Build___JDK_20_and_Scala_2_13___testThreadPoolResize__/)
[Build / JDK 20 and Scala 2.13 / org.apache.kafka.tiered.storage.integration.OffloadAndConsumeFromLeaderTest.initializationError](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tiered.storage.integration/OffloadAndConsumeFromLeaderTest/Build___JDK_20_and_Scala_2_13___initializationError/)
[Build / JDK 20 and Scala 2.13 / org.apache.kafka.streams.integration.JoinGracePeriodDurabilityIntegrationTest.shouldRecoverBufferAfterShutdown[at_least_once]](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.streams.integration/JoinGracePeriodDurabilityIntegrationTest/Build___JDK_20_and_Scala_2_13___shouldRecoverBufferAfterShutdown_at_least_once_/)
[Build / JDK 20 and Scala 2.13 / org.apache.kafka.streams.integration.JoinGracePeriodDurabilityIntegrationTest.shouldRecoverBufferAfterShutdown[exactly_once]](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.streams.integration/JoinGracePeriodDurabilityIntegrationTest/Build___JDK_20_and_Scala_2_13___shouldRecoverBufferAfterShutdown_exactly_once_/)
[Build / JDK 20 and Scala 2.13 / org.apache.kafka.streams.integration.JoinGracePeriodDurabilityIntegrationTest.shouldRecoverBufferAfterShutdown[exactly_once_v2]](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.streams.integration/JoinGracePeriodDurabilityIntegrationTest/Build___JDK_20_and_Scala_2_13___shouldRecoverBufferAfterShutdown_exactly_once_v2_/)
[Build / JDK 11 and Scala 2.13 / kafka.api.ClientIdQuotaTest.testProducerConsumerOverrideUnthrottled(String).quorum=zk](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/kafka.api/ClientIdQuotaTest/Build___JDK_11_and_Scala_2_13___testProducerConsumerOverrideUnthrottled_String__quorum_zk/)
[Build / JDK 11 and Scala 2.13 / kafka.server.DynamicBrokerReconfigurationTest.testThreadPoolResize()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/kafka.server/DynamicBrokerReconfigurationTest/Build___JDK_11_and_Scala_2_13___testThreadPoolResize__/)
[Build / JDK 11 and Scala 2.13 / org.apache.kafka.tiered.storage.integration.OffloadAndConsumeFromLeaderTest.initializationError](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tiered.storage.integration/OffloadAndConsumeFromLeaderTest/Build___JDK_11_and_Scala_2_13___initializationError/)
[Build / JDK 17 and Scala 2.13 / integration.kafka.server.FetchFromFollowerIntegrationTest.testRackAwareRangeAssignor()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/integration.kafka.server/FetchFromFollowerIntegrationTest/Build___JDK_17_and_Scala_2_13___testRackAwareRangeAssignor__/)
[Build / JDK 17 and Scala 2.13 / kafka.server.DynamicBrokerReconfigurationTest.testThreadPoolResize()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/kafka.server/DynamicBrokerReconfigurationTest/Build___JDK_17_and_Scala_2_13___testThreadPoolResize__/)
[Build / JDK 17 and Scala 2.13 / org.apache.kafka.controller.QuorumControllerTest.testBalancePartitionLeaders()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.controller/QuorumControllerTest/Build___JDK_17_and_Scala_2_13___testBalancePartitionLeaders__/)
[Build / JDK 17 and Scala 2.13 / org.apache.kafka.tiered.storage.integration.OffloadAndConsumeFromLeaderTest.initializationError](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tiered.storage.integration/OffloadAndConsumeFromLeaderTest/Build___JDK_17_and_Scala_2_13___initializationError/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsWithCustomForwardingAdminIntegrationTest.testReplicationIsCreatingTopicsUsingProvidedForwardingAdmin()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsWithCustomForwardingAdminIntegrationTest/Build___JDK_8_and_Scala_2_12___testReplicationIsCreatingTopicsUsingProvidedForwardingAdmin__/)
[Build / JDK 8 and Scala 2.12 / kafka.admin.ReassignPartitionsIntegrationTest.testAlterReassignmentThrottle(String).quorum=kraft](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/kafka.admin/ReassignPartitionsIntegrationTest/Build___JDK_8_and_Scala_2_12___testAlterReassignmentThrottle_String__quorum_kraft/)
[Build / JDK 8 and Scala 2.12 / kafka.server.DynamicBrokerReconfigurationTest.testThreadPoolResize()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/kafka.server/DynamicBrokerReconfigurationTest/Build___JDK_8_and_Scala_2_12___testThreadPoolResize__/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.controller.QuorumControllerTest.testBalancePartitionLeaders()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.controller/QuorumControllerTest/Build___JDK_8_and_Scala_2_12___testBalancePartitionLeaders__/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.tiered.storage.integration.OffloadAndConsumeFromLeaderTest.initializationError](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tiered.storage.integration/OffloadAndConsumeFromLeaderTest/Build___JDK_8_and_Scala_2_12___initializationError/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.streams.integration.EOSUncleanShutdownIntegrationTest.shouldWorkWithUncleanShutdownWipeOutStateStore[exactly_once]](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.streams.integration/EOSUncleanShutdownIntegrationTest/Build___JDK_8_and_Scala_2_12___shouldWorkWithUncleanShutdownWipeOutStateStore_exactly_once_/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.streams.integration.EosV2UpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosV2[true]](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.streams.integration/EosV2UpgradeIntegrationTest/Build___JDK_8_and_Scala_2_12___shouldUpgradeFromEosAlphaToEosV2_true_/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[1] Type=Raft-Combined, Name=testDescribeQuorumReplicationSuccessful, MetadataVersion=3.6-IV2, Security=PLAINTEXT](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tools/MetadataQuorumCommandTest/Build___JDK_8_and_Scala_2_12____1__Type_Raft_Combined__Name_testDescribeQuorumReplicationSuccessful__MetadataVersion_3_6_IV2__Security_PLAINTEXT/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[5] Type=Raft-Combined, Name=testDescribeQuorumReplicationSuccessful, MetadataVersion=3.6-IV2, Security=PLAINTEXT](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tools/MetadataQuorumCommandTest/Build___JDK_8_and_Scala_2_12____5__Type_Raft_Combined__Name_testDescribeQuorumReplicationSuccessful__MetadataVersion_3_6_IV2__Security_PLAINTEXT/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[1] Type=Raft-Combined, Name=testDescribeQuorumStatusSuccessful, MetadataVersion=3.6-IV2, Security=PLAINTEXT](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tools/MetadataQuorumCommandTest/Build___JDK_8_and_Scala_2_12____1__Type_Raft_Combined__Name_testDescribeQuorumStatusSuccessful__MetadataVersion_3_6_IV2__Security_PLAINTEXT/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[2] Type=Raft-Isolated, Name=testDescribeQuorumStatusSuccessful, MetadataVersion=3.6-IV2, Security=PLAINTEXT](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tools/MetadataQuorumCommandTest/Build___JDK_8_and_Scala_2_12____2__Type_Raft_Isolated__Name_testDescribeQuorumStatusSuccessful__MetadataVersion_3_6_IV2__Security_PLAINTEXT/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[5] Type=Raft-Combined, Name=testDescribeQuorumStatusSuccessful, MetadataVersion=3.6-IV2, Security=PLAINTEXT](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tools/MetadataQuorumCommandTest/Build___JDK_8_and_Scala_2_12____5__Type_Raft_Combined__Name_testDescribeQuorumStatusSuccessful__MetadataVersion_3_6_IV2__Security_PLAINTEXT/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[2] Type=Raft-Isolated, Name=testDescribeQuorumStatusSuccessful, MetadataVersion=3.6-IV2, Security=PLAINTEXT](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.tools/MetadataQuorumCommandTest/Build___JDK_8_and_Scala_2_12____2__Type_Raft_Isolated__Name_testDescribeQuorumStatusSuccessful__MetadataVersion_3_6_IV2__Security_PLAINTEXT_2/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14283/1/testReport/junit/org.apache.kafka.trogdor.coordinator/CoordinatorTest/Build___JDK_8_and_Scala_2_12___testTaskRequestWithOldStartMsGetsUpdated__/)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya merged pull request #14283: KAFKA-15400: Use readLock when removing an item from the RemoteIndexCache
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya merged PR #14283:
URL: https://github.com/apache/kafka/pull/14283
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] kamalcph commented on a diff in pull request #14283: KAFKA-15400: Use readLock when removing an item from the RemoteIndexCache
Posted by "kamalcph (via GitHub)" <gi...@apache.org>.
kamalcph commented on code in PR #14283:
URL: https://github.com/apache/kafka/pull/14283#discussion_r1303913325
##########
core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala:
##########
@@ -91,7 +91,11 @@ class RemoteIndexCacheTest {
Utils.closeQuietly(cache, "RemoteIndexCache created for unit test")
// best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
// will be deleted at the end of test.
- Utils.delete(logDir)
Review Comment:
since `logDir` is a tempDir, it will be removed on exit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org