You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/06/13 12:46:59 UTC
[GitHub] [kafka] bozhao12 opened a new pull request, #12286: KAFKA-13984: Fix TopicBasedRemoteLogMetadataManager#initializeResourc…
bozhao12 opened a new pull request, #12286:
URL: https://github.com/apache/kafka/pull/12286
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on a diff in pull request #12286: KAFKA-13984: Fix TopicBasedRemoteLogMetadataManager#initializeResources should exit immediately when partition size of __remote_log_metadata is not same as configured
Posted by GitBox <gi...@apache.org>.
divijvaidya commented on code in PR #12286:
URL: https://github.com/apache/kafka/pull/12286#discussion_r901679350
##########
storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerRestartTest.java:
##########
@@ -68,10 +72,27 @@ protected Map<String, Object> overrideRemoteLogMetadataManagerProps() {
remoteLogMetadataManagerHarness.initialize(Collections.emptySet(), true);
}
- private void startTopicBasedRemoteLogMetadataManagerHarness(boolean startConsumerThread) {
- remoteLogMetadataManagerHarness.initializeRemoteLogMetadataManager(Collections.emptySet(), startConsumerThread);
+ private void startTopicBasedRemoteLogMetadataManagerHarness(boolean startConsumerThread,
+ int remoteLogMetadataTopicPartitionCount) {
+ remoteLogMetadataManagerHarness.initializeRemoteLogMetadataManager(Collections.emptySet(), startConsumerThread,
+ remoteLogMetadataTopicPartitionCount);
}
+
+ @Test
+ public void testRLMMInitializeResources() throws Exception {
+ assertTrue(topicBasedRlmm().isInitialized());
+
+ stopTopicBasedRemoteLogMetadataManagerHarness();
+ assertThrows(KafkaException.class, () -> startTopicBasedRemoteLogMetadataManagerHarness(false, 4));
Review Comment:
please replace hard coded `4` with `METADATA_TOPIC_PARTITIONS_COUNT + 1`. This would make it explicit state that we are starting RLMM with a partition value different from existing one here. Even better would be if we can get the number of partitions of existing topic and use `that + 1` here instead of METADATA_TOPIC_PARTITIONS_COUNT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] bozhao12 commented on a diff in pull request #12286: KAFKA-13984: Fix TopicBasedRemoteLogMetadataManager#initializeResources should exit immediately when partition size of __remote_log_metadata is not same as configured
Posted by GitBox <gi...@apache.org>.
bozhao12 commented on code in PR #12286:
URL: https://github.com/apache/kafka/pull/12286#discussion_r901721861
##########
storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerRestartTest.java:
##########
@@ -68,10 +72,27 @@ protected Map<String, Object> overrideRemoteLogMetadataManagerProps() {
remoteLogMetadataManagerHarness.initialize(Collections.emptySet(), true);
}
- private void startTopicBasedRemoteLogMetadataManagerHarness(boolean startConsumerThread) {
- remoteLogMetadataManagerHarness.initializeRemoteLogMetadataManager(Collections.emptySet(), startConsumerThread);
+ private void startTopicBasedRemoteLogMetadataManagerHarness(boolean startConsumerThread,
+ int remoteLogMetadataTopicPartitionCount) {
+ remoteLogMetadataManagerHarness.initializeRemoteLogMetadataManager(Collections.emptySet(), startConsumerThread,
+ remoteLogMetadataTopicPartitionCount);
}
+
+ @Test
+ public void testRLMMInitializeResources() throws Exception {
+ assertTrue(topicBasedRlmm().isInitialized());
+
+ stopTopicBasedRemoteLogMetadataManagerHarness();
+ assertThrows(KafkaException.class, () -> startTopicBasedRemoteLogMetadataManagerHarness(false, 4));
Review Comment:
@divijvaidya Thanks for your suggestion, I updated the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] bozhao12 commented on pull request #12286: KAFKA-13984: Fix TopicBasedRemoteLogMetadataManager#initializeResources should exit immediately when partition size of __remote_log_metadata is not same as configured
Posted by GitBox <gi...@apache.org>.
bozhao12 commented on PR #12286:
URL: https://github.com/apache/kafka/pull/12286#issuecomment-1157064520
@divijvaidya Thanks for your review. I add a unit test based on your suggestion. Due to the restart operation involved, I put this unit test in `TopicBasedRemoteLogMetadataManagerRestartTest`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #12286: KAFKA-13984: Fix TopicBasedRemoteLogMetadataManager#initializeResources should exit immediately when partition size of __remote_log_metadata is not same as configured
Posted by GitBox <gi...@apache.org>.
divijvaidya commented on PR #12286:
URL: https://github.com/apache/kafka/pull/12286#issuecomment-1156415549
@bozhao12 can you please add a unit test in `TopicBasedRemoteLogMetadataManagerTest` that fails before this change and succeeds after this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #12286: KAFKA-13984: Fix TopicBasedRemoteLogMetadataManager#initializeResources should exit immediately when partition size of __remote_log_metadata is not same as configured
Posted by GitBox <gi...@apache.org>.
divijvaidya commented on PR #12286:
URL: https://github.com/apache/kafka/pull/12286#issuecomment-1168608018
@wyuka you might be interested to review this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] bozhao12 commented on pull request #12286: KAFKA-13984: Fix TopicBasedRemoteLogMetadataManager#initializeResources should exit immediately when partition size of __remote_log_metadata is not same as configured
Posted by GitBox <gi...@apache.org>.
bozhao12 commented on PR #12286:
URL: https://github.com/apache/kafka/pull/12286#issuecomment-1155171655
@satishd Please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org