You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "smjn (via GitHub)" <gi...@apache.org> on 2025/04/30 11:54:46 UTC
[PR] KAFKA-19204: Allow persister retry of initializing topics. [kafka]
smjn opened a new pull request, #19603:
URL: https://github.com/apache/kafka/pull/19603
* Currently in the share group heartbeat flow, if we see a TP subscribed for the first time, we move that TP to initializing state in GC and let the GC send a persister request to share group initialize the aforementioned TP.
* However, if the coordinator runtime request for share group heartbeat times out (maybe due to restarting/bad broker), the future completes exceptionally resulting in persiter request to not be sent.
* Now, we are in a bad state since the TP is in initializing state in GC but not persister initialized. Future heartbeats for the same share partitions will also not help since we do not allow retrying persister request for initializing TPs.
* This PR remedies the situation by allowing the same.
* A temporary fix to increase offset commit timeouts in system tests was added to fix the issue. In this PR, we revert that change as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19204: Allow persister retry of initializing topics. [kafka]
Posted by "smjn (via GitHub)" <gi...@apache.org>.
smjn commented on code in PR #19603:
URL: https://github.com/apache/kafka/pull/19603#discussion_r2071394535
##########
clients/clients-integration-tests/src/test/java/org/apache/kafka/clients/consumer/ShareConsumerTest.java:
##########
@@ -2058,7 +2058,8 @@ public void testComplexShareConsumer() throws Exception {
// all messages which can be read are read, some would be redelivered
TestUtils.waitForCondition(complexCons1::isDone, 45_000L, () -> "did not close!");
- assertTrue(prodState.count().get() < complexCons1.recordsRead());
+ assertTrue(prodState.count().get() <= complexCons1.recordsRead(),
Review Comment:
@AndrewJSchofield Thanks for the review.
I'll keep the condition as the same but add some info as part of failed message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19204: Allow persister retry of initializing topics. [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #19603:
URL: https://github.com/apache/kafka/pull/19603#discussion_r2070296330
##########
clients/clients-integration-tests/src/test/java/org/apache/kafka/clients/consumer/ShareConsumerTest.java:
##########
@@ -2058,7 +2058,8 @@ public void testComplexShareConsumer() throws Exception {
// all messages which can be read are read, some would be redelivered
TestUtils.waitForCondition(complexCons1::isDone, 45_000L, () -> "did not close!");
- assertTrue(prodState.count().get() < complexCons1.recordsRead());
+ assertTrue(prodState.count().get() <= complexCons1.recordsRead(),
Review Comment:
This assertion failed in your build :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19204: Allow persister retry of initializing topics. [kafka]
Posted by "smjn (via GitHub)" <gi...@apache.org>.
smjn commented on code in PR #19603:
URL: https://github.com/apache/kafka/pull/19603#discussion_r2071394535
##########
clients/clients-integration-tests/src/test/java/org/apache/kafka/clients/consumer/ShareConsumerTest.java:
##########
@@ -2058,7 +2058,8 @@ public void testComplexShareConsumer() throws Exception {
// all messages which can be read are read, some would be redelivered
TestUtils.waitForCondition(complexCons1::isDone, 45_000L, () -> "did not close!");
- assertTrue(prodState.count().get() < complexCons1.recordsRead());
+ assertTrue(prodState.count().get() <= complexCons1.recordsRead(),
Review Comment:
I'll keep the condition as the same but add some info as part of failed message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19204: Allow persister retry of initializing topics. [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield merged PR #19603:
URL: https://github.com/apache/kafka/pull/19603
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org