You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/06/03 14:49:15 UTC
[GitHub] [kafka] dajac commented on a diff in pull request #12244: HOTFIX: only try to clear discover-coordinator future upon commit
dajac commented on code in PR #12244:
URL: https://github.com/apache/kafka/pull/12244#discussion_r889014500
##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java:
##########
@@ -514,9 +514,50 @@ public void testCoordinatorNotAvailableWithUserAssignedType() {
coordinator.poll(time.timer(0));
assertTrue(coordinator.coordinatorUnknown());
- // should find an available node in next find coordinator request
+ // should not try to find coordinator since we are in manual assignment
+ // hence the prepared response should not be returned
client.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
coordinator.poll(time.timer(Long.MAX_VALUE));
+ assertTrue(coordinator.coordinatorUnknown());
+ }
+
+ @Test
+ public void testAutoCommitAsyncWithUserAssignedType() {
+ try (ConsumerCoordinator coordinator = buildCoordinator(rebalanceConfig, new Metrics(), assignors, true, subscriptions)
+ ) {
+ subscriptions.assignFromUser(Collections.singleton(t1p));
+ // should mark coordinator unknown after COORDINATOR_NOT_AVAILABLE error
+ client.prepareResponse(groupCoordinatorResponse(node, Errors.COORDINATOR_NOT_AVAILABLE));
+ // set timeout to 0 because we don't want to retry after the error
+ coordinator.poll(time.timer(0));
+ assertTrue(coordinator.coordinatorUnknown());
+
+ // elapse auto commit interval and set committable position
+ time.sleep(autoCommitIntervalMs);
+ subscriptions.seekUnvalidated(t1p, new SubscriptionState.FetchPosition(100L));
+
+ // should try to find coordinator since we are auto committing
+ client.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
+ coordinator.poll(time.timer(Long.MAX_VALUE));
+ assertFalse(coordinator.coordinatorUnknown());
+ }
+ }
+
+ @Test
+ public void testCommitAsyncWithUserAssignedType() {
+ subscriptions.assignFromUser(Collections.singleton(t1p));
+ // should mark coordinator unknown after COORDINATOR_NOT_AVAILABLE error
+ client.prepareResponse(groupCoordinatorResponse(node, Errors.COORDINATOR_NOT_AVAILABLE));
+ // set timeout to 0 because we don't want to retry after the error
+ coordinator.poll(time.timer(0));
+ assertTrue(coordinator.coordinatorUnknown());
Review Comment:
Should we also assert that there is not inflight requests after calling poll?
##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java:
##########
@@ -514,9 +514,50 @@ public void testCoordinatorNotAvailableWithUserAssignedType() {
coordinator.poll(time.timer(0));
assertTrue(coordinator.coordinatorUnknown());
- // should find an available node in next find coordinator request
+ // should not try to find coordinator since we are in manual assignment
+ // hence the prepared response should not be returned
client.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
coordinator.poll(time.timer(Long.MAX_VALUE));
+ assertTrue(coordinator.coordinatorUnknown());
+ }
+
+ @Test
+ public void testAutoCommitAsyncWithUserAssignedType() {
+ try (ConsumerCoordinator coordinator = buildCoordinator(rebalanceConfig, new Metrics(), assignors, true, subscriptions)
+ ) {
+ subscriptions.assignFromUser(Collections.singleton(t1p));
+ // should mark coordinator unknown after COORDINATOR_NOT_AVAILABLE error
+ client.prepareResponse(groupCoordinatorResponse(node, Errors.COORDINATOR_NOT_AVAILABLE));
Review Comment:
Same question as before.
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java:
##########
@@ -553,8 +554,8 @@ public boolean poll(Timer timer, boolean waitForJoinGroup) {
// requests result in polls returning immediately, causing a tight loop of polls. Without
// the wakeup, poll() with no channels would block for the timeout, delaying re-connection.
// awaitMetadataUpdate() in ensureCoordinatorReady initiates new connections with configured backoff and avoids the busy loop.
- if (coordinatorUnknownAndUnready(timer)) {
- return false;
+ if (metadata.updateRequested() && !client.hasReadyNodes(timer.currentTimeMs())) {
Review Comment:
nit: I think that we need to remove `if coordinator is unknown, make sure we lookup one and` from the above comment (first sentence).
##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java:
##########
@@ -514,9 +514,50 @@ public void testCoordinatorNotAvailableWithUserAssignedType() {
coordinator.poll(time.timer(0));
assertTrue(coordinator.coordinatorUnknown());
- // should find an available node in next find coordinator request
+ // should not try to find coordinator since we are in manual assignment
+ // hence the prepared response should not be returned
client.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
coordinator.poll(time.timer(Long.MAX_VALUE));
+ assertTrue(coordinator.coordinatorUnknown());
+ }
+
+ @Test
+ public void testAutoCommitAsyncWithUserAssignedType() {
+ try (ConsumerCoordinator coordinator = buildCoordinator(rebalanceConfig, new Metrics(), assignors, true, subscriptions)
+ ) {
Review Comment:
nit: I would bring back the closing parenthesis and the opening curly brace on the previous line.
##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java:
##########
@@ -514,9 +514,50 @@ public void testCoordinatorNotAvailableWithUserAssignedType() {
coordinator.poll(time.timer(0));
assertTrue(coordinator.coordinatorUnknown());
- // should find an available node in next find coordinator request
+ // should not try to find coordinator since we are in manual assignment
+ // hence the prepared response should not be returned
client.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
coordinator.poll(time.timer(Long.MAX_VALUE));
+ assertTrue(coordinator.coordinatorUnknown());
+ }
+
+ @Test
+ public void testAutoCommitAsyncWithUserAssignedType() {
+ try (ConsumerCoordinator coordinator = buildCoordinator(rebalanceConfig, new Metrics(), assignors, true, subscriptions)
+ ) {
+ subscriptions.assignFromUser(Collections.singleton(t1p));
+ // should mark coordinator unknown after COORDINATOR_NOT_AVAILABLE error
+ client.prepareResponse(groupCoordinatorResponse(node, Errors.COORDINATOR_NOT_AVAILABLE));
+ // set timeout to 0 because we don't want to retry after the error
+ coordinator.poll(time.timer(0));
+ assertTrue(coordinator.coordinatorUnknown());
+
+ // elapse auto commit interval and set committable position
+ time.sleep(autoCommitIntervalMs);
+ subscriptions.seekUnvalidated(t1p, new SubscriptionState.FetchPosition(100L));
+
+ // should try to find coordinator since we are auto committing
+ client.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
+ coordinator.poll(time.timer(Long.MAX_VALUE));
+ assertFalse(coordinator.coordinatorUnknown());
+ }
+ }
+
+ @Test
+ public void testCommitAsyncWithUserAssignedType() {
+ subscriptions.assignFromUser(Collections.singleton(t1p));
+ // should mark coordinator unknown after COORDINATOR_NOT_AVAILABLE error
+ client.prepareResponse(groupCoordinatorResponse(node, Errors.COORDINATOR_NOT_AVAILABLE));
Review Comment:
Do we really need this? I thought that the whole point was to ensure that no requests are sent out when the manual mode is used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org