You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "lianetm (via GitHub)" <gi...@apache.org> on 2024/01/16 15:36:02 UTC

[PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

lianetm opened a new pull request, #15202:
URL: https://github.com/apache/kafka/pull/15202

   This includes multiple fixes for offsets commit/fetch error handling:
   
   - ensure the right exceptions are thrown for each expected error
   - ensure KafkaException is thrown for all unexpected errors
   - properly handle disconnection exceptions (added for fetch, fixed for commit)  
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1463371625


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -536,12 +543,23 @@ public void onResponse(final ClientResponse response) {
                         continue;
                     }
 
-                    if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
+                    if (error == Errors.GROUP_AUTHORIZATION_FAILED) {
+                        future.completeExceptionally(GroupAuthorizationException.forGroupId(groupId));
+                        return;
+                    } else if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
                         error == Errors.NOT_COORDINATOR ||
                         error == Errors.REQUEST_TIMED_OUT) {
                         coordinatorRequestManager.markCoordinatorUnknown(error.message(), currentTimeMs);
                         maybeRetry(currentTimeMs, error.exception());
                         return;
+                    } else if (error == Errors.FENCED_INSTANCE_ID) {
+                        log.info("OffsetCommit failed due to group instance id {} fenced: {}", groupInstanceId, error.message());
+                        future.completeExceptionally(new CommitFailedException());

Review Comment:
   Ah. I was referring to adding a message to `new CommitFailedException()`. Sorry for the confusion.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1462700895


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -536,12 +543,23 @@ public void onResponse(final ClientResponse response) {
                         continue;
                     }
 
-                    if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
+                    if (error == Errors.GROUP_AUTHORIZATION_FAILED) {
+                        future.completeExceptionally(GroupAuthorizationException.forGroupId(groupId));
+                        return;
+                    } else if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
                         error == Errors.NOT_COORDINATOR ||
                         error == Errors.REQUEST_TIMED_OUT) {
                         coordinatorRequestManager.markCoordinatorUnknown(error.message(), currentTimeMs);
                         maybeRetry(currentTimeMs, error.exception());
                         return;
+                    } else if (error == Errors.FENCED_INSTANCE_ID) {
+                        log.info("OffsetCommit failed due to group instance id {} fenced: {}", groupInstanceId, error.message());
+                        future.completeExceptionally(new CommitFailedException());
+                        return;
+                    } else if (error == Errors.OFFSET_METADATA_TOO_LARGE ||
+                        error == Errors.INVALID_COMMIT_OFFSET_SIZE) {
+                        future.completeExceptionally(error.exception());
+                        return;
                     } else if (error == Errors.COORDINATOR_LOAD_IN_PROGRESS ||
                         error == Errors.UNKNOWN_TOPIC_OR_PARTITION) {
                         // just retry

Review Comment:
   Agree, but the intention was to bring the retry changes in a next PR after this one, is that ok? That one will include the refactoring to move all the retries to the specific caller functions in this same manager, and keep the response handling only completing/failing the future.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1463253788


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -536,12 +543,23 @@ public void onResponse(final ClientResponse response) {
                         continue;
                     }
 
-                    if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
+                    if (error == Errors.GROUP_AUTHORIZATION_FAILED) {
+                        future.completeExceptionally(GroupAuthorizationException.forGroupId(groupId));
+                        return;
+                    } else if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
                         error == Errors.NOT_COORDINATOR ||
                         error == Errors.REQUEST_TIMED_OUT) {
                         coordinatorRequestManager.markCoordinatorUnknown(error.message(), currentTimeMs);
                         maybeRetry(currentTimeMs, error.exception());
                         return;
+                    } else if (error == Errors.FENCED_INSTANCE_ID) {
+                        log.info("OffsetCommit failed due to group instance id {} fenced: {}", groupInstanceId, error.message());
+                        future.completeExceptionally(new CommitFailedException());
+                        return;
+                    } else if (error == Errors.OFFSET_METADATA_TOO_LARGE ||
+                        error == Errors.INVALID_COMMIT_OFFSET_SIZE) {
+                        future.completeExceptionally(error.exception());
+                        return;
                     } else if (error == Errors.COORDINATOR_LOAD_IN_PROGRESS ||
                         error == Errors.UNKNOWN_TOPIC_OR_PARTITION) {
                         // just retry

Review Comment:
   Sure.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on PR #15202:
URL: https://github.com/apache/kafka/pull/15202#issuecomment-1906359448

   Hey @dajac , thanks for the comments, all addressed. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1466381107


##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java:
##########
@@ -272,7 +270,35 @@ public void testSyncAutocommitRetriedAfterRetriableException(Errors error) {
 
         // We expect that request should have been retried on this sync commit.
         assertExceptionHandling(commitRequestManger, error, true);
-        assertCoordinatorDisconnect(error);
+    }
+
+    @ParameterizedTest
+    @MethodSource("commitSyncExpectedExceptions")
+    public void testCommitSyncFailsWithExpectedException(Errors commitError,
+                                                      Class<? extends Exception> expectedException) {

Review Comment:
   nit: Indentation is off here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1463372247


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -536,12 +543,23 @@ public void onResponse(final ClientResponse response) {
                         continue;
                     }
 
-                    if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
+                    if (error == Errors.GROUP_AUTHORIZATION_FAILED) {
+                        future.completeExceptionally(GroupAuthorizationException.forGroupId(groupId));
+                        return;
+                    } else if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
                         error == Errors.NOT_COORDINATOR ||
                         error == Errors.REQUEST_TIMED_OUT) {
                         coordinatorRequestManager.markCoordinatorUnknown(error.message(), currentTimeMs);
                         maybeRetry(currentTimeMs, error.exception());
                         return;
+                    } else if (error == Errors.FENCED_INSTANCE_ID) {
+                        log.info("OffsetCommit failed due to group instance id {} fenced: {}", groupInstanceId, error.message());
+                        future.completeExceptionally(new CommitFailedException());

Review Comment:
   Oh maybe just a confusion...I notice now that on the main PR page it still shows the `log.info`, but we do have the `log.error` I introduced in [this](https://github.com/apache/kafka/pull/15202/commits/f7b76fd75de6615ac089c6c8ff24166fc5683a2d) previous commit , so maybe just github tricking us?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1463496440


##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java:
##########
@@ -414,8 +495,69 @@ public void testOffsetFetchRequestErroredRequests(final Errors error, final bool
             testNonRetriable(futures);
             assertEmptyPendingRequests(commitRequestManger);
         }
+    }
 
-        assertCoordinatorDisconnect(error);
+    @Test

Review Comment:
   Sure, done for this and other similar cases that could be parametrized (That was the approach, not even sure why I did not use them in these)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1462456746


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -820,6 +848,8 @@ private void onFailure(final long currentTimeMs,
             } else if (responseError == Errors.GROUP_AUTHORIZATION_FAILED) {

Review Comment:
   It is indeed being treated as non-retriable in the legacy coordinator (fails with a KafkaException), but I truly don't know or see any reason for it. So I made the changes here to treat it just like the NOT_COORDINATOR (try to find a new coordinator and retry), agree they should be treated similarly. 
   Just for the record, in this sense we would have a different behaviour in the new consumer, on the `committed` API. The legacy one fails with `KafkaException` if it gets a COORDINATOR_NOT_AVAILABLE in the fetch response, vs the new async consumer, that would retry. I still think that should be ok given that the error is defined as retriable, and the behaviour  follows the API contract for `committed`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1463501761


##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java:
##########
@@ -272,7 +270,65 @@ public void testSyncAutocommitRetriedAfterRetriableException(Errors error) {
 
         // We expect that request should have been retried on this sync commit.
         assertExceptionHandling(commitRequestManger, error, true);
-        assertCoordinatorDisconnect(error);
+    }
+
+    @Test
+    public void testCommitSyncThrowsCommitFailedExceptionOnFencedInstanceId() {
+        testCommitSyncFailsWithCommitFailedExceptionOnError(Errors.FENCED_INSTANCE_ID);
+    }
+
+    @Test
+    public void testCommitSyncThrowsCommitFailedExceptionOnUnknownMemberId() {
+        testCommitSyncFailsWithCommitFailedExceptionOnError(Errors.UNKNOWN_MEMBER_ID);
+    }
+
+    private void testCommitSyncFailsWithCommitFailedExceptionOnError(Errors commitError) {
+        CommitRequestManager commitRequestManger = create(false, 100);
+        when(coordinatorRequestManager.coordinator()).thenReturn(Optional.of(mockedNode));
+
+        Map<TopicPartition, OffsetAndMetadata> offsets = Collections.singletonMap(
+            new TopicPartition("topic", 1),
+            new OffsetAndMetadata(0));
+
+        // Send sync offset commit request that fails with an error that is expected to propagate
+        // a CommitFailedException
+        Long expirationTimeMs = time.milliseconds() + retryBackoffMs;
+        CompletableFuture<Void> commitResult = commitRequestManger.addOffsetCommitRequest(offsets, Optional.of(expirationTimeMs), false);
+        completeOffsetCommitRequestWithError(commitRequestManger, commitError);
+        assertFutureThrows(commitResult, CommitFailedException.class);
+    }
+
+    @Test
+    public void testCommitSyncThrowsOffsetMetadataTooLargeException() {
+        // Error with metadata provided by the user should propagate the exception, so they can handle it.
+        testCommitSyncFailsWithErrorException(Errors.OFFSET_METADATA_TOO_LARGE);
+    }
+
+    @Test
+    public void testCommitSyncThrowsInvalidCommitOffsetSizeException() {
+        // Error with data provided by the user should propagate the exception, so they can handle it.
+        testCommitSyncFailsWithErrorException(Errors.INVALID_COMMIT_OFFSET_SIZE);
+    }
+
+    @Test
+    public void testCommitSyncThrowsGroupAuthorizationException() {
+        testCommitSyncFailsWithErrorException(Errors.GROUP_AUTHORIZATION_FAILED);
+    }
+
+    private void testCommitSyncFailsWithErrorException(Errors commitError) {

Review Comment:
   Agree, missed that. All done and it did simplified a lot.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1463282309


##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java:
##########
@@ -414,8 +495,69 @@ public void testOffsetFetchRequestErroredRequests(final Errors error, final bool
             testNonRetriable(futures);
             assertEmptyPendingRequests(commitRequestManger);
         }
+    }
 
-        assertCoordinatorDisconnect(error);
+    @Test

Review Comment:
   nit: Should we use parameterized tests instead of specifying all cases like this?



##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java:
##########
@@ -272,7 +270,65 @@ public void testSyncAutocommitRetriedAfterRetriableException(Errors error) {
 
         // We expect that request should have been retried on this sync commit.
         assertExceptionHandling(commitRequestManger, error, true);
-        assertCoordinatorDisconnect(error);
+    }
+
+    @Test
+    public void testCommitSyncThrowsCommitFailedExceptionOnFencedInstanceId() {
+        testCommitSyncFailsWithCommitFailedExceptionOnError(Errors.FENCED_INSTANCE_ID);
+    }
+
+    @Test
+    public void testCommitSyncThrowsCommitFailedExceptionOnUnknownMemberId() {
+        testCommitSyncFailsWithCommitFailedExceptionOnError(Errors.UNKNOWN_MEMBER_ID);
+    }
+
+    private void testCommitSyncFailsWithCommitFailedExceptionOnError(Errors commitError) {
+        CommitRequestManager commitRequestManger = create(false, 100);
+        when(coordinatorRequestManager.coordinator()).thenReturn(Optional.of(mockedNode));
+
+        Map<TopicPartition, OffsetAndMetadata> offsets = Collections.singletonMap(
+            new TopicPartition("topic", 1),
+            new OffsetAndMetadata(0));
+
+        // Send sync offset commit request that fails with an error that is expected to propagate
+        // a CommitFailedException
+        Long expirationTimeMs = time.milliseconds() + retryBackoffMs;
+        CompletableFuture<Void> commitResult = commitRequestManger.addOffsetCommitRequest(offsets, Optional.of(expirationTimeMs), false);
+        completeOffsetCommitRequestWithError(commitRequestManger, commitError);
+        assertFutureThrows(commitResult, CommitFailedException.class);
+    }
+
+    @Test
+    public void testCommitSyncThrowsOffsetMetadataTooLargeException() {
+        // Error with metadata provided by the user should propagate the exception, so they can handle it.
+        testCommitSyncFailsWithErrorException(Errors.OFFSET_METADATA_TOO_LARGE);
+    }
+
+    @Test
+    public void testCommitSyncThrowsInvalidCommitOffsetSizeException() {
+        // Error with data provided by the user should propagate the exception, so they can handle it.
+        testCommitSyncFailsWithErrorException(Errors.INVALID_COMMIT_OFFSET_SIZE);
+    }
+
+    @Test
+    public void testCommitSyncThrowsGroupAuthorizationException() {
+        testCommitSyncFailsWithErrorException(Errors.GROUP_AUTHORIZATION_FAILED);
+    }
+
+    private void testCommitSyncFailsWithErrorException(Errors commitError) {

Review Comment:
   This method is very similar to testCommitSyncFailsWithCommitFailedExceptionOnError. Could we share part of the implementation for both cases? For instance, one idea would be to pass the Errors and the exception Exception as params.



##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java:
##########
@@ -440,28 +582,59 @@ public void testOffsetCommitRequestErroredRequestsNotRetriedForAsyncCommit(final
 
         // We expect that the request should not have been retried on this async commit.
         assertExceptionHandling(commitRequestManger, error, false);
-        assertCoordinatorDisconnect(error);
     }
 
+    @Test
+    public void testCommitAsyncThrowsKafkaExceptionForUnexpectedRetriableError() {
+        testCommitAsyncThrowsKafkaException(Errors.CORRUPT_MESSAGE);
+    }
 
     @Test
-    public void testAsyncOffsetCommitThrowsRetriableCommitExceptionForUnhandledRetriable() {
+    public void testCommitAsyncThrowsKafkaExceptionForUnexpectedNonRetriableError() {
+        testCommitAsyncThrowsKafkaException(Errors.UNKNOWN_SERVER_ERROR);
+    }
+
+    private void testCommitAsyncThrowsKafkaException(Errors error) {
         CommitRequestManager commitRequestManger = create(true, 100);
         when(coordinatorRequestManager.coordinator()).thenReturn(Optional.of(mockedNode));
 
         Map<TopicPartition, OffsetAndMetadata> offsets = Collections.singletonMap(new TopicPartition("topic", 1),
             new OffsetAndMetadata(0));
 
-        // Send commit request without expiration (async commit) that fails with retriable
-        // network exception that has no specific handling. Should fail with
-        // RetriableCommitException.
+        // Send async commit that fails with unexpected error. Should fail with KafkaException.
         CompletableFuture<Void> commitResult = commitRequestManger.addOffsetCommitRequest(offsets, Optional.empty(), false);
-        completeOffsetCommitRequestWithError(commitRequestManger, Errors.NETWORK_EXCEPTION);
+        completeOffsetCommitRequestWithError(commitRequestManger, error);
         NetworkClientDelegate.PollResult res = commitRequestManger.poll(time.milliseconds());
         assertEquals(0, res.unsentRequests.size());
         assertTrue(commitResult.isDone());
         assertTrue(commitResult.isCompletedExceptionally());
-        assertFutureThrows(commitResult, RetriableCommitFailedException.class);
+        assertFutureThrows(commitResult, KafkaException.class);
+    }
+
+    @Test
+    public void testCommitSyncThrowsKafkaExceptionForUnexpectedRetriableError() {
+        testCommitSyncThrowsKafkaException(Errors.CORRUPT_MESSAGE);
+    }
+
+    @Test
+    public void testCommitSyncThrowsKafkaExceptionForUnexpectedNonRetriableError() {
+        testCommitSyncThrowsKafkaException(Errors.UNKNOWN_SERVER_ERROR);
+    }
+
+    private void testCommitSyncThrowsKafkaException(Errors error) {

Review Comment:
   This one is also very similar to testCommitSyncFailsWithCommitFailedExceptionOnError.



##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -820,6 +848,8 @@ private void onFailure(final long currentTimeMs,
             } else if (responseError == Errors.GROUP_AUTHORIZATION_FAILED) {

Review Comment:
   That seems to be fine.



##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -536,12 +543,23 @@ public void onResponse(final ClientResponse response) {
                         continue;
                     }
 
-                    if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
+                    if (error == Errors.GROUP_AUTHORIZATION_FAILED) {
+                        future.completeExceptionally(GroupAuthorizationException.forGroupId(groupId));
+                        return;
+                    } else if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
                         error == Errors.NOT_COORDINATOR ||
                         error == Errors.REQUEST_TIMED_OUT) {
                         coordinatorRequestManager.markCoordinatorUnknown(error.message(), currentTimeMs);
                         maybeRetry(currentTimeMs, error.exception());
                         return;
+                    } else if (error == Errors.FENCED_INSTANCE_ID) {
+                        log.info("OffsetCommit failed due to group instance id {} fenced: {}", groupInstanceId, error.message());
+                        future.completeExceptionally(new CommitFailedException());

Review Comment:
   This one has not been addressed yet.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1461922947


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -820,6 +848,8 @@ private void onFailure(final long currentTimeMs,
             } else if (responseError == Errors.GROUP_AUTHORIZATION_FAILED) {

Review Comment:
   Do you know why we don't handle `COORDINATOR_NOT_AVAILABLE`? I think that we should treat it like `NOT_COORDINATOR`.



##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -536,12 +543,23 @@ public void onResponse(final ClientResponse response) {
                         continue;
                     }
 
-                    if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
+                    if (error == Errors.GROUP_AUTHORIZATION_FAILED) {
+                        future.completeExceptionally(GroupAuthorizationException.forGroupId(groupId));
+                        return;
+                    } else if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
                         error == Errors.NOT_COORDINATOR ||
                         error == Errors.REQUEST_TIMED_OUT) {
                         coordinatorRequestManager.markCoordinatorUnknown(error.message(), currentTimeMs);
                         maybeRetry(currentTimeMs, error.exception());
                         return;
+                    } else if (error == Errors.FENCED_INSTANCE_ID) {
+                        log.info("OffsetCommit failed due to group instance id {} fenced: {}", groupInstanceId, error.message());
+                        future.completeExceptionally(new CommitFailedException());

Review Comment:
   nit: While we are here, should we put an error msg?



##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -536,12 +543,23 @@ public void onResponse(final ClientResponse response) {
                         continue;
                     }
 
-                    if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
+                    if (error == Errors.GROUP_AUTHORIZATION_FAILED) {
+                        future.completeExceptionally(GroupAuthorizationException.forGroupId(groupId));
+                        return;
+                    } else if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
                         error == Errors.NOT_COORDINATOR ||
                         error == Errors.REQUEST_TIMED_OUT) {
                         coordinatorRequestManager.markCoordinatorUnknown(error.message(), currentTimeMs);
                         maybeRetry(currentTimeMs, error.exception());
                         return;
+                    } else if (error == Errors.FENCED_INSTANCE_ID) {
+                        log.info("OffsetCommit failed due to group instance id {} fenced: {}", groupInstanceId, error.message());
+                        future.completeExceptionally(new CommitFailedException());
+                        return;
+                    } else if (error == Errors.OFFSET_METADATA_TOO_LARGE ||
+                        error == Errors.INVALID_COMMIT_OFFSET_SIZE) {
+                        future.completeExceptionally(error.exception());
+                        return;
                     } else if (error == Errors.COORDINATOR_LOAD_IN_PROGRESS ||
                         error == Errors.UNKNOWN_TOPIC_OR_PARTITION) {
                         // just retry

Review Comment:
   Should we fail the future and rely and the outer retry loop now?



##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -770,8 +785,22 @@ public NetworkClientDelegate.UnsentRequest toUnsentRequest() {
                         new ArrayList<>(this.requestedPartitions),
                         throwOnFetchStableOffsetUnsupported);
             }
-            return new NetworkClientDelegate.UnsentRequest(builder, coordinatorRequestManager.coordinator())
-                    .whenComplete((r, t) -> onResponse(r.receivedTimeMs(), (OffsetFetchResponse) r.responseBody()));
+            NetworkClientDelegate.UnsentRequest request =
+                new NetworkClientDelegate.UnsentRequest(builder, coordinatorRequestManager.coordinator());
+            return request.whenComplete((response, error) -> {
+                if (error == null) {
+                    onResponse(response.receivedTimeMs(), (OffsetFetchResponse) response.responseBody());
+                } else {
+                    log.debug("OffsetFetch completed with error for partitions {}", requestedPartitions, error);
+                    long currentTimeMs = request.handler().completionTimeMs();
+                    handleCoordinatorDisconnect(error, currentTimeMs);
+                    if (error instanceof RetriableException) {
+                        maybeRetry(currentTimeMs, error);
+                    } else {
+                        future.completeExceptionally(error);
+                    }
+                }

Review Comment:
   This code looks pretty similar to the one that we have for the offset commit. Is there an opportunity to consolidate/reuse?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on PR #15202:
URL: https://github.com/apache/kafka/pull/15202#issuecomment-1905258807

   Thanks for the comments @dajac , all addressed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac merged PR #15202:
URL: https://github.com/apache/kafka/pull/15202


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1463366293


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -536,12 +543,23 @@ public void onResponse(final ClientResponse response) {
                         continue;
                     }
 
-                    if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
+                    if (error == Errors.GROUP_AUTHORIZATION_FAILED) {
+                        future.completeExceptionally(GroupAuthorizationException.forGroupId(groupId));
+                        return;
+                    } else if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
                         error == Errors.NOT_COORDINATOR ||
                         error == Errors.REQUEST_TIMED_OUT) {
                         coordinatorRequestManager.markCoordinatorUnknown(error.message(), currentTimeMs);
                         maybeRetry(currentTimeMs, error.exception());
                         return;
+                    } else if (error == Errors.FENCED_INSTANCE_ID) {
+                        log.info("OffsetCommit failed due to group instance id {} fenced: {}", groupInstanceId, error.message());
+                        future.completeExceptionally(new CommitFailedException());

Review Comment:
   uhm I had addressed it by adding `log.error` instead of `log.info`, but maybe I misunderstood your comment. Were you suggesting something different?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1463379100


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -536,12 +543,23 @@ public void onResponse(final ClientResponse response) {
                         continue;
                     }
 
-                    if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
+                    if (error == Errors.GROUP_AUTHORIZATION_FAILED) {
+                        future.completeExceptionally(GroupAuthorizationException.forGroupId(groupId));
+                        return;
+                    } else if (error == Errors.COORDINATOR_NOT_AVAILABLE ||
                         error == Errors.NOT_COORDINATOR ||
                         error == Errors.REQUEST_TIMED_OUT) {
                         coordinatorRequestManager.markCoordinatorUnknown(error.message(), currentTimeMs);
                         maybeRetry(currentTimeMs, error.exception());
                         return;
+                    } else if (error == Errors.FENCED_INSTANCE_ID) {
+                        log.info("OffsetCommit failed due to group instance id {} fenced: {}", groupInstanceId, error.message());
+                        future.completeExceptionally(new CommitFailedException());

Review Comment:
   got it. Message added to the exception too.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1462698984


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -770,8 +785,22 @@ public NetworkClientDelegate.UnsentRequest toUnsentRequest() {
                         new ArrayList<>(this.requestedPartitions),
                         throwOnFetchStableOffsetUnsupported);
             }
-            return new NetworkClientDelegate.UnsentRequest(builder, coordinatorRequestManager.coordinator())
-                    .whenComplete((r, t) -> onResponse(r.receivedTimeMs(), (OffsetFetchResponse) r.responseBody()));
+            NetworkClientDelegate.UnsentRequest request =
+                new NetworkClientDelegate.UnsentRequest(builder, coordinatorRequestManager.coordinator());
+            return request.whenComplete((response, error) -> {
+                if (error == null) {
+                    onResponse(response.receivedTimeMs(), (OffsetFetchResponse) response.responseBody());
+                } else {
+                    log.debug("OffsetFetch completed with error for partitions {}", requestedPartitions, error);
+                    long currentTimeMs = request.handler().completionTimeMs();
+                    handleCoordinatorDisconnect(error, currentTimeMs);
+                    if (error instanceof RetriableException) {
+                        maybeRetry(currentTimeMs, error);
+                    } else {
+                        future.completeExceptionally(error);
+                    }
+                }

Review Comment:
   Totally. I moved all the logic for building the request and handling the client response to a common place.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16032: Fixes for commit/fetch error handling [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #15202:
URL: https://github.com/apache/kafka/pull/15202#discussion_r1463498713


##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java:
##########
@@ -440,28 +582,59 @@ public void testOffsetCommitRequestErroredRequestsNotRetriedForAsyncCommit(final
 
         // We expect that the request should not have been retried on this async commit.
         assertExceptionHandling(commitRequestManger, error, false);
-        assertCoordinatorDisconnect(error);
     }
 
+    @Test
+    public void testCommitAsyncThrowsKafkaExceptionForUnexpectedRetriableError() {
+        testCommitAsyncThrowsKafkaException(Errors.CORRUPT_MESSAGE);
+    }
 
     @Test
-    public void testAsyncOffsetCommitThrowsRetriableCommitExceptionForUnhandledRetriable() {
+    public void testCommitAsyncThrowsKafkaExceptionForUnexpectedNonRetriableError() {
+        testCommitAsyncThrowsKafkaException(Errors.UNKNOWN_SERVER_ERROR);
+    }
+
+    private void testCommitAsyncThrowsKafkaException(Errors error) {
         CommitRequestManager commitRequestManger = create(true, 100);
         when(coordinatorRequestManager.coordinator()).thenReturn(Optional.of(mockedNode));
 
         Map<TopicPartition, OffsetAndMetadata> offsets = Collections.singletonMap(new TopicPartition("topic", 1),
             new OffsetAndMetadata(0));
 
-        // Send commit request without expiration (async commit) that fails with retriable
-        // network exception that has no specific handling. Should fail with
-        // RetriableCommitException.
+        // Send async commit that fails with unexpected error. Should fail with KafkaException.
         CompletableFuture<Void> commitResult = commitRequestManger.addOffsetCommitRequest(offsets, Optional.empty(), false);
-        completeOffsetCommitRequestWithError(commitRequestManger, Errors.NETWORK_EXCEPTION);
+        completeOffsetCommitRequestWithError(commitRequestManger, error);
         NetworkClientDelegate.PollResult res = commitRequestManger.poll(time.milliseconds());
         assertEquals(0, res.unsentRequests.size());
         assertTrue(commitResult.isDone());
         assertTrue(commitResult.isCompletedExceptionally());
-        assertFutureThrows(commitResult, RetriableCommitFailedException.class);
+        assertFutureThrows(commitResult, KafkaException.class);
+    }
+
+    @Test
+    public void testCommitSyncThrowsKafkaExceptionForUnexpectedRetriableError() {
+        testCommitSyncThrowsKafkaException(Errors.CORRUPT_MESSAGE);
+    }
+
+    @Test
+    public void testCommitSyncThrowsKafkaExceptionForUnexpectedNonRetriableError() {
+        testCommitSyncThrowsKafkaException(Errors.UNKNOWN_SERVER_ERROR);
+    }
+
+    private void testCommitSyncThrowsKafkaException(Errors error) {

Review Comment:
   Agree, it does not exist anymore after the refactoring for the tests common logic and params



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org