You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "chickenchickenlove (via GitHub)" <gi...@apache.org> on 2026/10/06 10:02:59 UTC

[PR] KAFKA-21229: Preserve request offsets in async commit failure callbacks [kafka]

chickenchickenlove opened a new pull request, #23725:
URL: https://github.com/apache/kafka/pull/23725

   Fixes KAFKA-21229.
   
   When an asynchronous commit fails, AsyncKafkaConsumer currently passes null offsets to the user-provided OffsetCommitCallback. This affects both commitAsync(offsets, callback) and commitAsync(callback), preventing applications from identifying the failed request's offsets and potentially causing NullPointerException in error-handling code.
   
   The callback currently uses the commit future's result, which is null when the future completes exceptionally. This change retains the actual request offsets in CommitEvent and uses them for failure callbacks and error logs. Successful commits continue to use the future's result.
   
   For commits without explicit offsets, the retained map contains the consumer positions captured when preparing the request. Subsequent position changes therefore do not affect the offsets reported on failure. This aligns failure callbacks with the OffsetCommitCallback documentation and Classic Consumer behavior.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-21229: Preserve request offsets in async commit failure callbacks [kafka]

Posted by "liuliuOD (via GitHub)" <gi...@apache.org>.
liuliuOD commented on PR #23725:
URL: https://github.com/apache/kafka/pull/23725#issuecomment-6465446873

   @chickenchickenlove Could you please fix the failing checks first? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-21229: Preserve request offsets in async commit failure callbacks [kafka]

Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #23725:
URL: https://github.com/apache/kafka/pull/23725#discussion_r4297099589


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/CommitEvent.java:
##########
@@ -31,6 +32,11 @@ public abstract class CommitEvent extends CompletableApplicationEvent<Map<TopicP
      */
     private final Optional<Map<TopicPartition, OffsetAndMetadata>> offsets;
 
+    /**
+     * Offsets that is calculated.
+     */
+    private volatile Optional<Map<TopicPartition, OffsetAndMetadata>> calculatedOffsets;

Review Comment:
   agree with the question from @liuliuOD , if we can reuse the same `offsets` seems cleaner, in the end a commit event always has a single set of offsets to commit (it's just that we may not have it handy when we create the event and need to set it later on). Would just adding the missing setter be enough?



##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/CommitEvent.java:
##########
@@ -73,6 +80,14 @@ public void markOffsetsReady() {
         offsetsReady.complete(null);
     }
 
+    public void updateCalculatedOffsets(final Map<TopicPartition, OffsetAndMetadata> calculatedOffsets) {
+        this.calculatedOffsets = Optional.of(calculatedOffsets);

Review Comment:
   should we check that this is only called when `offsets` is empty? (to avoid accidentally "overwrites"..the commit event should always have a single set of non-empty offsets)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org