You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/08/02 02:37:54 UTC

[GitHub] [kafka] thomaslee opened a new pull request #9111: KAFKA-10337: await async commits in commitSync even if no offsets given

thomaslee opened a new pull request #9111:
URL: https://github.com/apache/kafka/pull/9111


   The contract for `commitSync()` guarantees that the callbacks for all
   prior async commits will be invoked before it (successfully?) returns.
   Prior to this change the contract could be violated if an empty offsets
   map were passed in to `commitSync()`.
   
   Also added a unit test in `ConsumerCoordinatorTest` exercising this particular path.
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] thomaslee edited a comment on pull request #9111: KAFKA-10337: await async commits in commitSync even if no offsets given

Posted by GitBox <gi...@apache.org>.
thomaslee edited a comment on pull request #9111:
URL: https://github.com/apache/kafka/pull/9111#issuecomment-669624137


   It's somewhat easy to do it accidentally. The scenario I was concerned about looks something like this with auto offset commits disabled:
   
   ```java
   KafkaConsumer<...> consumer = ...;
   Map<TopicPartition, OffsetAndMetadata> offsets = new HashMap<>();
   
   //
   // in a message processing loop
   //
   offsets.put(topicPartition, new OffsetAndMetadata(offset));
   consumer.commitAsync(offsets);
   offsets = new HashMap<>();
   
   //
   // before a rebalance (e.g. onPartitionsRevoked)
   //
   consumer.commitSync(offsets);
   ```
   
   Per the docs the `commitSync` call should ensure that the async commit finishes prior to the rebalance, but it may not if `offsets` is empty here.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] thomaslee commented on pull request #9111: KAFKA-10337: await async commits in commitSync even if no offsets given

Posted by GitBox <gi...@apache.org>.
thomaslee commented on pull request #9111:
URL: https://github.com/apache/kafka/pull/9111#issuecomment-669624137


   It's somewhat easy to do it accidentally. The scenario I was concerned about looks something like this with auto offset commits disabled:
   
   ```java
   KafkaConsumer<...> consumer = ...;
   Map<TopicPartition, OffsetAndMetadata> offsets = new HashMap<>();
   
   //
   // in a message processing loop
   //
   consumer.commitAsync(offsets);
   offsets = new HashMap<>();
   
   //
   // before a rebalance (e.g. onPartitionsRevoked)
   //
   consumer.commitSync(offsets);
   ```
   
   Per the docs the `commitSync` call should ensure that the async commit finishes prior to the rebalance, but it may not if `offsets` is empty here.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] thomaslee closed pull request #9111: KAFKA-10337: await async commits in commitSync even if no offsets given

Posted by GitBox <gi...@apache.org>.
thomaslee closed pull request #9111:
URL: https://github.com/apache/kafka/pull/9111


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org