You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/05/28 14:26:09 UTC

[GitHub] [kafka] Schmidt96u opened a new pull request #8740: Kip318

Schmidt96u opened a new pull request #8740:
URL: https://github.com/apache/kafka/pull/8740


   # KIP-318: Make Kafka Connect Source idempotent
   
   The main idea here was to add the idempotent configuration to Source. This pull request is trying to solve this [KIP](https://cwiki.apache.org/confluence/display/KAFKA/KIP-318%3A+Make+Kafka+Connect+Source+idempotent#KIP-318:MakeKafkaConnectSourceidempotent-Status)
   
   
   *Summary of testing strategy (including rationale)
   For now i'm just opening the discussion after the changes made and i'm waiting to any proposal 
   
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] ryannedolan commented on pull request #8740: KIP-318: Make Kafka Connect Source idempotent

Posted by GitBox <gi...@apache.org>.
ryannedolan commented on pull request #8740:
URL: https://github.com/apache/kafka/pull/8740#issuecomment-635399009


   This is a step in the right direction, but it doesn't really make source connectors idempotent unless we also fix the offset storage mechanism. If a task restarts for whatever reason (e.g. rebalance) it will resume at the last stored offsets and happily re-send any subsequent records.
   
   This _might_ be good to merge in order to increase throughput, but I'd like to see a benchmark to prove that's the case. If it doesn't significantly improve performance than I'd suggest it doesn't make sense to merge this, cuz it will tend to mislead people.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org