You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/02/02 23:02:43 UTC
[GitHub] [kafka] rhauch opened a new pull request #10027: KAFKA-12270: Handle race condition when Connect tasks attempt to create topic
rhauch opened a new pull request #10027:
URL: https://github.com/apache/kafka/pull/10027
When a source connector is configured to create missing topics has multiple tasks that generate records for the same topic, it is possible that multiple tasks may simultaneously describe the topic, find it does not exist, and attempt to create the task. One of those create topic requests will succeed, and the other concurrent tasks will receive the response from the topic admin as having not created the task and will fail unnecessarily.
This change corrects the logic after the create topic method returns false (signifying that it did not create the task) to check again whether the topic exists (i.e., was created since the worker task last described the topic) and to continue if it does exist and fail only if the topic did not exist after the second check. This is not as efficient as would be ideal, but this ensures that tasks do not fail in this race condition and that tasks will continue to fail if the topic could not be created and did not exist after the create topic request did not create the topic.
Modified one unit test that mocks the behavior, and added a new test with the new behavior.
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] rhauch commented on pull request #10027: KAFKA-12270: Handle race condition when Connect tasks attempt to create topic
Posted by GitBox <gi...@apache.org>.
rhauch commented on pull request #10027:
URL: https://github.com/apache/kafka/pull/10027#issuecomment-772700804
Closing in favor of #10032.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] rhauch closed pull request #10027: KAFKA-12270: Handle race condition when Connect tasks attempt to create topic
Posted by GitBox <gi...@apache.org>.
rhauch closed pull request #10027:
URL: https://github.com/apache/kafka/pull/10027
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org