You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/03/09 11:09:09 UTC
[GitHub] [kafka] sunshujie1990 opened a new pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
sunshujie1990 opened a new pull request #11869:
URL: https://github.com/apache/kafka/pull/11869
When kafka Connect restarts connector with includeTasks=true, DistributedHerder start all task without filter the currentAssignments. This results in duplicate tasks and duplicate data。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] C0urante commented on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
C0urante commented on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1067097745
Thanks for finding and fixing this @sunshujie1990. Changes like this one should usually come with a unit test to verify that the original bug exists, and that the proposed fix addresses it.
I've drafted a small addition to an existing test case that should accomplish this in a personal fork that also includes your commit for this PR: https://github.com/C0urante/kafka/commit/ddd64ebc6400b9044a1358cf18c23d3d914d1f90
If that looks good to you, feel free to cherry-pick onto your branch and make any adjustments that you'd like.
Given that I've reproduced the issue via unit test and verified that this fix does address it (by causing the test to pass), this LGTM once it has test coverage added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] C0urante edited a comment on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
C0urante edited a comment on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1067099906
cc @rhauch @kpatelatwork this is a pretty serious bug with the recently-added mass restart feature, worth taking a look if you have the time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] C0urante commented on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
C0urante commented on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1076799542
@showuon @mimaison would either of you have time for this? Might be worth trying to include in the 3.2.0 release if we're still considering non-regressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] sunshujie1990 commented on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
sunshujie1990 commented on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1062864985
@hachikuji Jason, please help to review it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mimaison merged pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
mimaison merged pull request #11869:
URL: https://github.com/apache/kafka/pull/11869
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] sunshujie1990 commented on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
sunshujie1990 commented on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1067660755
@C0urante Done. Please help me review it again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] C0urante edited a comment on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
C0urante edited a comment on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1077834292
No worries @kpatelatwork, good to hear from you!
@showuon no worries, thanks for taking a look. Definitely better to move cautiously than to merge something you're not confident in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] C0urante commented on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
C0urante commented on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1077834292
No worries @kpatelatwork, good to hear from you!
@showuon no worries, thanks for taking a look. Definitely better to move cautiously than to merge something you're not confident in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mimaison commented on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
mimaison commented on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1083415753
Backported to 3.2 and 3.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] C0urante edited a comment on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
C0urante edited a comment on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1067097745
Thanks for finding and fixing this @sunshujie1990! Changes like this one should usually come with a unit test to verify that the original bug exists, and that the proposed fix addresses it.
I've drafted a small addition to an existing test case that should accomplish this in a personal fork that also includes your commit for this PR: https://github.com/C0urante/kafka/commit/ddd64ebc6400b9044a1358cf18c23d3d914d1f90
If that looks good to you, feel free to cherry-pick onto your branch and make any adjustments that you'd like.
Given that the issue can be reproduced via unit test and that this fix does address it (by causing the test to pass), this LGTM once it has test coverage added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] kpatelatwork commented on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
kpatelatwork commented on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1077823237
@C0urante Thanks a lot this indeed seems like a bug and sorry for the late reply as I was busy. LGTM from my side
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] C0urante commented on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
C0urante commented on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1067099906
cc @rhauch @kpatelatwork
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] C0urante edited a comment on pull request #11869: KAFKA-13719: fix connector restart cause duplicate tasks
Posted by GitBox <gi...@apache.org>.
C0urante edited a comment on pull request #11869:
URL: https://github.com/apache/kafka/pull/11869#issuecomment-1067097745
Thanks for finding and fixing this @sunshujie1990! Changes like this one should usually come with a unit test to verify that the original bug exists, and that the proposed fix addresses it.
I've drafted a small addition to an existing test case that should accomplish this in a personal fork that also includes your commit for this PR: https://github.com/C0urante/kafka/commit/ddd64ebc6400b9044a1358cf18c23d3d914d1f90
If that looks good to you, feel free to cherry-pick onto your branch and make any adjustments that you'd like.
Given that I've reproduced the issue via unit test and verified that this fix does address it (by causing the test to pass), this LGTM once it has test coverage added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org