You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/08/16 15:46:34 UTC
[GitHub] [kafka] cadonna opened a new pull request, #12521: KAFKA-10199: Remove tasks from state updater on partition lost
cadonna opened a new pull request, #12521:
URL: https://github.com/apache/kafka/pull/12521
Removes tasks from the state updater when the input partitions
of the tasks are lost during a rebalance.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang merged pull request #12521: KAFKA-10199: Remove tasks from state updater on partition lost
Posted by GitBox <gi...@apache.org>.
guozhangwang merged PR #12521:
URL: https://github.com/apache/kafka/pull/12521
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on a diff in pull request #12521: KAFKA-10199: Remove tasks from state updater on partition lost
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on code in PR #12521:
URL: https://github.com/apache/kafka/pull/12521#discussion_r946985811
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/Tasks.java:
##########
@@ -111,12 +113,19 @@ void addPendingTaskToUpdateInputPartitions(final TaskId taskId, final Set<TopicP
pendingTasksToUpdateInputPartitions.put(taskId, inputPartitions);
}
- boolean removePendingTaskToClose(final TaskId taskId) {
- return pendingTasksToClose.remove(taskId);
+ boolean removePendingTaskToCloseDirty(final TaskId taskId) {
+ return pendingTasksToCloseDirty.remove(taskId);
+ }
+ void addPendingTaskToCloseDirty(final TaskId taskId) {
Review Comment:
nit: add new line.
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java:
##########
@@ -875,9 +886,16 @@ void handleLostAll() {
closeTaskDirty(task);
}
}
+ }
- if (processingMode == EXACTLY_ONCE_V2) {
- activeTaskCreator.reInitializeThreadProducer();
+ private void removeLostTasksFromStateUpdater() {
Review Comment:
nit: removeLostActiveTasksFromStateUpdater()
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/Tasks.java:
##########
@@ -55,7 +55,9 @@ class Tasks {
private final Map<TaskId, Set<TopicPartition>> pendingTasksToRecycle = new HashMap<>();
private final Map<TaskId, Set<TopicPartition>> pendingTasksToUpdateInputPartitions = new HashMap<>();
private final Set<Task> pendingTasksToInit = new HashSet<>();
- private final Set<TaskId> pendingTasksToClose = new HashSet<>();
+ private final Set<TaskId> pendingTasksToCloseClean = new HashSet<>();
+
Review Comment:
nit: delete new line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on pull request #12521: KAFKA-10199: Remove tasks from state updater on partition lost
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on PR #12521:
URL: https://github.com/apache/kafka/pull/12521#issuecomment-1218343047
Build tests succeeded actually, but timed out reporting. Merging to trunk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org