You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Guozhang Wang (Jira)" <ji...@apache.org> on 2020/01/30 02:04:00 UTC

[jira] [Created] (KAFKA-9481) Improve TaskMigratedException handling on Stream thread

Guozhang Wang created KAFKA-9481:
------------------------------------

             Summary: Improve TaskMigratedException handling on Stream thread
                 Key: KAFKA-9481
                 URL: https://issues.apache.org/jira/browse/KAFKA-9481
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Guozhang Wang
            Assignee: Guozhang Wang


Today we handle TaskMigratedException as one-task at a time, when 1) producer got fenced, 2) consumer got fenced, 3) adding records to closed tasks.

When 1) and 2) happens, all tasks hosted by that thread should have migrated; and for 3) it only happens when we are closing a task but not clearing its corresponding record buffer.

So a better exception handling is first better fixing 3) to also clear the record buffer when closing a task (clean or dirty), and then for 1/2) we can always treat it as all-tasks-are-migrated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)