[jira] [Updated] (SAMZA-778) KafkaCheckpointMigration should not start/stop the same instance of CoordinatorStreamSystemConsumer multiple times

["Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yi Pan (Data Infrastructure) updated SAMZA-778:
-----------------------------------------------
    Attachment: SAMZA-778-0.patch

Verified by running local migration of 0.9.1 test jobs to 0.10.0 test jobs.

> KafkaCheckpointMigration should not start/stop the same instance of CoordinatorStreamSystemConsumer multiple times
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: SAMZA-778
>                 URL: https://issues.apache.org/jira/browse/SAMZA-778
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Yi Pan (Data Infrastructure)
>            Assignee: Yi Pan (Data Infrastructure)
>         Attachments: SAMZA-778-0.patch
>
>
> The current implementation of KafkaSystemConsumer only allows start()/stop() once in the life cycle of the same SystemConsumer instance. Unfortunately, KafkaCheckpointMigration code start()/stop() the same instance of KafkaSystemConsumer multiple times, which results in an infinite loop in CoordinatorStreamSystemConsumer.start() in JobRunner.
> The quick fix is to avoid the sequence of SystemConsumer.start()->SystemConsumer.stop()->SystemConsumer.start() in the KafkaCheckpointMigration code.
> In long-term, we may also want to evaluate the use cases where we support start/stop multiple times in the life cycle of a single instance of SystemConsumer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)