You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/11/13 20:24:11 UTC
[jira] [Commented] (KAFKA-2811) Add standby tasks

    [ https://issues.apache.org/jira/browse/KAFKA-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004551#comment-15004551 ] 

ASF GitHub Bot commented on KAFKA-2811:
---------------------------------------

GitHub user ymatsuda opened a pull request:

    https://github.com/apache/kafka/pull/526

    KAFKA-2811: add standby tasks

    @guozhangwang 
    * added a new config param "num.standby.replicas" (the default value is 0).
    * added a new abstract class AbstractTask
    * added StandbyTask as a subclass of AbstractTask
    * modified StreamTask to a subclass of AbstractTask
    * StreamThread
      * standby tasks are created by calling StreamThread.addStandbyTask() from onPartitionsAssigned()
      * standby tasks are destroyed by calling StreamThread.removeStandbyTasks() from onPartitionRevoked()
      * In addStandbyTasks(), change log partitions are assigned to restoreConsumer.
      * In removeStandByTasks(), change log partitions are removed from restoreConsumer.
      * StreamThread polls change log records using restoreConsumer in the runLoop with timeout=0.
      * If records are returned, StreamThread calls StandbyTask.update and pass records to each standby tasks.
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ymatsuda/kafka standby_task

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/526.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #526
    
----
commit 4f015e107ede8b03cea9aeddd40c7f04bc930acc
Author: Yasuhiro Matsuda <ya...@confluent.io>
Date:   2015-11-13T18:51:20Z

    standby tasks

commit ea8814e3d668da63847650718d107b9a1a47ceb9
Author: Yasuhiro Matsuda <ya...@confluent.io>
Date:   2015-11-13T19:16:36Z

    cleanup

----


> Add standby tasks
> -----------------
>
>                 Key: KAFKA-2811
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2811
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: kafka streams
>            Reporter: Yasuhiro Matsuda
>            Assignee: Yasuhiro Matsuda
>
> Restoring local state from state change-log topics can be expensive. To alleviate this, we want to have an option to keep replications of local states that are kept up to date. The task assignment logic should be aware of existence of such replicas.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)