You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Prateek Maheshwari (JIRA)" <ji...@apache.org> on 2018/04/10 23:42:00 UTC

[jira] [Created] (SAMZA-1643) StreamPartitionCountMonitor should only restart/shut down the job if partition count increases

Prateek Maheshwari created SAMZA-1643:
-----------------------------------------

             Summary: StreamPartitionCountMonitor should only restart/shut down the job if partition count increases
                 Key: SAMZA-1643
                 URL: https://issues.apache.org/jira/browse/SAMZA-1643
             Project: Samza
          Issue Type: Bug
            Reporter: Prateek Maheshwari
            Assignee: Prateek Maheshwari


Make StreamPartitionCountMonitor less aggressive and only restart (stateless) or shut down (stateful) the job if partition count increases.

In some versions of KafkaConsumer, partitionFor(topic) can return a count lower than the configured number of partitions if the broker it asks has recently restarted.

The only downside of not restarting the job on partition count decreases is that some containers may be idle.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)