You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@samza.apache.org by Yong Yuan <yy...@gmail.com> on 2013/10/13 19:42:23 UTC

Can Samza scale up stream tasks at runtime?

Hi, folks,

Does Samza support scaling up processing at runtime? That is, if I add new
Kafka brokers or increase the number of partitions for a topic while my
Samza job is running, will Samza allocate more stream tasks accordingly? Or
do I have to relaunch the tasks?

Thanks,

Re: Can Samza scale up stream tasks at runtime?

Posted by Chris Riccomini <cr...@linkedin.com>.

Hey Yong,

Samza does not support dynamic scaling. You'd have to bump up your
yarn.container.count (assuming you're using YARN) and re-start the job.

Cheers,
Chris

On 10/13/13 10:42 AM, "Yong Yuan" <yy...@gmail.com> wrote:

>Hi, folks,
>
>Does Samza support scaling up processing at runtime? That is, if I add new
>Kafka brokers or increase the number of partitions for a topic while my
>Samza job is running, will Samza allocate more stream tasks accordingly?
>Or
>do I have to relaunch the tasks?
>
>Thanks,