You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Suraj Puvvada <su...@traceable.ai> on 2020/07/30 18:52:34 UTC

Does Flink automatically apply the rebalance operator ?

Hello

We are testing a simple use case where we read from kafka -> process and
write to kafka.

I have set parallelism of the job to 3 and parallelism for the process
function to 6. When I looked at the job graph in the Flink UI noticed that
between the source and process function a rebalance step was introduced. Is
this expected ?

I was expecting some instances of process tasks to be idle i.e. 3 tasks to
be idle.

Thanks
Suraj

Re: Does Flink automatically apply the rebalance operator ?

Posted by Robert Metzger <rm...@apache.org>.

Hi Suraj,

yes, this is the expected and desired behavior: a rebalance is introduced
so that all 6 process functions are performing work.

On Thu, Jul 30, 2020 at 8:53 PM Suraj Puvvada <su...@traceable.ai> wrote:

> Hello
>
> We are testing a simple use case where we read from kafka -> process and
> write to kafka.
>
> I have set parallelism of the job to 3 and parallelism for the process
> function to 6. When I looked at the job graph in the Flink UI noticed that
> between the source and process function a rebalance step was introduced. Is
> this expected ?
>
> I was expecting some instances of process tasks to be idle i.e. 3 tasks to
> be idle.
>
> Thanks
> Suraj
>