Load Distribution in specific Slot of Taskmanager in flink(version 1.15.2)

["harshit.varshney@iktara.ai" <Ha...@iktara.ai>]

Dear Team,

 

I need some advice on setting up load distribution of flink tasks. 

 

I have a flink task that processes transactions for users. Since load is
more than what can be handled on single machine, I want same task to be
executed on 3 machines. 

 

I am trying to use parallelism feature of flink for same. I am able to get
flink Job manager to start same task on 3 machines. I want task on each
machine to handle 1/3 of total user transaction, kindly suggest what
mechanism to use so each task handles only data for their own 1/3 of users.

 

 

Thanks,

Harshit Varshney

Re: Load Distribution in specific Slot of Taskmanager in flink(version 1.15.2)

[yidan zhao <hi...@gmail.com>]

If no data skew exists, you can set the job's parallelism any times of
the count of taskmanagers, and set `cluster.evenly-spread-out-slots`
to true in flink-conf.yaml of your flink cluster.

harshit.varshney@iktara.ai <Ha...@iktara.ai> 于2022年11月7日周一 20:41写道：
>
> Dear Team,
>
>
>
> I need some advice on setting up load distribution of flink tasks.
>
>
>
> I have a flink task that processes transactions for users. Since load is more than what can be handled on single machine, I want same task to be executed on 3 machines.
>
>
>
> I am trying to use parallelism feature of flink for same. I am able to get flink Job manager to start same task on 3 machines. I want task on each machine to handle 1/3 of total user transaction, kindly suggest what mechanism to use so each task handles only data for their own 1/3 of users.
>
>
>
>
>
> Thanks,
>
> Harshit Varshney