You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Vijayendra Yadav <co...@gmail.com> on 2020/08/28 19:39:19 UTC

Flink Task Slots

Hi Team,

https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/config.html#configuring-taskmanager-processing-slots


 In my Flinkkafkaconsumer streaming process. I have kafka with 3
partitions.

I am getting parallelism = 3  (-p 3). Here 3 taskmangers are launched but
each taskmanager gets only *one slot. *

When I try to provide property in flink yaml. as :
taskmanager.numberOfTaskSlots: 6

Flink on a yarn cluster launches only when taskmanager with specified
slots.

While I was expecting it to still launch 3 taskmanagers and just allocate
more slots to each of them.

Could you please suggest how to allocate more slots to each task manager
without reducing the number of taskmangers ?

Regards,
Vijay

Re: Flink Task Slots

Posted by Vijayendra Yadav <co...@gmail.com>.

Please ignore, last Email, Its working now by adding more parallelism.

On Fri, Aug 28, 2020 at 12:39 PM Vijayendra Yadav <co...@gmail.com>
wrote:

> Hi Team,
>
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/config.html#configuring-taskmanager-processing-slots
>
>
>  In my Flinkkafkaconsumer streaming process. I have kafka with 3
> partitions.
>
> I am getting parallelism = 3  (-p 3). Here 3 taskmangers are launched but
> each taskmanager gets only *one slot. *
>
> When I try to provide property in flink yaml. as :
> taskmanager.numberOfTaskSlots: 6
>
> Flink on a yarn cluster launches only when taskmanager with specified
> slots.
>
> While I was expecting it to still launch 3 taskmanagers and just allocate
> more slots to each of them.
>
> Could you please suggest how to allocate more slots to each task manager
> without reducing the number of taskmangers ?
>
> Regards,
> Vijay
>