You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Aljoscha Krettek <al...@apache.org> on 2016/12/01 11:59:21 UTC
Re: How Flink decide which slots to use?
+Till since he is working a lot in these parts.
Could you maybe shed some light on this?
On Mon, 28 Nov 2016 at 11:34 Renkai Ge <ga...@gmail.com> wrote:
> I have 8 machine and I started a taskmanager with 24 slots on every
> machine.At first, I submited two apps, each of them used 8 slots, the all
> consumed one slot on each machine in the cluster.
> [image: 屏幕快照 2016-11-28 18.18.51.png]
>
> Then I submited another application, but it behaves different, it used 8
> slots,too. But 4 of them are run on the same machine.
> [image: 屏幕快照 2016-11-28 18.21.52.png]
>
> [image: 屏幕快照 2016-11-28 18.22.31.png]
>
> Since this application used a lot of memory, such dispatch may cause a
> memory overflow. I want to know if I can custom the slot dispatcher.
>
Re: How Flink decide which slots to use?
Posted by Stephan Ewen <se...@apache.org>.
There is no deterministic rule what slots are picked by the scheduler.
For tasks that read data from other tasks, Flink will try to respect
locality. Otherwise, the choice is "first that is free".
Best,
Stephan
On Mon, Dec 5, 2016 at 12:07 PM, Robert Metzger <rm...@apache.org> wrote:
> While we are waiting for Till's response, I can recommend this
> documentation page: https://ci.apache.org/projects/flink/flink-docs-
> release-1.2/concepts/index.html#workers-slots-resources
>
> On Thu, Dec 1, 2016 at 12:59 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
>> +Till since he is working a lot in these parts.
>>
>> Could you maybe shed some light on this?
>>
>>
>> On Mon, 28 Nov 2016 at 11:34 Renkai Ge <ga...@gmail.com> wrote:
>>
>>> I have 8 machine and I started a taskmanager with 24 slots on every
>>> machine.At first, I submited two apps, each of them used 8 slots, the all
>>> consumed one slot on each machine in the cluster.
>>> [image: 屏幕快照 2016-11-28 18.18.51.png]
>>>
>>> Then I submited another application, but it behaves different, it used 8
>>> slots,too. But 4 of them are run on the same machine.
>>> [image: 屏幕快照 2016-11-28 18.21.52.png]
>>>
>>> [image: 屏幕快照 2016-11-28 18.22.31.png]
>>>
>>> Since this application used a lot of memory, such dispatch may cause a
>>> memory overflow. I want to know if I can custom the slot dispatcher.
>>>
>>
>
Re: How Flink decide which slots to use?
Posted by Robert Metzger <rm...@apache.org>.
While we are waiting for Till's response, I can recommend this
documentation page:
https://ci.apache.org/projects/flink/flink-docs-release-1.2/concepts/index.html#workers-slots-resources
On Thu, Dec 1, 2016 at 12:59 PM, Aljoscha Krettek <al...@apache.org>
wrote:
> +Till since he is working a lot in these parts.
>
> Could you maybe shed some light on this?
>
>
> On Mon, 28 Nov 2016 at 11:34 Renkai Ge <ga...@gmail.com> wrote:
>
>> I have 8 machine and I started a taskmanager with 24 slots on every
>> machine.At first, I submited two apps, each of them used 8 slots, the all
>> consumed one slot on each machine in the cluster.
>> [image: 屏幕快照 2016-11-28 18.18.51.png]
>>
>> Then I submited another application, but it behaves different, it used 8
>> slots,too. But 4 of them are run on the same machine.
>> [image: 屏幕快照 2016-11-28 18.21.52.png]
>>
>> [image: 屏幕快照 2016-11-28 18.22.31.png]
>>
>> Since this application used a lot of memory, such dispatch may cause a
>> memory overflow. I want to know if I can custom the slot dispatcher.
>>
>