Question about slot and yarn vcores

[sohimankotia <so...@gmail.com>]

Let's assume I have yarn cluster with 3 nodes, 3 vcores each nodes. So total
available cores = 9

Now if I spin a flink job with taskmanager = 3 and no. of slots per task
manager = 2 ,what will happen :


1. 3 Jvms will be initiated (for each task manager)
2. Each JVM will run 2 threads for tasks .
3. Will each thread will use 1 vcore from yarn or both threads will share
same vcore ??



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Question about slot and yarn vcores

[Dawid Wysakowicz <dw...@apache.org>]

Hi,

It is not as easy. To understand it well I would recommend going through
the docs[1]. In short slots do not equal thread/core it is just an
abstraction over a share of resources.

For you setup: It is true there will be 3 jvms each will be assigned 2
vcores ( by default it is equal to slot per taskamanger if you don't
overwrite it with[2]), but each of the jvm will have multiple threads.

Best,

Dawid

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.6/concepts/runtime.html#task-slots-and-resources

[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.6/ops/config.html#yarn-containers-vcores

On 02/11/2018 10:48, sohimankotia wrote:
> Let's assume I have yarn cluster with 3 nodes, 3 vcores each nodes. So total
> available cores = 9
>
> Now if I spin a flink job with taskmanager = 3 and no. of slots per task
> manager = 2 ,what will happen :
>
>
> 1. 3 Jvms will be initiated (for each task manager)
> 2. Each JVM will run 2 threads for tasks .
> 3. Will each thread will use 1 vcore from yarn or both threads will share
> same vcore ??
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/