You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Vincent Wang <fv...@gmail.com> on 2016/07/14 08:37:37 UTC

Questions about slot sharing

Hi all,

  I'm totally new to Flink and I got a question about Flink's slot sharing
feature.
  Suppose I have a pipeline like *Mapper-> Reducer, *and of cause the
operators are* not *chained to a single task.
  When the job is actually scheduled to the Flink cluster, there is a
Mapper task and a Reducer task scheduled to the same TaskManager's slot,
here slot sharing works.

  The question is:  the slot sharing is an abstract concept, sharing slot
doesn't means sharing thread, is that right? The Mapper task and Reducer
task are still launched in independent threads.

Thanks in advance.

Thanks,
Huafeng

Re: Questions about slot sharing

Posted by Vincent Wang <fv...@gmail.com>.
Got it, thank you, Robert.

2016-07-14 20:55 GMT+08:00 Robert Metzger <rm...@apache.org>:

> Hi Huafeng,
>
> yes, the mapper and reducer are running in different threads on the
> TaskManager. Slot sharing is an abstract concept of the scheduler.
>
> Flink also supports thread-sharing, for example when you have a series of
> mappers (or filters) running with the same parallelism. We call this
> feature "operator chaining" [1]
>
> Regards,
> Robert
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/concepts/concepts.html#tasks--operator-chains
>
>
>
> On Thu, Jul 14, 2016 at 10:37 AM, Vincent Wang <fv...@gmail.com>
> wrote:
>
>> Hi all,
>>
>>   I'm totally new to Flink and I got a question about Flink's slot
>> sharing feature.
>>   Suppose I have a pipeline like *Mapper-> Reducer, *and of cause the
>> operators are* not *chained to a single task.
>>   When the job is actually scheduled to the Flink cluster, there is a
>> Mapper task and a Reducer task scheduled to the same TaskManager's slot,
>> here slot sharing works.
>>
>>   The question is:  the slot sharing is an abstract concept, sharing slot
>> doesn't means sharing thread, is that right? The Mapper task and Reducer
>> task are still launched in independent threads.
>>
>> Thanks in advance.
>>
>> Thanks,
>> Huafeng
>>
>>
>
>

Re: Questions about slot sharing

Posted by Robert Metzger <rm...@apache.org>.
Hi Huafeng,

yes, the mapper and reducer are running in different threads on the
TaskManager. Slot sharing is an abstract concept of the scheduler.

Flink also supports thread-sharing, for example when you have a series of
mappers (or filters) running with the same parallelism. We call this
feature "operator chaining" [1]

Regards,
Robert

[1]
https://ci.apache.org/projects/flink/flink-docs-master/concepts/concepts.html#tasks--operator-chains



On Thu, Jul 14, 2016 at 10:37 AM, Vincent Wang <fv...@gmail.com> wrote:

> Hi all,
>
>   I'm totally new to Flink and I got a question about Flink's slot sharing
> feature.
>   Suppose I have a pipeline like *Mapper-> Reducer, *and of cause the
> operators are* not *chained to a single task.
>   When the job is actually scheduled to the Flink cluster, there is a
> Mapper task and a Reducer task scheduled to the same TaskManager's slot,
> here slot sharing works.
>
>   The question is:  the slot sharing is an abstract concept, sharing slot
> doesn't means sharing thread, is that right? The Mapper task and Reducer
> task are still launched in independent threads.
>
> Thanks in advance.
>
> Thanks,
> Huafeng
>
>