You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Radu Tudoran <ra...@huawei.com> on 2016/03/07 19:36:02 UTC
realtion between operator and task
Hi,
Can someone explain how and where a stream operator is mapped to a stream task.
I am particularly interested in the way the stream outputs are created and attached to the operators. I saw that this happen in OperatorChain functions but I do not have the picture of the lifecycle of an stream operator that you would create to its mapping to the task and assignment of the output binding.
Re: realtion between operator and task
Posted by Till Rohrmann <tr...@apache.org>.
Hi Radu,
the mapping which StreamOperator is executed by which StreamTask happens
first in the StreamGraph.addOperator method. However, there is a second
step in the StreamingJobGraphGenerator.createChain where chainable
operators are chained and then executed by a single StreamTask. The
construction of the actual operator chain happens in the class OperatorChain
.
Cheers,
Till
On Tue, Mar 8, 2016 at 10:51 AM, Radu Tudoran <ra...@huawei.com>
wrote:
> Hi,
>
>
>
> Thanks for the answer. Can you point me to the code where the operators
> are being assign to tasks.
>
> Thanks
>
>
>
> *From:* ewenstephan@gmail.com [mailto:ewenstephan@gmail.com] *On Behalf
> Of *Stephan Ewen
> *Sent:* Monday, March 07, 2016 8:29 PM
> *To:* user@flink.apache.org
> *Subject:* Re: realtion between operator and task
>
>
>
> Hi!
>
>
>
> A "task" is something that is deployed as one unit to the TaskManager and
> runs in one thread.
>
>
>
> A task can have multiple "operators" chained together, usually one per
> user function, for example "Map -> Filter -> FlatMap -> AssignTimestamps ->
> ..."
>
>
>
> Stephan
>
>
>
>
>
> On Mon, Mar 7, 2016 at 7:36 PM, Radu Tudoran <ra...@huawei.com>
> wrote:
>
> Hi,
>
>
>
> Can someone explain how and where a stream operator is mapped to a stream
> task.
>
> I am particularly interested in the way the stream outputs are created and
> attached to the operators. I saw that this happen in OperatorChain
> functions but I do not have the picture of the lifecycle of an stream
> operator that you would create to its mapping to the task and assignment of
> the output binding.
>
>
>
RE: realtion between operator and task
Posted by Radu Tudoran <ra...@huawei.com>.
Hi,
Thanks for the answer. Can you point me to the code where the operators are being assign to tasks.
Thanks
From: ewenstephan@gmail.com [mailto:ewenstephan@gmail.com] On Behalf Of Stephan Ewen
Sent: Monday, March 07, 2016 8:29 PM
To: user@flink.apache.org
Subject: Re: realtion between operator and task
Hi!
A "task" is something that is deployed as one unit to the TaskManager and runs in one thread.
A task can have multiple "operators" chained together, usually one per user function, for example "Map -> Filter -> FlatMap -> AssignTimestamps -> ..."
Stephan
On Mon, Mar 7, 2016 at 7:36 PM, Radu Tudoran <ra...@huawei.com>> wrote:
Hi,
Can someone explain how and where a stream operator is mapped to a stream task.
I am particularly interested in the way the stream outputs are created and attached to the operators. I saw that this happen in OperatorChain functions but I do not have the picture of the lifecycle of an stream operator that you would create to its mapping to the task and assignment of the output binding.
Re: realtion between operator and task
Posted by Stephan Ewen <se...@apache.org>.
Hi!
A "task" is something that is deployed as one unit to the TaskManager and
runs in one thread.
A task can have multiple "operators" chained together, usually one per user
function, for example "Map -> Filter -> FlatMap -> AssignTimestamps -> ..."
Stephan
On Mon, Mar 7, 2016 at 7:36 PM, Radu Tudoran <ra...@huawei.com>
wrote:
> Hi,
>
>
>
> Can someone explain how and where a stream operator is mapped to a stream
> task.
>
> I am particularly interested in the way the stream outputs are created and
> attached to the operators. I saw that this happen in OperatorChain
> functions but I do not have the picture of the lifecycle of an stream
> operator that you would create to its mapping to the task and assignment of
> the output binding.
>