You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Marvin777 <xy...@gmail.com> on 2018/10/29 02:18:40 UTC

Flink Task Allocation on Yarn

Hi all,

In the mode of on yarn, a node may contain more than one container,  is
there a scheme for assigning tasks to different nodes.

the version is 1.4.2

Thanks for your assistance.

Re: Flink Task Allocation on Yarn

Posted by vino yang <ya...@gmail.com>.

Hi Marvin,

Can you tell me why you want to do this?

Thanks, vino.

Marvin777 <xy...@gmail.com> 于2018年10月30日周二 上午10:24写道：

> Hi vino,
>
> The current generation environment is on yarn. We do not want to increase
> the operation and maintenance cost of the Standalone mode.
>
> Is there any other way to make better use of the resources of the yarn
> cluster, try to allocate tasks to containers on different nodes.
>
> Thanks, Marvin.
>
> vino yang <ya...@gmail.com> 于2018年10月29日周一 下午3:04写道：
>
>> Hi Marvin,
>>
>> YARN is a resource management and scheduling framework.
>> When you run Flink on YARN, Flink will hand over the container's
>> scheduling tasks to YARN.
>> This is also the reason why YARN is used.
>> If you want to control the start and stop of TM, then I recommend you use
>> standalone mode and set the number of slots per node to 1.
>>
>> Thanks, vino.
>>
>> Marvin777 <xy...@gmail.com> 于2018年10月29日周一 上午10:18写道：
>>
>>> Hi all,
>>>
>>> In the mode of on yarn, a node may contain more than one container,  is
>>> there a scheme for assigning tasks to different nodes.
>>>
>>> the version is 1.4.2
>>>
>>> Thanks for your assistance.
>>>
>>

Re: Flink Task Allocation on Yarn

Posted by Marvin777 <xy...@gmail.com>.

Hi vino,

The current generation environment is on yarn. We do not want to increase
the operation and maintenance cost of the Standalone mode.

Is there any other way to make better use of the resources of the yarn
cluster, try to allocate tasks to containers on different nodes.

Thanks, Marvin.

vino yang <ya...@gmail.com> 于2018年10月29日周一 下午3:04写道：

> Hi Marvin,
>
> YARN is a resource management and scheduling framework.
> When you run Flink on YARN, Flink will hand over the container's
> scheduling tasks to YARN.
> This is also the reason why YARN is used.
> If you want to control the start and stop of TM, then I recommend you use
> standalone mode and set the number of slots per node to 1.
>
> Thanks, vino.
>
> Marvin777 <xy...@gmail.com> 于2018年10月29日周一 上午10:18写道：
>
>> Hi all,
>>
>> In the mode of on yarn, a node may contain more than one container,  is
>> there a scheme for assigning tasks to different nodes.
>>
>> the version is 1.4.2
>>
>> Thanks for your assistance.
>>
>

Re: Flink Task Allocation on Yarn

Posted by vino yang <ya...@gmail.com>.

Hi Marvin,

YARN is a resource management and scheduling framework.
When you run Flink on YARN, Flink will hand over the container's scheduling
tasks to YARN.
This is also the reason why YARN is used.
If you want to control the start and stop of TM, then I recommend you use
standalone mode and set the number of slots per node to 1.

Thanks, vino.

Marvin777 <xy...@gmail.com> 于2018年10月29日周一 上午10:18写道：

> Hi all,
>
> In the mode of on yarn, a node may contain more than one container,  is
> there a scheme for assigning tasks to different nodes.
>
> the version is 1.4.2
>
> Thanks for your assistance.
>