You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Abhishek Agarwal <ab...@gmail.com> on 2016/08/29 11:41:13 UTC
Resource isolation in flink among multiple jobs
In a standalone flink cluster, the TaskManager has multiple slots and can
run different applications in different slot but still within same JVM.
Given that, two applications are running under same process, how is the cpu
and memory isolation achieved?
Are there any recommendations to achieve isolation between applications?
--
Regards,
Abhishek Agarwal
Re: Resource isolation in flink among multiple jobs
Posted by Robert Metzger <rm...@apache.org>.
Hi,
this is the JIRA for mesos support:
https://issues.apache.org/jira/browse/FLINK-1984
The first part of it has been merged today:
https://github.com/apache/flink/pull/2315
So its likely going to be fully functional for the 1.2 release.
You can run multiple TaskManager instances per machine, and give each only
one (or very few) slots.
On Mon, Aug 29, 2016 at 5:40 PM, Abhishek Agarwal <ab...@gmail.com>
wrote:
> Hi Robert,
> Thanks for your reply. Few follow up questions -
> Is there any timeline for mesos support?
>
> In standalone installations, how about having one Task slot per
> TaskManager and multiple TaskManager instances per machine?
>
> On Mon, Aug 29, 2016 at 7:13 PM, Robert Metzger <rm...@apache.org>
> wrote:
>
>> Hi,
>>
>> for isolation, we recommend using YARN, or soon Mesos.
>>
>> For standalone installations, you'll need to manually set up multiple
>> independent Flink clusters within one physical cluster if you want them to
>> be isolated.
>>
>> Regards,
>> Robert
>>
>>
>> On Mon, Aug 29, 2016 at 1:41 PM, Abhishek Agarwal <ab...@gmail.com>
>> wrote:
>>
>>> In a standalone flink cluster, the TaskManager has multiple slots and
>>> can run different applications in different slot but still within same JVM.
>>> Given that, two applications are running under same process, how is the cpu
>>> and memory isolation achieved?
>>>
>>> Are there any recommendations to achieve isolation between applications?
>>>
>>> --
>>> Regards,
>>> Abhishek Agarwal
>>>
>>>
>>
>
>
> --
> Regards,
> Abhishek Agarwal
>
>
Re: Resource isolation in flink among multiple jobs
Posted by Abhishek Agarwal <ab...@gmail.com>.
Hi Robert,
Thanks for your reply. Few follow up questions -
Is there any timeline for mesos support?
In standalone installations, how about having one Task slot per TaskManager
and multiple TaskManager instances per machine?
On Mon, Aug 29, 2016 at 7:13 PM, Robert Metzger <rm...@apache.org> wrote:
> Hi,
>
> for isolation, we recommend using YARN, or soon Mesos.
>
> For standalone installations, you'll need to manually set up multiple
> independent Flink clusters within one physical cluster if you want them to
> be isolated.
>
> Regards,
> Robert
>
>
> On Mon, Aug 29, 2016 at 1:41 PM, Abhishek Agarwal <ab...@gmail.com>
> wrote:
>
>> In a standalone flink cluster, the TaskManager has multiple slots and can
>> run different applications in different slot but still within same JVM.
>> Given that, two applications are running under same process, how is the cpu
>> and memory isolation achieved?
>>
>> Are there any recommendations to achieve isolation between applications?
>>
>> --
>> Regards,
>> Abhishek Agarwal
>>
>>
>
--
Regards,
Abhishek Agarwal
Re: Resource isolation in flink among multiple jobs
Posted by Robert Metzger <rm...@apache.org>.
Hi,
for isolation, we recommend using YARN, or soon Mesos.
For standalone installations, you'll need to manually set up multiple
independent Flink clusters within one physical cluster if you want them to
be isolated.
Regards,
Robert
On Mon, Aug 29, 2016 at 1:41 PM, Abhishek Agarwal <ab...@gmail.com>
wrote:
> In a standalone flink cluster, the TaskManager has multiple slots and can
> run different applications in different slot but still within same JVM.
> Given that, two applications are running under same process, how is the cpu
> and memory isolation achieved?
>
> Are there any recommendations to achieve isolation between applications?
>
> --
> Regards,
> Abhishek Agarwal
>
>