Is it a good practice to create lots of jobs in a flink cluster ?

[Christophe Julien <ch...@gmail.com>]

Hi,

I wonder if create and delete lots of jobs in a cluster can be a problem ?

From my point of view the impact will be limited, because the task Manager
won't be impacted during the process, it is only a task slot available or
not.
But is it really "free" or a good practice to do that ? Especially in
production, is it a "stable" practice ?


Thanks,
Christophe

Re: Is it a good practice to create lots of jobs in a flink cluster ?

[Maximilian Michels <mx...@apache.org>]

It is safe to run multiple jobs concurrently in a Flink cluster. If we
care about resource isolation, you are probably better off with Flink
on Yarn.

That said, we're moving away from this model in FLIP-6 towards a
cluster-per-job model:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077

-Max

On Thu, Oct 6, 2016 at 10:42 AM, Christophe Julien
<ch...@gmail.com> wrote:
> Hi,
>
> I wonder if create and delete lots of jobs in a cluster can be a problem ?
>
> From my point of view the impact will be limited, because the task Manager
> won't be impacted during the process, it is only a task slot available or
> not.
> But is it really "free" or a good practice to do that ? Especially in
> production, is it a "stable" practice ?
>
>
> Thanks,
> Christophe