Re: support launch more than one pipeline in a yarn session

[Till Rohrmann <tr...@apache.org>]

Hi Chen,

I'm not sure what you mean with a pipeline but Flink supports the
submission of multiple jobs to the same cluster (in standalone as well as
Yarn session mode). You simply have to make sure that there are enough
slots for all jobs to be executed at the same time.

Cheers,
Till
On Jun 28, 2016 6:47 AM, "Chen Qin" <qi...@gmail.com> wrote:

> Hi there,
>
> We are researching launch more than one pipeline in a flink yarn session.
> It's sort of like cluster mode where user pick off multiple pipelines on a
> job manager. Is there any plan to support this use case? If not, why not?
> Is it because of better isolation per pipeline or some implementation
> insights? What would be recommended way to add this part if some one plan
> to work on it?
>
> Speaking of our use case, we want to allow runs arbitrary numbers of
> pipeline in a given flink yarn session so that people don't need to chase
> rabbits and manage lots of web dashboards at same time.
>
> Thanks,
> Chen
>