Confirm concurrence of Zeppelin

[Fengdong Yu <fe...@everstring.com>]

Hi Dev Team,

I want to confirm with you guys:

1)  Does that Zeppelin only run one Job concurrently?  I mean if I submit a job on my notebook, so other’s submit will be pending until my job finished?
2)  How can I submit more than one job at the same time on my notebook?

Re: Confirm concurrence of Zeppelin

[moon soo Lee <mo...@apache.org>]

Hi,

In Zeppelin side,

Depending on what scheduler is returned from individual interpreter
implementation "Interpreter.getScheduler()", Zeppelin runs job sequentially
or concurrently.

For example, if interpreter implementation returns FIFO scheduler,
paragraphs using the interpreter will run one by one, sequentially. If
interpreter implementation return Parallel scheduler, paragraphs using the
interpreter will run concurrently.


In Spark Interpreter,

Basically, SparkInterpreter implementation uses FIFO scheduler. Because of
code passed to scala / python compiler need to have an order.
But SparkContext it self is thread safe and it's got fair scheduler inside.

So Zeppelin's SparkSQLInterpreter leverage fair scheduler of Spark to
execute query concurrently. Set zeppelin.spark.concurrentSQL true will make
SparkSQLInterpreter use Parallel scheduler, and it allow multiple
paragraphs running concurrently, and Spark processes the job with it's fair
scheduler.


So, Zeppelin is capable of running job concurrently. It depends on which
interpreter you use and which scheduler is returned from that particular
interpreter.

Hope this helps.

Best,
moon

On Thu, Nov 19, 2015 at 11:22 AM Fengdong Yu <fe...@everstring.com>
wrote:

> Hi Dev Team,
>
> I want to confirm with you guys:
>
> 1)  Does that Zeppelin only run one Job concurrently?  I mean if I submit
> a job on my notebook, so other’s submit will be pending until my job
> finished?
> 2)  How can I submit more than one job at the same time on my notebook?
>
>
>
>
>
>