LIVY VS Spark Job Server

[SamyaMaiti <sa...@gmail.com>]

Hi Team,

I am evaluating different ways to submit & monitor spark Jobs using REST
Interfaces. 

When to use Livy vs Spark Job Server?

Regards,
Sam



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/LIVY-VS-Spark-Job-Server-tp27722.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org

Re: LIVY VS Spark Job Server

[Vadim Semenov <va...@datadoghq.com>]

I have experience with both Livy & spark-jobserver.

spark-jobserver gives you better API, particularly, if you want to work
within a single spark context.

Livy supports submitting python & R code while spark-jobserver doesn't
support it.

spark-jobserver code is more complex, it actively uses Akka, and for me it
was difficult to debug and add additional features to it. Livy's
architecture is simpler in this sense.
I tried to add `yarn-cluster` support to spark-jobserver but got lost
halfway-through while Livy supports yarn-cluster mode out of the box and it
was easy for me to add missing features to Livy.

I'm not going into more details, but I'm ready to answer some specific
questions about usage of both Livy & spark-jobserver.

On Wed, Sep 14, 2016 at 3:32 PM, SamyaMaiti <sa...@gmail.com>
wrote:

> Hi Team,
>
> I am evaluating different ways to submit & monitor spark Jobs using REST
> Interfaces.
>
> When to use Livy vs Spark Job Server?
>
> Regards,
> Sam
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/LIVY-VS-Spark-Job-Server-tp27722.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>