You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by sunil m <26...@gmail.com> on 2015/12/07 09:42:23 UTC
Available options for Spark REST API
Dear Spark experts!
I would like to know the best practices used for invoking spark jobs via
REST API.
We tried out the hidden REST API mentioned here:
http://arturmkrtchyan.com/apache-spark-hidden-rest-api
It works fine for spark standalone mode but does not seem to be working
when i specify
"spark.master" : "YARN-CLUSTER" or "mesos://..."
did anyone encounter a similar problem?
Has anybody used:
https://github.com/spark-jobserver/spark-jobserver
If yes please share your experience. Does it work good with both Scala and
Java classes. I saw only scala example. Are there any known disadvantages
of using it?
Is there anything better available, which is used in Production environment?
Any advise is appreciated. We are using Spark 1.5.1.
Thanks in advance.
Warm regards,
Sunil M.
Re: Available options for Spark REST API
Posted by Василец Дмитрий <pr...@gmail.com>.
hello
if i correct understand - sparkui with rest api - for monitoring
spark-jobserver - for submit job.
On Mon, Dec 7, 2015 at 9:42 AM, sunil m <26...@gmail.com> wrote:
> Dear Spark experts!
>
> I would like to know the best practices used for invoking spark jobs via
> REST API.
>
> We tried out the hidden REST API mentioned here:
> http://arturmkrtchyan.com/apache-spark-hidden-rest-api
>
> It works fine for spark standalone mode but does not seem to be working
> when i specify
> "spark.master" : "YARN-CLUSTER" or "mesos://..."
> did anyone encounter a similar problem?
>
> Has anybody used:
> https://github.com/spark-jobserver/spark-jobserver
>
> If yes please share your experience. Does it work good with both Scala and
> Java classes. I saw only scala example. Are there any known disadvantages
> of using it?
>
> Is there anything better available, which is used in Production
> environment?
>
> Any advise is appreciated. We are using Spark 1.5.1.
>
> Thanks in advance.
>
> Warm regards,
> Sunil M.
>