You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Sagi r <st...@gmail.com> on 2015/07/06 22:58:28 UTC
Spark application with a RESTful API
Hi,
I've been researching spark for a couple of months now, and I strongly
believe it can solve our problem.
We are developing an application that allows the user to analyze various
sources of information. We are dealing with non-technical users, so simply
giving them and interactive shell won't do the trick.
To allow the users to execute queries, I have considered writing a Spark
application that exposes a RESTful api and runs on our cluster. This
application will execute the queries on demand on different threads.
We need to serve a few thousand users.
I should mention that I've looked into Spark Job-Server too, it looks
promising however it's not quite what we are looking for.
I wanted to here your input on this solution, and maybe if you can suggest a
better one.
Thank you.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-application-with-a-RESTful-API-tp23654.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Spark application with a RESTful API
Posted by Debasish Das <de...@gmail.com>.
How do you manage the spark context elastically when your load grows from
1000 users to 10000 users ?
On Tue, Jul 14, 2015 at 8:31 AM, Hafsa Asif <ha...@matchinguu.com>
wrote:
> I have almost the same case. I will tell you what I am actually doing, if
> it
> is according to your requirement, then I will love to help you.
>
> 1. my database is aerospike. I get data from it.
> 2. written standalone spark app (it does not run in standalone mode, but
> with simple java command or maven command), even I make its JAR file with
> simple java command and it runs spark without interactive shell.
> 3. Write different methods based on different queries in standalone spark
> app.
> 4. my restful API is based on NodeJS, user sends his request through
> NodeJS,
> request pass to my simple java maven project, execute particular spark
> method based on query, returns result to NodeJS in JSON format.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-application-with-a-RESTful-API-tp23654p23831.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: Spark application with a RESTful API
Posted by Hafsa Asif <ha...@matchinguu.com>.
I have almost the same case. I will tell you what I am actually doing, if it
is according to your requirement, then I will love to help you.
1. my database is aerospike. I get data from it.
2. written standalone spark app (it does not run in standalone mode, but
with simple java command or maven command), even I make its JAR file with
simple java command and it runs spark without interactive shell.
3. Write different methods based on different queries in standalone spark
app.
4. my restful API is based on NodeJS, user sends his request through NodeJS,
request pass to my simple java maven project, execute particular spark
method based on query, returns result to NodeJS in JSON format.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-application-with-a-RESTful-API-tp23654p23831.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
RE: Spark application with a RESTful API
Posted by Mohammed Guller <mo...@glassbeam.com>.
It is not a bad idea. Many people use this approach.
Mohammed
-----Original Message-----
From: Sagi r [mailto:stsagir@gmail.com]
Sent: Monday, July 6, 2015 1:58 PM
To: user@spark.apache.org
Subject: Spark application with a RESTful API
Hi,
I've been researching spark for a couple of months now, and I strongly believe it can solve our problem.
We are developing an application that allows the user to analyze various sources of information. We are dealing with non-technical users, so simply giving them and interactive shell won't do the trick.
To allow the users to execute queries, I have considered writing a Spark application that exposes a RESTful api and runs on our cluster. This application will execute the queries on demand on different threads.
We need to serve a few thousand users.
I should mention that I've looked into Spark Job-Server too, it looks promising however it's not quite what we are looking for.
I wanted to here your input on this solution, and maybe if you can suggest a better one.
Thank you.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-application-with-a-RESTful-API-tp23654.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org