You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Ralph Bergmann <ra...@dasralph.de> on 2015/06/18 13:03:30 UTC
connect mobile app with Spark backend
Hi,
I'm new to Spark and need some architecture tips :-)
I need a way to connect the mobile app with the Spark backend to upload
data to and download data from the Spark backend.
The use case is that the user do something with the app. This changes
are uploaded to the backend. Spark calculates something. If the user
uses the app again it download the new calculated data.
My plan is that the mobile app talks with a Jersey-Tomcat server and
this Jersey-Tomcat server loads the data into Spark and starts the jobs.
But what is the best way to upload the data to Spark and to start the job?
Currently Jersey, Tomcat and Spark are on the same machine.
I found this spark-jobserver[1] but I'm not sure if it is the right
choise. The mobile app uploads a JSON. Jersey converts it into POJOs to
do something with it. And than it converts it to JSON again to load it
into Spark witch converts it to POJOs.
I thought also about Spark streaming. But this means that this streaming
stuff runs 24/7?
[1] ... https://github.com/spark-jobserver/spark-jobserver
--
Ralph Bergmann
www http://www.dasralph.de | http://www.the4thFloor.eu
mail ralph@dasralph.de
skype dasralph
facebook https://www.facebook.com/dasralph
google+ https://plus.google.com/+RalphBergmann
xing https://www.xing.com/profile/Ralph_Bergmann3
linkedin https://www.linkedin.com/in/ralphbergmann
gulp https://www.gulp.de/Profil/RalphBergmann.html
github https://github.com/the4thfloor
pgp key id 0x421F9B78
pgp fingerprint CEE3 7AE9 07BE 98DF CD5A E69C F131 4A8E 421F 9B78
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: connect mobile app with Spark backend
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Why not something like your mobile app pushes data to your webserver which
pushes the data to Kafka or Cassandra or any other database and have a
Spark streaming job running all the time operating on the incoming data and
pushes the calculated values back. This way, you don't have to start a
spark job for every user request. (Which will end up awful if you have
thousands of user requests per second)
Thanks
Best Regards
On Thu, Jun 18, 2015 at 4:33 PM, Ralph Bergmann <ra...@dasralph.de> wrote:
> Hi,
>
>
> I'm new to Spark and need some architecture tips :-)
>
> I need a way to connect the mobile app with the Spark backend to upload
> data to and download data from the Spark backend.
>
> The use case is that the user do something with the app. This changes
> are uploaded to the backend. Spark calculates something. If the user
> uses the app again it download the new calculated data.
>
> My plan is that the mobile app talks with a Jersey-Tomcat server and
> this Jersey-Tomcat server loads the data into Spark and starts the jobs.
>
> But what is the best way to upload the data to Spark and to start the job?
>
> Currently Jersey, Tomcat and Spark are on the same machine.
>
> I found this spark-jobserver[1] but I'm not sure if it is the right
> choise. The mobile app uploads a JSON. Jersey converts it into POJOs to
> do something with it. And than it converts it to JSON again to load it
> into Spark witch converts it to POJOs.
>
> I thought also about Spark streaming. But this means that this streaming
> stuff runs 24/7?
>
>
>
> [1] ... https://github.com/spark-jobserver/spark-jobserver
>
> --
>
> Ralph Bergmann
>
>
> www http://www.dasralph.de | http://www.the4thFloor.eu
> mail ralph@dasralph.de
> skype dasralph
>
> facebook https://www.facebook.com/dasralph
> google+ https://plus.google.com/+RalphBergmann
> xing https://www.xing.com/profile/Ralph_Bergmann3
> linkedin https://www.linkedin.com/in/ralphbergmann
> gulp https://www.gulp.de/Profil/RalphBergmann.html
> github https://github.com/the4thfloor
>
>
> pgp key id 0x421F9B78
> pgp fingerprint CEE3 7AE9 07BE 98DF CD5A E69C F131 4A8E 421F 9B78
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>