Unable to lauch Python Web Application on Spark Cluster

[anjali gautam <an...@gmail.com>]

Hello Everyone,

I have developed a web application (say abc) in Python using web.py. I want
to deploy it on the Spark Cluster. Since this application is a project with
dependencies therefore I have made a zip file of the project (abc.zip) to
be deployed on the cluster. In the project abc I have to execute
application.py file which has the main method to start my web application.
The command to execute the project is :

bin/spark-submit --master spark://master:7077 --py-files abc.zip
application.py

The structure of the project abc is that it has a data folder which has a
file data.txt. The other python files in the project abc access that
data.txt.

On executing the above command and then opening the browser to access the
developed web application results in an error of "data/data.txt not found"
when actually it is present.

I have been trying this for long. but everytime it shows the same error.
Can anybody help me in this case. Plz let me know if any other info is
required.

Thanks & Regards,
Anjali

Re: Unable to lauch Python Web Application on Spark Cluster

[Daniel van der Ende <da...@gmail.com>]

Hi Anjali,

It would help to see the code. But more importantly: why do you want to
deploy a web application on a Spark cluster? Spark is meant for
distributed, in-memory computations. I don't know what you're application
is doing, but it would make more sense to run it outside the Spark cluster,
and then submit computation tasks if/when necessary.

Daniel

On Thu, Nov 10, 2016 at 9:03 AM, anjali gautam <an...@gmail.com>
wrote:

>
> ---------- Forwarded message ----------
> From: anjali gautam <an...@gmail.com>
> Date: Thu, Nov 10, 2016 at 12:01 PM
> Subject: Unable to lauch Python Web Application on Spark Cluster
> To: user@spark.apache.org
>
>
> Hello Everyone,
>
> I have developed a web application (say abc) in Python using web.py. I
> want to deploy it on the Spark Cluster. Since this application is a project
> with dependencies therefore I have made a zip file of the project (abc.zip)
> to be deployed on the cluster. In the project abc I have to execute
> application.py file which has the main method to start my web application.
> The command to execute the project is :
>
> bin/spark-submit --master spark://master:7077 --py-files abc.zip
> application.py
>
> The structure of the project abc is that it has a data folder which has a
> file data.txt. The other python files in the project abc access that
> data.txt.
>
> On executing the above command and then opening the browser to access the
> developed web application results in an error of "data/data.txt not found"
> when actually it is present.
>
> I have been trying this for long. but everytime it shows the same error.
> Can anybody help me in this case. Plz let me know if any other info is
> required.
>
> Thanks & Regards,
> Anjali
>
>
>
>
>


-- 
Daniel

Fwd: Unable to lauch Python Web Application on Spark Cluster

[anjali gautam <an...@gmail.com>]

---------- Forwarded message ----------
From: anjali gautam <an...@gmail.com>
Date: Thu, Nov 10, 2016 at 12:01 PM
Subject: Unable to lauch Python Web Application on Spark Cluster
To: user@spark.apache.org


Hello Everyone,

I have developed a web application (say abc) in Python using web.py. I want
to deploy it on the Spark Cluster. Since this application is a project with
dependencies therefore I have made a zip file of the project (abc.zip) to
be deployed on the cluster. In the project abc I have to execute
application.py file which has the main method to start my web application.
The command to execute the project is :

bin/spark-submit --master spark://master:7077 --py-files abc.zip
application.py

The structure of the project abc is that it has a data folder which has a
file data.txt. The other python files in the project abc access that
data.txt.

On executing the above command and then opening the browser to access the
developed web application results in an error of "data/data.txt not found"
when actually it is present.

I have been trying this for long. but everytime it shows the same error.
Can anybody help me in this case. Plz let me know if any other info is
required.

Thanks & Regards,
Anjali