You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by MEETHU MATHEW <me...@yahoo.co.in> on 2014/07/17 10:05:28 UTC
Pysparkshell are not listing in the web UI while running
Hi all,
I just upgraded to spark 1.0.1. In spark 1.0.0 when I start Ipython notebook using the following command,it used to come in the running applications tab in master:8080 web UI.
IPYTHON_OPTS="notebook --pylab inline" $SPARK_HOME/bin/pyspark
But now when I run it,its not getting listed under running application/completed application(once its closed).But I am able to see the spark stages at master:4040 while its running
Anyone have any idea why this
Thanks & Regards,
Meethu M
Re: Pysparkshell are not listing in the web UI while running
Posted by MEETHU MATHEW <me...@yahoo.co.in>.
Hi Akhil,
That fixed the problem...Thanks
Thanks & Regards,
Meethu M
On Thursday, 17 July 2014 2:26 PM, Akhil Das <ak...@sigmoidanalytics.com> wrote:
Hi Neethu,
Your application is running on local mode and that's the reason why you are not seeing the driver app in the 8080 webUI. You can pass the Master IP to your pyspark and get it running in cluster mode.
eg: IPYTHON_OPTS="notebook --pylab inline" $SPARK_HOME/bin/pyspark --master spark://master:7077
Replace master:7077 with the spark uri that you are seeing in top left of the 8080 webui.
Thanks
Best Regards
On Thu, Jul 17, 2014 at 1:35 PM, MEETHU MATHEW <me...@yahoo.co.in> wrote:
>
> Hi all,
>
>
>I just upgraded to spark 1.0.1. In spark 1.0.0 when I start Ipython notebook using the following command,it used to come in the running applications tab in master:8080 web UI.
>
>
>IPYTHON_OPTS="notebook --pylab inline" $SPARK_HOME/bin/pyspark
>
>
>But now when I run it,its not getting listed under running application/completed application(once its closed).But I am able to see the spark stages at master:4040 while its running
>
>
>Anyone have any idea why this
>
>
>
>
>
>
>Thanks & Regards,
>Meethu M
Re: Pysparkshell are not listing in the web UI while running
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Hi Neethu,
Your application is running on local mode and that's the reason why you are
not seeing the driver app in the 8080 webUI. You can pass the Master IP to
your pyspark and get it running in cluster mode.
eg: IPYTHON_OPTS="notebook --pylab inline" $SPARK_HOME/bin/pyspark --master
spark://master:7077
Replace master:7077 with the spark uri that you are seeing in top left of
the 8080 webui.
Thanks
Best Regards
On Thu, Jul 17, 2014 at 1:35 PM, MEETHU MATHEW <me...@yahoo.co.in>
wrote:
>
> Hi all,
>
> I just upgraded to spark 1.0.1. In spark 1.0.0 when I start Ipython
> notebook using the following command,it used to come in the running
> applications tab in master:8080 web UI.
>
> IPYTHON_OPTS="notebook --pylab inline" $SPARK_HOME/bin/pyspark
>
> But now when I run it,its not getting listed under running
> application/completed application(once its closed).But I am able to see the
> spark stages at master:4040 while its running
>
> Anyone have any idea why this
>
>
>
> Thanks & Regards,
> Meethu M
>