You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by rroxanaioana <rr...@gmail.com> on 2015/07/09 16:14:25 UTC
Accessing Spark Web UI from another place than where the job
actually ran
I have a spark cluster with 1 master 9nodes.I am running in standalone-mode.
I do not have access to a web browser from any of the nodes in the cluster
(I am connecting to the nodes through ssh --it is a grid5000 cluster). I was
wondering, is there any possibility to access Spark Web UI in this case? I
tried by copying the logs from my cluster in SPARK_PATH/work on my local
machine (leaving the impression that the jobs that ran in the cluster were
ran on my local machine). This idea came after reading this part from the
documentation:
If an application has logged events over the course of its lifetime, then
the Standalone master’s web UI will automatically re-render the
application’s UI after the application has finished.
But it did not work. What I can see in the UI is:
Applications: 0 Running, 0
Completed Drivers: 0 Running, 0 Completed
Status: ALIVE
Thank you!
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Accessing-Spark-Web-UI-from-another-place-than-where-the-job-actually-ran-tp23745.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Accessing Spark Web UI from another place than where the job
actually ran
Posted by Roxana Ioana Roman <rr...@gmail.com>.
Thank you for your answer!
The problem is, I cannot ssh to the master directly.
I have to ssh first to a frontend, then I have to ssh to another frontend.
And only from this last frontend I can ssh to my master.
Can I do this by ssh -ing with -L to the first two frontends and to the
master?
And maybe the traffic on master 8080 will be redirected from
master->frontend1->frontend2->myPC?
Thanks again,
Roxana Roman
2015-07-10 9:53 GMT+02:00 Akhil Das <ak...@sigmoidanalytics.com>:
> When you connect to the machines you can create an ssh tunnel to access
> the UI :
>
> ssh -L 8080:127.0.0.1:8080 MasterMachinesIP
>
>
> And then you can simply open localhost:8080 in your browser and it should
> show up the UI.
>
> Thanks
> Best Regards
>
> On Thu, Jul 9, 2015 at 7:44 PM, rroxanaioana <rr...@gmail.com>
> wrote:
>
>>
>>
>> I have a spark cluster with 1 master 9nodes.I am running in
>> standalone-mode.
>> I do not have access to a web browser from any of the nodes in the cluster
>> (I am connecting to the nodes through ssh --it is a grid5000 cluster). I
>> was
>> wondering, is there any possibility to access Spark Web UI in this case? I
>> tried by copying the logs from my cluster in SPARK_PATH/work on my local
>> machine (leaving the impression that the jobs that ran in the cluster were
>> ran on my local machine). This idea came after reading this part from the
>> documentation:
>>
>> If an application has logged events over the course of its lifetime, then
>> the Standalone master’s web UI will automatically re-render the
>> application’s UI after the application has finished.
>>
>> But it did not work. What I can see in the UI is:
>> Applications: 0 Running, 0
>> Completed Drivers: 0 Running, 0 Completed
>> Status: ALIVE
>>
>> Thank you!
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Accessing-Spark-Web-UI-from-another-place-than-where-the-job-actually-ran-tp23745.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
--
*Roxana Ioana Roman*
Re: Accessing Spark Web UI from another place than where the job
actually ran
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
When you connect to the machines you can create an ssh tunnel to access the
UI :
ssh -L 8080:127.0.0.1:8080 MasterMachinesIP
And then you can simply open localhost:8080 in your browser and it should
show up the UI.
Thanks
Best Regards
On Thu, Jul 9, 2015 at 7:44 PM, rroxanaioana <rr...@gmail.com> wrote:
>
>
> I have a spark cluster with 1 master 9nodes.I am running in
> standalone-mode.
> I do not have access to a web browser from any of the nodes in the cluster
> (I am connecting to the nodes through ssh --it is a grid5000 cluster). I
> was
> wondering, is there any possibility to access Spark Web UI in this case? I
> tried by copying the logs from my cluster in SPARK_PATH/work on my local
> machine (leaving the impression that the jobs that ran in the cluster were
> ran on my local machine). This idea came after reading this part from the
> documentation:
>
> If an application has logged events over the course of its lifetime, then
> the Standalone master’s web UI will automatically re-render the
> application’s UI after the application has finished.
>
> But it did not work. What I can see in the UI is:
> Applications: 0 Running, 0
> Completed Drivers: 0 Running, 0 Completed
> Status: ALIVE
>
> Thank you!
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Accessing-Spark-Web-UI-from-another-place-than-where-the-job-actually-ran-tp23745.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>