You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Heiko Braun <ik...@googlemail.com> on 2014/01/27 15:15:00 UTC
Purpose of the HTTP Server?
Can someone briefly explain the the purpose of the HTTP server in spark? Is it related to shipping the jars around in a cluster?
Regards, Heiko
Re: Purpose of the HTTP Server?
Posted by Mark Hamstra <ma...@clearstorydata.com>.
Used for broadcast variables and to distribute files or jars to worker
nodes. See HttpBroadcast.scala<https://github.com/apache/incubator-spark/blob/master/core/src/main/scala/org/apache/spark/broadcast/HttpBroadcast.scala>and
SparkContext.scala<https://github.com/apache/incubator-spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala>
On Mon, Jan 27, 2014 at 7:03 AM, Heiko Braun <ik...@googlemail.com>wrote:
> Yes, I've seen the one used for the UI. But there is also the HttpServer
> and HttpFileServer. Those are the ones I was wondering about.
>
> /Heiko
>
> On 27 Jan 2014, at 15:18, Cheng Lian <rh...@gmail.com> wrote:
>
> It's used for the Web UI. By default, you may access
> http://localhost:8080 to view the cluster information as well as
> individual job details.
>
>
> On Mon, Jan 27, 2014 at 10:15 PM, Heiko Braun <ik...@googlemail.com>wrote:
>
>>
>>
>> Can someone briefly explain the the purpose of the HTTP server in spark?
>> Is it related to shipping the jars around in a cluster?
>>
>> Regards, Heiko
>
>
>
>
Re: Purpose of the HTTP Server?
Posted by Heiko Braun <ik...@googlemail.com>.
Yes, I've seen the one used for the UI. But there is also the HttpServer and HttpFileServer. Those are the ones I was wondering about.
/Heiko
On 27 Jan 2014, at 15:18, Cheng Lian <rh...@gmail.com> wrote:
> It's used for the Web UI. By default, you may access http://localhost:8080 to view the cluster information as well as individual job details.
>
>
> On Mon, Jan 27, 2014 at 10:15 PM, Heiko Braun <ik...@googlemail.com> wrote:
>
>
> Can someone briefly explain the the purpose of the HTTP server in spark? Is it related to shipping the jars around in a cluster?
>
> Regards, Heiko
>
Re: Purpose of the HTTP Server?
Posted by Cheng Lian <rh...@gmail.com>.
It's used for the Web UI. By default, you may access
http://localhost:8080to view the cluster information as well as
individual job details.
On Mon, Jan 27, 2014 at 10:15 PM, Heiko Braun <ik...@googlemail.com>wrote:
>
>
> Can someone briefly explain the the purpose of the HTTP server in spark?
> Is it related to shipping the jars around in a cluster?
>
> Regards, Heiko