You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "DB Tsai (JIRA)" <ji...@apache.org> on 2014/06/11 00:03:02 UTC
[jira] [Updated] (SPARK-2100) Allow users to disable Jetty Spark UI
in local mode
[ https://issues.apache.org/jira/browse/SPARK-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
DB Tsai updated SPARK-2100:
---------------------------
Description:
Since we want to use Spark hadoop APIs in local mode for design time to explore the first couple hundred lines of data in HDFS. Also, we want to use Spark in our tomcat application, so starting a jetty UI will make our tomcat unhappy. In those scenarios, Spark UI is not necessary, and wasting resource.
As a result, for local mode, it's desirable that users are able to disable the spark UI.
Couple places I found where the jetty will be started.
In SparkEnv.scala
1) val broadcastManager = new BroadcastManager(isDriver, conf, securityManager)
2) val httpFileServer = new HttpFileServer(securityManager)
httpFileServer.initialize()
I don't know if broadcastManager is needed in local mode tho.
was:
Since we're using Spark hadoop APIs in local mode for design time to explore the first couple hundred lines of data in HDFS. Also, we use it in our tomcat application, and starting a jetty UI will make our tomcat unhappy. In those scenarios, Spark UI is not necessary, and wasting resource.
As a result, for local mode, users should be able to disable the spark UI.
Couple places I found that the jetty will be started.
In SparkEnv.scala
1) val broadcastManager = new BroadcastManager(isDriver, conf, securityManager)
2) val httpFileServer = new HttpFileServer(securityManager)
httpFileServer.initialize()
I don't know if broadcastManager is needed in local mode tho.
> Allow users to disable Jetty Spark UI in local mode
> ---------------------------------------------------
>
> Key: SPARK-2100
> URL: https://issues.apache.org/jira/browse/SPARK-2100
> Project: Spark
> Issue Type: Improvement
> Reporter: DB Tsai
>
> Since we want to use Spark hadoop APIs in local mode for design time to explore the first couple hundred lines of data in HDFS. Also, we want to use Spark in our tomcat application, so starting a jetty UI will make our tomcat unhappy. In those scenarios, Spark UI is not necessary, and wasting resource.
> As a result, for local mode, it's desirable that users are able to disable the spark UI.
> Couple places I found where the jetty will be started.
> In SparkEnv.scala
> 1) val broadcastManager = new BroadcastManager(isDriver, conf, securityManager)
> 2) val httpFileServer = new HttpFileServer(securityManager)
> httpFileServer.initialize()
> I don't know if broadcastManager is needed in local mode tho.
--
This message was sent by Atlassian JIRA
(v6.2#6252)