[jira] [Closed] (LIVY-133) For interactive sessions, spark.master is hardcoded

["Matthew Livesey (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LIVY-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew Livesey closed LIVY-133.
--------------------------------

> For interactive sessions, spark.master is hardcoded
> ---------------------------------------------------
>
>                 Key: LIVY-133
>                 URL: https://issues.apache.org/jira/browse/LIVY-133
>             Project: Livy
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.2
>         Environment: Centos Linux, Standalone Spark 1.5.1, Livy master branch as of 28-04-2016.
>            Reporter: Matthew Livesey
>            Assignee: Marcelo Vanzin
>            Priority: Major
>             Fix For: 0.2
>
>
> In the InteractiveSession class at this line:
> https://github.com/cloudera/livy/blob/a5f3cfc9ebf1a5b3caf9ac23583c105fcf49aa6a/server/src/main/scala/com/cloudera/livy/server/interactive/InteractiveSession.scala#L74
> The master of the Client Builder is hard coded to be "yarn-cluster". This overrides any value set in config as described in a comment within the client builder:
> {code:java}
>   /**
>    * Creates a new builder that will optionally load the default Livy and Spark configuration
>    * from the classpath.
>    *
>    * Livy client configuration is stored in a file called "livy-client.conf", and Spark client
>    * configuration is stored in a file called "spark-defaults.conf", both in the root of the
>    * application's classpath. Livy configuration takes precedence over Spark's (in case
>    * configuration entries are duplicated), and configuration set in this builder object will
>    * override the values in those files.
>    */
> {code}
> Locally, with this line removed, I could start an interactive session (using the simple example from the project README) with master set to 'local' in my spark-defaults.conf.
> It looks like an InteractiveSession would only work with YARN in the current version.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)