[jira] [Commented] (BIGTOP-2462) Make Spark events and history logs directory configurable

["Olaf Flebbe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334335#comment-15334335 ] 

Olaf Flebbe commented on BIGTOP-2462:
-------------------------------------

Same comment as in BIGTOP-2458: Please do not set default settings in cluster.yaml. Provide default values to the class definition.



> Make Spark events and history logs directory configurable
> ---------------------------------------------------------
>
>                 Key: BIGTOP-2462
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-2462
>             Project: Bigtop
>          Issue Type: Improvement
>          Components: spark
>    Affects Versions: 1.1.0
>            Reporter: Konstantinos Tsakalozos
>              Labels: easyfix, newbie, patch
>         Attachments: 0001-BIGTOP-2462-Make-Spark-events-and-history-logs-direc.patch, BIGTOP-2462.patch, BIGTOP-2462.patch
>
>
> Currently spark events and history logs are hard coded to a path inside HDFS. Our spark-defaults.conf contains:
> spark.eventLog.dir hdfs:///var/log/spark/apps
> spark.history.fs.logDirectory hdfs:///var/log/spark/apps
> As a result deploying Bigtop Spark forces you to also have HDFS. This is an unnecessary dependency since Spark can operate with a variety of storage solutions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)