[jira] [Comment Edited] (BIGTOP-2462) Make Spark events and history logs directory configurable

["Olaf Flebbe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345026#comment-15345026 ] 

Olaf Flebbe edited comment on BIGTOP-2462 at 6/22/16 7:36 PM:
--------------------------------------------------------------

Great! looks good now. +1


was (Author: oflebbe):
Great looks good now. +1

> Make Spark events and history logs directory configurable
> ---------------------------------------------------------
>
>                 Key: BIGTOP-2462
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-2462
>             Project: Bigtop
>          Issue Type: Improvement
>          Components: spark
>    Affects Versions: 1.1.0
>            Reporter: Konstantinos Tsakalozos
>            Assignee: Konstantinos Tsakalozos
>              Labels: easyfix, newbie, patch
>         Attachments: 0001-BIGTOP-2462-Make-Spark-events-and-history-logs-direc.patch, BIGTOP-2462.patch, BIGTOP-2462.patch, BIGTOP-2462.patch
>
>
> Currently spark events and history logs are hard coded to a path inside HDFS. Our spark-defaults.conf contains:
> spark.eventLog.dir hdfs:///var/log/spark/apps
> spark.history.fs.logDirectory hdfs:///var/log/spark/apps
> As a result deploying Bigtop Spark forces you to also have HDFS. This is an unnecessary dependency since Spark can operate with a variety of storage solutions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)