Error running Spark on Cloudera

[Vijayasarathy Kannan <kv...@vt.edu>]

I am trying to run a Spark application using spark-submit on a cluster
using Cloudera manager. I get the error

"Exception in thread "main" java.io.IOException: Error in creating log
directory: file:/user/spark/applicationHistory//app-20150408094126-0008"

Adding the below lines in /etc/spark/conf/spark-defaults.conf wouldn't
resolve it.

*spark.eventLog.dir=/user/spark/applicationHistory*

*spark.eventLog.enabled=true*

Any idea on what is missing?

Re: Error running Spark on Cloudera

[Marcelo Vanzin <va...@cloudera.com>]

"spark.eventLog.dir" should contain the full HDFS URL. In general,
this should be sufficient:

spark.eventLog.dir=hdfs:/user/spark/applicationHistory


On Wed, Apr 8, 2015 at 6:45 AM, Vijayasarathy Kannan <kv...@vt.edu> wrote:
> I am trying to run a Spark application using spark-submit on a cluster using
> Cloudera manager. I get the error
>
> "Exception in thread "main" java.io.IOException: Error in creating log
> directory: file:/user/spark/applicationHistory//app-20150408094126-0008"
>
> Adding the below lines in /etc/spark/conf/spark-defaults.conf wouldn't
> resolve it.
>
> spark.eventLog.dir=/user/spark/applicationHistory
> spark.eventLog.enabled=true
>
> Any idea on what is missing?
>
>



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org