You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by "Varadhan, Jawahar" <va...@yahoo.com.INVALID> on 2015/08/25 18:37:35 UTC

Spark (1.2.0) submit fails with exception saying log directory already exists

Here is the error
yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: Log directory hdfs://Sandbox/user/spark/applicationHistory/application_1438113296105_0302 already exists!)
I am using cloudera 5.3.2 with Spark 1.2.0
Any help is appreciated.
ThanksJay

Re: Spark (1.2.0) submit fails with exception saying log directory already exists

Posted by Marcelo Vanzin <va...@cloudera.com>.

This probably means your app is failing and the second attempt is
hitting that issue. You may fix the "directory already exists" error
by setting
spark.eventLog.overwrite=true in your conf, but most probably that
will just expose the actual error in your app.

On Tue, Aug 25, 2015 at 9:37 AM, Varadhan, Jawahar
<va...@yahoo.com.invalid> wrote:
> Here is the error
>
>
> yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason:
> User class threw exception: Log directory
> hdfs://Sandbox/user/spark/applicationHistory/application_1438113296105_0302
> already exists!)
>
>
> I am using cloudera 5.3.2 with Spark 1.2.0
>
>
> Any help is appreciated.
>
>
> Thanks
>
> Jay
>
>
>



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: Spark (1.2.0) submit fails with exception saying log directory already exists

Posted by Marcelo Vanzin <va...@cloudera.com>.

This probably means your app is failing and the second attempt is
hitting that issue. You may fix the "directory already exists" error
by setting
spark.eventLog.overwrite=true in your conf, but most probably that
will just expose the actual error in your app.

On Tue, Aug 25, 2015 at 9:37 AM, Varadhan, Jawahar
<va...@yahoo.com.invalid> wrote:
> Here is the error
>
>
> yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason:
> User class threw exception: Log directory
> hdfs://Sandbox/user/spark/applicationHistory/application_1438113296105_0302
> already exists!)
>
>
> I am using cloudera 5.3.2 with Spark 1.2.0
>
>
> Any help is appreciated.
>
>
> Thanks
>
> Jay
>
>
>



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org