You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Alexei Bakanov <ru...@gmail.com> on 2015/10/07 16:14:13 UTC

Temp files are not removed when done (Mesos)

Hi

I'm running Spark 1.5.1 on Mesos in coarse-grained mode. No dynamic
allocation or shuffle service. I see that there are two types of temporary
files under /tmp folder associated with every executor: /tmp/spark-<UUID>
and /tmp/blockmgr-<UUID>. When job is finished /tmp/spark-<UUID> is gone,
but blockmgr directory is left with all gigabytes in it. In Spark 1.4.1
blockmgr-<UUID> folder was under /tmp/spark-<UUID>, so when /tmp/spark
folder was removed blockmgr was gone too.
Is it a bug in 1.5.1?

By the way, in fine-grain mode /tmp/spark-<UUID> folder does not get
removed in neither 1.4.1 nor 1.5.1 for some reason.

Thanks,
Alexei

Re: Temp files are not removed when done (Mesos)

Posted by Iulian Dragoș <iu...@typesafe.com>.
https://issues.apache.org/jira/browse/SPARK-10975

On Wed, Oct 7, 2015 at 11:36 AM, Iulian Dragoș <iu...@typesafe.com>
wrote:

> It is indeed a bug. I believe the shutdown procedure in #7820 only kicks
> in when the external shuffle service is enabled (a pre-requisite of dynamic
> allocation). As a workaround you can use dynamic allocation (you can set
> spark.dynamicAllocation.maxExecutors and
> spark.dynamicAllocation.minExecutors to the same value.
>
> I'll file a Jira ticket.
>
> On Wed, Oct 7, 2015 at 10:14 AM, Alexei Bakanov <ru...@gmail.com> wrote:
>
>> Hi
>>
>> I'm running Spark 1.5.1 on Mesos in coarse-grained mode. No dynamic
>> allocation or shuffle service. I see that there are two types of temporary
>> files under /tmp folder associated with every executor: /tmp/spark-<UUID>
>> and /tmp/blockmgr-<UUID>. When job is finished /tmp/spark-<UUID> is gone,
>> but blockmgr directory is left with all gigabytes in it. In Spark 1.4.1
>> blockmgr-<UUID> folder was under /tmp/spark-<UUID>, so when /tmp/spark
>> folder was removed blockmgr was gone too.
>> Is it a bug in 1.5.1?
>>
>> By the way, in fine-grain mode /tmp/spark-<UUID> folder does not get
>> removed in neither 1.4.1 nor 1.5.1 for some reason.
>>
>> Thanks,
>> Alexei
>>
>
>
>
> --
>
> --
> Iulian Dragos
>
> ------
> Reactive Apps on the JVM
> www.typesafe.com
>
>


-- 

--
Iulian Dragos

------
Reactive Apps on the JVM
www.typesafe.com

Re: Temp files are not removed when done (Mesos)

Posted by Iulian Dragoș <iu...@typesafe.com>.
It is indeed a bug. I believe the shutdown procedure in #7820 only kicks in
when the external shuffle service is enabled (a pre-requisite of dynamic
allocation). As a workaround you can use dynamic allocation (you can set
spark.dynamicAllocation.maxExecutors and
spark.dynamicAllocation.minExecutors to the same value.

I'll file a Jira ticket.

On Wed, Oct 7, 2015 at 10:14 AM, Alexei Bakanov <ru...@gmail.com> wrote:

> Hi
>
> I'm running Spark 1.5.1 on Mesos in coarse-grained mode. No dynamic
> allocation or shuffle service. I see that there are two types of temporary
> files under /tmp folder associated with every executor: /tmp/spark-<UUID>
> and /tmp/blockmgr-<UUID>. When job is finished /tmp/spark-<UUID> is gone,
> but blockmgr directory is left with all gigabytes in it. In Spark 1.4.1
> blockmgr-<UUID> folder was under /tmp/spark-<UUID>, so when /tmp/spark
> folder was removed blockmgr was gone too.
> Is it a bug in 1.5.1?
>
> By the way, in fine-grain mode /tmp/spark-<UUID> folder does not get
> removed in neither 1.4.1 nor 1.5.1 for some reason.
>
> Thanks,
> Alexei
>



-- 

--
Iulian Dragos

------
Reactive Apps on the JVM
www.typesafe.com