You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by David Maddison <ma...@gmail.com> on 2020/03/10 17:39:39 UTC

Automatically Clearing Temporary Directories

Hi,

When a TaskManager is restarted it can leave behind unreferenced BlobServer
cache directories in the temporary storage that never get cleaned up.
Would it be safe to automatically clear the temporary storage every time
when a TaskManager is started?

(Note: the temporary volumes in use are dedicated to the TaskManager and
not shared :-)

Thanks in advance,

David.

Re: Automatically Clearing Temporary Directories

Posted by Yang Wang <da...@gmail.com>.

Hi David,

Currently, the TaskManager could cleanup the non-referenced files in blob
cache. It
could configured via `blob.service.cleanup.interval`[1].
Also when the TaskManager is shut down gracefully, the storage directory
will be deleted.
So do you stop your TaskManager forcibly(i.e. kill -9)?

[1].
https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#blob-service-cleanup-interval


Best,
Yang

David Maddison <ma...@gmail.com> 于2020年3月11日周三 上午1:39写道：

> Hi,
>
> When a TaskManager is restarted it can leave behind unreferenced
> BlobServer cache directories in the temporary storage that never get
> cleaned up.  Would it be safe to automatically clear the temporary storage
> every time when a TaskManager is started?
>
> (Note: the temporary volumes in use are dedicated to the TaskManager and
> not shared :-)
>
> Thanks in advance,
>
> David.
>

Re: Automatically Clearing Temporary Directories

Posted by David Maddison <ma...@gmail.com>.

Thanks for the responses and thanks Gary for the confirmation.

Just to give some background, we deploy Flink inside Kubernetes so there is
a chance that TaskManagers COULD be shut down in a non-graceful way leaving
cache artifacts on the temporary volumes.

With Gary's confirmation, we'll add an init container to make sure the
volumes are cleared before a TM starts.

/David/

On Thu, Mar 12, 2020 at 8:24 AM Gary Yao <ga...@apache.org> wrote:

> Hi David,
>
> > Would it be safe to automatically clear the temporary storage every time
> when a TaskManager is started?
> > (Note: the temporary volumes in use are dedicated to the TaskManager and
> not shared :-)
> Yes, it is safe in your case.
>
> Best,
> Gary
>
> On Tue, Mar 10, 2020 at 6:39 PM David Maddison <ma...@gmail.com>
> wrote:
>
>> Hi,
>>
>> When a TaskManager is restarted it can leave behind unreferenced
>> BlobServer cache directories in the temporary storage that never get
>> cleaned up.  Would it be safe to automatically clear the temporary storage
>> every time when a TaskManager is started?
>>
>> (Note: the temporary volumes in use are dedicated to the TaskManager and
>> not shared :-)
>>
>> Thanks in advance,
>>
>> David.
>>
>

Re: Automatically Clearing Temporary Directories

Posted by Gary Yao <ga...@apache.org>.

Hi David,

> Would it be safe to automatically clear the temporary storage every time
when a TaskManager is started?
> (Note: the temporary volumes in use are dedicated to the TaskManager and
not shared :-)
Yes, it is safe in your case.

Best,
Gary

On Tue, Mar 10, 2020 at 6:39 PM David Maddison <ma...@gmail.com>
wrote:

> Hi,
>
> When a TaskManager is restarted it can leave behind unreferenced
> BlobServer cache directories in the temporary storage that never get
> cleaned up.  Would it be safe to automatically clear the temporary storage
> every time when a TaskManager is started?
>
> (Note: the temporary volumes in use are dedicated to the TaskManager and
> not shared :-)
>
> Thanks in advance,
>
> David.
>