[jira] [Commented] (FLINK-5666) Blob files are not cleaned up from ZK storage directory

["Nico Kruber (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840187#comment-15840187 ] 

Nico Kruber commented on FLINK-5666:
------------------------------------

hmm - do you see a warning like "Failed to delete blob at..."? This is what the FileSystemBlobStore should log in case an exception is thrown during the delete() call. However, it completely ignores the result from the fileSystem.delete() call!

Both BlobRecoveryITCase and BlobLibraryCacheRecoveryITCase do test that the delete is actually deleting the files (although this verification is a bit displaced in each of them) but only work on the local filesystem (with HA mode set). I guess if the reference counting of the BlobLibraryCacheManager is not different in HA mode, than something may indeed be wrong in the BlobServer's use of the hdfs backend only(?).

> Blob files are not cleaned up from ZK storage directory
> -------------------------------------------------------
>
>                 Key: FLINK-5666
>                 URL: https://issues.apache.org/jira/browse/FLINK-5666
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>            Reporter: Ufuk Celebi
>
> When running a job with HA in an standalone cluster, the blob files are not cleaned up from the ZooKeeper storage directory.
> {{:zkStorageDir/blob/cache/blob_:rand}}
> [~NicoK] Have you seen such a behaviour while refactoring the blob server?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)