[jira] [Commented] (FLINK-7266) Don't attempt to delete parent directory on S3

["Stefan Richter (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110453#comment-16110453 ] 

Stefan Richter commented on FLINK-7266:
---------------------------------------

One comment about the "not necessary on S3" part: during the release 1.3.2 testing, I observed that I can see some empty directory entries remaining in S3. I added a screenshot in the testing document [here|https://docs.google.com/document/d/1dN9AM9FUPizIu4hTKAXJSbbAORRdrce-BqQ8AUHlOqE/edit?ts=59807985#]. If this is not a problem, can the issue be closed or is the merge into 1.4 still pending?





> Don't attempt to delete parent directory on S3
> ----------------------------------------------
>
>                 Key: FLINK-7266
>                 URL: https://issues.apache.org/jira/browse/FLINK-7266
>             Project: Flink
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.3.1
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>            Priority: Critical
>             Fix For: 1.4.0, 1.3.2
>
>
> Currently, every attempted release of an S3 state object also checks if the "parent directory" is empty and then tries to delete it.
> Not only is that unnecessary on S3, but it is prohibitively expensive and for example causes S3 to throttle calls by the JobManager on checkpoint cleanup.
> The {{FileState}} must only attempt parent directory cleanup when operating against real file systems, not when operating against object stores.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)