[jira] [Created] (FLINK-8539) Introduce "CompletedCheckpointStorageLocation" to explicitly handle disposal of checkpoint storage locations

["Stephan Ewen (JIRA)" <ji...@apache.org>]

Stephan Ewen created FLINK-8539:
-----------------------------------

             Summary: Introduce "CompletedCheckpointStorageLocation" to explicitly handle disposal of checkpoint storage locations
                 Key: FLINK-8539
                 URL: https://issues.apache.org/jira/browse/FLINK-8539
             Project: Flink
          Issue Type: Sub-task
          Components: State Backends, Checkpointing
            Reporter: Stephan Ewen
            Assignee: Stephan Ewen
             Fix For: 1.5.0


The storage location of completed checkpoints misses a proper representation. Because of that, there is no place that can handle the deletion of a checkpoint directory, or the dropping of a checkpoint specific table.

Current workaround for file systems is, for example, that every file disposal checks if the parent directory is now empty, and deletes it if that is the case. That is not only inefficient, but prohibitively expensive on some systems, like Amazon S3.

Properly representing the storage location for completed checkpoints allows us to add a disposal call for that location.

That {{CompletedCheckpointStorageLocation}} can also be used to capture "external pointers", metadata, and even allow us to use custom serialization and deserialization of the metadata in the future, making the abstraction more extensible by allowing users to introduce new types of state handles.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)