[GitHub] kl0u commented on a change in pull request #7161: [FLINK-10963][fs-connector, s3] Cleanup tmp S3 objects uploaded as backups of in-progress files.

[GitBox <gi...@apache.org>]

kl0u commented on a change in pull request #7161: [FLINK-10963][fs-connector, s3] Cleanup tmp S3 objects uploaded as backups of in-progress files.
URL: https://github.com/apache/flink/pull/7161#discussion_r237059956
 
 

 ##########
 File path: flink-core/src/main/java/org/apache/flink/core/fs/RecoverableWriter.java
 ##########
 @@ -121,6 +121,32 @@
 	 */
 	RecoverableFsDataOutputStream recover(ResumeRecoverable resumable) throws IOException;
 
+	/**
+	 * Marks if the writer requires to do any additional cleanup/freeing of resources occupied
+	 * as part of a {@link ResumeRecoverable}, e.g. temporarily files created or objects uploaded
+	 * to external systems.
+	 *
+	 * <p>In case cleanup is required, then {@link #cleanupRecoverableState(ResumeRecoverable)} should
+	 * be called.
+	 *
+	 * @return {@code true} if cleanup is required, {@code false} otherwise.
+	 */
+	boolean requiresCleanupOfRecoverableState();
+
+	/**
+	 * Frees up any resources that were were previously occupied in order to be able to
+	 * recover from a (potential) failure. This can be temporary files that we written or
+	 * objects that were uploaded (e.g. S3).
+	 *
+	 * <p><b>NOTE:</b> This operation should not through an exception if the resumable has already
+	 * been cleaned up and the resources have been freed.
+	 *
+	 * @param resumable The {@link ResumeRecoverable} whose state we want to clean-up.
+	 * @return {@code true} if the resources were successfully freed, {@code false} otherwise
+	 * (e.g. the file to be deleted was not there).
+	 */
+	boolean cleanupRecoverableState(ResumeRecoverable resumable) throws IOException;
 
 Review comment:
   The `requiresCleanupOfRecoverableState()` is just there for an optimisation. Essentially, using this, we avoid unnecessary iterations in the `Bucket#cleanupOutdatedResumables()`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services