[jira] [Commented] (HADOOP-12292) Make use of DeleteObjects optional

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14649473#comment-14649473 ] 

Steve Loughran commented on HADOOP-12292:
-----------------------------------------

There's actually one more way to do deletes: asynchronously (thanks to [~ndimiduk] for pointing this out to me). AWS S3 lets you specify an expiry time for data: if you PUT a new lifecycle specification into the bucket which declares everything under a path as expired, then AWS does the cleanup for you, async.

This argues for having an asyncDelete operation on object stores, which implementations are free to implement themselves (and which could be changed on an endpoint by endpoint basis)

> Make use of DeleteObjects optional
> ----------------------------------
>
>                 Key: HADOOP-12292
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12292
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Thomas Demoor
>            Assignee: Thomas Demoor
>
> The {{DeleteObjectsRequest}} was not part of the initial S3 API, but was added later. This patch allows one to configure s3a to replace each multidelete request by consecutive single deletes. Evidently, this setting is disabled by default as this causes slower deletes.
> The main motivation is to enable legacy S3-compatible object stores to make the transition from s3n (which does not use multidelete) to s3a, fully allowing the planned s3n deprecation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)