[jira] [Commented] (HADOOP-15575) Add a way for an FS instance to say "really, no trash interval at all"

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-15575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529752#comment-16529752 ] 

Steve Loughran commented on HADOOP-15575:
-----------------------------------------

You could do all of this with a custom trash policy; add an S3A aware one which asked the FS bucket conf, delegating to the default trash policy for all other stores.
You'd want to override {{Trash.getEmptier()}} to return a new Emptier...this would be instantiated in the NN of any (transient) HDFS cluster, so still need to do the normal cleanup for HDFS. As it never gets pointed at the object store (presumably), it's not doing any work there..and even if it was, assuming the ~/.Trash was never filled up, it'd not have anything to delete.

> Add a way for an FS instance to say "really, no trash interval at all"
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-15575
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15575
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs, fs/s3
>    Affects Versions: 3.1.0
>            Reporter: Steve Loughran
>            Priority: Major
>
> Object stores like S3 often offer [object versioning|https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectVersioning.html] as a way to recover from accidental deletions. There is no need to use the Fs Shell trash mechanism.
> Proposed: a way for an FS instance to tell the shell to not use trash, even if the client has configured it. The current {{getServerDefaults().getTrashInterval()}} returns a value from the server, but it *only* overrides the ""fs.trash.interval" if it !=0. So if a server says "don't use trash" it gets ignored.
> If a special value (-1) is returned (or we use a new field?), FS instances can return to the shell saying "no need for trash". Then you can turn it on for a bucket by bucket basis; any store with the feature enabled can do the same thing.
> Alternative option: A special S3A-aware trash policy which does a getFileSystem.getConf.getBoolean("fs.s3a.trash.skip") & skips trash if set. This will all you to disable it on a bucket-by-bucket basis. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org