You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/08/02 17:02:00 UTC

[jira] [Created] (HADOOP-16484) Add option to choose how to react on S3A instantiation

Steve Loughran created HADOOP-16484:
---------------------------------------

             Summary: Add option to choose how to react on S3A instantiation
                 Key: HADOOP-16484
                 URL: https://issues.apache.org/jira/browse/HADOOP-16484
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: 3.2.0
            Reporter: Steve Loughran


A seemingly recurrent problem with s3guard is "people who think S3Guard is turned on but really it isn't"

It's not immediately obvious this is the case, and the fact S3Guard is off tends to surface after some intermittent failure has actually been detected.

Propose: add a configuration parameter which chooses what to do when an S3A FS is instantiated without S3Guard

* silent : today; do nothing. 
* inform: log FS is instantiated without s3guard
* warn: Warn that data may be at risk in workflows
* fail

deployments could then choose which level of reaction they want. I'd make the default "inform" for now; any on-prem object store deployment should switch to silent, and if you really want strictness, fail is the ultimate option



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org