[jira] [Commented] (HADOOP-13651) S3Guard: S3AFileSystem Integration with MetadataStore

["Aaron Fabbri (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15624188#comment-15624188 ] 

Aaron Fabbri commented on HADOOP-13651:
---------------------------------------

Following up your security comments, [~stevel@apache.org].  To make sure I'm understanding, is it correct to say that:

- S3A FileSystem authorization is delegated to the AWS S3 SDK client.
- S3A code does not check hadoop user permissions, nor map hadoop users to AWS credentials.
   - So authorization is not "per user" in the hadoop sense, but "per configuration" as that is where S3A credentials / instance roles / etc. are defined.

- If a user tries to open a s3a:// FileSystem and they do not supply/config proper AWS credentials, S3AFileSystem.initialize() will throw an exception in verifyBucketExists() -> s3.doesBucketExist()

- It should be sufficient to only allow MetadataStore read/write operations after success of S3 read/write operation (respectively).

Questions:

- If a user has valid AWS credentials, but no read permissions for given bucket, what happens?  Does initialize() succeed? (I can test this if needed)

- What needs to be done before we can commit this patch (besides the LOG.isDebugEnabled thing)?  I'd like to get this basic support in the feature branch so [~eddyxu] and [~liuml07] can integrate with it.  I agree we need to address security and add tests to demonstrate its correctness.  I'd be happy to take a followup JIRA on that as well, or we can hold this patch up.







> S3Guard: S3AFileSystem Integration with MetadataStore
> -----------------------------------------------------
>
>                 Key: HADOOP-13651
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13651
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Aaron Fabbri
>            Assignee: Aaron Fabbri
>         Attachments: HADOOP-13651-HADOOP-13345.001.patch, HADOOP-13651-HADOOP-13345.002.patch, HADOOP-13651-HADOOP-13345.003.patch
>
>
> Modify S3AFileSystem et al. to optionally use a MetadataStore for metadata consistency and caching.
> Implementation should have minimal overhead when no MetadataStore is configured.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org