[jira] [Commented] (HADOOP-13887) Support for client-side encryption in S3A file system

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905695#comment-15905695 ] 

Steve Loughran commented on HADOOP-13887:
-----------------------------------------

First. the phrase "it should be trivial" is one of those things which all software engineers fear, especially in the context of adding a new feature to a system

The list of things which need to be done here are, at the very least

* provide an option
* wire it up
* document it
* test it
* make sure the tests can be bypassed when running against non-AWS endpoints.
* regression test *everything* with encryption turned on
* support the Hadoop credential providers API so that you can propagate secrets in a more secure mechanism than just configuration strings.

The good news is that [~Igor Mazur] has started on this with the first submission of code. I'm pulling his patch here from HADOOP-14171 to put it through the yetus review

Irrespective of what Yetus says, we have a strict policy here of "you must declare which s3 endpoint you ran the entire hadoop-aws test suite against". Jenkins can't automatically test the object stores, submitters have to. See [the test policy|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md]

Igor, thanks for getting started; I've moved the patch over because this has the most watchers. Let's target 2.9 for this, with an option to pull back into 2.8.1+

> Support for client-side encryption in S3A file system
> -----------------------------------------------------
>
>                 Key: HADOOP-13887
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13887
>             Project: Hadoop Common
>          Issue Type: New Feature
>            Reporter: Jeeyoung Kim
>            Assignee: Igor Mazur
>            Priority: Minor
>
> Expose the client-side encryption option documented in Amazon S3 documentation  - http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html
> Currently this is not exposed in Hadoop but it is exposed as an option in AWS Java SDK, which Hadoop currently includes. It should be trivial to propagate this as a parameter passed to the S3client used in S3AFileSystem.java



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org