[jira] [Commented] (HADOOP-19197) S3A: Support AWS KMS Encryption Context

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-19197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009965#comment-18009965 ] 

ASF GitHub Bot commented on HADOOP-19197:
-----------------------------------------

steveloughran commented on PR #6874:
URL: https://github.com/apache/hadoop/pull/6874#issuecomment-3118073395

   This is incompatible at the wire serialization level, as EncryptionSecrets says 
   ```
   
   " * <i>Important.</i>
    * If the wire format is ever changed incompatibly,
    * update the serial version UID to ensure that older clients get safely
    * rejected.
   "
   ```
   
   We've added a new string to the payload, so it needs a new serialization ID generated and set in the java source.
   
   
   @raphaelazzolini would you be able to change that serialization number ASAP? 
   
   I've updated the docs to indicate wire incompatibility.




> S3A: Support AWS KMS Encryption Context
> ---------------------------------------
>
>                 Key: HADOOP-19197
>                 URL: https://issues.apache.org/jira/browse/HADOOP-19197
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs/s3
>    Affects Versions: 3.4.0
>            Reporter: Raphael Azzolini
>            Assignee: Raphael Azzolini
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.5.0, 3.4.2
>
>
> S3A properties allow users to choose the AWS KMS key ({_}fs.s3a.encryption.key{_}) and S3 encryption algorithm to be used (f{_}s.s3a.encryption.algorithm{_}). In addition to the AWS KMS Key, an encryption context can be used as non-secret data that adds additional integrity and authenticity to check the encrypted data. However, there is no option to specify the [AWS KMS Encryption Context|https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context] in S3A.
> In AWS SDK v2 the encryption context in S3 requests is set by the parameter [ssekmsEncryptionContext.|https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/CreateMultipartUploadRequest.Builder.html#ssekmsEncryptionContext(java.lang.String)] It receives a base64-encoded UTF-8 string holding JSON with the encryption context key-value pairs. The value of this parameter could be set by the user in a new property {_}*fs.s3a.encryption.context*{_}, and be stored in the [EncryptionSecrets|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/EncryptionSecrets.java] to later be used when setting the encryption parameters in [RequestFactoryImpl|https://github.com/apache/hadoop/blob/f92a8ab8ae54f11946412904973eb60404dee7ff/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/RequestFactoryImpl.java].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org