[jira] [Commented] (YARN-4094) Add Configration to support encryption of Distributed Cache Data

["Vijay Singh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080757#comment-15080757 ] 

Vijay Singh commented on YARN-4094:
-----------------------------------

I have a working version available. The changes include following:-
1) Modify org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploader.java to chnage uploadFile to encrypt the file while getting copied instream based on yarn service wide property to encrypt sharedcache files.
2) While reading the data the same property is being evaluated to whether read the file after decrypt or read it as it is. FSDownload.java  file in org.apache.hadoop.yarn.util package will be modified to unpack the file accrodingly.

Please suggest if any alternative mechanism exists.

> Add Configration to support encryption of Distributed Cache Data
> ----------------------------------------------------------------
>
>                 Key: YARN-4094
>                 URL: https://issues.apache.org/jira/browse/YARN-4094
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: documentation, yarn
>    Affects Versions: 2.6.0, 2.7.0
>            Reporter: Vijay Singh
>
> Currently Ditributed cache does not allow mechanism to encrypt the data that gets copied over during processing. One attack vector is to process small files that contain sensitive data to use this mechanism to access contents of small files. 
> This requests aims to counter that by providing for configuration at service level that lets yarn encrypt all the data that gets to cache on each node. Yarn components should encrypt while copying the data on to disk and decrypt during the processing. Lets start by leveraging the symmetric key mechanism used by HDFS transparent mechanism similar to DEK (Data Encryption key) that could be generated as part of the process.
> Next step could be to setup Encryption zone key similar to transperent encryption mechanism.
> Please suggest if there is a better way.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)