[jira] [Commented] (HDFS-7256) Encryption Key created in Java Key Store after Namenode start unavailable for EZ Creation

["Yi Liu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-7256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14174669#comment-14174669 ] 

Yi Liu commented on HDFS-7256:
------------------------------

Thanks [~xyao] for testing this, this should be not an issue. Let me explain below.

HDFS encryption at rest requires user to configure a KMS, and the backing KeyProvider of KMS can be a {{JavaKeyStoreProvider}} or a third-party keystore which implements Hadoop {{KeyProvider}} interface.
In your case, {{JavaKeyStoreProvider}} is used directly, actually both FSN and DFSClient will have KeyProvider instance (different), FSN uses KeyProvider instance to get EncryptionZone key and get Encrypted data encryption keys, and DFSClient uses KeyProvider instance to decrypt the data encryption keys.  JavaKeyStoreProvider uses local java keystore file, it can't satisfy multiple nodes accessing. 
"hadoop key create ..." command constructs its KeyProvider instance in client side, and create/flush key to java keystore file, and FSN will not reload the java keystore file. That's the reason why you see the exception.

So please configure a KMS and the backing KeyProvider could be a {{JavaKeyStoreProvider}}, for more information, please refer to the fs-encryption/KMS user doc.

> Encryption Key created in Java Key Store after Namenode start unavailable for EZ Creation 
> ------------------------------------------------------------------------------------------
>
>                 Key: HDFS-7256
>                 URL: https://issues.apache.org/jira/browse/HDFS-7256
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: encryption, security
>    Affects Versions: 2.6.0
>            Reporter: Xiaoyu Yao
>
> Hit an error on "RemoteException: Key ezkey1 doesn't exist." when creating EZ with a Key created after NN starts.
> Briefly check the code and found that the KeyProivder is loaded by FSN only at the NN start. My work around is to restart the NN which triggers the reload of Key Provider. Is this expected?
> Repro Steps:
> Create a new Key after NN and KMS starts
> hadoop/bin/hadoop key create ezkey1 -size 256 -provider jceks://file/home/hadoop/kms.keystore
> List Keys
> hadoop@SaturnVm:~/deploy$ hadoop/bin/hadoop key list -provider jceks://file/home/hadoop/kms.keystore -metadata
> Listing keys for KeyProvider: jceks://file/home/hadoop/kms.keystore
> ezkey1 : cipher: AES/CTR/NoPadding, length: 256, description: null, created: Thu Oct 16 18:51:30 EDT 2014, version: 1, attributes: null
> key2 : cipher: AES/CTR/NoPadding, length: 128, description: null, created: Tue Oct 14 19:44:09 EDT 2014, version: 1, attributes: null
> key1 : cipher: AES/CTR/NoPadding, length: 128, description: null, created: Tue Oct 14 17:52:36 EDT 2014, version: 1, attributes: null
> Create Encryption Zone
> hadoop/bin/hdfs dfs -mkdir /Ez1
> hadoop@SaturnVm:~/deploy$ hadoop/bin/hdfs crypto -createZone -keyName ezkey1 -path /Ez1
> RemoteException: Key ezkey1 doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)