[jira] [Created] (LIVY-850) Deprecated HDFS config (dfs.encryption.key.provider.uri) in Livy

["Vishnu Chelle (Jira)" <ji...@apache.org>]

Vishnu Chelle created LIVY-850:
----------------------------------

             Summary: Deprecated HDFS config (dfs.encryption.key.provider.uri) in Livy 
                 Key: LIVY-850
                 URL: https://issues.apache.org/jira/browse/LIVY-850
             Project: Livy
          Issue Type: Bug
          Components: Core
    Affects Versions: 0.7.0
            Reporter: Vishnu Chelle


Hi, 

When using Livy with Hadoop3.0 Cluster where KMS is enabled, there is an issue with one of the HDFS parameters Livy is looking for. 

Jobs are failing with below exception: 

>>>

*21/03/16 06:41:19 ERROR KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!*

21/03/16 06:41:19 WARN FileSystemStateStore: Failed to read v1/batch/5027 from state store.

*java.io.IOException: No KeyProvider is configured, cannot access an encrypted file*

at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1372)

<<<

This is a deprecated config and is now replaced by "hadoop.security.key.provider.path".

How do we handle this is Livy?  

Ref: https://issues.apache.org/jira/browse/HDFS-10489?jql=text%20~%20%22dfs.encryption.key.provider.uri%22



--
This message was sent by Atlassian Jira
(v8.3.4#803005)