[jira] [Comment Edited] (HIVE-16490) Hive should not use private HDFS APIs for encryption

["Uma Maheswara Rao G (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-16490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157145#comment-17157145 ] 

Uma Maheswara Rao G edited comment on HIVE-16490 at 7/14/20, 2:32 PM:
----------------------------------------------------------------------

Any update on this?

{code}
public HdfsEncryptionShim(URI uri, Configuration conf) throws IOException {
      DistributedFileSystem dfs = (DistributedFileSystem)FileSystem.get(uri, conf);

      this.conf = conf;
      this.keyProvider = dfs.getClient().getKeyProvider();
      this.hdfsAdmin = new HdfsAdmin(uri, conf);
    }
{code}
Looks like Hadoop23Shims.java is still using DFSClient api directly. HDFS-11687 added getKeyProvider in HdfsAdmin class. Please use it and can void DistributedFileSystem casting as well.
Casting and DFS checks already present in HdfsAdmin constructor, so let's not do here. 
Thanks


was (Author: umamaheswararao):
Any update on this?

{code}
public HdfsEncryptionShim(URI uri, Configuration conf) throws IOException {
      DistributedFileSystem dfs = (DistributedFileSystem)FileSystem.get(uri, conf);

      this.conf = conf;
      this.keyProvider = dfs.getClient().getKeyProvider();
      this.hdfsAdmin = new HdfsAdmin(uri, conf);
    }
{code}
Looks like Hadoop23Shims.java is still using DFSClient api directly. HDFS-11687 added getKeyProvider in HdfsAdmin class. Please use it and can void DistributedFileSystem casting as well.
Casting and DFS checks already present in HdfsAdmin constructor, so we don't to do here. 
Thanks

> Hive should not use private HDFS APIs for encryption
> ----------------------------------------------------
>
>                 Key: HIVE-16490
>                 URL: https://issues.apache.org/jira/browse/HIVE-16490
>             Project: Hive
>          Issue Type: Improvement
>          Components: Encryption
>    Affects Versions: 2.2.0
>            Reporter: Andrew Wang
>            Assignee: Naveen Gangam
>            Priority: Critical
>
> When compiling against bleeding edge versions of Hive and Hadoop, we discovered that HIVE-16047 references a private HDFS API, DFSClient, to get at various encryption related information. The private API was recently changed by HADOOP-14104, which broke Hive compilation.
> It'd be better to instead use publicly supported APIs. HDFS-11687 has been filed to add whatever encryption APIs are needed by Hive. This JIRA is to move Hive over to these new APIs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)