[jira] [Commented] (HBASE-20797) hbase-spark erroneously sets authenticationMethod to PROXY

["Wei-Chiu Chuang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-20797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524970#comment-16524970 ] 

Wei-Chiu Chuang commented on HBASE-20797:
-----------------------------------------

Hi!
I'm not familiar with the development of the HBase-spark, but the KMSClientProvider code you pointed out actually has a bug, which is later fixed by HADOOP-13749 (KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used).

If your use case involves encrypted Hadoop cluster, I would strongly recommend you upgrade to Hadoop 2.8.x branches where numerous bugs were fixed.

> hbase-spark erroneously sets authenticationMethod to PROXY
> ----------------------------------------------------------
>
>                 Key: HBASE-20797
>                 URL: https://issues.apache.org/jira/browse/HBASE-20797
>             Project: HBase
>          Issue Type: Bug
>          Components: spark
>    Affects Versions: 3.0.0
>            Reporter: Lars Francke
>            Priority: Critical
>
> We're running into an issue using the spark integration when using Hadoop 2.7.2. The problem is this line of code from {{HBaseContext.scala}}
> {code:java}
> ugi.setAuthenticationMethod(AuthenticationMethod.PROXY)
> {code}
> I'm not an expert but I think that's wrong code. If we were to create a Proxy user then we'd need to use {{UserGroupInformation.createProxyUser(...) }} which would also set the realUser etc. Also: I don't think it makes sense to create a proxy user on the client side? The chances are good that the user we're authenticating as doesn't exen have proxy privileges as it's usually only granted to servers.
> We've tried to trace where this line of code came from in Git but it was a code drop back in Ted's original repo.
> The error we're seeing actually occurs when (in a Spark job) we access HDFS because KMSClientProvider has code like this:
> {code:java}
> actualUgi =
>     (UserGroupInformation.getCurrentUser().getAuthenticationMethod() ==
>     UserGroupInformation.AuthenticationMethod.PROXY) ? UserGroupInformation
>         .getCurrentUser().getRealUser() : UserGroupInformation
> {code}
> But we've never set up the realUser so actualUgi is null which later leads to a NullPointerException.
> I _think_ the proper fix is to just remove that line as I have no idea what its intention is. I can provide a patch but I'd like to get input first. Maybe I'm mistaken?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)