[jira] [Created] (HAWQ-1790) Kerberos HTTP SPNEGO Authentication not supported in LibHdfs3

["Krishna Gundamaraju (Jira)" <ji...@apache.org>]

Krishna Gundamaraju created HAWQ-1790:
-----------------------------------------

             Summary: Kerberos HTTP SPNEGO Authentication not supported in LibHdfs3
                 Key: HAWQ-1790
                 URL: https://issues.apache.org/jira/browse/HAWQ-1790
             Project: Apache HAWQ
          Issue Type: Bug
          Components: libhdfs
            Reporter: Krishna Gundamaraju
            Assignee: Ruilong Huo


Hi, I have integrated the latest LibHdfs3 from the git hub repo at [https://github.com/apache/hawq/tree/master/depends/libhdfs3] with my application that is expected to work like a HDFS Client.

I have verified that the following use cases work in my test setup:

1) Transparent Data Encryption (TDE) works when I configure the Hadoop cluster and KMS to use simple authentication. My application is able to both read files under an encryption zone and create and write to new files under an encryption zone. So all of the interactions between my application and the Hadoop KMS work as expected.

2) Non TDE use cases with a kerberized Hadoop cluster work as well. My application can successfully authenticate itself with the Hadoop cluster that is configured to use Kerberos. It can read and write files from this Hadoop cluster.

What doesn't work is when my application tries to read files under an encryption zone from a Hadoop cluster that is configured to use Kerberos authentication. I have created a HTTP service principal on the KDC and generated a keytab and installed it on the Linux host where my application runs. I have verified that using this keytab file my application is able to successfully get a TGT from the KDC.

 

  I stepped through the LibHdfs3 code and I see that the 

KmsClientProvider::buildKmsUrl() function throws the following exception when Authentication method is set to Kerberos.

 

 **     if (method == AuthMethod::KERBEROS) {

  **          // todo

  **          THROW(InvalidParameter, "KmsClientProvider : Not support kerberos yet.");

  **      } else if (method == AuthMethod::SIMPLE) {

 

My question is about whether LibHdfs3 supports Kerberos HTTP SPNEGO Authentication or not? If the answer is yes, then can you please help me in debugging this issue by pointing me to any relevant literature/documentation or by providing any other hints on what I could be missing? I can provide pcaps that show the packets exchanged between my application and the Hadoop KMS and I can also provide pcaps that show the packets exchanged between my application and the Hadoop NameNode.

 

Thanks in advance

Krishna



--
This message was sent by Atlassian Jira
(v8.3.4#803005)