[jira] [Created] (AMBARI-25174) OneFS Mpack missing configuration settings needed for Spark2

["Bjorn Olsen (JIRA)" <ji...@apache.org>]

Bjorn Olsen created AMBARI-25174:
------------------------------------

             Summary: OneFS Mpack missing configuration settings needed for Spark2
                 Key: AMBARI-25174
                 URL: https://issues.apache.org/jira/browse/AMBARI-25174
             Project: Ambari
          Issue Type: Bug
          Components: ambari-server
    Affects Versions: 2.7.3
         Environment: HDP 3.1.1 using OneFS instead of HDFS

Isilon 8.2.1.0 (simulator)

isilon-onefs-mpack-1.0.0.0
            Reporter: Bjorn Olsen


The OneFS management pack does not include the below HDFS configurations/parameters. If using the OneFS mpack as a replacement for the HDFS service, these parameters no longer exist. 

{{dfs.datanode.kerberos.principal=hdfs/_HOST@<MY REALM> dfs.datanode.keytab.file=/etc/security/keytabs/hdfs.service.keytab dfs.namenode.kerberos.principal=hdfs/_HOST@<MY REALM> dfs.namenode.keytab.file=/etc/security/keytabs/hdfs.service.keytab }}


These are required by Spark2 in a Kerberised environment, and Spark2 will fail with obscure errors "Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException: 
Failed to specify server's Kerberos principal name"
when these parameters do not exist.


See my answer on https://stackoverflow.com/questions/35325720/connecting-to-kerberrized-hdfs-java-lang-illegalargumentexception-failed-to-s/54926487



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)