You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Bjorn Olsen (JIRA)" <ji...@apache.org> on 2019/02/28 13:40:00 UTC
[jira] [Updated] (AMBARI-25174) OneFS Mpack missing configuration
settings needed for Spark2
[ https://issues.apache.org/jira/browse/AMBARI-25174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bjorn Olsen updated AMBARI-25174:
---------------------------------
Description:
The OneFS management pack does not include the below HDFS configurations/parameters. If using the OneFS mpack as a replacement for the HDFS service, these parameters no longer exist.
dfs.datanode.kerberos.principal=hdfs/_HOST@<MY REALM>
dfs.datanode.keytab.file=/etc/security/keytabs/hdfs.service.keytab
dfs.namenode.kerberos.principal=hdfs/_HOST@<MY REALM>
dfs.namenode.keytab.file=/etc/security/keytabs/hdfs.service.keytab
These are required by Spark2 in a Kerberised environment, and starting the Spark2 History Server will otherwise fail with obscure errors such as "Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException:
Failed to specify server's Kerberos principal name"
See my answer on [https://stackoverflow.com/questions/35325720/connecting-to-kerberrized-hdfs-java-lang-illegalargumentexception-failed-to-s/54926487]
was:
The OneFS management pack does not include the below HDFS configurations/parameters. If using the OneFS mpack as a replacement for the HDFS service, these parameters no longer exist.
{{dfs.datanode.kerberos.principal=hdfs/_HOST@<MY REALM> dfs.datanode.keytab.file=/etc/security/keytabs/hdfs.service.keytab dfs.namenode.kerberos.principal=hdfs/_HOST@<MY REALM> dfs.namenode.keytab.file=/etc/security/keytabs/hdfs.service.keytab }}
These are required by Spark2 in a Kerberised environment, and Spark2 will fail with obscure errors "Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException:
Failed to specify server's Kerberos principal name"
when these parameters do not exist.
See my answer on https://stackoverflow.com/questions/35325720/connecting-to-kerberrized-hdfs-java-lang-illegalargumentexception-failed-to-s/54926487
> OneFS Mpack missing configuration settings needed for Spark2
> ------------------------------------------------------------
>
> Key: AMBARI-25174
> URL: https://issues.apache.org/jira/browse/AMBARI-25174
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.7.3
> Environment: HDP 3.1.1 using OneFS instead of HDFS
> Isilon 8.2.1.0 (simulator)
> isilon-onefs-mpack-1.0.0.0
> Reporter: Bjorn Olsen
> Priority: Major
>
> The OneFS management pack does not include the below HDFS configurations/parameters. If using the OneFS mpack as a replacement for the HDFS service, these parameters no longer exist.
> dfs.datanode.kerberos.principal=hdfs/_HOST@<MY REALM>
> dfs.datanode.keytab.file=/etc/security/keytabs/hdfs.service.keytab
> dfs.namenode.kerberos.principal=hdfs/_HOST@<MY REALM>
> dfs.namenode.keytab.file=/etc/security/keytabs/hdfs.service.keytab
> These are required by Spark2 in a Kerberised environment, and starting the Spark2 History Server will otherwise fail with obscure errors such as "Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException:
> Failed to specify server's Kerberos principal name"
> See my answer on [https://stackoverflow.com/questions/35325720/connecting-to-kerberrized-hdfs-java-lang-illegalargumentexception-failed-to-s/54926487]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)