You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2015/10/26 21:15:28 UTC
[jira] [Commented] (AMBARI-13566) After upgrade Namenode fails to
start when Kerberos is enabled using HDP 2.2.8.0
[ https://issues.apache.org/jira/browse/AMBARI-13566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14974949#comment-14974949 ]
Hudson commented on AMBARI-13566:
---------------------------------
FAILURE: Integrated in Ambari-branch-2.1 #748 (See [https://builds.apache.org/job/Ambari-branch-2.1/748/])
AMBARI-13566. After upgrade Namenode fails to start when Kerberos is (aonishuk: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=e9ad2afea2de381414e4a19c733b69986f4f9d92])
* ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml
* ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java
* ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
* ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
* ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java
> After upgrade Namenode fails to start when Kerberos is enabled using HDP 2.2.8.0
> --------------------------------------------------------------------------------
>
> Key: AMBARI-13566
> URL: https://issues.apache.org/jira/browse/AMBARI-13566
> Project: Ambari
> Issue Type: Bug
> Reporter: Andrew Onischuk
> Assignee: Andrew Onischuk
> Fix For: 2.1.3
>
>
> After upgrading Ambari 1.7.0 (or 2.0.x) to Ambari 2.1.3 and then either
> enabling Kerberos or Kerberos was previously enabled, NameNode fails to start
> with the following error:
>
>
> 2015-10-16 13:46:04,499 ERROR namenode.NameNode (NameNode.java:main(1645)) - Failed to start namenode.
> java.io.IOException: Login failure for nn/localhost@EXAMPLE.COM from keytab /etc/security/keytabs/nn.service.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user
>
> at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:976)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:243)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser(NameNode.java:637)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:656)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:896)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:880)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1574)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1640)
> Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user
>
> at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:856)
> at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:719)
> at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
> at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
> at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:967)
> ... 7 more
> 2015-10-16 13:46:04,517 INFO util.ExitUtil (ExitUtil.java:terminate(124)) - Exiting with status 1
> 2015-10-16 13:46:04,532 INFO namenode.NameNode (LogAdapter.java:info(47)) - SHUTDOWN_MSG:
>
> I am using HDP 2.2.8.0-3150
>
>
> # hdp-select status
> accumulo-client - None
> accumulo-gc - None
> accumulo-master - None
> accumulo-monitor - None
> accumulo-tablet - None
> accumulo-tracer - None
> falcon-client - None
> falcon-server - None
> flume-server - None
> hadoop-client - 2.2.8.0-3150
> hadoop-hdfs-datanode - 2.2.8.0-3150
> hadoop-hdfs-journalnode - 2.2.8.0-3150
> hadoop-hdfs-namenode - 2.2.8.0-3150
> hadoop-hdfs-nfs3 - 2.2.8.0-3150
> hadoop-hdfs-portmap - 2.2.8.0-3150
> hadoop-hdfs-secondarynamenode - 2.2.8.0-3150
> hadoop-httpfs - 2.2.8.0-3150
> hadoop-mapreduce-historyserver - 2.2.8.0-3150
> hadoop-yarn-nodemanager - 2.2.8.0-3150
> hadoop-yarn-resourcemanager - 2.2.8.0-3150
> hadoop-yarn-timelineserver - 2.2.8.0-3150
> hbase-client - None
> hbase-master - None
> hbase-regionserver - None
> hive-metastore - None
> hive-server2 - None
> hive-webhcat - None
> kafka-broker - None
> knox-server - None
> mahout-client - None
> oozie-client - None
> oozie-server - None
> phoenix-client - None
> ranger-admin - None
> ranger-usersync - None
> slider-client - None
> spark-client - None
> spark-historyserver - None
> sqoop-client - None
> sqoop-server - None
> storm-client - None
> storm-nimbus - None
> storm-slider-client - None
> storm-supervisor - None
> zookeeper-client - 2.2.8.0-3150
> zookeeper-server - 2.2.8.0-3150
>
> This issue does not exist when using 2.3.4.0-3011.
> **Steps to reproduce #1**
> 1. Install Ambari 2.0.2 and HDP 2.2 (HDP-2.2.8.0)
> 2. Upgrade Ambari to 2.1.3
> 3. Restart HDFS
> 4. Enable Kerberos
> 5. See Failure
> **Steps to reproduce #2**
> 1. Install Ambari 2.0.2 and HDP 2.2 (HDP-2.2.8.0)
> 2. Enable Kerberos
> 3. Upgrade Ambari to 2.1.3
> 4. Restart HDFS
> 5. See Failure
> **Cause**
> In `org.apache.ambari.server.upgrade.UpgradeCatalog210#updateHdfsConfigs`,
> `dfs.namenode.rpc-address` is set to be updated to the proper value. However
> the call to `org.apache.ambari.server.upgrade.AbstractUpgradeCatalog#updateCon
> figurationPropertiesForCluster(org.apache.ambari.server.state.Cluster,
> java.lang.String, java.util.Map<java.lang.String,java.lang.String>, boolean,
> boolean)` is made with the `updateIfExists` flag set to **false**. Before
> getting to this point, new configs have been added from the hdfs-site.xml file
> via `org.apache.ambari.server.upgrade.AbstractUpgradeCatalog#addNewConfigurati
> onsFromXml`. This added `dfs.namenode.rpc-address` to the hdfs-site config
> with the value of "localhost:8020" and thus the calculated (correct) value was
> ignored.
> **Solution**
> Change the call to `org.apache.ambari.server.upgrade.AbstractUpgradeCatalog#up
> dateConfigurationPropertiesForCluster(org.apache.ambari.server.state.Cluster,
> java.lang.String, java.util.Map<java.lang.String,java.lang.String>, boolean,
> boolean)` so that it is made with the `updateIfExists` flag set to **true**.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)