You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Hari Sekhon (JIRA)" <ji...@apache.org> on 2014/12/18 13:12:13 UTC
[jira] [Updated] (AMBARI-8785) Resource Manager HA Kerberos
principal not handled and user not notified of requirement
[ https://issues.apache.org/jira/browse/AMBARI-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Sekhon updated AMBARI-8785:
--------------------------------
Summary: Resource Manager HA Kerberos principal not handled and user not notified of requirement (was: Resource Manager HA Kerberos principal not handled or even notified)
> Resource Manager HA Kerberos principal not handled and user not notified of requirement
> ---------------------------------------------------------------------------------------
>
> Key: AMBARI-8785
> URL: https://issues.apache.org/jira/browse/AMBARI-8785
> Project: Ambari
> Issue Type: Bug
> Affects Versions: 1.7.0
> Environment: HDP 2.1
> Reporter: Hari Sekhon
>
> When enabling Yarn Resource Manager HA in a Kerberos secured cluster, Ambari fails to tell the user about the required Kerberos principal + keytab for the new Resource Manager instance.
> As as result the new Resource Manager fails to start with the following logs:
> {code}2014-12-18 11:39:06,379 FATAL resourcemanager.ResourceManager (ResourceManager.java:main(1043)) - Error starting ResourceManager
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to login
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:910)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1041)
> Caused by: java.io.IOException: Login failure for rm/<host>@REALM from keytab /etc/security/keytabs/rm.service.keytab
> at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:920)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:242)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.doSecureLogin(ResourceManager.java:929)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:908)
> ... 2 more
> Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user
> at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:856)
> at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:719)
> at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
> at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
> at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:911)
> ... 5 more
> 2014-12-18 11:39:06,383 INFO resourcemanager.ResourceManager (StringUtils.java:run(640)) - SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down ResourceManager at <host>/x.x.x.x
> ************************************************************/
> {code}
> The fix is quite simple for experienced cluster administrators, create the principal, export the keytab, mimic the original RM permissions of yarn:hadoop 620 (not really correct octal but that's a separate issue I've already raised to fix keytab perms AMBARI-8138) and then it will start up.
> This is sort of related to AMBARI-8610 where Ambari should export a CSV for new hosts/services to allow existing automation scripts to generate the principals and and distributed the keytabs such as the basic kerberos_setup.sh shipped with Ambari or the more real world FreeIPA one I publish on my github (https://github.com/harisekhon/toolbox).
> Except in this instance it's not a new service, it's just HA enablement where it doesn't notify users that RM HA will require a new Keberos principal + keytab to allow them to do that, never mind provide a subset CSV for passing to a kerberos setup script. This issue will also affect NN HA if Ambari ever allows setting up NN HA without disabling Kerberos first.
> Regards,
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)