You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Andras Piros (JIRA)" <ji...@apache.org> on 2018/08/24 15:09:00 UTC
[jira] [Updated] (OOZIE-2809) Oozie HA with secure zookeeper access
fails because ZKUtils#setJaasConfiguration does not replace "_HOST" in
principal string
[ https://issues.apache.org/jira/browse/OOZIE-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andras Piros updated OOZIE-2809:
--------------------------------
Fix Version/s: (was: 5.1.0)
5.2.0
> Oozie HA with secure zookeeper access fails because ZKUtils#setJaasConfiguration does not replace "_HOST" in principal string
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: OOZIE-2809
> URL: https://issues.apache.org/jira/browse/OOZIE-2809
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: trunk, 4.2.0, 4.3.0
> Reporter: Thomas Larsson
> Priority: Major
> Fix For: 5.2.0
>
>
> Setting "oozie.zookeeper.secure" = true causes the oozie server not to start if "oozie.service.HadoopAccessorService.kerberos.principal" is defined with a "_HOST" token instead of an actual hostname.
> The symptom when this happens is that the oozie log shows something like this:
> {noformat}
> org.apache.oozie.service.ServiceException: E1700: Issue communicating with ZooKeeper: KeeperErrorCode = NoAuth for /oozie/services/servers
> at org.apache.oozie.service.ZKLocksService.init(ZKLocksService.java:76)
> at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
> at org.apache.oozie.service.Services.setService(Services.java:372)
> at org.apache.oozie.service.Services.loadServices(Services.java:305)
> at org.apache.oozie.service.Services.init(Services.java:213)
> at org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
> at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> at org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:802)
> at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
> at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:676)
> at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:602)
> at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:503)
> at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)
> at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)
> at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1068)
> at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
> at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1060)
> at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
> at org.apache.catalina.core.StandardService.start(StandardService.java:525)
> at org.apache.catalina.core.StandardServer.start(StandardServer.java:759)
> at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> Caused by: org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /oozie/services/servers
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
> at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:199)
> at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:682)
> at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:660)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
> at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:656)
> at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:441)
> at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:431)
> at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44)
> at org.apache.curator.x.discovery.details.ServiceDiscoveryImpl.internalRegisterService(ServiceDiscoveryImpl.java:176)
> at org.apache.curator.x.discovery.details.ServiceDiscoveryImpl.registerService(ServiceDiscoveryImpl.java:150)
> at org.apache.oozie.util.ZKUtils.advertiseService(ZKUtils.java:217)
> at org.apache.oozie.util.ZKUtils.<init>(ZKUtils.java:141)
> at org.apache.oozie.util.ZKUtils.register(ZKUtils.java:154)
> at org.apache.oozie.service.ZKLocksService.init(ZKLocksService.java:70)
> ... 29 more
> 2017-02-24 14:03:23,683 INFO Services:520 - SERVER[datavault-dev-app1.internal.machines] Shutdown
> {noformat}
> This seems to come from the class [org.apache.oozie.util.ZKUtils|https://git-wip-us.apache.org/repos/asf?p=oozie.git;a=blob;f=core/src/main/java/org/apache/oozie/util/ZKUtils.java;h=5835fb2bd7460809743d21ac16f2992c353ada71;hb=HEAD#l357]
> {code}
> private void setJaasConfiguration() throws ServiceException, IOException {
> String keytabFile = Services.get().getConf().get(KERBEROS_KEYTAB, System.getProperty("user.home") + "/oozie.keytab").trim();
> if (keytabFile.length() == 0) {
> throw new ServiceException(ErrorCode.E0026, KERBEROS_KEYTAB);
> }
> String principal = Services.get().getConf().get(KERBEROS_PRINCIPAL, "oozie/localhost@LOCALHOST");
> if (principal.length() == 0) {
> throw new ServiceException(ErrorCode.E0026, KERBEROS_PRINCIPAL);
> }
> // This is equivalent to writing a jaas.conf file and setting the system property, "java.security.auth.login.config", to
> // point to it (but this way we don't have to write a file, and it works better for the tests)
> JaasConfiguration.addEntry("Client", principal, keytabFile);
> Configuration.setConfiguration(JaasConfiguration.getInstance());
> }
> {code}
> Compare how the principal is set without any string replacement with how it is done in class [org.apache.oozie.service.HadoopAccessorService|https://git-wip-us.apache.org/repos/asf?p=oozie.git;a=blob;f=core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java;h=5377127736cf189bcf4aa3d62bbbeab6b3a29d52;hb=HEAD#l226]
> {code}
> private void kerberosInit(Configuration serviceConf) throws ServiceException {
> try {
> String keytabFile = ConfigurationService.get(serviceConf, KERBEROS_KEYTAB).trim();
> if (keytabFile.length() == 0) {
> throw new ServiceException(ErrorCode.E0026, KERBEROS_KEYTAB);
> }
> String principal = SecurityUtil.getServerPrincipal(
> serviceConf.get(KERBEROS_PRINCIPAL, "oozie/localhost@LOCALHOST"),
> InetAddress.getLocalHost().getCanonicalHostName());
> if (principal.length() == 0) {
> throw new ServiceException(ErrorCode.E0026, KERBEROS_PRINCIPAL);
> }
> Configuration conf = new Configuration();
> conf.set("hadoop.security.authentication", "kerberos");
> UserGroupInformation.setConfiguration(conf);
> UserGroupInformation.loginUserFromKeytab(principal, keytabFile);
> LOG.info("Got Kerberos ticket, keytab [{0}], Oozie principal principal [{1}]",
> keytabFile, principal);
> }
> catch (ServiceException ex) {
> throw ex;
> }
> catch (Exception ex) {
> throw new ServiceException(ErrorCode.E0100, getClass().getName(), ex.getMessage(), ex);
> }
> }
> {code}
> In the latter case, the SecurityUtil class is used to replace "_HOST" occurrences with an actual hostname.
> I guess something similar should be done in the ZKUtils class as well.
> Best Regards
> /Thomas
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)