You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Naveen Gangam (Jira)" <ji...@apache.org> on 2023/09/15 14:40:00 UTC
[jira] [Resolved] (HIVE-27675) Support keystore/truststore types for hive to zookeeper integration points
[ https://issues.apache.org/jira/browse/HIVE-27675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Naveen Gangam resolved HIVE-27675.
----------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Fix has been merged to master. Thank you for the review
> Support keystore/truststore types for hive to zookeeper integration points
> --------------------------------------------------------------------------
>
> Key: HIVE-27675
> URL: https://issues.apache.org/jira/browse/HIVE-27675
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, JDBC, Standalone Metastore
> Affects Versions: 3.1.0
> Reporter: Naveen Gangam
> Assignee: Naveen Gangam
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> In HIVE-24253, we added support for HS2/HMS/JDBC DRiver to support other store types like BCFKS (other than JKS). This allows JDBC Clients to connect to HS2 directly. However, with service discovery enabled, the clients have to connect zookeeper to determine HS2 endpoints. This connectivity currently does not support other store types. Similarly, HS2/HMS services also do not provide ability to use different store types for the zk registration process.
> {noformat}
> $ beeline
> Connecting to jdbc:hive2://<snip>:2181/default;httpPath=cliservice;principal=hive/_HOST@<SNIP>;retries=5;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;transportMode=http;trustStorePassword=RoeCFK11Pq54;trustStoreType=bcfks;zooKeeperNamespace=hiveserver2
> Error: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper (state=,code=0)
> {noformat}
> {noformat}
> Opening socket connection to server <SNIP>:2182. Will attempt to SASL-authenticate using Login Context section 'HiveZooKeeperClient'
> 2023-08-09 13:28:07,591 WARN io.netty.channel.ChannelInitializer: [nioEventLoopGroup-3-1]: Failed to initialize a channel. Closing: [id: 0x0937583f]
> org.apache.zookeeper.common.X509Exception$SSLContextException: Failed to create KeyManager
> at org.apache.zookeeper.common.X509Util.createSSLContextAndOptions(X509Util.java:346) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> at org.apache.zookeeper.common.X509Util.createSSLContext(X509Util.java:278) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initSSL(ClientCnxnSocketNetty.java:454) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:444) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:429) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> at io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:1114) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.DefaultChannelPipeline.access$100(DefaultChannelPipeline.java:46) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1463) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1115) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:650) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:514) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) [netty-common-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) [netty-common-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) [netty-common-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.86.Final.jar:4.1.86.Final]
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.86.Final.jar:4.1.86.Final]
> at java.lang.Thread.run(Thread.java:750) [?:1.8.0_382]
> Caused by: org.apache.zookeeper.common.X509Exception$KeyManagerException: java.io.IOException: Invalid keystore format
> at org.apache.zookeeper.common.X509Util.createKeyManager(X509Util.java:471) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> at org.apache.zookeeper.common.X509Util.createSSLContextAndOptions(X509Util.java:344) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> ... 23 more
> Caused by: java.io.IOException: Invalid keystore format
> at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:666) ~[?:1.8.0_382]
> at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:57) ~[?:1.8.0_382]
> at sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:224) ~[?:1.8.0_382]
> at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:71) ~[?:1.8.0_382]
> at java.security.KeyStore.load(KeyStore.java:1445) ~[?:1.8.0_382]
> at org.apache.zookeeper.common.StandardTypeFileKeyStoreLoader.loadKeyStore(StandardTypeFileKeyStoreLoader.java:54) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> at org.apache.zookeeper.common.X509Util.loadKeyStore(X509Util.java:400) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> at org.apache.zookeeper.common.X509Util.createKeyManager(X509Util.java:460) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> at org.apache.zookeeper.common.X509Util.createSSLContextAndOptions(X509Util.java:344) ~[zookeeper-3.5.5.7.2.16.300-7.jar:3.5.5.7.2.16.300-7]
> ... 23 more
> 2023-08-09 13:28:07,591 INFO org.apache.zookeeper.ClientCnxnSocketNetty: [nioEventLoopGroup-3-1]: future isn't success, cause:
> io.netty.channel.StacklessClosedChannelException: null
> at io.netty.channel.AbstractChannel$AbstractUnsafe.ensureOpen(ChannelPromise)(Unknown Source) ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)