You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/04/26 09:23:21 UTC
[GitHub] [incubator-kyuubi] turboFei opened a new issue, #2483: [Bug] Kyuubi engine does not got killed on connection failure
turboFei opened a new issue, #2483:
URL: https://github.com/apache/incubator-kyuubi/issues/2483
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
I met issue when open kyuubi connection with CONNECTION share level.
But the backend engine does not got killed in this case.
```
01:48:38.480 INFO org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient: Get service instance:hdc42-mcc10-01-0710-3501-027-tess0035:33563 and version:Some(1.6.0.1.2.0) under /kyuubistaging_1.6.0.1.2.0_CONNECTION_SPARK_SQL/b_stf/a9863889-86b0-4753-876c-31dd4d9a02e4
01:48:38.481 INFO org.apache.kyuubi.engine.ProcBuilder: Destroy the process, since waitCompletion is false.
01:48:38.520 ERROR org.apache.kyuubi.session.KyuubiSessionImpl: Opening engine [kyuubi_apollorno_CONNECTION_SPARK_SQL_b_stf_a9863889-86b0-4753-876c-31dd4d9a02e4 hdc42-mcc10-01-0710-3501-027-tess0035:33563] for b_stf session failed
org.apache.thrift.transport.TTransportException: java.net.UnknownHostException: hdc42-mcc10-01-0710-3501-027-tess0035
at org.apache.thrift.transport.TSocket.open(TSocket.java:226) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.kyuubi.client.KyuubiSyncThriftClient$.createTProtocol(KyuubiSyncThriftClient.scala:396) ~[kyuubi-server_2.12-1.6.0.1.2.0.jar:1.6.0.1.2.0]
at org.apache.kyuubi.client.KyuubiSyncThriftClient$.createClient(KyuubiSyncThriftClient.scala:414) ~[kyuubi-server_2.12-1.6.0.1.2.0.jar:1.6.0.1.2.0]
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1(KyuubiSessionImpl.scala:145) ~[kyuubi-server_2.12-1.6.0.1.2.0.jar:1.6.0.1.2.0]
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1$adapted(KyuubiSessionImpl.scala:136) ~[kyuubi-server_2.12-1.6.0.1.2.0.jar:1.6.0.1.2.0]
at org.apache.kyuubi.ha.client.DiscoveryClientProvider$.withDiscoveryClient(DiscoveryClientProvider.scala:35) ~[kyuubi-ha_2.12-1.6.0.1.2.0.jar:1.6.0.1.2.0]
at org.apache.kyuubi.session.KyuubiSessionImpl.openEngineSession(KyuubiSessionImpl.scala:136) ~[kyuubi-server_2.12-1.6.0.1.2.0.jar:1.6.0.1.2.0]
at org.apache.kyuubi.operation.LaunchEngine.$anonfun$runInternal$1(LaunchEngine.scala:61) ~[kyuubi-server_2.12-1.6.0.1.2.0.jar:1.6.0.1.2.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_202]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_202]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_202]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_202]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_202]
Caused by: java.net.UnknownHostException: hdc42-mcc10-01-0710-3501-027-tess0035
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) ~[?:1.8.0_202]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_202]
at java.net.Socket.connect(Socket.java:589) ~[?:1.8.0_202]
at org.apache.thrift.transport.TSocket.open(TSocket.java:221) ~[libthrift-0.9.3.jar:0.9.3]
... 14 more
01:48:38.521 INFO org.apache.curator.framework.imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting
01:48:38.539 INFO org.apache.zookeeper.ZooKeeper: Session: 0x57a610ea0e501c4 closed
01:48:38.539 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down for session: 0x57a610ea0e501c4
01:48:38.540 INFO org.apache.kyuubi.operation.LaunchEngine: Processing b_stf's query[a1413996-160d-4428-a815-d8e3687cbd46]: RUNNING_STATE -> ERROR_STATE, statement: LAUNCH_ENGINE, time taken: 66.371 seconds
22/04/26 01:48:37 INFO Client: Application report for application_1650420485508_475368 (state: ACCEPTED)
Error: org.apache.kyuubi.KyuubiSQLException: Error operating UNKNOWN_OPERATION: org.apache.thrift.transport.TTransportException: java.net.UnknownHostException: hdc42-mcc10-01-0710-3501-027-tess0035
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.kyuubi.client.KyuubiSyncThriftClient$.createTProtocol(KyuubiSyncThriftClient.scala:396)
at org.apache.kyuubi.client.KyuubiSyncThriftClient$.createClient(KyuubiSyncThriftClient.scala:414)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1(KyuubiSessionImpl.scala:145)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1$adapted(KyuubiSessionImpl.scala:136)
at org.apache.kyuubi.ha.client.DiscoveryClientProvider$.withDiscoveryClient(DiscoveryClientProvider.scala:35)
at org.apache.kyuubi.session.KyuubiSessionImpl.openEngineSession(KyuubiSessionImpl.scala:136)
at org.apache.kyuubi.operation.LaunchEngine.$anonfun$runInternal$1(LaunchEngine.scala:61)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.UnknownHostException: hdc42-mcc10-01-0710-3501-027-tess0035
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
... 14 more
at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69)
at org.apache.kyuubi.operation.KyuubiOperation$$anonfun$onError$1.applyOrElse(KyuubiOperation.scala:87)
at org.apache.kyuubi.operation.KyuubiOperation$$anonfun$onError$1.applyOrElse(KyuubiOperation.scala:68)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38)
at org.apache.kyuubi.operation.LaunchEngine.$anonfun$runInternal$1(LaunchEngine.scala:65)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException: java.net.UnknownHostException: hdc42-mcc10-01-0710-3501-027-tess0035
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.kyuubi.client.KyuubiSyncThriftClient$.createTProtocol(KyuubiSyncThriftClient.scala:396)
at org.apache.kyuubi.client.KyuubiSyncThriftClient$.createClient(KyuubiSyncThriftClient.scala:414)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1(KyuubiSessionImpl.scala:145)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1$adapted(KyuubiSessionImpl.scala:136)
at org.apache.kyuubi.ha.client.DiscoveryClientProvider$.withDiscoveryClient(DiscoveryClientProvider.scala:35)
at org.apache.kyuubi.session.KyuubiSessionImpl.openEngineSession(KyuubiSessionImpl.scala:136)
at org.apache.kyuubi.operation.LaunchEngine.$anonfun$runInternal$1(LaunchEngine.scala:61)
... 5 more
Caused by: java.net.UnknownHostException: hdc42-mcc10-01-0710-3501-027-tess0035
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
... 14 more (state=,code=0)
```
BTW, after met this issue, I set `kyuubi.engine.connection.url.use.hostname` to false to fix it.
### Affects Version(s)
1.6.0-SNAPSHOT
### Kyuubi Server Log Output
_No response_
### Kyuubi Engine Log Output
_No response_
### Kyuubi Server Configurations
_No response_
### Kyuubi Engine Configurations
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn closed issue #2483: [Bug] Kyuubi engine does not got killed on connection failure
Posted by GitBox <gi...@apache.org>.
yaooqinn closed issue #2483: [Bug] Kyuubi engine does not got killed on connection failure
URL: https://github.com/apache/incubator-kyuubi/issues/2483
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #2483: [Bug] Kyuubi engine does not got killed on connection failure
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #2483:
URL: https://github.com/apache/incubator-kyuubi/issues/2483#issuecomment-1109565126
#2419
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org