You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "zhouyifan279 (via GitHub)" <gi...@apache.org> on 2023/02/09 04:07:17 UTC
[GitHub] [kyuubi] zhouyifan279 opened a new issue, #4280: [Bug] ShutdownHook was not called when SparkSQLEngine exited gracefully
zhouyifan279 opened a new issue, #4280:
URL: https://github.com/apache/kyuubi/issues/4280
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
SparkSQLEngine did not call ShutdownHook when it was launched in connection share level and stopped gracefully when session was closed.
### Affects Version(s)
1.6.0/1.6.1
### Kyuubi Server Log Output
_No response_
### Kyuubi Engine Log Output
```logtalk
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkSQLSessionManager: Session stopped due to shared level is Connection.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkSQLEngine: Service: [SparkTBinaryFrontend] is stopping.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkTBinaryFrontendService: Service: [EngineServiceDiscovery] is stopping.
23/02/09 11:56:51 WARN [main-EventThread] ZookeeperDiscoveryClient: This Kyuubi instance 10.221.99.69:50439 is now de-registered from ZooKeeper. The server will be shut down after the last client session completes.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] EngineServiceDiscovery: Clean up discovery service due to this is connection share level.
23/02/09 11:56:51 INFO [Curator-Framework-0] CuratorFrameworkImpl: backgroundOperationsLoop exiting
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] ZooKeeper: Session: 0x10004e31d840004 closed
23/02/09 11:56:51 INFO [main-EventThread] ClientCnxn: EventThread shut down for session: 0x10004e31d840004
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] EngineServiceDiscovery: Service[EngineServiceDiscovery] is stopped.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkTBinaryFrontendService: Service[SparkTBinaryFrontend] is stopped.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkTBinaryFrontendService: SparkTBinaryFrontend has stopped
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkSQLEngine: Service: [SparkSQLBackendService] is stopping.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkSQLBackendService: Service: [SparkSQLSessionManager] is stopping.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkSQLSessionManager: Service: [SparkSQLOperationManager] is stopping.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkSQLOperationManager: Service[SparkSQLOperationManager] is stopped.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkSQLSessionManager: Service[SparkSQLSessionManager] is stopped.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkSQLBackendService: Service[SparkSQLBackendService] is stopped.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkSQLEngine: Service[SparkSQLEngine] is stopped.
23/02/09 11:56:51 INFO [SparkTBinaryFrontendHandler-Pool: Thread-79] SparkTBinaryFrontendService: Finished closing SessionHandle [b98cec81-6444-4803-8431-7dd6dc56a4e5]
23/02/09 11:56:51 INFO [main] SparkUI: Stopped Spark web UI at http://10.221.99.69:50428
```
### Kyuubi Server Configurations
_No response_
### Kyuubi Engine Configurations
_No response_
### Additional context
If SparkSQLEngine was killed by command, or was running in other share level, ShutdownHook was called.
```
23/02/09 11:58:26 ERROR [SIGTERM handler] SparkSQLEngine: RECEIVED SIGNAL 15: TERM
23/02/09 11:58:26 INFO [Thread-1] SparkSQLEngine: Service: [SparkTBinaryFrontend] is stopping.
23/02/09 11:58:26 INFO [Thread-1] SparkTBinaryFrontendService: Service: [EngineServiceDiscovery] is stopping.
23/02/09 11:58:26 INFO [Thread-1] EngineServiceDiscovery: Clean up discovery service due to this is connection share level.
23/02/09 11:58:26 WARN [main-EventThread] ZookeeperDiscoveryClient: This Kyuubi instance 10.221.99.69:50499 is now de-registered from ZooKeeper. The server will be shut down after the last client session completes.
23/02/09 11:58:26 INFO [Curator-Framework-0] CuratorFrameworkImpl: backgroundOperationsLoop exiting
23/02/09 11:58:26 INFO [main-EventThread] EngineServiceDiscovery: 1 connection(s) are active, delay shutdown
23/02/09 11:58:26 INFO [Thread-1] ZooKeeper: Session: 0x10004e31d840006 closed
23/02/09 11:58:26 INFO [Thread-1] EngineServiceDiscovery: Service[EngineServiceDiscovery] is stopped.
23/02/09 11:58:26 INFO [Thread-1] SparkTBinaryFrontendService: Service[SparkTBinaryFrontend] is stopped.
23/02/09 11:58:26 INFO [Thread-1] SparkTBinaryFrontendService: SparkTBinaryFrontend has stopped
23/02/09 11:58:26 INFO [Thread-1] SparkSQLEngine: Service: [SparkSQLBackendService] is stopping.
23/02/09 11:58:26 INFO [Thread-1] SparkSQLBackendService: Service: [SparkSQLSessionManager] is stopping.
23/02/09 11:58:26 INFO [Thread-1] SparkSQLSessionManager: Service: [SparkSQLOperationManager] is stopping.
23/02/09 11:58:26 INFO [Thread-1] SparkSQLOperationManager: Service[SparkSQLOperationManager] is stopped.
23/02/09 11:58:26 INFO [Thread-1] SparkSQLSessionManager: Service[SparkSQLSessionManager] is stopped.
23/02/09 11:58:26 INFO [Thread-1] SparkSQLBackendService: Service[SparkSQLBackendService] is stopped.
23/02/09 11:58:26 INFO [Thread-1] SparkSQLEngine: Service[SparkSQLEngine] is stopped.
23/02/09 11:58:26 INFO [Thread-1] DiskBlockManager: Shutdown hook called
23/02/09 11:58:26 INFO [main] SparkUI: Stopped Spark web UI at http://10.221.99.69:50487
23/02/09 11:58:26 INFO [Thread-1] ShutdownHookManager: Shutdown hook called
23/02/09 11:58:26 INFO [Thread-1] ShutdownHookManager: Deleting directory /private/var/folders/yj/25xqj6_52n51xmctftgl_77c0000gn/T/spark-92945c05-f7e3-4fae-86fe-f1a409f141fa
23/02/09 11:58:26 INFO [Thread-1] ShutdownHookManager: Deleting directory /private/var/folders/yj/25xqj6_52n51xmctftgl_77c0000gn/T/spark-dd00d8b3-d228-4750-83dd-191059e842a7
23/02/09 11:58:26 INFO [Thread-1] ShutdownHookManager: Deleting directory /private/var/folders/yj/25xqj6_52n51xmctftgl_77c0000gn/T/spark-92945c05-f7e3-4fae-86fe-f1a409f141fa/userFiles-676192b3-9da0-4bd0-8814-84d82abd1ad1
23/02/09 11:58:26 INFO [dispatcher-event-loop-3] MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
```
### Are you willing to submit PR?
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [ ] No. I cannot submit a PR at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org