You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "lordk911 (via GitHub)" <gi...@apache.org> on 2023/04/03 03:20:26 UTC
[GitHub] [kyuubi] lordk911 opened a new issue, #4591: [Bug] Spark connection level engines didn't be closed after all job finished
lordk911 opened a new issue, #4591:
URL: https://github.com/apache/kyuubi/issues/4591
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
Env is HDP3.1.4, Apache spark3.2.3, Kyuubi 1.7.0
Some ETL task didn't finish after all its sql statments finish. I'm using connection level spark engine, I found that the application on yarn didn't go to SUCCEEDED state until kyuubi.session.idle.timeout happen.
Especially，I found a task only have a `create table if not exists` statment ，the statment execute cost few millisecond, but the yarn application keep alive 2 hour.
please use `ecaced83-15d4-4ac6-9095-9e57bfa9e353` as key to search the log.
### Affects Version(s)
1.7.0
### Kyuubi Server Log Output
```logtalk
cat kyuubi-bigtop-org.apache.kyuubi.server.KyuubiServer-worker-10-0-xxx-24.out | grep ecaced83-15d4-4ac6-9095-9e57bfa9e353
2023-03-24 01:33:04.942 INFO org.apache.kyuubi.operation.log.OperationLog: Creating operation log file /data/soft/kyuubi/apache-kyuubi-1.7.0-bin/work/server_operation_logs/ecaced83-15d4-4ac6-9095-9e57bfa9e353/4846034a-c3d8-42b2-9e2b-d44061b70b61
2023-03-24 01:33:04.942 INFO org.apache.kyuubi.session.KyuubiSessionManager: x's session with SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] is opened, current opening sessions 36
--conf spark.kyuubi.ha.engine.ref.id=ecaced83-15d4-4ac6-9095-9e57bfa9e353 \
--conf spark.kyuubi.ha.namespace=/kyuubi_1.7.0_CONNECTION_SPARK_SQL/x/ecaced83-15d4-4ac6-9095-9e57bfa9e353 \
--conf spark.app.name=kyuubi_CONNECTION_SPARK_SQL_x_ecaced83-15d4-4ac6-9095-9e57bfa9e353 \
--conf spark.yarn.tags=KYUUBI,ecaced83-15d4-4ac6-9095-9e57bfa9e353 \
2023-03-24 01:33:05.009 INFO org.apache.kyuubi.session.KyuubiSessionImpl: [x:10.0.x.x] SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] - Starting to wait the launch engine operation finished
2023-03-24 01:33:26.656 INFO org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient: Get service instance:worker-10-0-161-23:46425 engine id:application_1677295542968_70353 and version:1.7.0 under /kyuubi_1.7.0_CONNECTION_SPARK_SQL/x/ecaced83-15d4-4ac6-9095-9e57bfa9e353
2023-03-24 01:33:26.945 INFO org.apache.kyuubi.session.KyuubiSessionImpl: [x:10.0.x.x] SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] - Connected to engine [worker-10-0-161-23:46425]/[application_1677295542968_70353] with SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]]
2023-03-24 01:33:26.948 INFO org.apache.kyuubi.session.KyuubiSessionImpl: [x:10.0.x.x] SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] - Engine has been launched, elapsed time: 21 s
2023-03-24 01:33:27.253 INFO org.apache.kyuubi.operation.log.OperationLog: Creating operation log file /data/soft/kyuubi/apache-kyuubi-1.7.0-bin/work/server_operation_logs/ecaced83-15d4-4ac6-9095-9e57bfa9e353/1baf73fb-53ec-4d60-85bf-7decf558bea4
2023-03-24 03:33:17.848 INFO org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Received request of closing SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]
2023-03-24 03:33:17.848 INFO org.apache.kyuubi.session.KyuubiSessionManager: x's session with SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] is closed, current opening sessions 37
2023-03-24 03:33:20.116 INFO org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Finished closing SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]
```
### Kyuubi Engine Log Output
```logtalk
23/03/24 01:33:27 INFO operation.ExecuteStatement:
Spark application name: kyuubi_CONNECTION_SPARK_SQL_x_ecaced83-15d4-4ac6-9095-9e57bfa9e353
application ID: application_1677295542968_70353
application web UI: http://master-10-0-x-x:8088/proxy/application_1677295542968_70353,http://master-10-0-x-x:8088/proxy/application_1677295542968_70353
master: yarn
deploy mode: cluster
version: 3.2.3
Start time: 2023-03-24T01:33:11.627
User: x
23/03/24 01:33:27 WARN conf.HiveConf: HiveConf of name hive.metastore.uri.selection does not exist
23/03/24 01:33:27 WARN conf.HiveConf: HiveConf of name hive.tez.cartesian-product.enabled does not exist
23/03/24 01:33:27 INFO sqlstd.SQLStdHiveAccessController: Created SQLStdHiveAccessController for session context : HiveAuthzSessionContext [sessionString=2e31a169-63e2-48d2-b061-a644e26b5330, clientType=HIVECLI]
23/03/24 01:33:27 WARN session.SessionState: METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Mestastore configuration metastore.filter.hook changed from org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Closed a connection to metastore, current connections: 0
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Trying to connect to metastore with URI thrift://worker-10-0-161-20:9083
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Opened a connection to metastore, current connections: 1
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Connected to metastore.
23/03/24 01:33:27 INFO metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=x (auth:SIMPLE) retries=1 delay=5 lifetime=0
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Trying to connect to metastore with URI thrift://worker-10-0-161-20:9083
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Opened a connection to metastore, current connections: 2
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Connected to metastore.
23/03/24 01:33:27 INFO metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=x (auth:SIMPLE) retries=1 delay=5 lifetime=0
23/03/24 01:33:27 INFO operation.ExecuteStatement: Execute in full collect mode
23/03/24 01:33:28 INFO operation.ExecuteStatement: Processing x's query[7dcbddbb-da82-42e6-9c08-033acf2d2b58]: RUNNING_STATE -> FINISHED_STATE, time taken: 0.676 seconds
23/03/24 01:33:56 INFO cluster.YarnClusterSchedulerBackend: Requesting to kill executor(s) 1
23/03/24 01:33:56 INFO cluster.YarnClusterSchedulerBackend: Actual list of executor(s) to be killed is 1
23/03/24 01:33:56 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 1.
23/03/24 01:33:56 INFO yarn.YarnAllocator: Resource profile 0 doesn't exist, adding it
23/03/24 01:33:56 INFO spark.ExecutorAllocationManager: Executors 1 removed due to idle timeout.
23/03/24 01:33:56 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Disabling executor 1.
23/03/24 01:33:56 INFO scheduler.DAGScheduler: Executor lost: 1 (epoch 0)
23/03/24 01:33:56 INFO cluster.YarnClusterScheduler: Executor 1 on worker-10-0-161-22 killed by driver.
23/03/24 01:33:56 INFO dynalloc.ExecutorMonitor: Executor 1 is removed. Remove reason statistics: (gracefully decommissioned: 0, decommision unfinished: 0, driver killed: 1, unexpectedly exited: 0).
23/03/24 01:33:56 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 1 from BlockManagerMaster.
23/03/24 01:33:56 INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(1, worker-10-0-161-22, 35597, None)
23/03/24 01:33:56 INFO storage.BlockManagerMaster: Removed 1 successfully in removeExecutor
23/03/24 01:37:13 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on worker-10-0-161-23:46596 in memory (size: 3.5 KiB, free: 2004.6 MiB)
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: Received request of closing SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]
23/03/24 03:33:20 INFO session.SparkSQLSessionManager: x's session with SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] is closed, current opening sessions 0
23/03/24 03:33:20 INFO session.SparkSQLSessionManager: Session stopped due to shared level is Connection.
23/03/24 03:33:20 INFO spark.SparkSQLEngine: Service: [SparkTBinaryFrontend] is stopping.
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: Service: [EngineServiceDiscovery] is stopping.
23/03/24 03:33:20 WARN zookeeper.ZookeeperDiscoveryClient: This Kyuubi instance worker-10-0-161-23:46425 is now de-registered from ZooKeeper. The server will be shut down after the last client session completes.
23/03/24 03:33:20 INFO client.EngineServiceDiscovery: Clean up discovery service due to this is connection share level.
23/03/24 03:33:20 INFO imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting
23/03/24 03:33:20 INFO zookeeper.ZooKeeper: Session: 0x38377a2b5e9d1df closed
23/03/24 03:33:20 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0x38377a2b5e9d1df
23/03/24 03:33:20 INFO client.EngineServiceDiscovery: Service[EngineServiceDiscovery] is stopped.
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: Service[SparkTBinaryFrontend] is stopped.
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: SparkTBinaryFrontend has stopped
23/03/24 03:33:20 INFO spark.SparkSQLEngine: Service: [SparkSQLBackendService] is stopping.
23/03/24 03:33:20 INFO spark.SparkSQLBackendService: Service: [SparkSQLSessionManager] is stopping.
23/03/24 03:33:20 INFO session.SparkSQLSessionManager: Service: [SparkSQLOperationManager] is stopping.
23/03/24 03:33:20 INFO operation.SparkSQLOperationManager: Service[SparkSQLOperationManager] is stopped.
23/03/24 03:33:20 INFO session.SparkSQLSessionManager: Service[SparkSQLSessionManager] is stopped.
23/03/24 03:33:20 INFO spark.SparkSQLBackendService: Service[SparkSQLBackendService] is stopped.
23/03/24 03:33:20 INFO spark.SparkSQLEngine: Service[SparkSQLEngine] is stopped.
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: Finished closing SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]
```
### Kyuubi Server Configurations
```yaml
kyuubi.ha.addresses xxx
kyuubi.authentication LDAP
kyuubi.authentication.ldap.base.dn ou=People,dc=xx,dc=com
kyuubi.authentication.ldap.url ldap://xxx:xxx/
kyuubi.authentication.ldap.guidKey cn
kyuubi.session.engine.initialize.timeout PT10M
kyuubi.session.idle.timeout PT2H
```
### Kyuubi Engine Configurations
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [ ] No. I cannot submit a PR at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] yaooqinn commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482377182
> I've upgrade to kyuubi 1.7.0 about two weeks,
May I ask which version you used before? There may also an upgrade for log4j2 from log4j1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] lordk911 commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482309686
I have discovered another situation. The etl task also use spark's beeline to submit a sql file to kyuubi with connection level. This Sql file have three sql , The first sql is also a statment like `create table if not exists ... stored as orc;` After this statment finished , the task was stuck until kyuubi.operation.idle.timeout happen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] lordk911 closed issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 closed issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
URL: https://github.com/apache/kyuubi/issues/4591
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] lordk911 commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482510255
I almost make sure, all this two kind of issue happened only on one kyuubi server(I have 3 kyuubi server).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] lordk911 commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482288559
Thank you for your attention. Actually the ETL task script is very simple , and I can not reproduce this issue too.
ETL task script is :
`/data/soft/spark3/bin/beeline -n xxx -p xxx -u 'jdbc:hive2://worker-10-0-xxx:2181,worker-10-0-xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;?kyuubi.engine.share.level=connection;' -f '/data/scripts/datax/hdp-doc/scripts/gdwd/01.ods/ods_key_customer_touch_create.sql'`
and the the sql file has only one sql :
```
create table if not exists ods.ods__touch
(
...
update_time timestamp comment '更新时间',
etltime timestamp comment 'etl时间'
)
stored as orc;
```
And from the DolphinScheduler's task history I can see the task's script not be changed for a long time , also the sql file submit to kyuubi not be changed for a long time by seach git history .
from the picture above , the colum '工作流实例' has value like 'ods_key_customer_touch-2-20230324010001985' which 'ods_key_customer_touch' is process name and '2' is process version. This process runs once a day using version 2 and has been running for a long time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482351550
@lordk911 There is a known issue which may produce unnecessary logs https://github.com/apache/kyuubi/pull/4527
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] yaooqinn commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482293936
It may need the jstack information of the beeline process or the one who calls it during such a long time pause
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] lordk911 commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482349746
I've upgrade to kyuubi 1.7.0 about two weeks, These kind of issue happen first time. And I found the log file : `${KYUUBI_HOME}/logs/kyuubi-${kyuubi user}-org.apache.kyuubi.server.KyuubiServer-${kyuubi server host name}.out` is much larger than other versions（two weeks, each server generated approximately 3GB), is this will cause some performance issue？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] lordk911 closed issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 closed issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
URL: https://github.com/apache/kyuubi/issues/4591
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] lordk911 commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482458788
> > I've upgrade to kyuubi 1.7.0 about two weeks,
>
> May I ask which version you used before? There may also an upgrade for log4j2 from log4j1.
1.4.1 -> 1.6.1 -> 1.7.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] yaooqinn commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482213890
I've tried both spark'beeline and kyuubi's beeline against v1.7.0, but can not reproduce your issue. You may check whether your client-side has something blocking your connection to be closed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482263573
> didn't go to SUCCEEDED state until `kyuubi.session.idle.timeout` happen
Typically, it is because the client does not close the session(JDBC connection) properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] lordk911 commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1482331609
> the task was stuck until kyuubi.operation.idle.timeout happen.
add beeline out put :
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] lordk911 commented on issue #4591: [Bug] Spark connection level engines didn't be closed after all job finished
Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #4591:
URL: https://github.com/apache/kyuubi/issues/4591#issuecomment-1484327695
> I almost make sure, all this two kind of issue happened only on one kyuubi server(I have 3 kyuubi server). And `Operation OperationHandle [1ff4d7ea-dfa5-4be8-b375-9bf8fdfb3067] is timed-out and will be closed` this kind of issue be the most part.
After stopped the kyuubi server which met these kind of issue, everything goes well.
Close this issue, because it's hard to reproduce, And I can't provide any more clues at the moment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org