You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/10/27 08:26:27 UTC
[GitHub] [incubator-kyuubi] jiaoqingbo opened a new issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
jiaoqingbo opened a new issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
show test demo as below
CREATE DATABASE IF NOT EXISTS aqe_demo_db;
USE `aqe_demo_db;
CREATE TABLE items
USING parquet
AS
SELECT id AS i_item_id,
CAST(rand() * 1000 AS INT) AS i_price
FROM RANGE(30000000);
then use beeline to connect
./bin/beeline -u 'jdbc:hive2://ocdp39:2181,vm01.asia.com:2181,vm02.asia.com:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;principal=spark/_HOST@ocdp;'
and then execute
use aqe_demo_db;
select * from items ;
after that，I saw an exception message
<img width="834" alt="微信截图_20211027161048" src="https://user-images.githubusercontent.com/14961757/139026504-5c864ac4-d412-4acc-9efe-4c22edacad11.png">
### Affects Version(s)
1.3.0
### Kyuubi Server Log Output
```logtalk
2021-10-27 16:08:56.235 INFO operation.ExecuteStatement: Processing ambari-qa's query[d45d9f09-ea2a-4367-9983-9ddec491fa0c]: RUNNING_STATE -> ERROR_STATE, statement: select * from items, time taken: 25.232 seconds
2021-10-27 16:08:56.236 DEBUG server.KyuubiBackendService: OperationHandle [type=EXECUTE_STATEMENT, identifier: d45d9f09-ea2a-4367-9983-9ddec491fa0c]: The background operation was aborted, org.apache.kyuubi.KyuubiSQLException: Error operating EXECUTE_STATEMENT: java.net.SocketTimeoutException: Read timed out
2021-10-27 16:09:00.826 WARN operation.ExecuteStatement: Ignore exception in terminal state with d45d9f09-ea2a-4367-9983-9ddec491fa0c: org.apache.thrift.TApplicationException: CloseOperation failed: out of sequence response
```
### Kyuubi Engine Log Output
```logtalk
No abnormal information
```
### Kyuubi Server Configurations
```yaml
kyuubi.ha.enabled true
kyuubi.ha.zookeeper.quorum ocdp39,vm01.asia.com,vm02.asia.com
kyuubi.ha.zookeeper.client.port 2181
kyuubi.ha.zookeeper.session.timeout 600000
kyuubi.session.engine.initialize.timeout 120000
kyuubi.authentication KERBEROS
kyuubi.kinit.principal spark/vm02.asia.com@ocdp
kyuubi.kinit.keytab /etc/security/keytabs/spark.service.keytab
kyuubi.ha.zookeeper.acl.enabled false
spark.master yarn
spark.submit.deployMode cluster
spark.yarn.queue default
spark.yarn.am.memory 4g
spark.driver.memory 4g
spark.executor.memory 4g
spark.sql.extensions org.apache.submarine.spark.security.api.RangerSparkSQLExtension
```
### Kyuubi Engine Configurations
```yaml
spark.acls.enable true
spark.admin.acls
spark.driver.extraLibraryPath /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64
spark.eventLog.dir hdfs:///spark-history/
spark.eventLog.enabled true
spark.executor.extraJavaOptions -XX:+UseNUMA
spark.executor.extraLibraryPath /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64
spark.history.fs.cleaner.enabled true
spark.history.fs.cleaner.interval 7d
spark.history.fs.cleaner.maxAge 90d
spark.history.fs.logDirectory hdfs:///spark-history/
spark.history.kerberos.enabled true
spark.history.kerberos.keytab /etc/security/keytabs/spark.service.keytab
spark.history.kerberos.principal spark/vm02.asia.com@ocdp
# Generated by Apache Ambari. Wed Oct 27 16:01:23 2021
spark.acls.enable true
spark.admin.acls
spark.driver.extraLibraryPath /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64
spark.eventLog.dir hdfs:///spark-history/
spark.eventLog.enabled true
spark.executor.extraJavaOptions -XX:+UseNUMA
spark.executor.extraLibraryPath /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64
spark.history.fs.cleaner.enabled true
spark.history.fs.cleaner.interval 7d
spark.history.fs.cleaner.maxAge 90d
spark.history.fs.logDirectory hdfs:///spark-history/
spark.history.kerberos.enabled true
spark.history.kerberos.keytab /etc/security/keytabs/spark.service.keytab
spark.history.kerberos.principal spark/vm02.asia.com@ocdp
spark.history.provider org.apache.spark.deploy.history.FsHistoryProvider
spark.history.store.path /var/lib/spark/shs_db
spark.history.ui.acls.enable true
spark.history.ui.admin.acls
spark.history.ui.port 18081
spark.io.compression.lz4.blockSize 128kb
spark.master yarn
spark.shuffle.file.buffer 1m
spark.shuffle.io.backLog 8192
spark.shuffle.io.serverThreads 128
spark.shuffle.unsafe.file.output.buffer 5m
spark.sql.autoBroadcastJoinThreshold 26214400
spark.sql.hive.convertMetastoreOrc true
spark.sql.hive.metastore.jars /usr/hdp/current/spark-client/standalone-metastore/*
spark.sql.hive.metastore.version 3.0
spark.sql.orc.filterPushdown true
spark.sql.orc.impl native
spark.sql.statistics.fallBackToHdfs true
spark.sql.warehouse.dir /apps/spark/warehouse
spark.unsafe.sorter.spill.reader.buffer.size 1m
spark.yarn.historyServer.address vm02.asia.com:18081
spark.yarn.queue default
```
### Additional context
OOM due to insufficient client memory？Can someone tell me how to configure？thx
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300#issuecomment-958781165
Please file a new issue and describe the problem their instead of using this one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300#issuecomment-958766845
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300#issuecomment-958766845
Found a clue, reopen this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300#issuecomment-958781165
Please file a new issue and describe the problem their instead of using this one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn closed issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
yaooqinn closed issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn closed issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
yaooqinn closed issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 closed issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
pan3793 closed issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300#issuecomment-958781165
Please file a new issue and describe the problem their instead of using this one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn closed issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
yaooqinn closed issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 closed issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
pan3793 closed issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300#issuecomment-952723037
cc @turboFei
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 closed issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
pan3793 closed issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300#issuecomment-958766845
Found a clue, reopen this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #1300: [Bug] Prompt read timeout when executing select * query in beeline
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #1300:
URL: https://github.com/apache/incubator-kyuubi/issues/1300#issuecomment-958781165
Please file a new issue and describe the problem their instead of using this one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org