You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "GPF199541 (via GitHub)" <gi...@apache.org> on 2024/02/06 08:10:59 UTC
[I] Streaming query shardingsphere-proxy always interrupted [shardingsphere]
GPF199541 opened a new issue, #30014:
URL: https://github.com/apache/shardingsphere/issues/30014
## Bug Report
streaming query always interrupted
ShardingSphere-Proxy version:5.4.1
Java 1.8
### ShardingSphere-Proxy ERROR
```[ERROR] 2024-02-06 15:57:24.574 [ShardingSphere-Command-6] o.a.s.p.f.c.CommandExecutorTask - Exception occur:
java.sql.SQLException: null
at org.apache.shardingsphere.proxy.backend.connector.DatabaseConnector.close(DatabaseConnector.java:401)
at org.apache.shardingsphere.proxy.frontend.mysql.command.query.text.query.MySQLComQueryPacketExecutor.close(MySQLComQueryPacketExecutor.java:128)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.doExecuteCommand$original$OA8Nqe3z(CommandExecutorTask.java:139)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.doExecuteCommand$original$OA8Nqe3z$accessor$Vbh5tAMm(CommandExecutorTask.java)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask$auxiliary$7v0fw8rO.call(Unknown Source)
at org.apache.shardingsphere.agent.core.advisor.executor.type.InstanceMethodAdviceExecutor.advice(InstanceMethodAdviceExecutor.java:72)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.doExecuteCommand(CommandExecutorTask.java)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.executeCommand(CommandExecutorTask.java:121)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run$original$OA8Nqe3z(CommandExecutorTask.java:78)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run$original$OA8Nqe3z$accessor$Vbh5tAMm(CommandExecutorTask.java)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask$auxiliary$DzxI85sW.call(Unknown Source)
at org.apache.shardingsphere.agent.core.advisor.executor.type.InstanceMethodAdviceExecutor.advice(InstanceMethodAdviceExecutor.java:72)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run(CommandExecutorTask.java)
at com.alibaba.ttl.TtlRunnable.run(TtlRunnable.java:60)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
-----
### ShardingSphere-Proxy-CONFIG:
```
mode:
type: Cluster
repository:
type: ZooKeeper
props:
namespace: test54101
server-lists: localhost:2181
retryIntervalMilliseconds: 500
timeToLiveSeconds: 60
maxRetries: 3
operationTimeoutMilliseconds: 500
authority:
users:
- user: root@%
password: root
privilege:
type: ALL_PERMITTED
sqlParser:
sqlCommentParseEnabled: false
sqlStatementCache:
initialCapacity: 2000
maximumSize: 65535
parseTreeCache:
initialCapacity: 128
maximumSize: 1024
logging:
loggers:
- loggerName: ShardingSphere-SQL
additivity: true
level: DEBUG
props:
enable: true
sqlFederation:
sqlFederationEnabled: false
executionPlanCache:
initialCapacity: 2000
maximumSize: 65535
props:
system-log-level: DEBUG
max-connections-size-per-query: 1
kernel-executor-size: 1
proxy-frontend-flush-threshold: 1
sql-show: true
check-table-metadata-enabled: false
proxy-backend-query-fetch-size: -1
proxy-frontend-executor-size: 0
proxy-frontend-max-connections: 0
proxy-default-port: 3307
proxy-netty-backlog: 1024
cdc-server-port: 33071
proxy-frontend-ssl-enabled: false
proxy-frontend-ssl-cipher: ''
proxy-frontend-ssl-version: TLSv1.2,TLSv1.3
```
------
### JAVA-CODE
```
public class tools {
private static Driver driver;
static {
try {
driver = new Driver();
} catch (Exception e) {
log.error("error", e);
}
}
@Autowired
BaseConfig baseConfig;
@PostConstruct
private void init() {
try {
DataSource ds = dbPool(baseConfig.getJdbcUrl(), baseConfig.getUserName(), baseConfig.getPassWord(), baseConfig.getDbDriverName());
Connection connection = ds.getConnection();
connection.setAutoCommit(false);
String dbSql = "use " + baseConfig.getDbName();
PreparedStatement preparedStatement = connection.prepareStatement(dbSql,ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
preparedStatement.setFetchSize(Integer.MIN_VALUE);
preparedStatement.execute();
ResultSet resultSet = preparedStatement.executeQuery(baseConfig.getQuerySql());
dealResultSet(resultSet);
connection.commit();
resultSet.close();
preparedStatement.close();
connection.close();
}catch (Exception e){
log.error(e.getMessage());
e.printStackTrace();
}
}
private HikariDataSource dbPool(String url,String userName,String passWd,String driverClassName){
DataSource dataSource = DataSourceBuilder
.create()
.url(url)
.username(userName)
.password(passWd)
.driverClassName(driverClassName)
.build();
HikariDataSource ds = (HikariDataSource) dataSource;
ds.setConnectionTimeout(baseConfig.getConnectTimeout());
ds.setMaximumPoolSize(baseConfig.getMaxPoolSize());
ds.setIdleTimeout(baseConfig.getIdleTimeout());
ds.setConnectionTestQuery("select 1");
ds.setKeepaliveTime(baseConfig.getKeepalivedTimeout());
return ds;
}
private void dealResultSet(ResultSet resultSet){
try {
ResultSetMetaData resultSetMetaData = resultSet.getMetaData();
int columnCount = resultSetMetaData.getColumnCount();
if (columnCount == 0) {
return;
}
Integer rowNum = 0;
while (resultSet.next()) {
List<Object> data = new ArrayList<>();
for (int i = 1; i <= columnCount; i++) {
data.add(resultSet.getString(i));
}
rowNum++;
sleep(baseConfig.getSleep());
log.info("rowNum:{},data is:{}",rowNum,data);
}
}catch (Exception e){
log.error(e.getMessage());
e.printStackTrace();
}
}
}
```
------
### code error:
````
** BEGIN NESTED EXCEPTION **
java.io.EOFException
MESSAGE: Can not read response from server. Expected to read 152 bytes, read 29 bytes before connection was unexpectedly lost.
STACKTRACE:
java.io.EOFException: Can not read response from server. Expected to read 152 bytes, read 29 bytes before connection was unexpectedly lost.
at com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:67)
at com.mysql.cj.protocol.a.SimplePacketReader.readMessageLocal(SimplePacketReader.java:137)
at com.mysql.cj.protocol.a.SimplePacketReader.readMessage(SimplePacketReader.java:102)
at com.mysql.cj.protocol.a.SimplePacketReader.readMessage(SimplePacketReader.java:45)
at com.mysql.cj.protocol.a.MultiPacketReader.readMessage(MultiPacketReader.java:66)
at com.mysql.cj.protocol.a.MultiPacketReader.readMessage(MultiPacketReader.java:44)
at com.mysql.cj.protocol.a.ResultsetRowReader.read(ResultsetRowReader.java:75)
at com.mysql.cj.protocol.a.ResultsetRowReader.read(ResultsetRowReader.java:42)
at com.mysql.cj.protocol.a.NativeProtocol.read(NativeProtocol.java:1651)
at com.mysql.cj.protocol.a.result.ResultsetRowsStreaming.next(ResultsetRowsStreaming.java:194)
at com.mysql.cj.protocol.a.result.ResultsetRowsStreaming.close(ResultsetRowsStreaming.java:116)
at com.mysql.cj.jdbc.result.ResultSetImpl.realClose(ResultSetImpl.java:1950)
at com.mysql.cj.jdbc.result.ResultSetImpl.close(ResultSetImpl.java:564)
at com.zaxxer.hikari.pool.HikariProxyResultSet.close(HikariProxyResultSet.java)
at org.apache.shardingsphere.proxy.backend.connector.DatabaseConnector.closeResultSets(DatabaseConnector.java:410)
at org.apache.shardingsphere.proxy.backend.connector.DatabaseConnector.close(DatabaseConnector.java:395)
at org.apache.shardingsphere.proxy.frontend.mysql.command.query.text.query.MySQLComQueryPacketExecutor.close(MySQLComQueryPacketExecutor.java:128)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.doExecuteCommand$original$OA8Nqe3z(CommandExecutorTask.java:139)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.doExecuteCommand$original$OA8Nqe3z$accessor$Vbh5tAMm(CommandExecutorTask.java)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask$auxiliary$7v0fw8rO.call(Unknown Source)
at org.apache.shardingsphere.agent.core.advisor.executor.type.InstanceMethodAdviceExecutor.advice(InstanceMethodAdviceExecutor.java:72)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.doExecuteCommand(CommandExecutorTask.java)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.executeCommand(CommandExecutorTask.java:121)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run$original$OA8Nqe3z(CommandExecutorTask.java:78)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run$original$OA8Nqe3z$accessor$Vbh5tAMm(CommandExecutorTask.java)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask$auxiliary$DzxI85sW.call(Unknown Source)
at org.apache.shardingsphere.agent.core.advisor.executor.type.InstanceMethodAdviceExecutor.advice(InstanceMethodAdviceExecutor.java:72)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run(CommandExecutorTask.java)
at com.alibaba.ttl.TtlRunnable.run(TtlRunnable.java:60)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
** END NESTED EXCEPTION **
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Streaming query shardingsphere-proxy always interrupted java.sql.SQLException: null [shardingsphere]
Posted by "GPF199541 (via GitHub)" <gi...@apache.org>.
GPF199541 commented on issue #30014:
URL: https://github.com/apache/shardingsphere/issues/30014#issuecomment-1960630995
This is useful,thanks @TeslaCN
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Streaming query shardingsphere-proxy always interrupted java.sql.SQLException: null [shardingsphere]
Posted by "TeslaCN (via GitHub)" <gi...@apache.org>.
TeslaCN commented on issue #30014:
URL: https://github.com/apache/shardingsphere/issues/30014#issuecomment-1937766183
Try setting `netTimeoutForStreamingResults`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Streaming query shardingsphere-proxy always interrupted java.sql.SQLException: null [shardingsphere]
Posted by "GPF199541 (via GitHub)" <gi...@apache.org>.
GPF199541 closed issue #30014: Streaming query shardingsphere-proxy always interrupted java.sql.SQLException: null
URL: https://github.com/apache/shardingsphere/issues/30014
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Streaming query shardingsphere-proxy always interrupted [shardingsphere]
Posted by "GPF199541 (via GitHub)" <gi...@apache.org>.
GPF199541 commented on issue #30014:
URL: https://github.com/apache/shardingsphere/issues/30014#issuecomment-1928983918
I want to stream query 50 million rows of data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org