You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/01/17 08:25:09 UTC
[GitHub] [shardingsphere] strongduanmu opened a new issue #14827: Add setFetchSize logic for JDBC adaptor when use stream query
strongduanmu opened a new issue #14827:
URL: https://github.com/apache/shardingsphere/issues/14827
When I investigate stream resultset not close exception in PR#14822, I found that this exception did not occur in JDBC adaptor, and Proxy adaptor had an exception that the streaming result set was not closed.
It can be found from source code that in Proxy adaptor, we implement the setting of the fetchSize parameter through the `StatementMemoryStrictlyFetchSizeSetter` SPI to implement streaming query, but the JDBC adaptor lacks similar logic, so streaming query cannot be used.
Proxy adaptor:
```java
@Override
public Statement createStorageResource(final Connection connection, final ConnectionMode connectionMode, final StatementOption option) throws SQLException {
Statement result = connection.createStatement();
if (ConnectionMode.MEMORY_STRICTLY == connectionMode) {
setFetchSize(result);
}
return result;
}
```
JDBC adaptor:
```java
@SuppressWarnings("MagicConstant")
@Override
public Statement createStorageResource(final Connection connection, final ConnectionMode connectionMode, final StatementOption option) throws SQLException {
return connection.createStatement(option.getResultSetType(), option.getResultSetConcurrency(), option.getResultSetHoldability());
}
```
In order to reduce the memory usage, we need to perform the same logic on the JDBC adaptor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #14827: Add setFetchSize logic for JDBC adaptor when use stream query
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #14827:
URL: https://github.com/apache/shardingsphere/issues/14827#issuecomment-1019386194
The pr corresponding to this issue caused an unexpected streaming query exception, so we rolled back the code. For more exception info, you can refer #14975.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero closed issue #14827: Add setFetchSize logic for JDBC adaptor when use stream query
Posted by GitBox <gi...@apache.org>.
tristaZero closed issue #14827:
URL: https://github.com/apache/shardingsphere/issues/14827
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #14827: Add setFetchSize logic for JDBC adaptor when use stream query
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #14827:
URL: https://github.com/apache/shardingsphere/issues/14827#issuecomment-1014299889
After I added the `setFetchSize` logic to the JDBC access, the same streaming query exception appeared.
```
Exception in thread "main" java.sql.SQLException: exception while executing query: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@7433ca19 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:577)
at org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137)
at org.apache.shardingsphere.infra.federation.executor.original.OriginalFilterableExecutor.executeQuery(OriginalFilterableExecutor.java:80)
at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.executeFederationQuery(ShardingSpherePreparedStatement.java:252)
at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.execute(ShardingSpherePreparedStatement.java:334)
at com.strongduanmu.jdbc.MySQLShardingJDBCTest.execute(MySQLShardingJDBCTest.java:46)
at com.strongduanmu.jdbc.MySQLShardingJDBCTest.main(MySQLShardingJDBCTest.java:31)
Caused by: org.apache.shardingsphere.infra.exception.ShardingSphereException: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@7433ca19 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
at org.apache.shardingsphere.infra.federation.executor.original.table.FilterableTableScanExecutor.execute(FilterableTableScanExecutor.java:155)
at org.apache.shardingsphere.infra.federation.executor.original.table.FilterableTable.scan(FilterableTable.java:53)
at org.apache.calcite.schema.Schemas.enumerable(Schemas.java:249)
at Baz.bind(Unknown Source)
at org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:363)
at org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:333)
at org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:578)
at org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:569)
at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:182)
at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64)
at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43)
at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:573)
... 6 more
Caused by: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@7433ca19 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:869)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:865)
at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:3217)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2453)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2495)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1903)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2011)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
at org.apache.shardingsphere.driver.executor.callback.impl.PreparedStatementExecuteQueryCallback.executeQuery(PreparedStatementExecuteQueryCallback.java:40)
at org.apache.shardingsphere.driver.executor.callback.ExecuteQueryCallback.executeSQL(ExecuteQueryCallback.java:44)
at org.apache.shardingsphere.driver.executor.callback.ExecuteQueryCallback.executeSQL(ExecuteQueryCallback.java:36)
at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutorCallback.execute(JDBCExecutorCallback.java:85)
at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutorCallback.execute(JDBCExecutorCallback.java:64)
at org.apache.shardingsphere.infra.executor.kernel.ExecutorEngine.syncExecute(ExecutorEngine.java:101)
at org.apache.shardingsphere.infra.executor.kernel.ExecutorEngine.parallelExecute(ExecutorEngine.java:97)
at org.apache.shardingsphere.infra.executor.kernel.ExecutorEngine.execute(ExecutorEngine.java:82)
at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutor.execute(JDBCExecutor.java:65)
at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutor.execute(JDBCExecutor.java:49)
at org.apache.shardingsphere.infra.federation.executor.original.table.FilterableTableScanExecutor.execute(FilterableTableScanExecutor.java:149)
... 17 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu edited a comment on issue #14827: Add setFetchSize logic for JDBC adaptor when use stream query
Posted by GitBox <gi...@apache.org>.
strongduanmu edited a comment on issue #14827:
URL: https://github.com/apache/shardingsphere/issues/14827#issuecomment-1014299889
After I added the `setFetchSize` logic to the JDBC adaptor, the same streaming query exception appeared.
```
Exception in thread "main" java.sql.SQLException: exception while executing query: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@7433ca19 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:577)
at org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137)
at org.apache.shardingsphere.infra.federation.executor.original.OriginalFilterableExecutor.executeQuery(OriginalFilterableExecutor.java:80)
at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.executeFederationQuery(ShardingSpherePreparedStatement.java:252)
at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.execute(ShardingSpherePreparedStatement.java:334)
at com.strongduanmu.jdbc.MySQLShardingJDBCTest.execute(MySQLShardingJDBCTest.java:46)
at com.strongduanmu.jdbc.MySQLShardingJDBCTest.main(MySQLShardingJDBCTest.java:31)
Caused by: org.apache.shardingsphere.infra.exception.ShardingSphereException: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@7433ca19 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
at org.apache.shardingsphere.infra.federation.executor.original.table.FilterableTableScanExecutor.execute(FilterableTableScanExecutor.java:155)
at org.apache.shardingsphere.infra.federation.executor.original.table.FilterableTable.scan(FilterableTable.java:53)
at org.apache.calcite.schema.Schemas.enumerable(Schemas.java:249)
at Baz.bind(Unknown Source)
at org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:363)
at org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:333)
at org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:578)
at org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:569)
at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:182)
at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64)
at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43)
at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:573)
... 6 more
Caused by: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@7433ca19 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:869)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:865)
at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:3217)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2453)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2495)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1903)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2011)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
at org.apache.shardingsphere.driver.executor.callback.impl.PreparedStatementExecuteQueryCallback.executeQuery(PreparedStatementExecuteQueryCallback.java:40)
at org.apache.shardingsphere.driver.executor.callback.ExecuteQueryCallback.executeSQL(ExecuteQueryCallback.java:44)
at org.apache.shardingsphere.driver.executor.callback.ExecuteQueryCallback.executeSQL(ExecuteQueryCallback.java:36)
at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutorCallback.execute(JDBCExecutorCallback.java:85)
at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutorCallback.execute(JDBCExecutorCallback.java:64)
at org.apache.shardingsphere.infra.executor.kernel.ExecutorEngine.syncExecute(ExecutorEngine.java:101)
at org.apache.shardingsphere.infra.executor.kernel.ExecutorEngine.parallelExecute(ExecutorEngine.java:97)
at org.apache.shardingsphere.infra.executor.kernel.ExecutorEngine.execute(ExecutorEngine.java:82)
at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutor.execute(JDBCExecutor.java:65)
at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutor.execute(JDBCExecutor.java:49)
at org.apache.shardingsphere.infra.federation.executor.original.table.FilterableTableScanExecutor.execute(FilterableTableScanExecutor.java:149)
... 17 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org