You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "itxiangkui (Jira)" <ji...@apache.org> on 2022/04/19 04:04:00 UTC
[jira] [Comment Edited] (CALCITE-5099) ResultSetEnumerable.enumeratorBasedOnStatement cause the backend connection leak
[ https://issues.apache.org/jira/browse/CALCITE-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17524036#comment-17524036 ]
itxiangkui edited comment on CALCITE-5099 at 4/19/22 4:03 AM:
--------------------------------------------------------------
We have extended JdbcSchema and JdbcCatalogSchema in order to connect to the back-end mysql database.
At the same time, HikariDataSource was used to replace the original BasicDataSource. When the connection pool leak detection of HikariDataSource was activated, a connection leak was detected.
{code:java}
java.lang.Exception: Apparent connection leak detected
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128)
at org.apache.calcite.runtime.ResultSetEnumerable.enumeratorBasedOnStatement(ResultSetEnumerable.java:267)
at org.apache.calcite.runtime.ResultSetEnumerable.enumerator(ResultSetEnumerable.java:257)
at org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
at org.apache.calcite.jdbc.CalciteMetaImpl.fetch(CalciteMetaImpl.java:674)
at org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:245)
at org.apache.calcite.avatica.remote.Service$FetchRequest.accept(Service.java:1395)
at org.apache.calcite.avatica.remote.Service$FetchRequest.accept(Service.java:1362)
at org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:94)
at org.apache.calcite.avatica.remote.JsonHandler.apply(JsonHandler.java:52)
at org.apache.calcite.avatica.server.AvaticaJsonHandler.handle(AvaticaJsonHandler.java:133)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:500)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.lang.Thread.run(Thread.java:748)
{code}
Navigating to the snippet is
{code:java}
private Enumerator<T> enumeratorBasedOnStatement() {
Connection connection = null;
Statement statement = null;
try {
connection = dataSource.getConnection();
statement = connection.createStatement();
setTimeoutIfPossible(statement);
if (statement.execute(sql)) {
final ResultSet resultSet = statement.getResultSet();
statement = null;
connection = null;
return new ResultSetEnumerator<>(resultSet, rowBuilderFactory);
} else {
Integer updateCount = statement.getUpdateCount();
//noinspection unchecked
return Linq4j.singletonEnumerator((T) updateCount);
}
} catch (SQLException e) {
throw Static.RESOURCE.exceptionWhilePerformingQueryOnJdbcSubSchema(sql)
.ex(e);
} finally {
closeIfPossible(connection, statement);
}
}
{code}
obviously when you set:
*statement = null;*
*connection = null;*
{code:java}
if (statement.execute(sql)) {
final ResultSet resultSet = statement.getResultSet();
statement = null;
connection = null;
return new ResultSetEnumerator<>(resultSet, rowBuilderFactory);
}
{code}
the connection may leak,closeIfPossible(connection, statement); will close a null connection
was (Author: JIRAUSER286541):
We have extended JdbcSchema and JdbcCatalogSchema in order to connect to the back-end mysql database.
At the same time, HikariDataSource was used to replace the original BasicDataSource. When the connection pool leak detection of HikariDataSource was activated, a connection leak was detected.
{code:java}
java.lang.Exception: Apparent connection leak detected
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128)
at org.apache.calcite.runtime.ResultSetEnumerable.enumeratorBasedOnStatement(ResultSetEnumerable.java:267)
at org.apache.calcite.runtime.ResultSetEnumerable.enumerator(ResultSetEnumerable.java:257)
at org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
at org.apache.calcite.jdbc.CalciteMetaImpl.fetch(CalciteMetaImpl.java:674)
at org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:245)
at org.apache.calcite.avatica.remote.Service$FetchRequest.accept(Service.java:1395)
at org.apache.calcite.avatica.remote.Service$FetchRequest.accept(Service.java:1362)
at org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:94)
at org.apache.calcite.avatica.remote.JsonHandler.apply(JsonHandler.java:52)
at org.apache.calcite.avatica.server.AvaticaJsonHandler.handle(AvaticaJsonHandler.java:133)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:500)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.lang.Thread.run(Thread.java:748)
{code}
Navigating to the snippet is
{code:java}
private Enumerator<T> enumeratorBasedOnStatement() {
Connection connection = null;
Statement statement = null;
try {
connection = dataSource.getConnection();
statement = connection.createStatement();
setTimeoutIfPossible(statement);
if (statement.execute(sql)) {
final ResultSet resultSet = statement.getResultSet();
statement = null;
connection = null;
return new ResultSetEnumerator<>(resultSet, rowBuilderFactory);
} else {
Integer updateCount = statement.getUpdateCount();
//noinspection unchecked
return Linq4j.singletonEnumerator((T) updateCount);
}
} catch (SQLException e) {
throw Static.RESOURCE.exceptionWhilePerformingQueryOnJdbcSubSchema(sql)
.ex(e);
} finally {
closeIfPossible(connection, statement);
}
}
{code}
obviously when you set:
*statement = null;*
*connection = null;*
the connection may leak,closeIfPossible(connection, statement); will close a null connection
> ResultSetEnumerable.enumeratorBasedOnStatement cause the backend connection leak
> ---------------------------------------------------------------------------------
>
> Key: CALCITE-5099
> URL: https://issues.apache.org/jira/browse/CALCITE-5099
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.30.0
> Reporter: itxiangkui
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)