You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Bertrand Guay-Paquet <be...@step.polymtl.ca> on 2013/03/19 20:48:50 UTC
Tomcat jdbc-pool not closing statements
Hello,
I'm using Tomcat 7.0.34 via TomEE 1.5.1 with MySQL. I noticed a memory
leak in my web application which uses jdbc connection pooling with
Tomcat's jdbc-pool.
The com.mysql.jdbc.JDBC4Connection class has a field named
"openStatements" which holds, as you can imagine, open sql statements.
This structure grows continuously over time and no statements are ever
released. I stepped into my code to verify that I closed opened
statements and it is the case.
Digging some more, I downloaded Tomcat's source and it seems that
jdbc-pool discards all calls to java.sql.Statement.close() in
StatementDecoratorInterceptor#invoke(Object proxy, Method method,
Object[] args)
I see what could be a bug in StatementCache#closeInvoked() which is
called by the above method. Here is the code with my own comments added:
@Override
public void closeInvoked() {
boolean shouldClose = true;
if (cacheSize.get() < maxCacheSize) {
// omitted for brievety
}
closed = true;
// [1] I think "delegate = null" is done too soon
delegate = null;
if (shouldClose) {
// check its body below
super.closeInvoked();
}
}
// This is super.closeInvoked()
public void closeInvoked() {
if (getDelegate()!=null) {
// never true when coming from
// StatementCache#closeInvoked()
// because of [1]
try {
getDelegate().close();
}catch (SQLException ignore) {
}
}
closed = true;
delegate = null;
}
Regards,
Bertrand
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat jdbc-pool not closing statements
Posted by Bertrand Guay-Paquet <be...@step.polymtl.ca>.
> Can you post some example of your code? Can you also post your pool's
> configuration?
Here is my configuration from tomee.xml. I'm not 100% sure how it maps
to Tomcat values though.
<Resource id="jdbc/my-db" type="javax.sql.DataSource">
JdbcDriver=com.mysql.jdbc.Driver
JdbcUrl=jdbc:mysql://localhost:3306/DBNAME
UserName=....
Password=....
JtaManaged=true
ConnectionProperties=characterEncoding=UTF-8;useLegacyDatetimeCode=false
initialSize=10
maxActive=100
maxIdle=30
validationQuery=/* ping */
testOnBorrow=true
testWhileIdle=true
timeBetweenEvictionRunsMillis=10000
minEvictableIdleTimeMillis=60000
</Resource>
Adding "DataSourceCreator=dbcp" to the configuration stops the leak
without any other change to my code. This option reverts the connection
pooling of TomEE to commons-dbcp.
My code uses MyBatis 3.2.1 for the actual management of jdbc, so I'm not
sure which information would help. I use a mapper and I can guarantee
that sqlSession.close() is called which in turn calls
Connection#close(). The code which closes Statements is inside MyBatis.
Let me know if you need more information.
Regards,
Bertrand
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat jdbc-pool not closing statements
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Bertrand,
On 3/19/13 3:48 PM, Bertrand Guay-Paquet wrote:
> I'm using Tomcat 7.0.34 via TomEE 1.5.1 with MySQL. I noticed a
> memory leak in my web application which uses jdbc connection
> pooling with Tomcat's jdbc-pool.
>
> The com.mysql.jdbc.JDBC4Connection class has a field named
> "openStatements" which holds, as you can imagine, open sql
> statements. This structure grows continuously over time and no
> statements are ever released. I stepped into my code to verify that
> I closed opened statements and it is the case.
Can you post some example of your code? Can you also post your pool's
configuration?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEAREIAAYFAlFIxyEACgkQ9CaO5/Lv0PCdOQCgotmWjkcCxnZvknexMq3ZY2Dp
t64An1WxIm9ptEK24Yq2ub2qbjdw+u7v
=RyXf
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat jdbc-pool not closing statements
Posted by Bertrand Guay-Paquet <be...@step.polymtl.ca>.
Bug reported at https://issues.apache.org/bugzilla/show_bug.cgi?id=54732
Due to another bug in TomEE, StatementCache is always enabled. That bug
is reported here: https://issues.apache.org/jira/browse/TOMEE-837
Thanks for your help
On 20/03/2013 7:28 AM, Felix Schumacher wrote:
> Am 19.03.2013 22:20, schrieb Bertrand Guay-Paquet:
>> On 19/03/2013 5:05 PM, Felix Schumacher wrote:
>>> Have you looked at
>>> http://grokbase.com/t/openejb/users/13135d2a0v/jdbc-connection-pool-memory-leak
>>> ? It seems like your problem. Regards Felix
>>
>> Indeed, this is extremely similar to my issue. Thanks for sharing this.
>>
>> It does seem however like the StatementFinalizer Tomcat interceptor
>> should not be necessary if an application closes its connections,
>> statements and result sets properly. From what I could see by stepping
>> in the code, this is the case with MyBatis. The actual source of the
>> problem really seems to be that Tomcat's jdbc pool swallows calls to
>> Statement.close() like I showed in my original message.
> Now that I had time to look more closely, I believe you are right and
> the assignment of 'closed' and 'delegate' before (or even after) the
> call to super.closeInvoked() looks like a bug, too.
>
> So I think you should go ahead and file one in bugzilla. You should
> keep in mind, that afaik the StatementCache is not enabled by default
> in tomcat.
>
> Regards
> Felix
>>
>> Regards,
>> Bertrand
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat jdbc-pool not closing statements
Posted by Bertrand Guay-Paquet <be...@step.polymtl.ca>.
Bug reported at https://issues.apache.org/bugzilla/show_bug.cgi?id=54732
Due to another bug in TomEE, StatementCache is always enabled. That bug
is reported here: https://issues.apache.org/jira/browse/TOMEE-837
Thanks for your help
On 20/03/2013 7:28 AM, Felix Schumacher wrote:
> Am 19.03.2013 22:20, schrieb Bertrand Guay-Paquet:
>> On 19/03/2013 5:05 PM, Felix Schumacher wrote:
>>> Have you looked at
>>> http://grokbase.com/t/openejb/users/13135d2a0v/jdbc-connection-pool-memory-leak
>>> ? It seems like your problem. Regards Felix
>>
>> Indeed, this is extremely similar to my issue. Thanks for sharing this.
>>
>> It does seem however like the StatementFinalizer Tomcat interceptor
>> should not be necessary if an application closes its connections,
>> statements and result sets properly. From what I could see by stepping
>> in the code, this is the case with MyBatis. The actual source of the
>> problem really seems to be that Tomcat's jdbc pool swallows calls to
>> Statement.close() like I showed in my original message.
> Now that I had time to look more closely, I believe you are right and
> the assignment of 'closed' and 'delegate' before (or even after) the
> call to super.closeInvoked() looks like a bug, too.
>
> So I think you should go ahead and file one in bugzilla. You should
> keep in mind, that afaik the StatementCache is not enabled by default
> in tomcat.
>
> Regards
> Felix
>>
>> Regards,
>> Bertrand
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat jdbc-pool not closing statements
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 19.03.2013 22:20, schrieb Bertrand Guay-Paquet:
> On 19/03/2013 5:05 PM, Felix Schumacher wrote:
>> Have you looked at
>> http://grokbase.com/t/openejb/users/13135d2a0v/jdbc-connection-pool-memory-leak
>> ? It seems like your problem. Regards Felix
>
> Indeed, this is extremely similar to my issue. Thanks for sharing
> this.
>
> It does seem however like the StatementFinalizer Tomcat interceptor
> should not be necessary if an application closes its connections,
> statements and result sets properly. From what I could see by stepping
> in the code, this is the case with MyBatis. The actual source of the
> problem really seems to be that Tomcat's jdbc pool swallows calls to
> Statement.close() like I showed in my original message.
Now that I had time to look more closely, I believe you are right and
the assignment of 'closed' and 'delegate' before (or even after) the
call to super.closeInvoked() looks like a bug, too.
So I think you should go ahead and file one in bugzilla. You should
keep in mind, that afaik the StatementCache is not enabled by default in
tomcat.
Regards
Felix
>
> Regards,
> Bertrand
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat jdbc-pool not closing statements
Posted by Bertrand Guay-Paquet <be...@step.polymtl.ca>.
On 19/03/2013 5:05 PM, Felix Schumacher wrote:
> Have you looked at
> http://grokbase.com/t/openejb/users/13135d2a0v/jdbc-connection-pool-memory-leak
> ? It seems like your problem. Regards Felix
Indeed, this is extremely similar to my issue. Thanks for sharing this.
It does seem however like the StatementFinalizer Tomcat interceptor
should not be necessary if an application closes its connections,
statements and result sets properly. From what I could see by stepping
in the code, this is the case with MyBatis. The actual source of the
problem really seems to be that Tomcat's jdbc pool swallows calls to
Statement.close() like I showed in my original message.
Regards,
Bertrand
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat jdbc-pool not closing statements
Posted by Felix Schumacher <fe...@internetallee.de>.
Bertrand Guay-Paquet <be...@step.polymtl.ca> schrieb:
>Hello,
>
>I'm using Tomcat 7.0.34 via TomEE 1.5.1 with MySQL. I noticed a memory
>leak in my web application which uses jdbc connection pooling with
>Tomcat's jdbc-pool.
>
>The com.mysql.jdbc.JDBC4Connection class has a field named
>"openStatements" which holds, as you can imagine, open sql statements.
>This structure grows continuously over time and no statements are ever
>released. I stepped into my code to verify that I closed opened
>statements and it is the case.
>
Have you looked at http://grokbase.com/t/openejb/users/13135d2a0v/jdbc-connection-pool-memory-leak ? It seems like your problem.
Regards
Felix
>Digging some more, I downloaded Tomcat's source and it seems that
>jdbc-pool discards all calls to java.sql.Statement.close() in
>StatementDecoratorInterceptor#invoke(Object proxy, Method method,
>Object[] args)
>
>I see what could be a bug in StatementCache#closeInvoked() which is
>called by the above method. Here is the code with my own comments
>added:
>@Override
>public void closeInvoked() {
> boolean shouldClose = true;
> if (cacheSize.get() < maxCacheSize) {
> // omitted for brievety
> }
> closed = true;
> // [1] I think "delegate = null" is done too soon
> delegate = null;
> if (shouldClose) {
> // check its body below
> super.closeInvoked();
> }
>}
>
>// This is super.closeInvoked()
>public void closeInvoked() {
> if (getDelegate()!=null) {
> // never true when coming from
> // StatementCache#closeInvoked()
> // because of [1]
> try {
> getDelegate().close();
> }catch (SQLException ignore) {
> }
> }
> closed = true;
> delegate = null;
>}
>
>Regards,
>Bertrand
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org