You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Patricio Echague (JIRA)" <ji...@apache.org> on 2011/09/23 02:15:28 UTC
[jira] [Created] (CASSANDRA-3244) JDBC CassandraConnection may lead
to memory leak when used in a pool
JDBC CassandraConnection may lead to memory leak when used in a pool
--------------------------------------------------------------------
Key: CASSANDRA-3244
URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
Project: Cassandra
Issue Type: Improvement
Components: Drivers
Affects Versions: 1.0.0
Reporter: Patricio Echague
Priority: Minor
I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Patricio Echague (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117565#comment-13117565 ]
Patricio Echague commented on CASSANDRA-3244:
---------------------------------------------
Rick, looking again at JDBCTemplate, I realized that Spring closes automatically the Statement. And since you added a method to remove the statement from the connection once the statement is closed, I think it should be ok for now.
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Fix For: 0.8.7
>
> Attachments: 3244-v1.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Rick Shaw (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119884#comment-13119884 ]
Rick Shaw commented on CASSANDRA-3244:
--------------------------------------
Good suggestion. Re-patched.
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Fix For: 0.8.7
>
> Attachments: 3244-v1.txt, 3244-v2.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119546#comment-13119546 ]
Jonathan Ellis commented on CASSANDRA-3244:
-------------------------------------------
- It would be cleaner to make the collection of statements a Set, since ordering is not important and we do "random access" removes on manual close()
- using ConcurrentSkipListSet would allow both thread safety of using a single Connection (with separate Statements) across multiple threads, as well as obviate the need for cloning the statements before iterating over them
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Fix For: 0.8.7
>
> Attachments: 3244-v1.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Rick Shaw (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117529#comment-13117529 ]
Rick Shaw commented on CASSANDRA-3244:
--------------------------------------
I think this driver assumes its connection is not reusable by a pool. It maintains state and JDBC currently has no interface methods to manage the notion of clearing and cleaning up its state separate from closing the physical connection, just {{close()}}. It needs to be done by cooperating {{protected}} methods. This will be fine if we implement our own {{PooledConnection}} classes but is not really practical for frameworks that want to wrap our {{Connection}} implementation. The drastic solution is to give up maintaining state and therefor (at least in my opinion) breaking the best effort contract to adhere to the spec where we can to make an external cooperating framework function.
Note that there is really no problem with not closing the connection if the {{Statement}} is closed. It will just leave an empty list of statements. The patch forces the {{Statement}} to clean up the list when it closes. But of course the _potential_ problem remains. This is all done because the spec says the statements should be closed on premature close of the connection. There is a whole section in the spec on how pools should be handled.
The only approach I can think of is to split {{CassanadraConnection}} in two and wrap the physical connection and keep state in the wrapper. That (I think) is the approach of {{PooledConnection}} anyway.
Do you think this is urgent? Can it wait toll we implement our own JDBC compliant pooling classes?
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Fix For: 0.8.7
>
> Attachments: 3244-v1.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Rick Shaw (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rick Shaw reassigned CASSANDRA-3244:
------------------------------------
Assignee: Rick Shaw
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Attachments: 3244-v1.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3244) JDBC
CassandraConnection may lead to memory leak when used in a pool
Posted by "Rick Shaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113452#comment-13113452 ]
Rick Shaw edited comment on CASSANDRA-3244 at 9/23/11 2:11 PM:
---------------------------------------------------------------
Creating a {{Statement}} every time you need one is unnecessary but not illegal. The assumption is the {{Statement}} is closed when Spring is done, which should be just fine. The problem is the list in {{Connection}} _should_ be a list of open {{Statements}}. The implication being that closing a {{Statement}} removes itself from that list. Unfortunately this is not being done.
The list is necessary. But it needs to be maintained by {{Statement}} because only it knows when it is asked to close.
Thanks for catching this bug. I'll create a fix for this.
was (Author: ardot):
Creating a {{Statement}} every time you need one is unnecessary but not illegal. The assumption is the {{Statement}} is closed when Spring is done, which should be just fine. The problem is the list in {{Connection}} _should_ be a list of open {{Statements}}. The implication being that closing a {{Statement}} removes itself from that list. Unfortunately this is not being done.
The list is necessary. But it needs to be maintained by {{Statement}} because only it knows when it is asked to close.
As a note this has nothing to do with pooling. The JDBC driver pooling ({{PooledConnection}}) will handle the issues around {{Statement}} creation and does not engage the physical {{Connection}} so the connection can be reused because none of that state is placed at the physical level and is therefore reusable without closing it.
Thanks for catching this bug. I'll create a fix for this.
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Priority: Minor
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3244) JDBC CassandraConnection may lead
to memory leak when used in a pool
Posted by "Rick Shaw (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rick Shaw updated CASSANDRA-3244:
---------------------------------
Attachment: 3244-v1.txt
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Priority: Minor
> Attachments: 3244-v1.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Rick Shaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113452#comment-13113452 ]
Rick Shaw commented on CASSANDRA-3244:
--------------------------------------
Creating a {{Statement}} every time you need one is unnecessary but not illegal. The assumption is the {{Statement}} is closed when Spring is done, which should be just fine. The problem is the list in {{Connection}} _should_ be a list of open {{Statements}}. The implecation being that closing a {{Statement}} removes itself from that list. Unfortunately this is not being done.
The list is necessary. But it needs to be maintained by {{Statement}} because only it knows when it is asked to close.
As a note this has nothing to do with pooling. The JDBC driver pooling ({{PooledConnection}}) will handle the issues around {{Statement}} creation and does not engage the physical {{Connection}} so the connection can be reused because none of that state is placed at the physical level and is therefore reusable without closing it.
Thanks for catching this bug. I'll create a fix for this.
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Priority: Minor
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121442#comment-13121442 ]
Hudson commented on CASSANDRA-3244:
-----------------------------------
Integrated in Cassandra #1142 (See [https://builds.apache.org/job/Cassandra/1142/])
remove statements from connection's list, when closed manually
patch by Rick Shaw; reviewed by Patricio Echague and jbellis for CASSANDRA-3244
jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179399
Files :
* /cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraConnection.java
* /cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraStatement.java
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Attachments: 3244-v1.txt, 3244-v2.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3244) JDBC CassandraConnection may lead
to memory leak when used in a pool
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3244:
--------------------------------------
Affects Version/s: (was: 1.0.0)
Fix Version/s: (was: 0.8.8)
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Attachments: 3244-v1.txt, 3244-v2.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Patricio Echague (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117484#comment-13117484 ]
Patricio Echague commented on CASSANDRA-3244:
---------------------------------------------
Rick, thanks for the patch.
I think think it will lead to a leak. The statement cache needs to be handled by the pool instead of by the driver itself.
Pools like BoneCP do not close the connection.
Also DBCP doesn't either. See:
http://svn.apache.org/viewvc/commons/proper/dbcp/trunk/src/java/org/apache/commons/dbcp2/PoolableConnection.java?view=markup
Thoughts?
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Fix For: 0.8.7
>
> Attachments: 3244-v1.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Patricio Echague (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113151#comment-13113151 ]
Patricio Echague commented on CASSANDRA-3244:
---------------------------------------------
Hi Rick. I browsed Spring source code, and I saw the JDBCTemplate creates a statement every time a query is executed.
This is a snippet of code:
{code}
//-------------------------------------------------------------------------
// Methods dealing with static SQL (java.sql.Statement)
//-------------------------------------------------------------------------
public <T> T execute(StatementCallback<T> action) throws DataAccessException {
Assert.notNull(action, "Callback object must not be null");
Connection con = DataSourceUtils.getConnection(getDataSource());
Statement stmt = null;
try {
Connection conToUse = con;
if (this.nativeJdbcExtractor != null &&
this.nativeJdbcExtractor.isNativeConnectionNecessaryForNativeStatements()) {
conToUse = this.nativeJdbcExtractor.getNativeConnection(con);
}
stmt = conToUse.createStatement(); // <-------- here ---------
applyStatementSettings(stmt);
Statement stmtToUse = stmt;
if (this.nativeJdbcExtractor != null) {
stmtToUse = this.nativeJdbcExtractor.getNativeStatement(stmt);
}
T result = action.doInStatement(stmtToUse);
handleWarnings(stmt);
return result;
}
{code}
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Priority: Minor
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3244) JDBC
CassandraConnection may lead to memory leak when used in a pool
Posted by "Rick Shaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113452#comment-13113452 ]
Rick Shaw edited comment on CASSANDRA-3244 at 9/23/11 2:04 PM:
---------------------------------------------------------------
Creating a {{Statement}} every time you need one is unnecessary but not illegal. The assumption is the {{Statement}} is closed when Spring is done, which should be just fine. The problem is the list in {{Connection}} _should_ be a list of open {{Statements}}. The implication being that closing a {{Statement}} removes itself from that list. Unfortunately this is not being done.
The list is necessary. But it needs to be maintained by {{Statement}} because only it knows when it is asked to close.
As a note this has nothing to do with pooling. The JDBC driver pooling ({{PooledConnection}}) will handle the issues around {{Statement}} creation and does not engage the physical {{Connection}} so the connection can be reused because none of that state is placed at the physical level and is therefore reusable without closing it.
Thanks for catching this bug. I'll create a fix for this.
was (Author: ardot):
Creating a {{Statement}} every time you need one is unnecessary but not illegal. The assumption is the {{Statement}} is closed when Spring is done, which should be just fine. The problem is the list in {{Connection}} _should_ be a list of open {{Statements}}. The implecation being that closing a {{Statement}} removes itself from that list. Unfortunately this is not being done.
The list is necessary. But it needs to be maintained by {{Statement}} because only it knows when it is asked to close.
As a note this has nothing to do with pooling. The JDBC driver pooling ({{PooledConnection}}) will handle the issues around {{Statement}} creation and does not engage the physical {{Connection}} so the connection can be reused because none of that state is placed at the physical level and is therefore reusable without closing it.
Thanks for catching this bug. I'll create a fix for this.
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Priority: Minor
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3244) JDBC CassandraConnection may lead
to memory leak when used in a pool
Posted by "Rick Shaw (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rick Shaw updated CASSANDRA-3244:
---------------------------------
Attachment: 3244-v2.txt
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Fix For: 0.8.7
>
> Attachments: 3244-v1.txt, 3244-v2.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3244) JDBC CassandraConnection may lead
to memory leak when used in a pool
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3244:
--------------------------------------
Reviewer: patricioe
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Fix For: 0.8.7
>
> Attachments: 3244-v1.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Patricio Echague (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113091#comment-13113091 ]
Patricio Echague commented on CASSANDRA-3244:
---------------------------------------------
The list just holds statement objects
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Priority: Minor
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Patricio Echague (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117540#comment-13117540 ]
Patricio Echague commented on CASSANDRA-3244:
---------------------------------------------
Appreciate your thorough analysis.
The reason I brought it up is because I'm working on a connection pool for Cassandra JDBC Driver based on the way Hector handles failover, load balancing and retry mechanism. I also based the design in the way BoneCP works (although I haven't implemented yet the statement cache, which is freed up when the connection is released).
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Assignee: Rick Shaw
> Priority: Minor
> Labels: JDBC
> Fix For: 0.8.7
>
> Attachments: 3244-v1.txt
>
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113064#comment-13113064 ]
Jonathan Ellis commented on CASSANDRA-3244:
-------------------------------------------
what is the list used for?
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Priority: Minor
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3244) JDBC CassandraConnection may
lead to memory leak when used in a pool
Posted by "Rick Shaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113139#comment-13113139 ]
Rick Shaw commented on CASSANDRA-3244:
--------------------------------------
It keeps them on a list because {{Statement}} structures are associated with the creating {{Connection}}. Closing the {{Connection}} closes all {{Statement}} items. {{Statement}} structures are re-usable; you _usually_ only open one or maybe a few. But it does keep track of all you create. I don't see how that implies a memory leak?
> JDBC CassandraConnection may lead to memory leak when used in a pool
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3244
> Project: Cassandra
> Issue Type: Improvement
> Components: Drivers
> Affects Versions: 1.0.0
> Reporter: Patricio Echague
> Priority: Minor
>
> I may be wrong here but I noticed that the implementations of CassandraConnection#createStatement() and CassandraConnection#prepareStatement() keep(cache) the created Statement/PrepareStatement internally in a List.
> They list is freed up only during CassandraConnection.close() which makes me think that, if the connection object is used in a pool implementation, it will lead to a memory leak as it will hold every single statement that is used to interact with the DB until the connection gets closed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira