You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-user@db.apache.org by Clebert Suconic <cl...@gmail.com> on 2023/03/16 15:29:10 UTC
Memory Leak in org.apache.derby.impl.io.vfmem.DataStore
I work in ActiveMQ Artemis, and we have a series of tests that will
use derby memory, with the uri as
"jdbc:derby:memory:testName;create=true"
So, every test will have its own database, and we do that for
isolating the tests.
at the end of the test, we call destroy and shutdown of derby with:
try {
DriverManager.getConnection("jdbc:derby:memory:test;destroy=true");
} catch (Exception ignored) {
// it always throws an exception on shutdown
}
try {
DriverManager.getConnection("jdbc:derby:;destroy=true;shutdown=true;destroy=true");
} catch (Exception e) {
So, no matter how hard I try, there is always a memory leak from
DataStore being held at a HashMap, from VFMemoryStorageFactory:tables
I presume. Or there is a "leaked" list (a reference of a reference
somewhere).
I recently developed a framework / library to test and find for memory
leaks, and I have this reproduced here:
https://github.com/clebertsuconic/check-leak/blob/derby-test/examples/junit-example/src/test/java/io/github/checkleak/junitexample/DerbyLeakTest.java
So, either I'm supposed to do something else in order to Derby to
remove the hashmap, or there's an actual leak.
So, I need to either identify what I'm doing wrong...
or if this is an actual issue, I need help to build derby and debug
the issue further.
I tried building with:
ant buildjars
mvn -f maven2/pom.xml install
( I need to consume derby in Maven, so I can test it in my process).
I wasn't sure this would be appropriate for the user's list as it
might be an actual bug that I need help with how to develop a fix.
*******************************************************************************************************************************
References to obj[4]=TOSTRING(org.apache.derby.impl.io.vfmem.DataStore@4dbad37),
class=org.apache.derby.impl.io.vfmem.DataStore
!-- FieldReference name='value'::=java.lang.Object
java.util.HashMap$Node.value on object
TOSTRING(/Users/clebertsuconic/work/apache/check-leak/examples/junit-example/test4=org.apache.derby.impl.io.vfmem.DataStore@4dbad37),
class=java.util.HashMap$Node
!--!-- arrayRef [Ljava.util.HashMap$Node;[5] id=@1883788127
!--!--!-- FieldReference name='table'::=transient
java.util.HashMap$Node[] java.util.HashMap.table on object
OBJ(java.util.HashMap@1492156162)
!--!--!--!-- StaticFieldReference public static final int
org.apache.derby.io.StorageFactory.VERSION_NUMBER
!-- arrayRef [Ljava.lang.Object;[4] id=@987950392
!--!-- Reference inside a method -
io.github.checkleak.core.CheckLeak::exploreObjectReferences
!--!-- Reference inside a method -
io.github.checkleak.core.CheckLeak::exploreObjectReferences
--
Clebert Suconic
Re: Memory Leak in org.apache.derby.impl.io.vfmem.DataStore
Posted by Clebert Suconic <cl...@gmail.com>.
Thank you so much... the drop did the trick.
On Thu, Mar 16, 2023 at 2:49 PM Stanimir Stamenkov via derby-user
<de...@db.apache.org> wrote:
>
> Thu, 16 Mar 2023 20:06:09 +0200, /Stanimir Stamenkov/:
>
> > I'm not finding destroy=true attribute but a drop=true one:
> >
> > * Setting attributes for the database connection URL
> > <https://db.apache.org/derby/docs/10.16/ref/rrefattrib24612.html> (Derby
> > Reference Manual)
> >
> > that is specific for in-memory databases. destroy=true is likely doing
> > nothing but have you logged any possible exception in the catch blocks
> > to see if Derby isn't actually failing the operation altogether?
>
> You may use drop=true with a specific in-memory database:
>
> try {
> DriverManager.getConnection(
> "jdbc:derby:memory:TestDB;drop=true");
> } catch (SQLException e) {
> if (e.getSQLState().equals("08006")) {
> // Database 'TestDB' dropped.
> System.out.println(e.getMessage());
> } else {
> throw e; // unexpected
> }
> }
>
> You may use shutdown=true with any embedded database (in-memory or
> persisted):
>
> try {
> DriverManager.getConnection(
> "jdbc:derby:memory:TestDB;shutdown=true");
> } catch (SQLException e) {
> if (e.getSQLState().equals("08006")) {
> // Database 'TestDB' shutdown.
> System.out.println(e.getMessage());
> } else {
> throw e; // unexpected
> }
> }
>
> You may shut the complete engine down:
>
> try {
> DriverManager.getConnection(
> "jdbc:derby:;shutdown=true");
> } catch (SQLException e) {
> if (e.getSQLState().equals("XJ015")) {
> // Derby engine shutdown.
> System.out.println(e.getMessage());
> } else {
> throw e; // unexpected
> }
> }
>
> You may want deregister=false with shutdown=true:
>
> try {
> DriverManager.getConnection(
> "jdbc:derby:;shutdown=true;deregister=false");
> } catch (SQLException e) {
> if (e.getSQLState().equals("XJ015")) {
> // Derby engine shutdown.
> System.out.println(e.getMessage());
> } else {
> throw e; // unexpected
> }
> }
>
> --
> Stanimir
--
Clebert Suconic
Re: Memory Leak in org.apache.derby.impl.io.vfmem.DataStore
Posted by Stanimir Stamenkov via derby-user <de...@db.apache.org>.
Thu, 16 Mar 2023 20:06:09 +0200, /Stanimir Stamenkov/:
> I'm not finding destroy=true attribute but a drop=true one:
>
> * Setting attributes for the database connection URL
> <https://db.apache.org/derby/docs/10.16/ref/rrefattrib24612.html> (Derby
> Reference Manual)
>
> that is specific for in-memory databases. destroy=true is likely doing
> nothing but have you logged any possible exception in the catch blocks
> to see if Derby isn't actually failing the operation altogether?
You may use drop=true with a specific in-memory database:
try {
DriverManager.getConnection(
"jdbc:derby:memory:TestDB;drop=true");
} catch (SQLException e) {
if (e.getSQLState().equals("08006")) {
// Database 'TestDB' dropped.
System.out.println(e.getMessage());
} else {
throw e; // unexpected
}
}
You may use shutdown=true with any embedded database (in-memory or
persisted):
try {
DriverManager.getConnection(
"jdbc:derby:memory:TestDB;shutdown=true");
} catch (SQLException e) {
if (e.getSQLState().equals("08006")) {
// Database 'TestDB' shutdown.
System.out.println(e.getMessage());
} else {
throw e; // unexpected
}
}
You may shut the complete engine down:
try {
DriverManager.getConnection(
"jdbc:derby:;shutdown=true");
} catch (SQLException e) {
if (e.getSQLState().equals("XJ015")) {
// Derby engine shutdown.
System.out.println(e.getMessage());
} else {
throw e; // unexpected
}
}
You may want deregister=false with shutdown=true:
try {
DriverManager.getConnection(
"jdbc:derby:;shutdown=true;deregister=false");
} catch (SQLException e) {
if (e.getSQLState().equals("XJ015")) {
// Derby engine shutdown.
System.out.println(e.getMessage());
} else {
throw e; // unexpected
}
}
--
Stanimir
Re: Memory Leak in org.apache.derby.impl.io.vfmem.DataStore
Posted by Stanimir Stamenkov via derby-user <de...@db.apache.org>.
Thu, 16 Mar 2023 11:29:10 -0400, /Clebert Suconic/:
> at the end of the test, we call destroy and shutdown of derby with:
>
> try {
> DriverManager.getConnection("jdbc:derby:memory:test;destroy=true");
> } catch (Exception ignored) {
> // it always throws an exception on shutdown
> }
>
> try {
> DriverManager.getConnection("jdbc:derby:;destroy=true;shutdown=true;destroy=true");
> } catch (Exception e) {
>
> So, no matter how hard I try, there is always a memory leak from
> DataStore being held at a HashMap, from VFMemoryStorageFactory:tables
> I presume. Or there is a "leaked" list (a reference of a reference
> somewhere).
I'm not finding destroy=true attribute but a drop=true one:
* Setting attributes for the database connection URL
<https://db.apache.org/derby/docs/10.16/ref/rrefattrib24612.html> (Derby
Reference Manual)
that is specific for in-memory databases. destroy=true is likely doing
nothing but have you logged any possible exception in the catch blocks
to see if Derby isn't actually failing the operation altogether? I'm
always logging the SQLException message (but not the stack trace, if the
SQL code is as expected) in a catch block.
FWIW, I'm using just shutdown=true for a specific in-memory database
instance and not the entire engine (drop=true appears to result in
unwanted time overhead for me) for similar test setup and I haven't
experienced memory leaks. It could be my test suite is not big enough
for me to notice, also.
--
Stanimir