You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by de...@db.apache.org on 2004/09/29 21:21:32 UTC
[jira] Created: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Message:
A new issue has been created in JIRA.
---------------------------------------------------------------------
View the issue:
http://issues.apache.org/jira/browse/DERBY-23
Here is an overview of the issue:
---------------------------------------------------------------------
Key: DERBY-23
Summary: just booting jdbc driver and shutting down seem to leak memory
Type: Bug
Status: Unassigned
Priority: Minor
Project: Derby
Components:
Services
Versions:
10.0.2.0
Assignee:
Reporter: Tulika Agrawal
Created: Wed, 29 Sep 2004 12:20 PM
Updated: Wed, 29 Sep 2004 12:20 PM
Description:
Reporting for Daniel John Debrunner.
Doing simple boot and shutdown of the driver in a loop
seem to grow the heap forever:
new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
DriverManager.getConnection("jdbc:derby:;shutdown=true");
//shutdown the cloudscape instance completely
after booting and shutting down 1516 times, memory used is
~~41931056 bytes.
Sample GC Output:
1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
---------------------------------------------------------------------
JIRA INFORMATION:
This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
Re: [jira] Updated: (DERBY-23) just booting jdbc driver and shutting
down seem to leak memory
Posted by "Bernt M. Johnsen" <Be...@Sun.COM>.
It looks sound to me, but I would prefer a second opinion from
somebody more familiar with this area of the code before I commit the
patch.
Bernt
>>>>>>>>>>>> Knut Anders Hatlen (JIRA) wrote (2005-09-19 09:59:55):
> [ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
>
> Knut Anders Hatlen updated DERBY-23:
> ------------------------------------
>
> Attachment: DERBY-23.diff
> DERBY-23.stat
>
> The memory leak comes from three sources:
>
> 1) The AntiGC thread is sometimes not terminated. This happens when
> Derby is shut down right after the driver is loaded, because the
> run() method in AntiGC starts with setting a boolean variable which
> is used to decide when the thread should stop. The same variable is
> set when a request to shut down Derby is sent, and if the shutdown
> is requested before the AntiGC thread has started running, the
> request to shut down Derby might not be noticed by AntiGC which
> will run forever.
>
> Solution: Don't set the boolean variable in AntiGC's run() method. It
> is enough that it is being set when the object is initialized.
>
> 2) Every time the driver is loaded, a new ThreadGroup is created. They
> don't seem to be garbage collected even when all their threads have
> finished.
>
> Solution: Set the daemon property on the ThreadGroups. This way a
> ThreadGroup will be destroyed when its last running thread (or thread
> group) is destroyed.
>
> 3) The Java Heap Analysis Tool reported that a huge number of
> ContextManager and ContextService objects were kept in the
> heap. The objects were not accessible from the root set, but they
> were not garbage collected, probably because of some cyclic
> references which the gc cannot handle.
>
> Solution: Break the reference cycle by nulling out the references to
> the lists of ContextManagers when ContextService objects are stopped.
>
> The attached patch seems to fix the problems mentioned above. Derbyall
> runs fine, and top reports that the memory usage does not increase
> when running a loop where the driver is loaded (with
> Class.forName("org...EmbeddedDriver").newInstance()) and unloaded
> (with DriverManager.getConnection("jdbc:derby:;shutdown=true")). That
> is, the memory usage of course increased during the first iteration,
> but not during the approximately 8.5 million next ones.
>
> I will submit tests later.
>
> > just booting jdbc driver and shutting down seem to leak memory
> > --------------------------------------------------------------
> >
> > Key: DERBY-23
> > URL: http://issues.apache.org/jira/browse/DERBY-23
> > Project: Derby
> > Type: Bug
> > Components: Services
> > Versions: 10.0.2.0
> > Reporter: Tulika Agrawal
> > Assignee: Knut Anders Hatlen
> > Priority: Minor
> > Attachments: DERBY-23.diff, DERBY-23.stat
> >
> > Reporting for Daniel John Debrunner.
> > Doing simple boot and shutdown of the driver in a loop
> > seem to grow the heap forever:
> > new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> > DriverManager.getConnection("jdbc:derby:;shutdown=true");
> > //shutdown the cloudscape instance completely
> > after booting and shutting down 1516 times, memory used is
> > ~~41931056 bytes.
> > Sample GC Output:
> > 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> > 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> > 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> > 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> > 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> > 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> > 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> > 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
>
> --
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the administrators:
> http://issues.apache.org/jira/secure/Administrators.jspa
> -
> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
>
--
Bernt Marius Johnsen, Database Technology Group,
Sun Microsystems, Trondheim, Norway
Re: [jira] Commented: (DERBY-23) just booting jdbc driver and shutting
down seem to leak memory
Posted by "Bernt M. Johnsen" <Be...@Sun.COM>.
>>>>>>>>>>>> Satheesh Bandaram (JIRA) wrote (2005-09-19 20:06:32):
> [ http://issues.apache.org/jira/browse/DERBY-23?page=comments#action_12329889 ]
>
> Satheesh Bandaram commented on DERBY-23:
> ----------------------------------------
>
> Good fix. Wonder if adding the explanations in the code, where
> possible, would be more useful than leaving them in Jira entry.
I agree.
>
> Bernt expressed interest in commiting the change, so I am not
> commiting this to SVN.
Knut: Add explanations, and then I'll commit.
>
> > just booting jdbc driver and shutting down seem to leak memory
> > --------------------------------------------------------------
> >
> > Key: DERBY-23
> > URL: http://issues.apache.org/jira/browse/DERBY-23
> > Project: Derby
> > Type: Bug
> > Components: Services
> > Versions: 10.0.2.0
> > Reporter: Tulika Agrawal
> > Assignee: Knut Anders Hatlen
> > Priority: Minor
> > Attachments: DERBY-23.diff, DERBY-23.stat
> >
> > Reporting for Daniel John Debrunner.
> > Doing simple boot and shutdown of the driver in a loop
> > seem to grow the heap forever:
> > new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> > DriverManager.getConnection("jdbc:derby:;shutdown=true");
> > //shutdown the cloudscape instance completely
> > after booting and shutting down 1516 times, memory used is
> > ~~41931056 bytes.
> > Sample GC Output:
> > 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> > 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> > 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> > 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> > 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> > 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> > 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> > 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
>
> --
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the administrators:
> http://issues.apache.org/jira/secure/Administrators.jspa
> -
> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
>
--
Bernt Marius Johnsen, Database Technology Group,
Sun Microsystems, Trondheim, Norway
Re: [jira] Updated: (DERBY-23) just booting jdbc driver and shutting
down seem to leak memory
Posted by "Bernt M. Johnsen" <Be...@Sun.COM>.
Reviewing and testing for commit now.
>>>>>>>>>>>> Knut Anders Hatlen (JIRA) wrote (2006-01-19 17:28:43):
> [ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
>
> Knut Anders Hatlen updated DERBY-23:
> ------------------------------------
>
> Attachment: DERBY-23-npe.diff
> DERBY-23-npe.stat
> derbyall_report.txt
>
> I have attached a patch (DERBY-23-npe.diff) that addresses the
> NullPointerException messages from rawStoreDaemon threads we have seen
> in derbyall on some occasions.
>
> The NullPointerExceptions are thrown because
> ContextService.threadContextList is set to null in
> ContextService.stop(), but some of the daemon threads might still try
> to access the variable. The patch adds checks for threadContextList
> being null, and returns null or false in those cases instead of
> failing with a NullPointerException.
>
> Five tests in derbyall failed when this patch was applied, but that
> seems to be what one should expect these days. All of the failures are
> also seen in the daily regression tests.
>
> Feel free to review and comment on the patch. Thanks.
>
> > just booting jdbc driver and shutting down seem to leak memory
> > --------------------------------------------------------------
> >
> > Key: DERBY-23
> > URL: http://issues.apache.org/jira/browse/DERBY-23
> > Project: Derby
> > Type: Bug
> > Components: Services
> > Versions: 10.0.2.0
> > Reporter: Tulika Agrawal
> > Assignee: Knut Anders Hatlen
> > Priority: Minor
> > Fix For: 10.2.0.0
> > Attachments: DERBY-23-npe.diff, DERBY-23-npe.stat, DERBY-23-with_comments.diff, DERBY-23.diff, DERBY-23.stat, derbyall_report.txt
> >
> > Reporting for Daniel John Debrunner.
> > Doing simple boot and shutdown of the driver in a loop
> > seem to grow the heap forever:
> > new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> > DriverManager.getConnection("jdbc:derby:;shutdown=true");
> > //shutdown the cloudscape instance completely
> > after booting and shutting down 1516 times, memory used is
> > ~~41931056 bytes.
> > Sample GC Output:
> > 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> > 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> > 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> > 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> > 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> > 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> > 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> > 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
>
> --
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the administrators:
> http://issues.apache.org/jira/secure/Administrators.jspa
> -
> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
>
--
Bernt Marius Johnsen, Database Technology Group,
Staff Engineer, Technical Lead Derby/Java DB
Sun Microsystems, Trondheim, Norway
Re: [jira] Resolved: (DERBY-23) just booting jdbc driver and shutting
down seem to leak memory
Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Kathey Marsden <km...@sbcglobal.net> writes:
> Knut, What do you think of this one for 10.1.2?
I don't think it is ready for 10.1.2 yet. Ole pointed out to me that
after the patch was committed, the Tinderbox test on Solaris 10 x86
has failed twice.
http://www.multinet.no/~solberg/public/Apache/TinderBox_Derby/testlog/SunOS-5.10_i86pc-i386/290646-encryptionCFB_diff.txt
encryptionCFB fail *************************************************************
encryptionCFB/encryptionCFB.fail:store/encryptDatabase.sql
encryptionCFB diff -------------------------------------------------------------
********* Diff file encryptionCFB/encryptionCFB/encryptDatabase.diff
*** Start: encryptDatabase jdk1.5.0_03 encryptionCFB:encryptionCFB 2005-09-21 10:05:39 ***
119 del
< ij>
119 add
> ij> Exception in thread "derby.rawStoreDaemon" java.lang.NullPointerException
> Exception in thread "derby.rawStoreDaemon" java.lang.NullPointerException
> Exception in thread "derby.rawStoreDaemon" java.lang.NullPointerException
Test Failed.
*** End: encryptDatabase jdk1.5.0_03 encryptionCFB:encryptionCFB 2005-09-21 10:05:43 ***
I guess rawStoreDaemon tries to access one of the variables I set to
null in ContextService after the the ContextService is stopped.
Although this has happened twice with the Tinderbox test, none of the
nightly tests (six platforms) have had problems with the patch.
--
Knut Anders
Re: [jira] Resolved: (DERBY-23) just booting jdbc driver and shutting
down seem to leak memory
Posted by Kathey Marsden <km...@sbcglobal.net>.
Knut, What do you think of this one for 10.1.2?
[jira] Resolved: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Posted by "Knut Anders Hatlen (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
Knut Anders Hatlen resolved DERBY-23:
-------------------------------------
Resolution: Fixed
Fixed in revision 290644.
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assignee: Knut Anders Hatlen
> Priority: Minor
> Fix For: 10.2.0.0
> Attachments: DERBY-23-with_comments.diff, DERBY-23.diff, DERBY-23.stat
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Posted by "Daniel John Debrunner (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=comments#action_12329861 ]
Daniel John Debrunner commented on DERBY-23:
--------------------------------------------
Patch looks good to me, the explanation was clear and very useful.
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assignee: Knut Anders Hatlen
> Priority: Minor
> Attachments: DERBY-23.diff, DERBY-23.stat
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Posted by "Knut Anders Hatlen (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=comments#action_12329592 ]
Knut Anders Hatlen commented on DERBY-23:
-----------------------------------------
When I looked into this issue, I found a problem with the synchronization. If Derby is shut down right after the driver is loaded, the AntiGC thread will continue to run in most cases. Fixing this problem made the heap grow slower, but it was not enough to stop the memory leak entirely.
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Priority: Minor
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (DERBY-23) just booting jdbc driver and shutting
down seem to leak memory
Posted by "Knut Anders Hatlen (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
Knut Anders Hatlen resolved DERBY-23.
-------------------------------------
Resolution: Fixed
I'm marking this issue as resolved.
The reason why I haven't resolved it before, is that I still see OutOfMemoryError on some platforms. These errors are caused by the background threads running with a lower priority than the the main thread, and therefore many of the background threads don't get enough CPU time to shut down. If the main thread takes a break now and then, I don't see OutOfMemoryError on any platform. That is,
while (true) {
Class.forName("org.apache.derby.jdbc.EmbeddedDriver").newInstance();
DriverManager.getConnection("jdbc:derby:;shutdown=true");
Thread.sleep(1);
}
runs reliably with no memory leak on all platforms I have tested. I don't want to add a regression test for this issue, since I fear it will be timing dependent and might cause noise in the nightly testing.
I think the issue can be closed because the remaining problem is unlikely to be seen in a real-world application (who will spend all their CPU time on loading and unloading the driver?). If it turns out to be a problem, I see these possible solutions:
1) On shutdown, wait for all background threads to terminate.
2) On shutdown, raise the priority of the background threads.
If someone feels this should be addressed, please open a new issue.
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Issue Type: Bug
> Components: Services
> Affects Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assigned To: Knut Anders Hatlen
> Priority: Minor
> Fix For: 10.2.1.0
>
> Attachments: DERBY-23-npe.diff, DERBY-23-npe.stat, DERBY-23-with_comments.diff, DERBY-23.diff, DERBY-23.stat, derbyall_report.txt
>
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Posted by "Knut Anders Hatlen (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
Knut Anders Hatlen updated DERBY-23:
------------------------------------
Attachment: DERBY-23-with_comments.diff
Attached patch with comments where appropriate.
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assignee: Knut Anders Hatlen
> Priority: Minor
> Attachments: DERBY-23-with_comments.diff, DERBY-23.diff, DERBY-23.stat
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-23) just booting jdbc driver and shutting
down seem to leak memory
Posted by "Rick Hillegas (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
Rick Hillegas updated DERBY-23:
-------------------------------
Urgency: Low
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Issue Type: Bug
> Components: Services
> Affects Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assigned To: Knut Anders Hatlen
> Priority: Minor
> Fix For: 10.2.0.0
>
> Attachments: DERBY-23-npe.diff, DERBY-23-npe.stat, DERBY-23-with_comments.diff, DERBY-23.diff, DERBY-23.stat, derbyall_report.txt
>
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Posted by "Satheesh Bandaram (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=comments#action_12329889 ]
Satheesh Bandaram commented on DERBY-23:
----------------------------------------
Good fix. Wonder if adding the explanations in the code, where possible, would be more useful than leaving them in Jira entry.
Bernt expressed interest in commiting the change, so I am not commiting this to SVN.
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assignee: Knut Anders Hatlen
> Priority: Minor
> Attachments: DERBY-23.diff, DERBY-23.stat
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Posted by "Knut Anders Hatlen (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
Knut Anders Hatlen updated DERBY-23:
------------------------------------
Attachment: DERBY-23.diff
DERBY-23.stat
The memory leak comes from three sources:
1) The AntiGC thread is sometimes not terminated. This happens when
Derby is shut down right after the driver is loaded, because the
run() method in AntiGC starts with setting a boolean variable which
is used to decide when the thread should stop. The same variable is
set when a request to shut down Derby is sent, and if the shutdown
is requested before the AntiGC thread has started running, the
request to shut down Derby might not be noticed by AntiGC which
will run forever.
Solution: Don't set the boolean variable in AntiGC's run() method. It
is enough that it is being set when the object is initialized.
2) Every time the driver is loaded, a new ThreadGroup is created. They
don't seem to be garbage collected even when all their threads have
finished.
Solution: Set the daemon property on the ThreadGroups. This way a
ThreadGroup will be destroyed when its last running thread (or thread
group) is destroyed.
3) The Java Heap Analysis Tool reported that a huge number of
ContextManager and ContextService objects were kept in the
heap. The objects were not accessible from the root set, but they
were not garbage collected, probably because of some cyclic
references which the gc cannot handle.
Solution: Break the reference cycle by nulling out the references to
the lists of ContextManagers when ContextService objects are stopped.
The attached patch seems to fix the problems mentioned above. Derbyall
runs fine, and top reports that the memory usage does not increase
when running a loop where the driver is loaded (with
Class.forName("org...EmbeddedDriver").newInstance()) and unloaded
(with DriverManager.getConnection("jdbc:derby:;shutdown=true")). That
is, the memory usage of course increased during the first iteration,
but not during the approximately 8.5 million next ones.
I will submit tests later.
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assignee: Knut Anders Hatlen
> Priority: Minor
> Attachments: DERBY-23.diff, DERBY-23.stat
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-23) just booting jdbc driver and shutting
down seem to leak memory
Posted by "Knut Anders Hatlen (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
Knut Anders Hatlen updated DERBY-23:
------------------------------------
Attachment: DERBY-23-npe.diff
DERBY-23-npe.stat
derbyall_report.txt
I have attached a patch (DERBY-23-npe.diff) that addresses the
NullPointerException messages from rawStoreDaemon threads we have seen
in derbyall on some occasions.
The NullPointerExceptions are thrown because
ContextService.threadContextList is set to null in
ContextService.stop(), but some of the daemon threads might still try
to access the variable. The patch adds checks for threadContextList
being null, and returns null or false in those cases instead of
failing with a NullPointerException.
Five tests in derbyall failed when this patch was applied, but that
seems to be what one should expect these days. All of the failures are
also seen in the daily regression tests.
Feel free to review and comment on the patch. Thanks.
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assignee: Knut Anders Hatlen
> Priority: Minor
> Fix For: 10.2.0.0
> Attachments: DERBY-23-npe.diff, DERBY-23-npe.stat, DERBY-23-with_comments.diff, DERBY-23.diff, DERBY-23.stat, derbyall_report.txt
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Reopened: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Posted by "Knut Anders Hatlen (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
Knut Anders Hatlen reopened DERBY-23:
-------------------------------------
As I have mentioned on derby-dev earlier, there is a problem with the
current fix. After the fix was committed, store/encryptDatabase.sql
sometimes failed because one or more rawStoreDaemon threads printed a
NullPointerException to System.err.
For an example, see:
http://www.multinet.no/~solberg/public/Apache/TinderBox_Derby/testlog/SunOS-5.10_i86pc-i386/290646-encryptionCFB_diff.txt
The reason for this is that the fix nulls out the reference from the
ContextService to the list of ContextManagers when the ContextService
is stopped. Some threads with references to the ContextService might
still be alive when the ContextService is stopped, and they get
NullPointerExceptions when accessing the stopped ContextService.
I have only been able to reproduce this behaviour with the
store/encryptDatabase.sql test on multi-cpu machines running with a
heavy background load. It seems like the threads that get
NullPointerExceptions are orphan threads (like the ones described in
DERBY-594) that have not yet been able to start because of the heavy
background load. The problem might go away when DERBY-594 is resolved,
but I think it needs further investigation.
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assignee: Knut Anders Hatlen
> Priority: Minor
> Fix For: 10.2.0.0
> Attachments: DERBY-23-with_comments.diff, DERBY-23.diff, DERBY-23.stat
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Assigned: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Posted by "Knut Anders Hatlen (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
Knut Anders Hatlen reassigned DERBY-23:
---------------------------------------
Assign To: Knut Anders Hatlen
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assignee: Knut Anders Hatlen
> Priority: Minor
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-23) just booting jdbc driver and shutting down seem to leak memory
Posted by "Knut Anders Hatlen (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-23?page=all ]
Knut Anders Hatlen updated DERBY-23:
------------------------------------
Fix Version: 10.2.0.0
Description:
Reporting for Daniel John Debrunner.
Doing simple boot and shutdown of the driver in a loop
seem to grow the heap forever:
new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
DriverManager.getConnection("jdbc:derby:;shutdown=true");
//shutdown the cloudscape instance completely
after booting and shutting down 1516 times, memory used is
~~41931056 bytes.
Sample GC Output:
1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
was:
Reporting for Daniel John Debrunner.
Doing simple boot and shutdown of the driver in a loop
seem to grow the heap forever:
new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
DriverManager.getConnection("jdbc:derby:;shutdown=true");
//shutdown the cloudscape instance completely
after booting and shutting down 1516 times, memory used is
~~41931056 bytes.
Sample GC Output:
1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
Environment:
> just booting jdbc driver and shutting down seem to leak memory
> --------------------------------------------------------------
>
> Key: DERBY-23
> URL: http://issues.apache.org/jira/browse/DERBY-23
> Project: Derby
> Type: Bug
> Components: Services
> Versions: 10.0.2.0
> Reporter: Tulika Agrawal
> Assignee: Knut Anders Hatlen
> Priority: Minor
> Fix For: 10.2.0.0
> Attachments: DERBY-23-with_comments.diff, DERBY-23.diff, DERBY-23.stat
>
> Reporting for Daniel John Debrunner.
> Doing simple boot and shutdown of the driver in a loop
> seem to grow the heap forever:
> new org.apache.derby.jdbc.EmbeddedDriver(); //boot the driver
> DriverManager.getConnection("jdbc:derby:;shutdown=true");
> //shutdown the cloudscape instance completely
> after booting and shutting down 1516 times, memory used is
> ~~41931056 bytes.
> Sample GC Output:
> 1931.93: [Full GC 43484K->43243K(65088K), 0.1779751 secs]
> 1933.13: [Full GC 43511K->43270K(65088K), 0.1914383 secs]
> 1934.47: [Full GC 43538K->43297K(65088K), 0.1808878 secs]
> 1935.68: [Full GC 43564K->43324K(65088K), 0.4461623 secs]
> 1937.43: [Full GC 43591K->43350K(65088K), 0.1842980 secs]
> 1938.63: [Full GC 43617K->43377K(65088K), 0.1873431 secs]
> 1939.85: [Full GC 43644K->43404K(65088K), 0.1948505 secs]
> 1941.07: [Full GC 43671K->43430K(65088K), 0.1790895 secs]
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira