I am having an out of memory condition on Derby 10.9.2.0 in our production environment. Derby is given 8G maximum heap but I am able to get a heap dump periodically and analyze it via Eclipse Memory Analyzer. I see a couple of strange things and was wondering if I can attach a screen shot here or not? Maybe someone might see something obvious. Brett ________________________________ Canoga Perkins 20600 Prairie Street Chatsworth, CA 91311 (818) 718-6300 This e-mail and any attached document(s) is confidential and is intended only for the review of the party to whom it is addressed. If you have received this transmission in error, please notify the sender immediately and discard the original message and any attachment(s).
On 5/19/2015 11:00 AM, Bergquist, Brett wrote:
> I found how to export to a text file. So here is one of the areas that the leak detection pointed out:
>
> '- buf, lock java.io.StringWriter @ 0xfffffffd86693928 | 48 | 706,545,952
> '- stringWriter org.apache.derby.iapi.error.ErrorStringBuilder @ 0xfffffffd86693900
> '- errorStringBuilder org.apache.derby.iapi.services.context.ContextManager @ 0xfffffffd86637d80
I'm not totally sure how to read this, but I think it's saying that the
errorStringBuilder field in the ContextManager has 706 meg of string data in it.
From what I can tell by looking at ContextManager.java, the
errorStringBuilder field is only supposed to contain data if
a fairly severe error has occurred, and Derby is trying to
write that error to derby.log:
/**
* clean up error and print it to derby.log. Extended diagnosis including
* thread dump to derby.log and javaDump if available, will print if the
* database is active and severity is greater than or equals to
* SESSTION_SEVERITY or as configured by
* derby.stream.error.extendedDiagSeverityLevel property
*
* @param error the error we want to clean up
* @param diagActive
* true if extended diagnostics should be considered,
* false not interested of extended diagnostic information
* @return true if the context manager is shutdown, false otherwise.
*/
public boolean cleanupOnError(Throwable error, boolean diagActive)
Now, 706 meg is a LOT of string data, but it's still a long ways from 8GB.
Did you end up getting any information in your derby.log?
I'm totally speculating, but it's possible that the ContextManager heap usage
is a secondary problem, that the real problem is:
what caused Derby to invoke ContextManager.cleanupOnError() in the first place?
Hope this is of some help and gives you some things to pursue.
thanks,
bryan
I found how to export to a text file. So here is one of the areas that the leak detection pointed out:
Class Name | Shallow Heap | Retained Heap
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| |
char[353272922] @ 0xfffffffd86696ae8 Fri Apr 17 15:50:47 GMT 2015 Thread[DRDAConnThread_16,5,main] (XID = 56622608086), (SESSIONID = 196298978), (DATABASE = csemdb), (DRDAID = \ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd.\ufffd\ufffd-881859672748842583{534017}), Cleanup action starting\u0...| 706,545,872 | 706,545,872
'- value java.lang.StringBuffer @ 0xfffffffd86693958 | 32 | 706,545,904
'- buf, lock java.io.StringWriter @ 0xfffffffd86693928 | 48 | 706,545,952
'- stringWriter org.apache.derby.iapi.error.ErrorStringBuilder @ 0xfffffffd86693900 | 40 | 706,546,072
'- errorStringBuilder org.apache.derby.iapi.services.context.ContextManager @ 0xfffffffd86637d80 | 104 | 706,549,528
'- myContextManager org.apache.derby.jdbc.XATransactionState @ 0xfffffffd86637fa8 | 96 | 168
'- value java.util.Hashtable$Entry @ 0xfffffffd86674f68 | 48 | 48
'- [46] java.util.Hashtable$Entry[95] @ 0xfffffffd75603cc8 | 784 | 1,120
'- table java.util.Hashtable @ 0xfffffffd747970d8 | 64 | 1,184
'- connectionTable org.apache.derby.jdbc.ResourceAdapterImpl @ 0xfffffffd74797090 | 40 | 1,224
|- ra org.apache.derby.impl.drda.XADatabase @ 0xfffffffec6a88120 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xfffffffec6b32ee8 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xfffffffec6bb8e68 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xfffffffec6c6fff8 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xfffffffec6d14008 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xfffffffec6dc05f8 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff56c462b8 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff56c72008 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff56cc3a30 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff56d68050 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff56e0bd00 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff56e5f190 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff56f11490 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff56f88c00 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff56fc5278 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff57077808 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff5708c580 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff5720a8c0 | 240 | 4,112
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff573b3908 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff575db540 | 240 | 4,112
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff57713920 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff57891350 | 240 | 4,104
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff57a6e8b8 | 240 | 4,112
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff57ac4eb0 | 240 | 4,112
|- ra org.apache.derby.impl.drda.XADatabase @ 0xffffffff57c3e500 | 240 | 4,104
From: Bergquist, Brett [mailto:BBergquist@canoga.com]
Sent: Tuesday, May 19, 2015 1:39 PM
To: derby-dev@db.apache.org
Subject: Having an out of memory condition on Derby 10.9.2.0
I am having an out of memory condition on Derby 10.9.2.0 in our production environment. Derby is given 8G maximum heap but I am able to get a heap dump periodically and analyze it via Eclipse Memory Analyzer.
I see a couple of strange things and was wondering if I can attach a screen shot here or not? Maybe someone might see something obvious.
Brett
________________________________
Canoga Perkins
20600 Prairie Street
Chatsworth, CA 91311
(818) 718-6300
This e-mail and any attached document(s) is confidential and is intended only for the review of the party to whom it is addressed. If you have received this transmission in error, please notify the sender immediately and discard the original message and any attachment(s).
________________________________
Canoga Perkins
20600 Prairie Street
Chatsworth, CA 91311
(818) 718-6300
This e-mail and any attached document(s) is confidential and is intended only for the review of the party to whom it is addressed. If you have received this transmission in error, please notify the sender immediately and discard the original message and any attachment(s).