You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@xmlbeans.apache.org by Jamie Zyskowski <jz...@bea.com> on 2005/01/05 19:46:11 UTC
RE: Memory Leak Problems in v1.0.1 w/selectPath()
There is a bug open against select path in a multi-threaded env causing
a JavaLangIllegalStateException:
http://issues.apache.org/jira/browse/XMLBEANS-84
This could be related to the memory leak issue that Murali and Venkatesh
have found. Thanx, Jacob for pointing this out to me.
Jamie
-----Original Message-----
From: Jamie Zyskowski
Sent: Wednesday, January 05, 2005 9:51 AM
To: dev@xmlbeans.apache.org
Cc: muvar33@yahoo.com; svejb@yahoo.com
Subject: FW: Memory Leak Problems in v1.0.1 w/selectPath()
Hi dev list,
Murali and Venkatesh (see user list thread below) are working on a
project that uses a compiled schema in a web application. They are
finding memory leaks while calling selectPath() in a custom ThreadPool
(all the threads are pulled out from a custom thread pool). They are
using v1.0.3. The schema is attached.
The main question they have is:
> What is the right way to use the "selectPath" in
> XmlObjects in a "Thread Pooled" environment (say in
> Tomcat server) so that there won't be any dangling
> references of xmlbeans objects referenced by the
> Thread Locals.
I'm going to try to put a repro together from the info they have
provided, but I am not familiar with the cursor implementation, so I'm
unable to answer their question about changes to the v1 source code to
fix their problem. Can someone take a look and help them out?
Thanx,
Jamie
-----Original Message-----
From: MV [mailto:muvar33@yahoo.com]
Sent: Tuesday, January 04, 2005 11:21 PM
To: user@xmlbeans.apache.org
Subject: Memory Leak Problems
Hi Jamie,
I have another update.
I did couple of changes in the xmlbeans source code
and the problem has gone. Kindly let me know whether
the changes might have any side effects.
- Modified the t1_CachedCursorData from "private" to
"public" in CursorData
- Added a method in the XmlBeans to reset the
"QNameCache".
public static void resetQNameCache ( )
{
_threadLocalLoaderQNameCache.set(new
QNameCache(32));
}
Then i added the following snippet in my control
servlet, so that i gets executed before processing the
request.
ArrayList arrList = (ArrayList)
CursorData.tl_CachedCursorData.get();
if( arrList != null) arrList.clear();
XmlBeans.resetQNameCache();
By doing the above changes the threads are pristine
without any threadlocal data due the earlier
"selectpaths" and now those objects are getting
garbage collected properly.
I will be doing some further testing tomorrow morning.
Would this cause any issues ?
thanks
Murali
--- MV <mu...@yahoo.com> wrote:
> Hi Jamie,
> Myself and Venkatesh are working on the same
> project.
> We did some further analysis and here is our update.
>
> We tried to create a test program to load these xmls
> and do some "selectPath" on these XmlObjects. We
> created multiple threads and did quite a number of
> select paths. In all these cases the "Cursor"
> objects
> where perfectly garbage collected. But when we try
> to
> run the same test using a "ThreadPool" (all the
> threads are pulled out from a custom thread pool )
> we
> noticed these "Cursor" objects were not garbage
> collected.
>
> We profiled them and noticed the following. All the
> org.apache.xmlbeans.impl.store.Cursor instances has
> incoming reference from
> org.apache.xmlbeans.impl.store.Root$ChangeClient.
> There seems to be too many of these
> "Root$ChangeClient" and finally they are referenced
> by
> org.apache.xmlbeans.impl.store.Root which in turn is
> referenced by
> org.apache.xmlbeans.impl.store.CursorData which is
> stored in the "ThreadLocal". Hence none of them are
> Garbage collected.
>
> We walked the source code of "Cursor" and noticed
> that
> every time we do a 'selectPath' it is trying to
> register itself to the "Root" thru the
> "Root$ChangeClient". We removed this dependency and
> tested it . Now the "Cursors" are garbage collected,
> but still we see
> org.apache.xmlbeans.impl.store.Splay$Begin and other
> classes.
>
> Not sure whether we are hitting in right direction.
>
> What is the right way to use the "selectPath" in
> XmlObjects in a "Thread Pooled" environment ( say in
> Tomcat server) so that there won't be any dangling
> references of xmlbeans objects referenced by the
> Thread Locals.
>
> Any pointers would be very much appreciated.
>
> thanks
> MV
>
>
>
> Date: Tue, 4 Jan 2005 10:44:57 -0800 (PST)
> From: Java SV <sv...@yahoo.com>
> Subject: Memory Leak Problems
> Content-Type: multipart/mixed;
> boundary="0-2099955324-1104864297=:69138"
>
>
> Hi Jamie,
>
> Thanks again for your quick response. Please find
> the
> schema attached.
>
> Thanks,
> Venkatesh
>
> --- Jamie Zyskowski <jz...@bea.com> wrote:
>
> > Hi Venkat,
> >
> > Could you provide the schema you compiled to get
> > these xmlbean classes?
> > With it, I can try to create a repro scenario.
> >
> > Thanx,
> > Jamie
> >
> > -----Original Message-----
> > From: Java SV [mailto:svejb@yahoo.com]
> > Sent: Monday, January 03, 2005 5:54 PM
> > To: user@xmlbeans.apache.org
> > Cc: Jamie Zyskowski; svejb@yahoo.com
> > Subject: RE: Memory Leak Problems
> >
> > Hi Jamie,
> >
> > Thanks for your response.
> >
> > We use XMLBeans as a Java Data Binding tool to
> > essentially load the XML documents and get the
> > objectified representation of XML.
> >
> > Environment: Tomcat 5.0.16, Solaris as well as
> > Windows
> > 2000
> >
> > Sample Code:
> >
> > URL urlHits = new URL(hitsURL); // Get the XML
> > Stream
> > InputStream inputStream =
> > urlHits.openConnection().getInputStream();
> >
> > HITSDocument hitsDocument =
> > HITSDocument.Factory.parse(inputStream);
> >
> > HITSDocument.HITS body =
> hitsDocument.getHITS();
> > HITSDocument.HITS.MSGHEAD msgHead =
> > body.getMSGHEAD();
> >
> > HITSConnection hitsConnection = null;
> > if (msgHead == null) { // Connection Creation
> is
> > Success
> > hitsConnection = new HITSConnection();
> > XmlObject[] xmlObjects =
> > body.getGENHEAD().selectPath("CONNID");
> >
> >
>
hitsConnection.setConnectionID(((XmlObjectBase)xmlObjects[0]).getStringV
> > alue());
> > } else {
> > String errorCode =
> > translateHitsErrorCode(msgHead.getMSGCODE(),
> > HITSErrorCodes.CONNECTION_NOT_AVAILABLE);
> > throw new HITSRuntimeException(errorCode);
> > }
> >
> > We basically peek into the result XML and look for
> > certain elements of interest. Never get explicit
> > cuursors but only execute a bunch of selectPath()
> > methods.
> >
> > We finally call xmlObject.newDomNode() to send the
> > result Node to the XSLT Transformer.
> >
> > Again, your help is greatly appreciated.
> >
> > Thanks,
> > Venkat
>
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - Helps protect you from nasty viruses.
> http://promotions.yahoo.com/new_mail
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> user-unsubscribe@xmlbeans.apache.org
> For additional commands, e-mail:
> user-help@xmlbeans.apache.org
>
>
__________________________________
Do you Yahoo!?
All your favorites on one personal page - Try My Yahoo!
http://my.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org
RE: Memory Leak Problems in v1.0.1 w/selectPath()
Posted by Venkat Seeth <sv...@yahoo.com>.
Hi Jamie/Jacob,
This is related to Jaxen when executing complex XPath
queries. We did uncover this bug with Jaxen and
reported to you guys. Hence we are fully aware of this
bug.
But, in our case, we are using simple XPath queries
that do not use Jaxen.
Cursor is holding on to cache in ThreadLocals which is
not cleared at all. Hence, there are dangling
references which cannot be collected.
Its very easy to reproduce what we are talking about.
Create a ThreadPool. Assign each thread in the pool to
perform the following.
* Open a cursor,
* walk thru some selections (iterate)
* Dispose the cursor
Profile to see that CursorData's ThreadLocal holds on
to Cursor objects and many more related to xmlbeans.
There is no callback to indicate to xmlbeans that I'm
done and please do away with your cache.
Waiting for your input, please advise.
Thanks,
Venkatesh
--- Jamie Zyskowski <jz...@bea.com> wrote:
> There is a bug open against select path in a
> multi-threaded env causing
> a JavaLangIllegalStateException:
> http://issues.apache.org/jira/browse/XMLBEANS-84
>
> This could be related to the memory leak issue that
> Murali and Venkatesh
> have found. Thanx, Jacob for pointing this out to
> me.
>
> Jamie
>
> -----Original Message-----
> From: Jamie Zyskowski
> Sent: Wednesday, January 05, 2005 9:51 AM
> To: dev@xmlbeans.apache.org
> Cc: muvar33@yahoo.com; svejb@yahoo.com
> Subject: FW: Memory Leak Problems in v1.0.1
> w/selectPath()
>
> Hi dev list,
>
> Murali and Venkatesh (see user list thread below)
> are working on a
> project that uses a compiled schema in a web
> application. They are
> finding memory leaks while calling selectPath() in a
> custom ThreadPool
> (all the threads are pulled out from a custom thread
> pool). They are
> using v1.0.3. The schema is attached.
>
> The main question they have is:
> > What is the right way to use the "selectPath" in
> > XmlObjects in a "Thread Pooled" environment (say
> in
> > Tomcat server) so that there won't be any dangling
> > references of xmlbeans objects referenced by the
> > Thread Locals.
>
> I'm going to try to put a repro together from the
> info they have
> provided, but I am not familiar with the cursor
> implementation, so I'm
> unable to answer their question about changes to the
> v1 source code to
> fix their problem. Can someone take a look and help
> them out?
>
> Thanx,
> Jamie
>
>
>
> -----Original Message-----
> From: MV [mailto:muvar33@yahoo.com]
> Sent: Tuesday, January 04, 2005 11:21 PM
> To: user@xmlbeans.apache.org
> Subject: Memory Leak Problems
>
> Hi Jamie,
> I have another update.
>
> I did couple of changes in the xmlbeans source code
> and the problem has gone. Kindly let me know whether
> the changes might have any side effects.
>
> - Modified the t1_CachedCursorData from "private" to
> "public" in CursorData
> - Added a method in the XmlBeans to reset the
> "QNameCache".
> public static void resetQNameCache ( )
> {
> _threadLocalLoaderQNameCache.set(new
> QNameCache(32));
> }
>
> Then i added the following snippet in my control
> servlet, so that i gets executed before processing
> the
> request.
>
> ArrayList arrList = (ArrayList)
> CursorData.tl_CachedCursorData.get();
> if( arrList != null) arrList.clear();
> XmlBeans.resetQNameCache();
>
> By doing the above changes the threads are pristine
> without any threadlocal data due the earlier
> "selectpaths" and now those objects are getting
> garbage collected properly.
>
> I will be doing some further testing tomorrow
> morning.
>
> Would this cause any issues ?
>
> thanks
> Murali
> --- MV <mu...@yahoo.com> wrote:
>
> > Hi Jamie,
> > Myself and Venkatesh are working on the same
> > project.
> > We did some further analysis and here is our
> update.
> >
> > We tried to create a test program to load these
> xmls
> > and do some "selectPath" on these XmlObjects. We
> > created multiple threads and did quite a number of
> > select paths. In all these cases the "Cursor"
> > objects
> > where perfectly garbage collected. But when we try
> > to
> > run the same test using a "ThreadPool" (all the
> > threads are pulled out from a custom thread pool )
> > we
> > noticed these "Cursor" objects were not garbage
> > collected.
> >
> > We profiled them and noticed the following. All
> the
> > org.apache.xmlbeans.impl.store.Cursor instances
> has
> > incoming reference from
> > org.apache.xmlbeans.impl.store.Root$ChangeClient.
> > There seems to be too many of these
> > "Root$ChangeClient" and finally they are
> referenced
> > by
> > org.apache.xmlbeans.impl.store.Root which in turn
> is
> > referenced by
> > org.apache.xmlbeans.impl.store.CursorData which
> is
> > stored in the "ThreadLocal". Hence none of them
> are
> > Garbage collected.
> >
> > We walked the source code of "Cursor" and noticed
> > that
> > every time we do a 'selectPath' it is trying to
> > register itself to the "Root" thru the
> > "Root$ChangeClient". We removed this dependency
> and
> > tested it . Now the "Cursors" are garbage
> collected,
> > but still we see
> > org.apache.xmlbeans.impl.store.Splay$Begin and
> other
> > classes.
> >
> > Not sure whether we are hitting in right
> direction.
> >
> > What is the right way to use the "selectPath" in
> > XmlObjects in a "Thread Pooled" environment ( say
> in
> > Tomcat server) so that there won't be any dangling
> > references of xmlbeans objects referenced by the
> > Thread Locals.
> >
> > Any pointers would be very much appreciated.
> >
> > thanks
> > MV
> >
> >
> >
> > Date: Tue, 4 Jan 2005 10:44:57 -0800 (PST)
> > From: Java SV <sv...@yahoo.com>
> > Subject: Memory Leak Problems
> > Content-Type: multipart/mixed;
> > boundary="0-2099955324-1104864297=:69138"
> >
> >
> > Hi Jamie,
> >
> > Thanks again for your quick response. Please find
> > the
> > schema attached.
> >
> > Thanks,
> > Venkatesh
> >
> > --- Jamie Zyskowski <jz...@bea.com> wrote:
> >
> > > Hi Venkat,
> > >
> > > Could you provide the schema you compiled to get
> > > these xmlbean classes?
> > > With it, I can try to create a repro scenario.
> > >
> > > Thanx,
> > > Jamie
> > >
> > > -----Original Message-----
> > > From: Java SV [mailto:svejb@yahoo.com]
> > > Sent: Monday, January 03, 2005 5:54 PM
> > > To: user@xmlbeans.apache.org
>
=== message truncated ===
__________________________________
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.
http://promotions.yahoo.com/new_mail
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org