You are viewing a plain text version of this content. The canonical link for it is here.
Posted to xindice-dev@xml.apache.org by Alain Javier Guarnieri del Gesu <ja...@ajgdg.com> on 2003/11/01 19:03:50 UTC
Re: Other sticky issues
* Henrik Vendelbo <hv...@bluprints.com> [2003-11-01 16:29]:
> ----- Original Message -----
> From: "Alain Javier Guarnieri del Gesu" <ja...@ajgdg.com>
> To: <xi...@xml.apache.org>; <xi...@xml.apache.org>
> Sent: Thursday, October 30, 2003 1:50 PM
> Subject: Re: Predicates are ignored in Xindice XPATH?
>
>
> > * Kevin Ross <ke...@iverticalleap.com> [2003-10-22 17:18]:
> >
> > > From: Kurt Kavanaugh [mailto:kkavanaugh@teletrol.com]
> > > Sent: Wednesday, October 22, 2003 11:00 AM
> > > To: xindice-users@xml.apache.org
> > > Subject: RE: Predicates are ignored in Xindice XPATH?
> > >
> > > BTW- Has anyone from the xindice team ever contacted the saxon guys, to
> > > check on how their xquery might be integrated? I believe they may have
> > > licensing conflicts, but maybe the saxon group would be interested in
> > > adjusting the license for xindice use? (HINT: maybe this would solve the
> > > whole issue)
> > Is there a roadmap for XQuery support? Besides aggregation across
> > documents, are the any other major sticking points in Xindice?
> > What needs tweaking in the Saxon license? (IANAL)
> Memory consumption is sticky in my book. It uses a huge amount of
> memory it seems from my tests. It could be the way I access, but
> it's fairly straightforward though.
> Create an object through JAXB once. Then persist it to a Xindice
> collection 1000 times via SAX. That appears to use beyond 1Gb RAM
> according to java.lang.Runtime. And takes 17 secs.
> Admitiably I do use a unit test that isn't very kind, since every
> time I run a test I create the collection, get the collection, do
> the test, and then remove all resources in the collection
I've been reading the code for XQuery. They use WeakHashMaps to
cache the page hits. If the test runs fast enough, it doesn't seem
like there would be a need to collect that memory. Or perhaps, their
is a leak somewhere, meaning, that a pages are read over and over
again, and never dereferenced.
--
Alain Javier Guarnieri del Gesu - javi@ajgdg.com
Re: Other sticky issues
Posted by JC Tchitchiama <jc...@panonet.net>.
All,
1) Does Henrik finds the same memory consumption when the resource is
returned as a String e.g.
...
XMLResource res = col.getResource ();
String document Contents = res.getContent().toString();
...
2) what are the JVM switch you have to help the garbage collection kick-in
more aggressively
3) Has anyone run a profiling tool with xindice at all ?
On Sunday 02 Nov 2003 5:03 pm, Henrik Vendelbo wrote:
> I was referring to Xindice. I looked at XMLResourceImpl, and if it is
> anything to go by, then the memory consumption is understandable. There has
> also been some concern about collection.close() being a nil implemention (I
> asume it still is). The getContentAsSAX gets a new XMLReader for every call
> relying on the garbage collector and Xerxes for memory efficiency. That in
> itself might be quite a hit to my knowledge.
>
> ----- Original Message -----
> From: "Alain Javier Guarnieri del Gesu" <ja...@ajgdg.com>
> To: "Henrik Vendelbo" <hv...@bluprints.com>
> Cc: <xi...@xml.apache.org>
> Sent: Saturday, November 01, 2003 6:03 PM
> Subject: Re: Other sticky issues
>
> > * Henrik Vendelbo <hv...@bluprints.com> [2003-11-01 16:29]:
> > > ----- Original Message -----
> > > From: "Alain Javier Guarnieri del Gesu" <ja...@ajgdg.com>
> > > To: <xi...@xml.apache.org>; <xi...@xml.apache.org>
> > > Sent: Thursday, October 30, 2003 1:50 PM
> > > Subject: Re: Predicates are ignored in Xindice XPATH?
> > >
> > > > * Kevin Ross <ke...@iverticalleap.com> [2003-10-22 17:18]:
> > > > > From: Kurt Kavanaugh [mailto:kkavanaugh@teletrol.com]
> > > > > Sent: Wednesday, October 22, 2003 11:00 AM
> > > > > To: xindice-users@xml.apache.org
> > > > > Subject: RE: Predicates are ignored in Xindice XPATH?
> > > > >
> > > > > BTW- Has anyone from the xindice team ever contacted the saxon
> > > > > guys,
>
> to
>
> > > > > check on how their xquery might be integrated? I believe they may
>
> have
>
> > > > > licensing conflicts, but maybe the saxon group would be interested
>
> in
>
> > > > > adjusting the license for xindice use? (HINT: maybe this would
> > > > > solve
>
> the
>
> > > > > whole issue)
> > > >
> > > > Is there a roadmap for XQuery support? Besides aggregation across
> > > > documents, are the any other major sticking points in Xindice?
> > > >
> > > > What needs tweaking in the Saxon license? (IANAL)
> > >
> > > Memory consumption is sticky in my book. It uses a huge amount of
> > > memory it seems from my tests. It could be the way I access, but
> > > it's fairly straightforward though.
> > >
> > > Create an object through JAXB once. Then persist it to a Xindice
> > > collection 1000 times via SAX. That appears to use beyond 1Gb RAM
> > > according to java.lang.Runtime. And takes 17 secs.
> > >
> > > Admitiably I do use a unit test that isn't very kind, since every
> > > time I run a test I create the collection, get the collection, do
> > > the test, and then remove all resources in the collection
> >
> > I've been reading the code for XQuery. They use WeakHashMaps to
> > cache the page hits. If the test runs fast enough, it doesn't seem
> > like there would be a need to collect that memory. Or perhaps, their
> > is a leak somewhere, meaning, that a pages are read over and over
> > again, and never dereferenced.
> >
> > --
> > Alain Javier Guarnieri del Gesu - javi@ajgdg.com
--
Best Regards.
JC.
\\- - -//
( @ @ )
===oOOo-(_)-oOOo=================================================
jct@panonet.net
=================================================================
Re: Other sticky issues
Posted by Alain Javier Guarnieri del Gesu <ja...@ajgdg.com>.
* Henrik Vendelbo <hv...@bluprints.com> [2003-11-02 13:05]:
> ----- Original Message -----
> From: "Alain Javier Guarnieri del Gesu" <ja...@ajgdg.com>
> To: "Henrik Vendelbo" <hv...@bluprints.com>
> Cc: <xi...@xml.apache.org>
> Sent: Saturday, November 01, 2003 6:03 PM
> Subject: Re: Other sticky issues
>
>
> > * Henrik Vendelbo <hv...@bluprints.com> [2003-11-01 16:29]:
> >
> > > Memory consumption is sticky in my book. It uses a huge amount of
> > > memory it seems from my tests. It could be the way I access, but
> > > it's fairly straightforward though.
> >
> > > Create an object through JAXB once. Then persist it to a Xindice
> > > collection 1000 times via SAX. That appears to use beyond 1Gb RAM
> > > according to java.lang.Runtime. And takes 17 secs.
> >
> > > Admitiably I do use a unit test that isn't very kind, since every
> > > time I run a test I create the collection, get the collection, do
> > > the test, and then remove all resources in the collection
> >
> > I've been reading the code for XQuery. They use WeakHashMaps to
> > cache the page hits. If the test runs fast enough, it doesn't seem
> > like there would be a need to collect that memory. Or perhaps, their
> > is a leak somewhere, meaning, that a pages are read over and over
> > again, and never dereferenced.
> I was referring to Xindice. I looked at XMLResourceImpl, and if it
> is anything to go by, then the memory consumption is
> understandable. There has also been some concern about
> collection.close() being a nil implemention (I asume it still is).
> The getContentAsSAX gets a new XMLReader for every call relying on
> the garbage collector and Xerxes for memory efficiency. That in
> itself might be quite a hit to my knowledge.
I was refering to Xindice too, typo, sorry.
--
Alain Javier Guarnieri del Gesu - javi@ajgdg.com
Re: Other sticky issues
Posted by Henrik Vendelbo <hv...@bluprints.com>.
I was referring to Xindice. I looked at XMLResourceImpl, and if it is
anything to go by, then the memory consumption is understandable. There has
also been some concern about collection.close() being a nil implemention (I
asume it still is). The getContentAsSAX gets a new XMLReader for every call
relying on the garbage collector and Xerxes for memory efficiency. That in
itself might be quite a hit to my knowledge.
----- Original Message -----
From: "Alain Javier Guarnieri del Gesu" <ja...@ajgdg.com>
To: "Henrik Vendelbo" <hv...@bluprints.com>
Cc: <xi...@xml.apache.org>
Sent: Saturday, November 01, 2003 6:03 PM
Subject: Re: Other sticky issues
> * Henrik Vendelbo <hv...@bluprints.com> [2003-11-01 16:29]:
> > ----- Original Message -----
> > From: "Alain Javier Guarnieri del Gesu" <ja...@ajgdg.com>
> > To: <xi...@xml.apache.org>; <xi...@xml.apache.org>
> > Sent: Thursday, October 30, 2003 1:50 PM
> > Subject: Re: Predicates are ignored in Xindice XPATH?
> >
> >
> > > * Kevin Ross <ke...@iverticalleap.com> [2003-10-22 17:18]:
> > >
> > > > From: Kurt Kavanaugh [mailto:kkavanaugh@teletrol.com]
> > > > Sent: Wednesday, October 22, 2003 11:00 AM
> > > > To: xindice-users@xml.apache.org
> > > > Subject: RE: Predicates are ignored in Xindice XPATH?
> > > >
> > > > BTW- Has anyone from the xindice team ever contacted the saxon guys,
to
> > > > check on how their xquery might be integrated? I believe they may
have
> > > > licensing conflicts, but maybe the saxon group would be interested
in
> > > > adjusting the license for xindice use? (HINT: maybe this would solve
the
> > > > whole issue)
>
> > > Is there a roadmap for XQuery support? Besides aggregation across
> > > documents, are the any other major sticking points in Xindice?
>
> > > What needs tweaking in the Saxon license? (IANAL)
>
> > Memory consumption is sticky in my book. It uses a huge amount of
> > memory it seems from my tests. It could be the way I access, but
> > it's fairly straightforward though.
>
> > Create an object through JAXB once. Then persist it to a Xindice
> > collection 1000 times via SAX. That appears to use beyond 1Gb RAM
> > according to java.lang.Runtime. And takes 17 secs.
>
> > Admitiably I do use a unit test that isn't very kind, since every
> > time I run a test I create the collection, get the collection, do
> > the test, and then remove all resources in the collection
>
> I've been reading the code for XQuery. They use WeakHashMaps to
> cache the page hits. If the test runs fast enough, it doesn't seem
> like there would be a need to collect that memory. Or perhaps, their
> is a leak somewhere, meaning, that a pages are read over and over
> again, and never dereferenced.
>
> --
> Alain Javier Guarnieri del Gesu - javi@ajgdg.com
>
>