You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucenenet.apache.org by Philip Withington <ph...@gmail.com> on 2006/09/27 21:23:31 UTC
Memory leak with sorted searches
I'm finding that the memory leak problem with sorted searches means that I
cannot use this feature of Lucene.Net. I'm using version 1.9.1. Is there a
version out there with a fix for this problem or will it be fixed in a
forthcoming release?
Regards
Phil
RE: Memory leak with sorted searches
Posted by George Aroush <ge...@aroush.net>.
Hi Phil,
What will really help is if you can provide a working code that demonstrate
the problem. This will help me and others to better see what the problem is
and we can use it for debugging.
Once you have a minimal working code demonstrating the problem, please
submit a JIRA issue and post the code there.
Btw, what version and build of Lucene.Net are you seeing this in? Please
provide this info too.
Regards,
-- George
-----Original Message-----
From: Philip Withington [mailto:philip.withington@gmail.com]
Sent: Wednesday, October 04, 2006 10:03 AM
To: lucene-net-user@incubator.apache.org
Subject: Re: Memory leak with sorted searches
George
The fixes made to solve the memory problem during indexing seem to work
great. However during sorted searching the problem remains. I think I've
narrowed the problem down to the fact that I'm creating and closing a new
IndexSearcher for every search. If I keep the the IndexSearcher open, the
problem does not occur. Therefore I'm guessing that IndexSearcher.Close()
is not releasing all of its resources.
Do what I've described repeatedly and memory usage will increase
indefinitely. However if I don't apply the sort I get no problems
whatsoever.
Regards
Phil
On 9/28/06, George Aroush <ge...@aroush.net> wrote:
>
> Hi Phil and Michael,
>
> I don't know about that proposed fix you referred to and I don't know
> if the known memory leak, during indexing, in Lucene.Net 1.9, 1.9.1
> and 2.0 is the same leak as what you are observing with the sort.
>
> To help me track this down, can you:
>
> 1) Try the fix that I found to fix a leak (see my earlier post.)
> 2) If #1 above doesn't fix your problem, try with .NET 2.0
> 3) If you are still seeing the problem, can you post an sample working
> code which demonstrates the problem?
>
> Regards,
>
> -- George Aroush
>
>
> -----Original Message-----
> From: Philip Withington [mailto:philip.withington@gmail.com]
> Sent: Wednesday, September 27, 2006 6:58 PM
> To: lucene-net-user@incubator.apache.org
> Subject: Re: Memory leak with sorted searches
>
> I am using .Net 1.1. I found some proposed fixes to the problem here
> http://blog.dotlucene.net/2006/06/lucenenet-19-rc1-build-003.html but
> would rather not take the risk of modifying code I do not fully
> understand. From what I understand from reading the lists, the next
> release of Lucene.Net (
> 2.0) will initially be geared towards the 1.1 version of the framework
> so I wonder if this will fix the problem.
>
> Michael
>
> Sorry if I've misunderstood but are you talking about re-sorting the
> hits after you've initially retrieved them ordered by relevance?
>
> Regards
>
> Phil
>
>
> On 9/27/06, Michael Mitiaguin <mi...@gmail.com> wrote:
> >
> > Philip,
> >
> > Personally, I found feature a very slow ( comparing to search itself
> > ) , so I just sort memory collection after iterating Hits.
> > You may have noticed messages regarding memory leak for indexing in
> > .Net
> > 1.1
> > Are you using .Net 1.1 ? I don't remember any memory leak when I
> > used it witn .Net 2
> >
> > Regards
> > Michael
> >
> > On 9/28/06, Philip Withington <ph...@gmail.com> wrote:
> > >
> > > I'm finding that the memory leak problem with sorted searches
> > > means that
> > I
> > > cannot use this feature of Lucene.Net. I'm using version 1.9.1.
> > > Is
> > there
> > > a
> > > version out there with a fix for this problem or will it be fixed
> > > in a forthcoming release?
> > >
> > > Regards
> > >
> > > Phil
> > >
> > >
> >
> >
>
>
RE: Memory leak with sorted searches
Posted by George Aroush <ge...@aroush.net>.
Hi Phil,
What will really help is if you can provide a working code that demonstrate
the problem. This will help me and others to better see what the problem is
and we can use it for debugging.
Once you have a minimal working code demonstrating the problem, please
submit a JIRA issue and post the code there.
Btw, what version and build of Lucene.Net are you seeing this in? Please
provide this info too.
Regards,
-- George
-----Original Message-----
From: Philip Withington [mailto:philip.withington@gmail.com]
Sent: Wednesday, October 04, 2006 10:03 AM
To: lucene-net-user@incubator.apache.org
Subject: Re: Memory leak with sorted searches
George
The fixes made to solve the memory problem during indexing seem to work
great. However during sorted searching the problem remains. I think I've
narrowed the problem down to the fact that I'm creating and closing a new
IndexSearcher for every search. If I keep the the IndexSearcher open, the
problem does not occur. Therefore I'm guessing that IndexSearcher.Close()
is not releasing all of its resources.
Do what I've described repeatedly and memory usage will increase
indefinitely. However if I don't apply the sort I get no problems
whatsoever.
Regards
Phil
On 9/28/06, George Aroush <ge...@aroush.net> wrote:
>
> Hi Phil and Michael,
>
> I don't know about that proposed fix you referred to and I don't know
> if the known memory leak, during indexing, in Lucene.Net 1.9, 1.9.1
> and 2.0 is the same leak as what you are observing with the sort.
>
> To help me track this down, can you:
>
> 1) Try the fix that I found to fix a leak (see my earlier post.)
> 2) If #1 above doesn't fix your problem, try with .NET 2.0
> 3) If you are still seeing the problem, can you post an sample working
> code which demonstrates the problem?
>
> Regards,
>
> -- George Aroush
>
>
> -----Original Message-----
> From: Philip Withington [mailto:philip.withington@gmail.com]
> Sent: Wednesday, September 27, 2006 6:58 PM
> To: lucene-net-user@incubator.apache.org
> Subject: Re: Memory leak with sorted searches
>
> I am using .Net 1.1. I found some proposed fixes to the problem here
> http://blog.dotlucene.net/2006/06/lucenenet-19-rc1-build-003.html but
> would rather not take the risk of modifying code I do not fully
> understand. From what I understand from reading the lists, the next
> release of Lucene.Net (
> 2.0) will initially be geared towards the 1.1 version of the framework
> so I wonder if this will fix the problem.
>
> Michael
>
> Sorry if I've misunderstood but are you talking about re-sorting the
> hits after you've initially retrieved them ordered by relevance?
>
> Regards
>
> Phil
>
>
> On 9/27/06, Michael Mitiaguin <mi...@gmail.com> wrote:
> >
> > Philip,
> >
> > Personally, I found feature a very slow ( comparing to search itself
> > ) , so I just sort memory collection after iterating Hits.
> > You may have noticed messages regarding memory leak for indexing in
> > .Net
> > 1.1
> > Are you using .Net 1.1 ? I don't remember any memory leak when I
> > used it witn .Net 2
> >
> > Regards
> > Michael
> >
> > On 9/28/06, Philip Withington <ph...@gmail.com> wrote:
> > >
> > > I'm finding that the memory leak problem with sorted searches
> > > means that
> > I
> > > cannot use this feature of Lucene.Net. I'm using version 1.9.1.
> > > Is
> > there
> > > a
> > > version out there with a fix for this problem or will it be fixed
> > > in a forthcoming release?
> > >
> > > Regards
> > >
> > > Phil
> > >
> > >
> >
> >
>
>
Re: Memory leak with sorted searches
Posted by Philip Withington <ph...@gmail.com>.
George
The fixes made to solve the memory problem during indexing seem to work
great. However during sorted searching the problem remains. I think I've
narrowed the problem down to the fact that I'm creating and closing a new
IndexSearcher for every search. If I keep the the IndexSearcher open, the
problem does not occur. Therefore I'm guessing that IndexSearcher.Close()
is not releasing all of its resources.
Do what I've described repeatedly and memory usage will increase
indefinitely. However if I don't apply the sort I get no problems
whatsoever.
Regards
Phil
On 9/28/06, George Aroush <ge...@aroush.net> wrote:
>
> Hi Phil and Michael,
>
> I don't know about that proposed fix you referred to and I don't know if
> the
> known memory leak, during indexing, in Lucene.Net 1.9, 1.9.1 and 2.0 is
> the
> same leak as what you are observing with the sort.
>
> To help me track this down, can you:
>
> 1) Try the fix that I found to fix a leak (see my earlier post.)
> 2) If #1 above doesn't fix your problem, try with .NET 2.0
> 3) If you are still seeing the problem, can you post an sample working
> code
> which demonstrates the problem?
>
> Regards,
>
> -- George Aroush
>
>
> -----Original Message-----
> From: Philip Withington [mailto:philip.withington@gmail.com]
> Sent: Wednesday, September 27, 2006 6:58 PM
> To: lucene-net-user@incubator.apache.org
> Subject: Re: Memory leak with sorted searches
>
> I am using .Net 1.1. I found some proposed fixes to the problem here
> http://blog.dotlucene.net/2006/06/lucenenet-19-rc1-build-003.html but
> would
> rather not take the risk of modifying code I do not fully
> understand. From
> what I understand from reading the lists, the next release of Lucene.Net (
> 2.0) will initially be geared towards the 1.1 version of the framework so
> I
> wonder if this will fix the problem.
>
> Michael
>
> Sorry if I've misunderstood but are you talking about re-sorting the hits
> after you've initially retrieved them ordered by relevance?
>
> Regards
>
> Phil
>
>
> On 9/27/06, Michael Mitiaguin <mi...@gmail.com> wrote:
> >
> > Philip,
> >
> > Personally, I found feature a very slow ( comparing to search itself )
> > , so I just sort memory collection after iterating Hits.
> > You may have noticed messages regarding memory leak for indexing in
> > .Net
> > 1.1
> > Are you using .Net 1.1 ? I don't remember any memory leak when I used
> > it witn .Net 2
> >
> > Regards
> > Michael
> >
> > On 9/28/06, Philip Withington <ph...@gmail.com> wrote:
> > >
> > > I'm finding that the memory leak problem with sorted searches means
> > > that
> > I
> > > cannot use this feature of Lucene.Net. I'm using version 1.9.1. Is
> > there
> > > a
> > > version out there with a fix for this problem or will it be fixed in
> > > a forthcoming release?
> > >
> > > Regards
> > >
> > > Phil
> > >
> > >
> >
> >
>
>
Re: Memory leak with sorted searches
Posted by Michael Mitiaguin <mi...@gmail.com>.
>
>
> Sorry if I've misunderstood but are you talking about re-sorting the hits
> after you've initially retrieved them ordered by relevance?
That's right . It seems to be faster than do it in Lucene.
Michael
>
>
> On 9/27/06, Michael Mitiaguin <mi...@gmail.com> wrote:
> >
> > Philip,
> >
> > Personally, I found feature a very slow ( comparing to search itself ) ,
> > so
> > I just sort memory collection after iterating Hits.
> > You may have noticed messages regarding memory leak for indexing in .Net
> > 1.1
> > Are you using .Net 1.1 ? I don't remember any memory leak when I used
> it
> > witn .Net 2
> >
> > Regards
> > Michael
> >
> > On 9/28/06, Philip Withington <ph...@gmail.com> wrote:
> > >
> > > I'm finding that the memory leak problem with sorted searches means
> that
> > I
> > > cannot use this feature of Lucene.Net. I'm using version 1.9.1. Is
> > there
> > > a
> > > version out there with a fix for this problem or will it be fixed in a
> > > forthcoming release?
> > >
> > > Regards
> > >
> > > Phil
> > >
> > >
> >
> >
>
>
RE: Memory leak with sorted searches
Posted by George Aroush <ge...@aroush.net>.
Hi Phil and Michael,
I don't know about that proposed fix you referred to and I don't know if the
known memory leak, during indexing, in Lucene.Net 1.9, 1.9.1 and 2.0 is the
same leak as what you are observing with the sort.
To help me track this down, can you:
1) Try the fix that I found to fix a leak (see my earlier post.)
2) If #1 above doesn't fix your problem, try with .NET 2.0
3) If you are still seeing the problem, can you post an sample working code
which demonstrates the problem?
Regards,
-- George Aroush
-----Original Message-----
From: Philip Withington [mailto:philip.withington@gmail.com]
Sent: Wednesday, September 27, 2006 6:58 PM
To: lucene-net-user@incubator.apache.org
Subject: Re: Memory leak with sorted searches
I am using .Net 1.1. I found some proposed fixes to the problem here
http://blog.dotlucene.net/2006/06/lucenenet-19-rc1-build-003.html but would
rather not take the risk of modifying code I do not fully understand. From
what I understand from reading the lists, the next release of Lucene.Net (
2.0) will initially be geared towards the 1.1 version of the framework so I
wonder if this will fix the problem.
Michael
Sorry if I've misunderstood but are you talking about re-sorting the hits
after you've initially retrieved them ordered by relevance?
Regards
Phil
On 9/27/06, Michael Mitiaguin <mi...@gmail.com> wrote:
>
> Philip,
>
> Personally, I found feature a very slow ( comparing to search itself )
> , so I just sort memory collection after iterating Hits.
> You may have noticed messages regarding memory leak for indexing in
> .Net
> 1.1
> Are you using .Net 1.1 ? I don't remember any memory leak when I used
> it witn .Net 2
>
> Regards
> Michael
>
> On 9/28/06, Philip Withington <ph...@gmail.com> wrote:
> >
> > I'm finding that the memory leak problem with sorted searches means
> > that
> I
> > cannot use this feature of Lucene.Net. I'm using version 1.9.1. Is
> there
> > a
> > version out there with a fix for this problem or will it be fixed in
> > a forthcoming release?
> >
> > Regards
> >
> > Phil
> >
> >
>
>
RE: Memory leak with sorted searches
Posted by George Aroush <ge...@aroush.net>.
Hi Phil and Michael,
I don't know about that proposed fix you referred to and I don't know if the
known memory leak, during indexing, in Lucene.Net 1.9, 1.9.1 and 2.0 is the
same leak as what you are observing with the sort.
To help me track this down, can you:
1) Try the fix that I found to fix a leak (see my earlier post.)
2) If #1 above doesn't fix your problem, try with .NET 2.0
3) If you are still seeing the problem, can you post an sample working code
which demonstrates the problem?
Regards,
-- George Aroush
-----Original Message-----
From: Philip Withington [mailto:philip.withington@gmail.com]
Sent: Wednesday, September 27, 2006 6:58 PM
To: lucene-net-user@incubator.apache.org
Subject: Re: Memory leak with sorted searches
I am using .Net 1.1. I found some proposed fixes to the problem here
http://blog.dotlucene.net/2006/06/lucenenet-19-rc1-build-003.html but would
rather not take the risk of modifying code I do not fully understand. From
what I understand from reading the lists, the next release of Lucene.Net (
2.0) will initially be geared towards the 1.1 version of the framework so I
wonder if this will fix the problem.
Michael
Sorry if I've misunderstood but are you talking about re-sorting the hits
after you've initially retrieved them ordered by relevance?
Regards
Phil
On 9/27/06, Michael Mitiaguin <mi...@gmail.com> wrote:
>
> Philip,
>
> Personally, I found feature a very slow ( comparing to search itself )
> , so I just sort memory collection after iterating Hits.
> You may have noticed messages regarding memory leak for indexing in
> .Net
> 1.1
> Are you using .Net 1.1 ? I don't remember any memory leak when I used
> it witn .Net 2
>
> Regards
> Michael
>
> On 9/28/06, Philip Withington <ph...@gmail.com> wrote:
> >
> > I'm finding that the memory leak problem with sorted searches means
> > that
> I
> > cannot use this feature of Lucene.Net. I'm using version 1.9.1. Is
> there
> > a
> > version out there with a fix for this problem or will it be fixed in
> > a forthcoming release?
> >
> > Regards
> >
> > Phil
> >
> >
>
>
Re: Memory leak with sorted searches
Posted by Philip Withington <ph...@gmail.com>.
I am using .Net 1.1. I found some proposed fixes to the problem here
http://blog.dotlucene.net/2006/06/lucenenet-19-rc1-build-003.html but would
rather not take the risk of modifying code I do not fully understand. From
what I understand from reading the lists, the next release of Lucene.Net (
2.0) will initially be geared towards the 1.1 version of the framework so I
wonder if this will fix the problem.
Michael
Sorry if I've misunderstood but are you talking about re-sorting the hits
after you've initially retrieved them ordered by relevance?
Regards
Phil
On 9/27/06, Michael Mitiaguin <mi...@gmail.com> wrote:
>
> Philip,
>
> Personally, I found feature a very slow ( comparing to search itself ) ,
> so
> I just sort memory collection after iterating Hits.
> You may have noticed messages regarding memory leak for indexing in .Net
> 1.1
> Are you using .Net 1.1 ? I don't remember any memory leak when I used it
> witn .Net 2
>
> Regards
> Michael
>
> On 9/28/06, Philip Withington <ph...@gmail.com> wrote:
> >
> > I'm finding that the memory leak problem with sorted searches means that
> I
> > cannot use this feature of Lucene.Net. I'm using version 1.9.1. Is
> there
> > a
> > version out there with a fix for this problem or will it be fixed in a
> > forthcoming release?
> >
> > Regards
> >
> > Phil
> >
> >
>
>
Re: Memory leak with sorted searches
Posted by Michael Mitiaguin <mi...@gmail.com>.
Philip,
Personally, I found feature a very slow ( comparing to search itself ) , so
I just sort memory collection after iterating Hits.
You may have noticed messages regarding memory leak for indexing in .Net 1.1
Are you using .Net 1.1 ? I don't remember any memory leak when I used it
witn .Net 2
Regards
Michael
On 9/28/06, Philip Withington <ph...@gmail.com> wrote:
>
> I'm finding that the memory leak problem with sorted searches means that I
> cannot use this feature of Lucene.Net. I'm using version 1.9.1. Is there
> a
> version out there with a fix for this problem or will it be fixed in a
> forthcoming release?
>
> Regards
>
> Phil
>
>