AW: [VOTE] Release of Apache Lucene.Net 2.0.0

[Erich Eichinger <E....@diamonddogs.cc>]

Hi,
 
I attached code containing some tests for memoryleaks. Unlike Lucene.NET 1.4.3, the current 2.0 codebase from SVN passes these tests without problems.
 
cheers,
Erich

________________________________

Von: Erich Eichinger [mailto:E.Eichinger@diamonddogs.cc]
Gesendet: Mi 2007-03-07 21:53
An: lucene-net-user@incubator.apache.org; lucene-net-dev@incubator.apache.org
Betreff: RE: [VOTE] Release of Apache Lucene.Net 2.0.0




Hi all,

I attached the patch for 1.4.3-final-004. It fixes a couple of memory-/resource leaks and some bugs in GermanStemmer. I have this version in use for over 1 year now in the "wilderness" and never had any problems (at least with Lucene) since then.

Unfortunately I didn't have time to look into 2.0 more detailed yet.

hope this helps,
Erich

> -----Original Message-----
> From: George Aroush [mailto:george@aroush.net]
> Sent: Thursday, March 01, 2007 4:00 PM
> To: lucene-net-user@incubator.apache.org;
> lucene-net-dev@incubator.apache.org
> Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
>
> Hi Erich,
>
> Now that you mentioned the version you are using, I believe
> those are fixed in 2.0 -- not necessarily in the same way as
> your fix.  Can you:
>
> 1) Try Lucene.Net 1.9.1 and see if you are still seeing the
> leaks.  I rather have you try 2.0 but some API's have changed in 2.0.
>
> 2) Share with us the patch you used for 1.4.3 -- others may
> find it useful.
>
> Regards,
>
> -- George Aroush
>
> -----Original Message-----
> From: Erich Eichinger [mailto:E.Eichinger@diamonddogs.cc]
> Sent: Thursday, March 01, 2007 4:12 AM
> To: lucene-net-user@incubator.apache.org;
> lucene-net-dev@incubator.apache.org
> Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
>
>
> Hi George,
>
> > The memory and resource leaks that you mentioned, have you observed
> > them or do you think they exist?
>
> Yes, I experienced them in a couple of high loaded
> web-applications and the
> fixes I mentioned took me 2 weeks to find them. TBH I
> experienced them using
> Lucene.NET 1.4.3 but checking the codebase showed, that the
> respective code
> didn't change to 2.0.
>
> > Do you have sample code that demonstrates them? 
> > I ask this because you said "possible".
>
> This is misleading. I said "possible" because the underlying
> IndexInput
> might be a RAMDirectory. In case of FileDirectory (or the
> DBDirectory I
> mentioned in my private email) there is almost certainly a
> leak because of
> unclosed file-handles or db-connections.
> I can write some sample code to show up the leaks if you
> like. Btw: I found
> and fixed the leaks in v1.4.3 using SciTech's ".NET Memory
> Profiler" (don't
> want to ad here, but it has been the only useful tool I found for this
> purpose)
>
> > As for your WeakHashtable suggestion and the use of
> NHibernate, this
> > isn't possible.
>
> Another misunderstanding ;-). I didn't want you to link Lucene.NET to
> NHibernate. I just wanted to point you to an implementation I
> am aware of.
>
>
> > In my opinion, none of those issues are critical to stop
> this release.
>
> Since I have to deal almost 100% with web-applications, for
> me these leaks
> are definitely a showstopper. Depending on the application's traffic I
> experienced OutOfMemoryExceptions every few hours. After
> applying the fixes
> I described, everything is running smooth.
>
> cheers,
> Erich
>
> > -----Original Message-----
> > From: George Aroush [mailto:george@aroush.net]
> > Sent: Thursday, March 01, 2007 2:19 AM
> > To: lucene-net-dev@incubator.apache.org;
> > lucene-net-user@incubator.apache.org
> > Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
> >
> > Hi Erich,
> >
> > In my opinion, none of those issues are critical to stop
> this release.
> >
> > The memory and resource leaks that you mentioned, have you observed
> > them or do you think they exist?  Do you have sample code that
> > demonstrates them?  I ask this because you said "possible".
>  Yes, I am
> > aware of a leak issue during sorting when Lucene.Net is
> compiled using
> > .NET 1.1, but with .NET 2.0, it disappears.
> >
> > As for your WeakHashtable suggestion and the use of
> NHibernate, this
> > isn't possible.  This release is supporting .NET 1.1 so we are
> > somewhat limited with what we have.  No, I can't use NHibernate as
> > it's a 3rd party artifact and none ASF.
> >
> > Regards,
> >
> > -- George
> >
> >
> > -----Original Message-----
> > From: Erich Eichinger [mailto:E.Eichinger@diamonddogs.cc]
> > Sent: Tuesday, February 27, 2007 9:04 AM
> > To: lucene-net-user@incubator.apache.org;
> > lucene-net-dev@incubator.apache.org
> > Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
> >
> > Hi,
> >
> > there are some resource leaks that lead to really nasty problems in
> > highloaded webapplications. I summed up the most required changes:
> >
> > *)
> > Index/CompoundFileReader.cs:260:
> > Lucene.Net.Index.Compound.FileReader.CSIndexInput doesn't close
> > underlying IndexInput (possible resource leak)
> >
> >
> > *)
> > Index/TermInfosReader.cs
> > Index/SegmentReader.cs
> >
> > Usage of System.Threading.Thread.GetData()/.SetData() may result in
> > memory leaks in web-applications. Using
> > System.Runtime.Remoting.Messaging.CallContext is a much
> better choice.
> >
> >
> > *)
> > Search/FieldCacheImpl.cs
> >
> > cache should be a "WeakHashtable" instead of Hashtable.
> > Otherwise cached readers can't ever be collected and cause
> > memory/resource leaks.
> >
> > *)
> > FieldSortedHitQueue.cs
> >
> > the same problem as in FieldCacheImpl but with "Comparators" table.
> >
> > Asfaik a possible implementation of a "WeakHashtable" can
> be found in
> > the NHibernate project.
> >
> >
> > cheers,
> > Erich
> >
> >
> > > -----Original Message-----
> > > From: George Aroush [mailto:george@aroush.net]
> > > Sent: Tuesday, February 27, 2007 4:06 AM
> > > To: lucene-net-user@incubator.apache.org;
> > > lucene-net-dev@incubator.apache.org
> > > Subject: [VOTE] Release of Apache Lucene.Net 2.0.0
> > >
> > > Hi folks,
> > >
> > > To follow the proper release process of ASF, please take a
> > moment to
> > > cast your vote to release Lucene.Net 2.0.0.  I have
> placed both the
> > > release candidate of source code and binary
> > > here: http://people.apache.org/~aroush/
> > >
> > > The change history for this release can be found here:
> > > https://svn.apache.org/repos/asf/incubator/lucene.net/trunk/C%
> > > 23/src/HISTORY
> > > .txt
> > >
> > > In addition to Lucene.Net release, this release also
> > contains ported
> > > code in "contrib".
> > >
> > > Please cast your vote on releasing Apache Lucene.Net
> 2.0.0 no later
> > > then this coming Saturday.
> > >
> > > [ ] +1 Approve release
> > > [x] -1 Veto release (please give reason)
> > >
> > > Regards,
> > >
> > > -- George Aroush
> > >
> > >
> >
> >
>
>