You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucenenet.apache.org by Michael Mitiaguin <mi...@gmail.com> on 2006/05/05 15:19:35 UTC
1.9 indexing
I noticed in development list a comment about using Lucene.Net 1.9 with .Net
2 without any problem. Quite clearly there is a memory problem when
indexing more than 50000+ records from database. The same surely applies to
50000+ files ( 50000 is just approximate number given as indication )
I tried a new and deprecated syntax without success.having "Out of
memory" in both cases.
Memory consumption grows to 1G and then program bombs.
I didn't have such a problem with 1.4.3 , having indexed up to 1 500 000
records from database.
Remarkably when I use interop calling Lucene from a COM client memory is
released for 1.9 for some reason.
Any comments ?
TIA
Michael
Re: 1.9 indexing
Posted by Jeff Rodenburg <je...@gmail.com>.
I believe the plan relative to Lucene 1.9 is to focus community development
using the 1.1 Framework. There will be a transition in the future to the
2.0 Framework, but for now I believe we're targeting the 1.1 Framework for
Lucene 1.9.
-- j
On 5/5/06, George Aroush <ge...@aroush.net> wrote:
>
> Hi Michael,
>
> The problem you noticed with 1.9 is not due to .NET 2.0, but due to a bug
> in
> Lucene 1.9
>
> Regards,
>
> -- George
>
> -----Original Message-----
> From: Michael Mitiaguin [mailto:mitiaguin@gmail.com]
> Sent: Friday, May 05, 2006 9:20 AM
> To: lucene-net-user@incubator.apache.org
> Subject: 1.9 indexing
>
> I noticed in development list a comment about using Lucene.Net 1.9 with
> .Net
> 2 without any problem. Quite clearly there is a memory problem when
> indexing more than 50000+ records from database. The same surely applies
> to
> 50000+ files ( 50000 is just approximate number given as indication )
> I tried a new and deprecated syntax without success.having "Out of
> memory" in both cases.
> Memory consumption grows to 1G and then program bombs.
>
> I didn't have such a problem with 1.4.3 , having indexed up to 1 500 000
> records from database.
>
> Remarkably when I use interop calling Lucene from a COM client memory is
> released for 1.9 for some reason.
>
> Any comments ?
>
> TIA
> Michael
>
>
RE: 1.9 indexing
Posted by George Aroush <ge...@aroush.net>.
Hi Michael,
You can tell which version/build you are using by examining the file
AssemblyInfo.cs. Look for the line with [assembly:
AssemblyVersion("1.9.0.002")] -- the last number, 002, is the build which is
what you should be using.
In the next day or two I will be moving the binary downloads to ASF so this
should help complete the transition to ASF from SourceForge.net.
Also, I am not too far off releasing build 3 of 1.9 which is mainly the port
of the Lucene Test code to C#.
Regards,
-- George Aroush
-----Original Message-----
From: Michael Mitiaguin [mailto:mitiaguin@gmail.com]
Sent: Sunday, May 07, 2006 11:39 PM
To: lucene-net-user@incubator.apache.org
Subject: Re: 1.9 indexing
George,
It is hard to say. Firstly, I am not sure how to download the most recent
version now Secondly, I seem to have run directory comparison at some stage
and just applied file changes, but whether it was build 2 against build 1
just don't remember. I am quite sure though , that the problem existed and
has gone for the same Lucene code , but different computer.
Regards
Michael
On 5/8/06, George Aroush <ge...@aroush.net> wrote:
>
> Hi Michael,
>
> I'm glad to see the problem disappeared, but it would have been better
> to know what caused it.
>
> May I ask, what build version of 1.9 are you using? Is it build 1 or 2?
> There are known memory leaks in build 1 which I fixed in build 2.
>
> Regards,
>
> -- George Aroush
>
>
> -----Original Message-----
> From: Michael Mitiaguin [mailto:mitiaguin@gmail.com]
> Sent: Sunday, May 07, 2006 10:15 PM
> To: lucene-net-user@incubator.apache.org
> Subject: Re: 1.9 indexing
>
> Sorry for being too quick to attribute a memory problem to Lucene. My
> box has changed and now I have a difficulty to reproduce a problem.
> Test case worked successfully , then running the old code which caused
> a memory grief didn't create any problem either.
> It is hard to explain what made it disbehave , but I am a happy man now.
>
> Michael
>
>
> On 5/8/06, Jeff Rodenburg <je...@gmail.com> wrote:
> >
> > Michael -
> >
> > Would you mind posting a test case that reproduces the problem?
> >
> > -- j
> >
> > On 5/6/06, Michael Mitiaguin <mi...@gmail.com> wrote:
> > >
> > > Workaround in my case was to use a COM client which is not
> > > suitable for general use. Another workaround is to write Java
> > > indexing code
> > > :) , but
> > if
> > > there is such a problem in Java I am flabbergasted then.
> > > Also bug seems to be critical and does exist ( according to George
> > > ) irrespective of 1.1 or 2 framework.
> > > So even the focus on 1.1 still without fixing that, Lucene.Net
> > > 1.9 can hardly be considered usable
> > > 1.9 was accessible on SourceForge for quite while and if it was a
> > > known problem just wondering whether the cause still remains unknown.
> > >
> > > Michael
> > >
> > > On 5/6/06, Eyal Post <ey...@epocalipse.com> wrote:
> > > >
> > > > Where's the leak? Is there a simple workaround?
> > > >
> > > > Eyal
> > > >
> > > >
> > >
> > >
> >
> >
>
>
Re: 1.9 indexing
Posted by Michael Mitiaguin <mi...@gmail.com>.
George,
It is hard to say. Firstly, I am not sure how to download the most recent
version now
Secondly, I seem to have run directory comparison at some stage and just
applied file changes, but whether it was build 2 against build 1 just don't
remember. I am quite sure though , that the problem existed and has gone
for the same Lucene code , but different computer.
Regards
Michael
On 5/8/06, George Aroush <ge...@aroush.net> wrote:
>
> Hi Michael,
>
> I'm glad to see the problem disappeared, but it would have been better to
> know what caused it.
>
> May I ask, what build version of 1.9 are you using? Is it build 1 or 2?
> There are known memory leaks in build 1 which I fixed in build 2.
>
> Regards,
>
> -- George Aroush
>
>
> -----Original Message-----
> From: Michael Mitiaguin [mailto:mitiaguin@gmail.com]
> Sent: Sunday, May 07, 2006 10:15 PM
> To: lucene-net-user@incubator.apache.org
> Subject: Re: 1.9 indexing
>
> Sorry for being too quick to attribute a memory problem to Lucene. My box
> has changed and now I have a difficulty to reproduce a problem. Test case
> worked successfully , then running the old code which caused a memory
> grief
> didn't create any problem either.
> It is hard to explain what made it disbehave , but I am a happy man now.
>
> Michael
>
>
> On 5/8/06, Jeff Rodenburg <je...@gmail.com> wrote:
> >
> > Michael -
> >
> > Would you mind posting a test case that reproduces the problem?
> >
> > -- j
> >
> > On 5/6/06, Michael Mitiaguin <mi...@gmail.com> wrote:
> > >
> > > Workaround in my case was to use a COM client which is not suitable
> > > for general use. Another workaround is to write Java indexing code
> > > :) , but
> > if
> > > there is such a problem in Java I am flabbergasted then.
> > > Also bug seems to be critical and does exist ( according to George )
> > > irrespective of 1.1 or 2 framework.
> > > So even the focus on 1.1 still without fixing that, Lucene.Net 1.9
> > > can hardly be considered usable
> > > 1.9 was accessible on SourceForge for quite while and if it was a
> > > known problem just wondering whether the cause still remains unknown.
> > >
> > > Michael
> > >
> > > On 5/6/06, Eyal Post <ey...@epocalipse.com> wrote:
> > > >
> > > > Where's the leak? Is there a simple workaround?
> > > >
> > > > Eyal
> > > >
> > > >
> > >
> > >
> >
> >
>
>
RE: 1.9 indexing
Posted by George Aroush <ge...@aroush.net>.
Hi Michael,
I'm glad to see the problem disappeared, but it would have been better to
know what caused it.
May I ask, what build version of 1.9 are you using? Is it build 1 or 2?
There are known memory leaks in build 1 which I fixed in build 2.
Regards,
-- George Aroush
-----Original Message-----
From: Michael Mitiaguin [mailto:mitiaguin@gmail.com]
Sent: Sunday, May 07, 2006 10:15 PM
To: lucene-net-user@incubator.apache.org
Subject: Re: 1.9 indexing
Sorry for being too quick to attribute a memory problem to Lucene. My box
has changed and now I have a difficulty to reproduce a problem. Test case
worked successfully , then running the old code which caused a memory grief
didn't create any problem either.
It is hard to explain what made it disbehave , but I am a happy man now.
Michael
On 5/8/06, Jeff Rodenburg <je...@gmail.com> wrote:
>
> Michael -
>
> Would you mind posting a test case that reproduces the problem?
>
> -- j
>
> On 5/6/06, Michael Mitiaguin <mi...@gmail.com> wrote:
> >
> > Workaround in my case was to use a COM client which is not suitable
> > for general use. Another workaround is to write Java indexing code
> > :) , but
> if
> > there is such a problem in Java I am flabbergasted then.
> > Also bug seems to be critical and does exist ( according to George )
> > irrespective of 1.1 or 2 framework.
> > So even the focus on 1.1 still without fixing that, Lucene.Net 1.9
> > can hardly be considered usable
> > 1.9 was accessible on SourceForge for quite while and if it was a
> > known problem just wondering whether the cause still remains unknown.
> >
> > Michael
> >
> > On 5/6/06, Eyal Post <ey...@epocalipse.com> wrote:
> > >
> > > Where's the leak? Is there a simple workaround?
> > >
> > > Eyal
> > >
> > >
> >
> >
>
>
Re: 1.9 indexing
Posted by Michael Mitiaguin <mi...@gmail.com>.
Sorry for being too quick to attribute a memory problem to Lucene. My box
has changed and now I have a difficulty to reproduce a problem. Test case
worked successfully , then running the old code which caused a memory grief
didn't create any problem either.
It is hard to explain what made it disbehave , but I am a happy man now.
Michael
On 5/8/06, Jeff Rodenburg <je...@gmail.com> wrote:
>
> Michael -
>
> Would you mind posting a test case that reproduces the problem?
>
> -- j
>
> On 5/6/06, Michael Mitiaguin <mi...@gmail.com> wrote:
> >
> > Workaround in my case was to use a COM client which is not suitable for
> > general use. Another workaround is to write Java indexing code :) , but
> if
> > there is such a problem in Java I am flabbergasted then.
> > Also bug seems to be critical and does exist ( according to George )
> > irrespective of 1.1 or 2 framework.
> > So even the focus on 1.1 still without fixing that, Lucene.Net 1.9 can
> > hardly be considered usable
> > 1.9 was accessible on SourceForge for quite while and if it was a known
> > problem just wondering whether the cause still remains unknown.
> >
> > Michael
> >
> > On 5/6/06, Eyal Post <ey...@epocalipse.com> wrote:
> > >
> > > Where's the leak? Is there a simple workaround?
> > >
> > > Eyal
> > >
> > >
> >
> >
>
>
Re: 1.9 indexing
Posted by Jeff Rodenburg <je...@gmail.com>.
Michael -
Would you mind posting a test case that reproduces the problem?
-- j
On 5/6/06, Michael Mitiaguin <mi...@gmail.com> wrote:
>
> Workaround in my case was to use a COM client which is not suitable for
> general use. Another workaround is to write Java indexing code :) , but if
> there is such a problem in Java I am flabbergasted then.
> Also bug seems to be critical and does exist ( according to George )
> irrespective of 1.1 or 2 framework.
> So even the focus on 1.1 still without fixing that, Lucene.Net 1.9 can
> hardly be considered usable
> 1.9 was accessible on SourceForge for quite while and if it was a known
> problem just wondering whether the cause still remains unknown.
>
> Michael
>
> On 5/6/06, Eyal Post <ey...@epocalipse.com> wrote:
> >
> > Where's the leak? Is there a simple workaround?
> >
> > Eyal
> >
> >
>
>
Re: 1.9 indexing
Posted by Michael Mitiaguin <mi...@gmail.com>.
Workaround in my case was to use a COM client which is not suitable for
general use. Another workaround is to write Java indexing code :) , but if
there is such a problem in Java I am flabbergasted then.
Also bug seems to be critical and does exist ( according to George )
irrespective of 1.1 or 2 framework.
So even the focus on 1.1 still without fixing that, Lucene.Net 1.9 can
hardly be considered usable
1.9 was accessible on SourceForge for quite while and if it was a known
problem just wondering whether the cause still remains unknown.
Michael
On 5/6/06, Eyal Post <ey...@epocalipse.com> wrote:
>
> Where's the leak? Is there a simple workaround?
>
> Eyal
>
>
RE: 1.9 indexing
Posted by Eyal Post <ey...@epocalipse.com>.
Where's the leak? Is there a simple workaround?
Eyal
> -----Original Message-----
> From: George Aroush [mailto:george@aroush.net]
> Sent: Friday, May 05, 2006 17:46 PM
> To: lucene-net-user@incubator.apache.org
> Subject: RE: 1.9 indexing
>
> Hi Michael,
>
> The problem you noticed with 1.9 is not due to .NET 2.0, but
> due to a bug in Lucene 1.9
>
> Regards,
>
> -- George
>
> -----Original Message-----
> From: Michael Mitiaguin [mailto:mitiaguin@gmail.com]
> Sent: Friday, May 05, 2006 9:20 AM
> To: lucene-net-user@incubator.apache.org
> Subject: 1.9 indexing
>
> I noticed in development list a comment about using
> Lucene.Net 1.9 with .Net
> 2 without any problem. Quite clearly there is a memory
> problem when indexing more than 50000+ records from database.
> The same surely applies to
> 50000+ files ( 50000 is just approximate number given as
> indication )
> I tried a new and deprecated syntax without success.having "Out of
> memory" in both cases.
> Memory consumption grows to 1G and then program bombs.
>
> I didn't have such a problem with 1.4.3 , having indexed up
> to 1 500 000 records from database.
>
> Remarkably when I use interop calling Lucene from a COM
> client memory is released for 1.9 for some reason.
>
> Any comments ?
>
> TIA
> Michael
>
>
>
RE: 1.9 indexing
Posted by George Aroush <ge...@aroush.net>.
Hi Michael,
The problem you noticed with 1.9 is not due to .NET 2.0, but due to a bug in
Lucene 1.9
Regards,
-- George
-----Original Message-----
From: Michael Mitiaguin [mailto:mitiaguin@gmail.com]
Sent: Friday, May 05, 2006 9:20 AM
To: lucene-net-user@incubator.apache.org
Subject: 1.9 indexing
I noticed in development list a comment about using Lucene.Net 1.9 with .Net
2 without any problem. Quite clearly there is a memory problem when
indexing more than 50000+ records from database. The same surely applies to
50000+ files ( 50000 is just approximate number given as indication )
I tried a new and deprecated syntax without success.having "Out of
memory" in both cases.
Memory consumption grows to 1G and then program bombs.
I didn't have such a problem with 1.4.3 , having indexed up to 1 500 000
records from database.
Remarkably when I use interop calling Lucene from a COM client memory is
released for 1.9 for some reason.
Any comments ?
TIA
Michael