You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by George Aroush <ge...@aroush.net> on 2010/11/01 21:55:43 UTC

RE: Lucene.NET Community Status

Let me jump in here and offer some perspective about Lucene.Net (btw,  
it's not Lucene.NET :-) ).  This is based on my past involvement with  
the project -- since 2003 when it was on SourceForge.net and called  
dotLucene.

1) Up until early this year, I have been porting and supporting  
Lucene.Net since ver 1.4 (back in 2004 on SourceForge.net) to the  
current release on trunk ver. 2.9.2.  This is in NO WAY to say that  
others have not helped or contributed.  I'm just saying that I know  
the history and have the experience (I wrote and worked on search  
engines from 1998 to 2002).

2) Doing an initial port of a new Java Lucene release to C# Lucene is  
very hard; it's the most complex part of the port even using automated  
tools such as JLCA and my own customize scripts which I use pre-and  
port JLCA (you can search the listing on how I do the port).  What  
used to take me about 1 months with 90% of tests passing took me well  
over 4 months (for 2.9.x) with only 10% of tests passing.  This was no  
easy effort and won't be easier now since Java Lucene is using new  
Java language features that JLCA is not aware of (MS is not  
maintaining JLCA).  Put another way, porting is hard especially when  
you are dealing with > 5.6 GB source code consistent of > 610 source  
files.  You will know this ONLY if you have tried it out and  
maintained it -- this is why no one has stepped up to do an initial  
port otherwise there would be a port by now not only of Java Lucene  
but other projects too.

3) To simplify ports of new release, maintaining as small as possible  
delta between release is very important. This was a main pain point  
when I ported from 2.4 to 2.9.  The in-between ports were never done  
due to lack of time on my end.  See point #2.

4) Diverging away from Java Lucene, both API base and algorithm is  
risky and will just make point #2 more evidence.  Not only will you  
now need a deep knowledge of search engines to catch bugs, but also a  
deep knowledge of Lucene's internals.  Also, you risk compatibility as  
well as books and existing resources on the web that cover Lucene --  
hack, one can take any Java Lucene example and easily read it as a  
Lucene.Net code or use Luke to debug an index.  Keep in mind, the  
current port model that we have for Lucene.Net keeps the API  
one-to-one in sync with Java Lucene; just upper case method names.   
Yes, it's not fully .NET'es, but if you are looking for a search  
engine that is compatible with the open source search engine standard,  
and it is available in C#, Lucene.Net is it.

5) Beside making the port simpler, and per point #3 above, doing a  
line-per-line port, and maintaining API naming as well as the  
algorithm and file format of Java Lucene in C# Lucene means a Lucene  
index created by Java Lucene is usable, concurrently, by C# Lucene.  I  
have worked on one such project where a Java and C# code accessing the  
same index.  I'm not too interested in making Lucene.Net .NET'es and  
end up adding more risk to the project.

6) If anyone wants a different flavor of Lucene.Net, the code is on  
Apache, just fork it and start a new project.  Make it more .NET'es,  
use the latest that .NET has to offer, and all.  However, until when  
you have first hand experience with the port, and a good knowledge of  
Lucene and search engines, and the cycles to work on it, I really  
don't want to exercise this idea it will die as I know few folks have  
tried.

7) I can't speak for the other committers or those who contributed,  
but for me, I do this totally during my own time.  Each hour I spent  
on Lucene.Net is an hour away from my family or anything else.  I  
don't get paid, and I hardly get much off my Luene.Net work on the  
side.  As you may know, I was active with Lucene.Net till about early  
this year, (I had a family emergency).  I want to step up again, but  
we need more participation than just an offer to help or request  
divergence from the goal of the project, per the points that I made  
above.

I can go on, but the above are to clarify some of the issues and  
background of Lucene.Net.  Please keep those in mind when thinking  
about this project and how you can contribute -- especially comments  
about making Lucene.Net more .NET'es -- can't start that till when you  
first achieve commit-per-commit port of Java Lucene to C# Lucene.

If you agree with the above, and it makes sense to you, my suggestion  
is as follows:

1) Lucene.Net goes back into incubation and start all over again.
2) Start with cleaning up the webpage and make it more like other  
Apache project site.
3) Put together an official Lucene.Net 2.9.2 and get it released.
4) Start working on the next port.

#2, #3 can happen right away, and all that it takes to do them is  
coming up to speed on how-to using existing Apache documentation.  Who  
is up to this task?

#4 is a bit more complicated.  I don't want to go through the port  
pain that I had with 2.9.0 -- it was too much.  JLCA that comes with  
VS 2005 is out of date; I would love to try out a newer version from  
www.artinsoft.com, but it is $$.

I hope the above helps and I have not offended or discouraged anyone  
as it isn't my intention.  I just want to clarify few things about  
Lucene.Net

PS: One final point.  Look at CLucene, NLucene and few other variation  
of Java Lucene ports that were done at Lucene internal level with the  
goal of maintaining language look feature and look-and-fell, such as  
C++, those projects are either way out of date in terms of release  
version support or offer only partial support (index read only).  I  
don't want to use this to bad mouth another project, but to make a  
point that porting is hard if you diverge from the core.  As is,  
Lucene.Net is not dead, it's slow and needs contributors who will  
step-up.

Thanks,

-- George


Re: Lucene.NET Community Status

Posted by Vincent Daron <vd...@ask.be>.
Hi all

Sharpen from db4o could maybe replace jlca

My 2cents...

Vincent

Le 1 nov. 2010 à 21:55, George Aroush <ge...@aroush.net> a écrit :

> Let me jump in here and offer some perspective about Lucene.Net  
> (btw, it's not Lucene.NET :-) ).  This is based on my past  
> involvement with the project -- since 2003 when it was on SourceForge.net 
>  and called dotLucene.
>
> 1) Up until early this year, I have been porting and supporting Lucene.Net 
>  since ver 1.4 (back in 2004 on SourceForge.net) to the current  
> release on trunk ver. 2.9.2.  This is in NO WAY to say that others  
> have not helped or contributed.  I'm just saying that I know the  
> history and have the experience (I wrote and worked on search  
> engines from 1998 to 2002).
>
> 2) Doing an initial port of a new Java Lucene release to C# Lucene  
> is very hard; it's the most complex part of the port even using  
> automated tools such as JLCA and my own customize scripts which I  
> use pre-and port JLCA (you can search the listing on how I do the  
> port).  What used to take me about 1 months with 90% of tests  
> passing took me well over 4 months (for 2.9.x) with only 10% of  
> tests passing.  This was no easy effort and won't be easier now  
> since Java Lucene is using new Java language features that JLCA is  
> not aware of (MS is not maintaining JLCA).  Put another way, porting  
> is hard especially when you are dealing with > 5.6 GB source code  
> consistent of > 610 source files.  You will know this ONLY if you  
> have tried it out and maintained it -- this is why no one has  
> stepped up to do an initial port otherwise there would be a port by  
> now not only of Java Lucene but other projects too.
>
> 3) To simplify ports of new release, maintaining as small as  
> possible delta between release is very important. This was a main  
> pain point when I ported from 2.4 to 2.9.  The in-between ports were  
> never done due to lack of time on my end.  See point #2.
>
> 4) Diverging away from Java Lucene, both API base and algorithm is  
> risky and will just make point #2 more evidence.  Not only will you  
> now need a deep knowledge of search engines to catch bugs, but also  
> a deep knowledge of Lucene's internals.  Also, you risk  
> compatibility as well as books and existing resources on the web  
> that cover Lucene -- hack, one can take any Java Lucene example and  
> easily read it as a Lucene.Net code or use Luke to debug an index.   
> Keep in mind, the current port model that we have for Lucene.Net  
> keeps the API one-to-one in sync with Java Lucene; just upper case  
> method names.  Yes, it's not fully .NET'es, but if you are looking  
> for a search engine that is compatible with the open source search  
> engine standard, and it is available in C#, Lucene.Net is it.
>
> 5) Beside making the port simpler, and per point #3 above, doing a  
> line-per-line port, and maintaining API naming as well as the  
> algorithm and file format of Java Lucene in C# Lucene means a Lucene  
> index created by Java Lucene is usable, concurrently, by C# Lucene.   
> I have worked on one such project where a Java and C# code accessing  
> the same index.  I'm not too interested in making Lucene.Net .NET'es  
> and end up adding more risk to the project.
>
> 6) If anyone wants a different flavor of Lucene.Net, the code is on  
> Apache, just fork it and start a new project.  Make it more .NET'es,  
> use the latest that .NET has to offer, and all.  However, until when  
> you have first hand experience with the port, and a good knowledge  
> of Lucene and search engines, and the cycles to work on it, I really  
> don't want to exercise this idea it will die as I know few folks  
> have tried.
>
> 7) I can't speak for the other committers or those who contributed,  
> but for me, I do this totally during my own time.  Each hour I spent  
> on Lucene.Net is an hour away from my family or anything else.  I  
> don't get paid, and I hardly get much off my Luene.Net work on the  
> side.  As you may know, I was active with Lucene.Net till about  
> early this year, (I had a family emergency).  I want to step up  
> again, but we need more participation than just an offer to help or  
> request divergence from the goal of the project, per the points that  
> I made above.
>
> I can go on, but the above are to clarify some of the issues and  
> background of Lucene.Net.  Please keep those in mind when thinking  
> about this project and how you can contribute -- especially comments  
> about making Lucene.Net more .NET'es -- can't start that till when  
> you first achieve commit-per-commit port of Java Lucene to C# Lucene.
>
> If you agree with the above, and it makes sense to you, my  
> suggestion is as follows:
>
> 1) Lucene.Net goes back into incubation and start all over again.
> 2) Start with cleaning up the webpage and make it more like other  
> Apache project site.
> 3) Put together an official Lucene.Net 2.9.2 and get it released.
> 4) Start working on the next port.
>
> #2, #3 can happen right away, and all that it takes to do them is  
> coming up to speed on how-to using existing Apache documentation.   
> Who is up to this task?
>
> #4 is a bit more complicated.  I don't want to go through the port  
> pain that I had with 2.9.0 -- it was too much.  JLCA that comes with  
> VS 2005 is out of date; I would love to try out a newer version from www.artinsoft.com 
> , but it is $$.
>
> I hope the above helps and I have not offended or discouraged anyone  
> as it isn't my intention.  I just want to clarify few things about Lucene.Net
>
> PS: One final point.  Look at CLucene, NLucene and few other  
> variation of Java Lucene ports that were done at Lucene internal  
> level with the goal of maintaining language look feature and look- 
> and-fell, such as C++, those projects are either way out of date in  
> terms of release version support or offer only partial support  
> (index read only).  I don't want to use this to bad mouth another  
> project, but to make a point that porting is hard if you diverge  
> from the core.  As is, Lucene.Net is not dead, it's slow and needs  
> contributors who will step-up.
>
> Thanks,
>
> -- George
>

RE: Lucene.NET Community Status

Posted by Phil Haack <ph...@microsoft.com>.
Hi George, 

Thanks for your input on this and setting the context straight. Quick question before I dive in.

How much is the www.artinsoft.com software that you would need? Maybe I can help look for sponsorship. I looked at the website and it's a bit of a mess. I don't know exactly which product you would need and what it would cost.

As for #2/#3 what is required to get started? Do you know if the documentation must be on the Apache site? Could we create a mirror CodePlex.com project that just houses releases and docs, but the official site would remain on Apache? The reason I ask is that CodePlex.com provides support for documentation using the MetaWeblog API (and thus Windows Live Writer) which is a fantastic tool for documentation. It lowers the bar to documentation and I've been using it very productively for my projects.

If it must stay on Apache, a little nudge in the right direction and I'll see what I can do to help.

Phil

-----Original Message-----
From: George Aroush [mailto:george@aroush.net] 
Sent: Monday, November 01, 2010 1:56 PM
To: lucene-net-user@lucene.apache.org; lucene-net-dev@incubator.apache.org
Subject: RE: Lucene.NET Community Status

Let me jump in here and offer some perspective about Lucene.Net (btw, it's not Lucene.NET :-) ).  This is based on my past involvement with the project -- since 2003 when it was on SourceForge.net and called dotLucene.

1) Up until early this year, I have been porting and supporting Lucene.Net since ver 1.4 (back in 2004 on SourceForge.net) to the current release on trunk ver. 2.9.2.  This is in NO WAY to say that others have not helped or contributed.  I'm just saying that I know the history and have the experience (I wrote and worked on search engines from 1998 to 2002).

2) Doing an initial port of a new Java Lucene release to C# Lucene is very hard; it's the most complex part of the port even using automated tools such as JLCA and my own customize scripts which I use pre-and port JLCA (you can search the listing on how I do the port).  What used to take me about 1 months with 90% of tests passing took me well over 4 months (for 2.9.x) with only 10% of tests passing.  This was no easy effort and won't be easier now since Java Lucene is using new Java language features that JLCA is not aware of (MS is not maintaining JLCA).  Put another way, porting is hard especially when you are dealing with > 5.6 GB source code consistent of > 610 source files.  You will know this ONLY if you have tried it out and maintained it -- this is why no one has stepped up to do an initial port otherwise there would be a port by now not only of Java Lucene but other projects too.

3) To simplify ports of new release, maintaining as small as possible delta between release is very important. This was a main pain point when I ported from 2.4 to 2.9.  The in-between ports were never done due to lack of time on my end.  See point #2.

4) Diverging away from Java Lucene, both API base and algorithm is risky and will just make point #2 more evidence.  Not only will you now need a deep knowledge of search engines to catch bugs, but also a deep knowledge of Lucene's internals.  Also, you risk compatibility as well as books and existing resources on the web that cover Lucene -- hack, one can take any Java Lucene example and easily read it as a Lucene.Net code or use Luke to debug an index.  Keep in mind, the current port model that we have for Lucene.Net keeps the API  
one-to-one in sync with Java Lucene; just upper case method names.   
Yes, it's not fully .NET'es, but if you are looking for a search engine that is compatible with the open source search engine standard, and it is available in C#, Lucene.Net is it.

5) Beside making the port simpler, and per point #3 above, doing a line-per-line port, and maintaining API naming as well as the algorithm and file format of Java Lucene in C# Lucene means a Lucene index created by Java Lucene is usable, concurrently, by C# Lucene.  I have worked on one such project where a Java and C# code accessing the same index.  I'm not too interested in making Lucene.Net .NET'es and end up adding more risk to the project.

6) If anyone wants a different flavor of Lucene.Net, the code is on Apache, just fork it and start a new project.  Make it more .NET'es, use the latest that .NET has to offer, and all.  However, until when you have first hand experience with the port, and a good knowledge of Lucene and search engines, and the cycles to work on it, I really don't want to exercise this idea it will die as I know few folks have tried.

7) I can't speak for the other committers or those who contributed, but for me, I do this totally during my own time.  Each hour I spent on Lucene.Net is an hour away from my family or anything else.  I don't get paid, and I hardly get much off my Luene.Net work on the side.  As you may know, I was active with Lucene.Net till about early this year, (I had a family emergency).  I want to step up again, but we need more participation than just an offer to help or request divergence from the goal of the project, per the points that I made above.

I can go on, but the above are to clarify some of the issues and background of Lucene.Net.  Please keep those in mind when thinking about this project and how you can contribute -- especially comments about making Lucene.Net more .NET'es -- can't start that till when you first achieve commit-per-commit port of Java Lucene to C# Lucene.

If you agree with the above, and it makes sense to you, my suggestion is as follows:

1) Lucene.Net goes back into incubation and start all over again.
2) Start with cleaning up the webpage and make it more like other Apache project site.
3) Put together an official Lucene.Net 2.9.2 and get it released.
4) Start working on the next port.

#2, #3 can happen right away, and all that it takes to do them is coming up to speed on how-to using existing Apache documentation.  Who is up to this task?

#4 is a bit more complicated.  I don't want to go through the port pain that I had with 2.9.0 -- it was too much.  JLCA that comes with VS 2005 is out of date; I would love to try out a newer version from www.artinsoft.com, but it is $$.

I hope the above helps and I have not offended or discouraged anyone as it isn't my intention.  I just want to clarify few things about Lucene.Net

PS: One final point.  Look at CLucene, NLucene and few other variation of Java Lucene ports that were done at Lucene internal level with the goal of maintaining language look feature and look-and-fell, such as  
C++, those projects are either way out of date in terms of release
version support or offer only partial support (index read only).  I don't want to use this to bad mouth another project, but to make a point that porting is hard if you diverge from the core.  As is, Lucene.Net is not dead, it's slow and needs contributors who will step-up.

Thanks,

-- George



Re: Lucene.NET Community Status

Posted by Michael Mitiaguin <mi...@gmail.com>.
<George Aroush>
porting is hard especially when you are dealing with > 5.6 GB source code
consistent of > 610 source files
</George Aroush>
I suppose, 5.6MB consisting of  more  610 files  was meant, otherwise  an
aspiring new comitter may run away after reading this <g> .

On Tue, Nov 2, 2010 at 7:55 AM, George Aroush <ge...@aroush.net> wrote:

> Let me jump in here and offer some perspective about Lucene.Net (btw, it's
> not Lucene.NET :-) ).  This is based on my past involvement with the project
> -- since 2003 when it was on SourceForge.net and called dotLucene.
>
> 1) Up until early this year, I have been porting and supporting Lucene.Net
> since ver 1.4 (back in 2004 on SourceForge.net) to the current release on
> trunk ver. 2.9.2.  This is in NO WAY to say that others have not helped or
> contributed.  I'm just saying that I know the history and have the
> experience (I wrote and worked on search engines from 1998 to 2002).
>
> 2) Doing an initial port of a new Java Lucene release to C# Lucene is very
> hard; it's the most complex part of the port even using automated tools such
> as JLCA and my own customize scripts which I use pre-and port JLCA (you can
> search the listing on how I do the port).  What used to take me about 1
> months with 90% of tests passing took me well over 4 months (for 2.9.x) with
> only 10% of tests passing.  This was no easy effort and won't be easier now
> since Java Lucene is using new Java language features that JLCA is not aware
> of (MS is not maintaining JLCA).  Put another way, porting is hard
> especially when you are dealing with > 5.6 GB source code consistent of >
> 610 source files.  You will know this ONLY if you have tried it out and
> maintained it -- this is why no one has stepped up to do an initial port
> otherwise there would be a port by now not only of Java Lucene but other
> projects too.
>
> 3) To simplify ports of new release, maintaining as small as possible delta
> between release is very important. This was a main pain point when I ported
> from 2.4 to 2.9.  The in-between ports were never done due to lack of time
> on my end.  See point #2.
>
> 4) Diverging away from Java Lucene, both API base and algorithm is risky
> and will just make point #2 more evidence.  Not only will you now need a
> deep knowledge of search engines to catch bugs, but also a deep knowledge of
> Lucene's internals.  Also, you risk compatibility as well as books and
> existing resources on the web that cover Lucene -- hack, one can take any
> Java Lucene example and easily read it as a Lucene.Net code or use Luke to
> debug an index.  Keep in mind, the current port model that we have for
> Lucene.Net keeps the API one-to-one in sync with Java Lucene; just upper
> case method names.  Yes, it's not fully .NET'es, but if you are looking for
> a search engine that is compatible with the open source search engine
> standard, and it is available in C#, Lucene.Net is it.
>
> 5) Beside making the port simpler, and per point #3 above, doing a
> line-per-line port, and maintaining API naming as well as the algorithm and
> file format of Java Lucene in C# Lucene means a Lucene index created by Java
> Lucene is usable, concurrently, by C# Lucene.  I have worked on one such
> project where a Java and C# code accessing the same index.  I'm not too
> interested in making Lucene.Net .NET'es and end up adding more risk to the
> project.
>
> 6) If anyone wants a different flavor of Lucene.Net, the code is on Apache,
> just fork it and start a new project.  Make it more .NET'es, use the latest
> that .NET has to offer, and all.  However, until when you have first hand
> experience with the port, and a good knowledge of Lucene and search engines,
> and the cycles to work on it, I really don't want to exercise this idea it
> will die as I know few folks have tried.
>
> 7) I can't speak for the other committers or those who contributed, but for
> me, I do this totally during my own time.  Each hour I spent on Lucene.Net
> is an hour away from my family or anything else.  I don't get paid, and I
> hardly get much off my Luene.Net work on the side.  As you may know, I was
> active with Lucene.Net till about early this year, (I had a family
> emergency).  I want to step up again, but we need more participation than
> just an offer to help or request divergence from the goal of the project,
> per the points that I made above.
>
> I can go on, but the above are to clarify some of the issues and background
> of Lucene.Net.  Please keep those in mind when thinking about this project
> and how you can contribute -- especially comments about making Lucene.Net
> more .NET'es -- can't start that till when you first achieve
> commit-per-commit port of Java Lucene to C# Lucene.
>
> If you agree with the above, and it makes sense to you, my suggestion is as
> follows:
>
> 1) Lucene.Net goes back into incubation and start all over again.
> 2) Start with cleaning up the webpage and make it more like other Apache
> project site.
> 3) Put together an official Lucene.Net 2.9.2 and get it released.
> 4) Start working on the next port.
>
> #2, #3 can happen right away, and all that it takes to do them is coming up
> to speed on how-to using existing Apache documentation.  Who is up to this
> task?
>
> #4 is a bit more complicated.  I don't want to go through the port pain
> that I had with 2.9.0 -- it was too much.  JLCA that comes with VS 2005 is
> out of date; I would love to try out a newer version from
> www.artinsoft.com, but it is $$.
>
> I hope the above helps and I have not offended or discouraged anyone as it
> isn't my intention.  I just want to clarify few things about Lucene.Net
>
> PS: One final point.  Look at CLucene, NLucene and few other variation of
> Java Lucene ports that were done at Lucene internal level with the goal of
> maintaining language look feature and look-and-fell, such as C++, those
> projects are either way out of date in terms of release version support or
> offer only partial support (index read only).  I don't want to use this to
> bad mouth another project, but to make a point that porting is hard if you
> diverge from the core.  As is, Lucene.Net is not dead, it's slow and needs
> contributors who will step-up.
>
> Thanks,
>
> -- George
>
>

Re: Lucene.NET Community Status

Posted by Arne Claassen <ar...@mindtouch.com>.
On Nov 1, 2010, at 1:55 PM, George Aroush wrote:

> 3) Put together an official Lucene.Net 2.9.2 and get it released.


George,

What's involved in #3? Just build the binaries, strip out the source  
and zip up similar to the 2.0 release? I'd like to help get this out  
the door.

thanks,
Arne Claassen

MindTouch
San Diego, CA
http://twitter.com/sdether


RE: Lucene.NET Community Status

Posted by Digy <di...@gmail.com>.
Hi Neal,

> The "brand recognition" ...... 
I Agree

> If the project were moved elsewhere, I do not think it would survive.
Probably you are right but I am not going to take any step (personally) for
acceptance to the incubator.


DIGY

-----Original Message-----
From: Granroth, Neal V. [mailto:neal.granroth@thermofisher.com] 
Sent: Tuesday, November 02, 2010 9:58 PM
To: lucene-net-dev@lucene.apache.org
Subject: RE: Lucene.NET Community Status

DIGY,

I agree in that I am surprised that Lucene PMC has not been following the
on-going activity of the project.  There has been significant activity after
George stepped away from the project and with his help I thought had taken
the project beyond incubator status.

While the ASF formal procedures might appear somewhat tedious, they do
provide project users with confidence in the quality and durability of the
project. The "brand recognition" Grant mentioned.  You don't get this
confidence from Open Source code projects managed elsewhere.

That web site and documentation are a bit light is of no consequence; the
Java source, documentation, and examples work well, and the ability to
re-use tools like Luke are a great development advantage.  Developers who
are not comfortable with this type of cross-platform approach should
probably seek further opportunities to advance their development skills.
 
If the project were moved elsewhere, I do not think it would survive.

- Neal


-----Original Message-----
From: Digy [mailto:digydigy@gmail.com] 
Sent: Tuesday, November 02, 2010 2:01 PM
To: lucene-net-user@lucene.apache.org
Subject: RE: Lucene.NET Community Status

Hi Neal,

 

You could write up a Board proposal to go straight to TLP status.  I'd have
a hard time recommending to the Board that they pass it but maybe they would
b/c as I outlined in my original email, this project isn't up to ASF
community standards and not only that you basically only have one current
person who is an active committer/PMC member.  As I also outlined in earlier
emails, the current Lucene PMC is not the appropriate place for Lucene.NET
b/c the members of the PMC are not interested in .NET.  George is the only
one and he has been gone for the past few months (if not more).  That isn't
to pick on George, it's to point out that a project has to be more than just
one committer to be a part of the ASF, especially one that has been around
this long.  So, in order  for this project to get more committers, people
need to step up and contribute.  Therein lies the conundrum.  The current
PMC is not equipped to judge those contributions since none of us use .NET.
Hence, going back to incubation gets you a new set of committers and it gets
you your own PMC where you can set the criteria for committership (within
ASF guidelines) and where the PMC is made up of the stakeholders in the
project.  Being a part of a project is about more than just the name, it's
about the community of people who use and contribute to that project.  The
.NET community is distinct from the Lucene Java community, despite it being
a port, therefore they should be separate.

 

See http://www.apache.org/foundation/how-it-works.html

 

As to those questions about forking somewhere else, that is certainly
something that can be done under a different name.  Lucene.NET is owned by
the ASF.  You can take the code and go call it something else, no problem.

 

As to what the ASF brings, that's up to the community to decide.  The number
one thing I think is our "community over code" approach.  Anyone can throw
code up on Github/Google Code, etc. and call it open source.  If you are
lucky, you might attract a following.  If the person who started that
project is nice, they might even allow other committers.  At the end of the
day, however, I think the ASF's meritocracy is why I choose to put my open
source efforts into the ASF.  It is just one way, not _the_ way.  Having
started other projects here at the ASF (Mahout), I can tell you the ASF is
one of the few orgs. out there that can attract large bases of
users/contributors almost instantaneously.  In other words, the ASF has
brand recognition like few other places.  Again, this is just my view.  I'm
not going to force it on you, but you are already here, so it seems like
it's less friction to go back to the incubator and graduate to TLP than to
fork and try to get people to go find you under a different name.

 

My two cents,

Grant=

 

 

 

 

Besides PMC's not knowing the activity of the project(see the svn log for
the last years. What was done and who has done to keep the project alive),
They also don't want any .NET app. 

Is this reason enough?

 

DIGY

 

 

-----Original Message-----
From: Granroth, Neal V. [mailto:neal.granroth@thermofisher.com] 
Sent: Tuesday, November 02, 2010 7:36 PM
To: lucene-net-user@lucene.apache.org
Subject: RE: Lucene.NET Community Status

 

Why another location?

Keeping it as close as possible to the location of Java-Lucene makes the
most sense.

- Neal

 

 

-----Original Message-----

From: Digy [mailto:digydigy@gmail.com] 

Sent: Monday, November 01, 2010 4:45 PM

To: lucene-net-user@lucene.apache.org

Subject: RE: Lucene.NET Community Status

 

We a still think of hosting the project in an another location.

 

DIGY

 

-----Original Message-----

From: George Aroush [mailto:george@aroush.net] 

Sent: Monday, November 01, 2010 10:56 PM

To: lucene-net-user@lucene.apache.org; lucene-net-dev@incubator.apache.org

Subject: RE: Lucene.NET Community Status

 

Let me jump in here and offer some perspective about Lucene.Net (btw,  

it's not Lucene.NET :-) ).  This is based on my past involvement with  

the project -- since 2003 when it was on SourceForge.net and called  

dotLucene.

 

1) Up until early this year, I have been porting and supporting  

Lucene.Net since ver 1.4 (back in 2004 on SourceForge.net) to the  

current release on trunk ver. 2.9.2.  This is in NO WAY to say that  

others have not helped or contributed.  I'm just saying that I know  

the history and have the experience (I wrote and worked on search  

engines from 1998 to 2002).

 

2) Doing an initial port of a new Java Lucene release to C# Lucene is  

very hard; it's the most complex part of the port even using automated  

tools such as JLCA and my own customize scripts which I use pre-and  

port JLCA (you can search the listing on how I do the port).  What  

used to take me about 1 months with 90% of tests passing took me well  

over 4 months (for 2.9.x) with only 10% of tests passing.  This was no  

easy effort and won't be easier now since Java Lucene is using new  

Java language features that JLCA is not aware of (MS is not  

maintaining JLCA).  Put another way, porting is hard especially when  

you are dealing with > 5.6 GB source code consistent of > 610 source  

files.  You will know this ONLY if you have tried it out and  

maintained it -- this is why no one has stepped up to do an initial  

port otherwise there would be a port by now not only of Java Lucene  

but other projects too.

 

3) To simplify ports of new release, maintaining as small as possible  

delta between release is very important. This was a main pain point  

when I ported from 2.4 to 2.9.  The in-between ports were never done  

due to lack of time on my end.  See point #2.

 

4) Diverging away from Java Lucene, both API base and algorithm is  

risky and will just make point #2 more evidence.  Not only will you  

now need a deep knowledge of search engines to catch bugs, but also a  

deep knowledge of Lucene's internals.  Also, you risk compatibility as  

well as books and existing resources on the web that cover Lucene --  

hack, one can take any Java Lucene example and easily read it as a  

Lucene.Net code or use Luke to debug an index.  Keep in mind, the  

current port model that we have for Lucene.Net keeps the API  

one-to-one in sync with Java Lucene; just upper case method names.   

Yes, it's not fully .NET'es, but if you are looking for a search  

engine that is compatible with the open source search engine standard,  

and it is available in C#, Lucene.Net is it.

 

5) Beside making the port simpler, and per point #3 above, doing a  

line-per-line port, and maintaining API naming as well as the  

algorithm and file format of Java Lucene in C# Lucene means a Lucene  

index created by Java Lucene is usable, concurrently, by C# Lucene.  I  

have worked on one such project where a Java and C# code accessing the  

same index.  I'm not too interested in making Lucene.Net .NET'es and  

end up adding more risk to the project.

 

6) If anyone wants a different flavor of Lucene.Net, the code is on  

Apache, just fork it and start a new project.  Make it more .NET'es,  

use the latest that .NET has to offer, and all.  However, until when  

you have first hand experience with the port, and a good knowledge of  

Lucene and search engines, and the cycles to work on it, I really  

don't want to exercise this idea it will die as I know few folks have  

tried.

 

7) I can't speak for the other committers or those who contributed,  

but for me, I do this totally during my own time.  Each hour I spent  

on Lucene.Net is an hour away from my family or anything else.  I  

don't get paid, and I hardly get much off my Luene.Net work on the  

side.  As you may know, I was active with Lucene.Net till about early  

this year, (I had a family emergency).  I want to step up again, but  

we need more participation than just an offer to help or request  

divergence from the goal of the project, per the points that I made  

above.

 

I can go on, but the above are to clarify some of the issues and  

background of Lucene.Net.  Please keep those in mind when thinking  

about this project and how you can contribute -- especially comments  

about making Lucene.Net more .NET'es -- can't start that till when you  

first achieve commit-per-commit port of Java Lucene to C# Lucene.

 

If you agree with the above, and it makes sense to you, my suggestion  

is as follows:

 

1) Lucene.Net goes back into incubation and start all over again.

2) Start with cleaning up the webpage and make it more like other  

Apache project site.

3) Put together an official Lucene.Net 2.9.2 and get it released.

4) Start working on the next port.

 

#2, #3 can happen right away, and all that it takes to do them is  

coming up to speed on how-to using existing Apache documentation.  Who  

is up to this task?

 

#4 is a bit more complicated.  I don't want to go through the port  

pain that I had with 2.9.0 -- it was too much.  JLCA that comes with  

VS 2005 is out of date; I would love to try out a newer version from  

www.artinsoft.com, but it is $$.

 

I hope the above helps and I have not offended or discouraged anyone  

as it isn't my intention.  I just want to clarify few things about  

Lucene.Net

 

PS: One final point.  Look at CLucene, NLucene and few other variation  

of Java Lucene ports that were done at Lucene internal level with the  

goal of maintaining language look feature and look-and-fell, such as  

C++, those projects are either way out of date in terms of release  

version support or offer only partial support (index read only).  I  

don't want to use this to bad mouth another project, but to make a  

point that porting is hard if you diverge from the core.  As is,  

Lucene.Net is not dead, it's slow and needs contributors who will  

step-up.

 

Thanks,

 

-- George


RE: Lucene.NET Community Status

Posted by "Granroth, Neal V." <ne...@thermofisher.com>.
DIGY,

I agree in that I am surprised that Lucene PMC has not been following the on-going activity of the project.  There has been significant activity after George stepped away from the project and with his help I thought had taken the project beyond incubator status.

While the ASF formal procedures might appear somewhat tedious, they do provide project users with confidence in the quality and durability of the project. The "brand recognition" Grant mentioned.  You don't get this confidence from Open Source code projects managed elsewhere.

That web site and documentation are a bit light is of no consequence; the Java source, documentation, and examples work well, and the ability to re-use tools like Luke are a great development advantage.  Developers who are not comfortable with this type of cross-platform approach should probably seek further opportunities to advance their development skills.
 
If the project were moved elsewhere, I do not think it would survive.

- Neal


-----Original Message-----
From: Digy [mailto:digydigy@gmail.com] 
Sent: Tuesday, November 02, 2010 2:01 PM
To: lucene-net-user@lucene.apache.org
Subject: RE: Lucene.NET Community Status

Hi Neal,

 

You could write up a Board proposal to go straight to TLP status.  I'd have
a hard time recommending to the Board that they pass it but maybe they would
b/c as I outlined in my original email, this project isn't up to ASF
community standards and not only that you basically only have one current
person who is an active committer/PMC member.  As I also outlined in earlier
emails, the current Lucene PMC is not the appropriate place for Lucene.NET
b/c the members of the PMC are not interested in .NET.  George is the only
one and he has been gone for the past few months (if not more).  That isn't
to pick on George, it's to point out that a project has to be more than just
one committer to be a part of the ASF, especially one that has been around
this long.  So, in order  for this project to get more committers, people
need to step up and contribute.  Therein lies the conundrum.  The current
PMC is not equipped to judge those contributions since none of us use .NET.
Hence, going back to incubation gets you a new set of committers and it gets
you your own PMC where you can set the criteria for committership (within
ASF guidelines) and where the PMC is made up of the stakeholders in the
project.  Being a part of a project is about more than just the name, it's
about the community of people who use and contribute to that project.  The
.NET community is distinct from the Lucene Java community, despite it being
a port, therefore they should be separate.

 

See http://www.apache.org/foundation/how-it-works.html

 

As to those questions about forking somewhere else, that is certainly
something that can be done under a different name.  Lucene.NET is owned by
the ASF.  You can take the code and go call it something else, no problem.

 

As to what the ASF brings, that's up to the community to decide.  The number
one thing I think is our "community over code" approach.  Anyone can throw
code up on Github/Google Code, etc. and call it open source.  If you are
lucky, you might attract a following.  If the person who started that
project is nice, they might even allow other committers.  At the end of the
day, however, I think the ASF's meritocracy is why I choose to put my open
source efforts into the ASF.  It is just one way, not _the_ way.  Having
started other projects here at the ASF (Mahout), I can tell you the ASF is
one of the few orgs. out there that can attract large bases of
users/contributors almost instantaneously.  In other words, the ASF has
brand recognition like few other places.  Again, this is just my view.  I'm
not going to force it on you, but you are already here, so it seems like
it's less friction to go back to the incubator and graduate to TLP than to
fork and try to get people to go find you under a different name.

 

My two cents,

Grant=

 

 

 

 

Besides PMC's not knowing the activity of the project(see the svn log for
the last years. What was done and who has done to keep the project alive),
They also don't want any .NET app. 

Is this reason enough?

 

DIGY

 

 

-----Original Message-----
From: Granroth, Neal V. [mailto:neal.granroth@thermofisher.com] 
Sent: Tuesday, November 02, 2010 7:36 PM
To: lucene-net-user@lucene.apache.org
Subject: RE: Lucene.NET Community Status

 

Why another location?

Keeping it as close as possible to the location of Java-Lucene makes the
most sense.

- Neal

 

 

-----Original Message-----

From: Digy [mailto:digydigy@gmail.com] 

Sent: Monday, November 01, 2010 4:45 PM

To: lucene-net-user@lucene.apache.org

Subject: RE: Lucene.NET Community Status

 

We a still think of hosting the project in an another location.

 

DIGY

 

-----Original Message-----

From: George Aroush [mailto:george@aroush.net] 

Sent: Monday, November 01, 2010 10:56 PM

To: lucene-net-user@lucene.apache.org; lucene-net-dev@incubator.apache.org

Subject: RE: Lucene.NET Community Status

 

Let me jump in here and offer some perspective about Lucene.Net (btw,  

it's not Lucene.NET :-) ).  This is based on my past involvement with  

the project -- since 2003 when it was on SourceForge.net and called  

dotLucene.

 

1) Up until early this year, I have been porting and supporting  

Lucene.Net since ver 1.4 (back in 2004 on SourceForge.net) to the  

current release on trunk ver. 2.9.2.  This is in NO WAY to say that  

others have not helped or contributed.  I'm just saying that I know  

the history and have the experience (I wrote and worked on search  

engines from 1998 to 2002).

 

2) Doing an initial port of a new Java Lucene release to C# Lucene is  

very hard; it's the most complex part of the port even using automated  

tools such as JLCA and my own customize scripts which I use pre-and  

port JLCA (you can search the listing on how I do the port).  What  

used to take me about 1 months with 90% of tests passing took me well  

over 4 months (for 2.9.x) with only 10% of tests passing.  This was no  

easy effort and won't be easier now since Java Lucene is using new  

Java language features that JLCA is not aware of (MS is not  

maintaining JLCA).  Put another way, porting is hard especially when  

you are dealing with > 5.6 GB source code consistent of > 610 source  

files.  You will know this ONLY if you have tried it out and  

maintained it -- this is why no one has stepped up to do an initial  

port otherwise there would be a port by now not only of Java Lucene  

but other projects too.

 

3) To simplify ports of new release, maintaining as small as possible  

delta between release is very important. This was a main pain point  

when I ported from 2.4 to 2.9.  The in-between ports were never done  

due to lack of time on my end.  See point #2.

 

4) Diverging away from Java Lucene, both API base and algorithm is  

risky and will just make point #2 more evidence.  Not only will you  

now need a deep knowledge of search engines to catch bugs, but also a  

deep knowledge of Lucene's internals.  Also, you risk compatibility as  

well as books and existing resources on the web that cover Lucene --  

hack, one can take any Java Lucene example and easily read it as a  

Lucene.Net code or use Luke to debug an index.  Keep in mind, the  

current port model that we have for Lucene.Net keeps the API  

one-to-one in sync with Java Lucene; just upper case method names.   

Yes, it's not fully .NET'es, but if you are looking for a search  

engine that is compatible with the open source search engine standard,  

and it is available in C#, Lucene.Net is it.

 

5) Beside making the port simpler, and per point #3 above, doing a  

line-per-line port, and maintaining API naming as well as the  

algorithm and file format of Java Lucene in C# Lucene means a Lucene  

index created by Java Lucene is usable, concurrently, by C# Lucene.  I  

have worked on one such project where a Java and C# code accessing the  

same index.  I'm not too interested in making Lucene.Net .NET'es and  

end up adding more risk to the project.

 

6) If anyone wants a different flavor of Lucene.Net, the code is on  

Apache, just fork it and start a new project.  Make it more .NET'es,  

use the latest that .NET has to offer, and all.  However, until when  

you have first hand experience with the port, and a good knowledge of  

Lucene and search engines, and the cycles to work on it, I really  

don't want to exercise this idea it will die as I know few folks have  

tried.

 

7) I can't speak for the other committers or those who contributed,  

but for me, I do this totally during my own time.  Each hour I spent  

on Lucene.Net is an hour away from my family or anything else.  I  

don't get paid, and I hardly get much off my Luene.Net work on the  

side.  As you may know, I was active with Lucene.Net till about early  

this year, (I had a family emergency).  I want to step up again, but  

we need more participation than just an offer to help or request  

divergence from the goal of the project, per the points that I made  

above.

 

I can go on, but the above are to clarify some of the issues and  

background of Lucene.Net.  Please keep those in mind when thinking  

about this project and how you can contribute -- especially comments  

about making Lucene.Net more .NET'es -- can't start that till when you  

first achieve commit-per-commit port of Java Lucene to C# Lucene.

 

If you agree with the above, and it makes sense to you, my suggestion  

is as follows:

 

1) Lucene.Net goes back into incubation and start all over again.

2) Start with cleaning up the webpage and make it more like other  

Apache project site.

3) Put together an official Lucene.Net 2.9.2 and get it released.

4) Start working on the next port.

 

#2, #3 can happen right away, and all that it takes to do them is  

coming up to speed on how-to using existing Apache documentation.  Who  

is up to this task?

 

#4 is a bit more complicated.  I don't want to go through the port  

pain that I had with 2.9.0 -- it was too much.  JLCA that comes with  

VS 2005 is out of date; I would love to try out a newer version from  

www.artinsoft.com, but it is $$.

 

I hope the above helps and I have not offended or discouraged anyone  

as it isn't my intention.  I just want to clarify few things about  

Lucene.Net

 

PS: One final point.  Look at CLucene, NLucene and few other variation  

of Java Lucene ports that were done at Lucene internal level with the  

goal of maintaining language look feature and look-and-fell, such as  

C++, those projects are either way out of date in terms of release  

version support or offer only partial support (index read only).  I  

don't want to use this to bad mouth another project, but to make a  

point that porting is hard if you diverge from the core.  As is,  

Lucene.Net is not dead, it's slow and needs contributors who will  

step-up.

 

Thanks,

 

-- George


RE: Lucene.NET Community Status

Posted by Digy <di...@gmail.com>.
Hi Neal,

 

You could write up a Board proposal to go straight to TLP status.  I'd have
a hard time recommending to the Board that they pass it but maybe they would
b/c as I outlined in my original email, this project isn't up to ASF
community standards and not only that you basically only have one current
person who is an active committer/PMC member.  As I also outlined in earlier
emails, the current Lucene PMC is not the appropriate place for Lucene.NET
b/c the members of the PMC are not interested in .NET.  George is the only
one and he has been gone for the past few months (if not more).  That isn't
to pick on George, it's to point out that a project has to be more than just
one committer to be a part of the ASF, especially one that has been around
this long.  So, in order  for this project to get more committers, people
need to step up and contribute.  Therein lies the conundrum.  The current
PMC is not equipped to judge those contributions since none of us use .NET.
Hence, going back to incubation gets you a new set of committers and it gets
you your own PMC where you can set the criteria for committership (within
ASF guidelines) and where the PMC is made up of the stakeholders in the
project.  Being a part of a project is about more than just the name, it's
about the community of people who use and contribute to that project.  The
.NET community is distinct from the Lucene Java community, despite it being
a port, therefore they should be separate.

 

See http://www.apache.org/foundation/how-it-works.html

 

As to those questions about forking somewhere else, that is certainly
something that can be done under a different name.  Lucene.NET is owned by
the ASF.  You can take the code and go call it something else, no problem.

 

As to what the ASF brings, that's up to the community to decide.  The number
one thing I think is our "community over code" approach.  Anyone can throw
code up on Github/Google Code, etc. and call it open source.  If you are
lucky, you might attract a following.  If the person who started that
project is nice, they might even allow other committers.  At the end of the
day, however, I think the ASF's meritocracy is why I choose to put my open
source efforts into the ASF.  It is just one way, not _the_ way.  Having
started other projects here at the ASF (Mahout), I can tell you the ASF is
one of the few orgs. out there that can attract large bases of
users/contributors almost instantaneously.  In other words, the ASF has
brand recognition like few other places.  Again, this is just my view.  I'm
not going to force it on you, but you are already here, so it seems like
it's less friction to go back to the incubator and graduate to TLP than to
fork and try to get people to go find you under a different name.

 

My two cents,

Grant=

 

 

 

 

Besides PMC's not knowing the activity of the project(see the svn log for
the last years. What was done and who has done to keep the project alive),
They also don't want any .NET app. 

Is this reason enough?

 

DIGY

 

 

-----Original Message-----
From: Granroth, Neal V. [mailto:neal.granroth@thermofisher.com] 
Sent: Tuesday, November 02, 2010 7:36 PM
To: lucene-net-user@lucene.apache.org
Subject: RE: Lucene.NET Community Status

 

Why another location?

Keeping it as close as possible to the location of Java-Lucene makes the
most sense.

- Neal

 

 

-----Original Message-----

From: Digy [mailto:digydigy@gmail.com] 

Sent: Monday, November 01, 2010 4:45 PM

To: lucene-net-user@lucene.apache.org

Subject: RE: Lucene.NET Community Status

 

We a still think of hosting the project in an another location.

 

DIGY

 

-----Original Message-----

From: George Aroush [mailto:george@aroush.net] 

Sent: Monday, November 01, 2010 10:56 PM

To: lucene-net-user@lucene.apache.org; lucene-net-dev@incubator.apache.org

Subject: RE: Lucene.NET Community Status

 

Let me jump in here and offer some perspective about Lucene.Net (btw,  

it's not Lucene.NET :-) ).  This is based on my past involvement with  

the project -- since 2003 when it was on SourceForge.net and called  

dotLucene.

 

1) Up until early this year, I have been porting and supporting  

Lucene.Net since ver 1.4 (back in 2004 on SourceForge.net) to the  

current release on trunk ver. 2.9.2.  This is in NO WAY to say that  

others have not helped or contributed.  I'm just saying that I know  

the history and have the experience (I wrote and worked on search  

engines from 1998 to 2002).

 

2) Doing an initial port of a new Java Lucene release to C# Lucene is  

very hard; it's the most complex part of the port even using automated  

tools such as JLCA and my own customize scripts which I use pre-and  

port JLCA (you can search the listing on how I do the port).  What  

used to take me about 1 months with 90% of tests passing took me well  

over 4 months (for 2.9.x) with only 10% of tests passing.  This was no  

easy effort and won't be easier now since Java Lucene is using new  

Java language features that JLCA is not aware of (MS is not  

maintaining JLCA).  Put another way, porting is hard especially when  

you are dealing with > 5.6 GB source code consistent of > 610 source  

files.  You will know this ONLY if you have tried it out and  

maintained it -- this is why no one has stepped up to do an initial  

port otherwise there would be a port by now not only of Java Lucene  

but other projects too.

 

3) To simplify ports of new release, maintaining as small as possible  

delta between release is very important. This was a main pain point  

when I ported from 2.4 to 2.9.  The in-between ports were never done  

due to lack of time on my end.  See point #2.

 

4) Diverging away from Java Lucene, both API base and algorithm is  

risky and will just make point #2 more evidence.  Not only will you  

now need a deep knowledge of search engines to catch bugs, but also a  

deep knowledge of Lucene's internals.  Also, you risk compatibility as  

well as books and existing resources on the web that cover Lucene --  

hack, one can take any Java Lucene example and easily read it as a  

Lucene.Net code or use Luke to debug an index.  Keep in mind, the  

current port model that we have for Lucene.Net keeps the API  

one-to-one in sync with Java Lucene; just upper case method names.   

Yes, it's not fully .NET'es, but if you are looking for a search  

engine that is compatible with the open source search engine standard,  

and it is available in C#, Lucene.Net is it.

 

5) Beside making the port simpler, and per point #3 above, doing a  

line-per-line port, and maintaining API naming as well as the  

algorithm and file format of Java Lucene in C# Lucene means a Lucene  

index created by Java Lucene is usable, concurrently, by C# Lucene.  I  

have worked on one such project where a Java and C# code accessing the  

same index.  I'm not too interested in making Lucene.Net .NET'es and  

end up adding more risk to the project.

 

6) If anyone wants a different flavor of Lucene.Net, the code is on  

Apache, just fork it and start a new project.  Make it more .NET'es,  

use the latest that .NET has to offer, and all.  However, until when  

you have first hand experience with the port, and a good knowledge of  

Lucene and search engines, and the cycles to work on it, I really  

don't want to exercise this idea it will die as I know few folks have  

tried.

 

7) I can't speak for the other committers or those who contributed,  

but for me, I do this totally during my own time.  Each hour I spent  

on Lucene.Net is an hour away from my family or anything else.  I  

don't get paid, and I hardly get much off my Luene.Net work on the  

side.  As you may know, I was active with Lucene.Net till about early  

this year, (I had a family emergency).  I want to step up again, but  

we need more participation than just an offer to help or request  

divergence from the goal of the project, per the points that I made  

above.

 

I can go on, but the above are to clarify some of the issues and  

background of Lucene.Net.  Please keep those in mind when thinking  

about this project and how you can contribute -- especially comments  

about making Lucene.Net more .NET'es -- can't start that till when you  

first achieve commit-per-commit port of Java Lucene to C# Lucene.

 

If you agree with the above, and it makes sense to you, my suggestion  

is as follows:

 

1) Lucene.Net goes back into incubation and start all over again.

2) Start with cleaning up the webpage and make it more like other  

Apache project site.

3) Put together an official Lucene.Net 2.9.2 and get it released.

4) Start working on the next port.

 

#2, #3 can happen right away, and all that it takes to do them is  

coming up to speed on how-to using existing Apache documentation.  Who  

is up to this task?

 

#4 is a bit more complicated.  I don't want to go through the port  

pain that I had with 2.9.0 -- it was too much.  JLCA that comes with  

VS 2005 is out of date; I would love to try out a newer version from  

www.artinsoft.com, but it is $$.

 

I hope the above helps and I have not offended or discouraged anyone  

as it isn't my intention.  I just want to clarify few things about  

Lucene.Net

 

PS: One final point.  Look at CLucene, NLucene and few other variation  

of Java Lucene ports that were done at Lucene internal level with the  

goal of maintaining language look feature and look-and-fell, such as  

C++, those projects are either way out of date in terms of release  

version support or offer only partial support (index read only).  I  

don't want to use this to bad mouth another project, but to make a  

point that porting is hard if you diverge from the core.  As is,  

Lucene.Net is not dead, it's slow and needs contributors who will  

step-up.

 

Thanks,

 

-- George


RE: Lucene.NET Community Status

Posted by "Granroth, Neal V." <ne...@thermofisher.com>.
Why another location?
Keeping it as close as possible to the location of Java-Lucene makes the most sense.
- Neal


-----Original Message-----
From: Digy [mailto:digydigy@gmail.com] 
Sent: Monday, November 01, 2010 4:45 PM
To: lucene-net-user@lucene.apache.org
Subject: RE: Lucene.NET Community Status

We a still think of hosting the project in an another location.

DIGY

-----Original Message-----
From: George Aroush [mailto:george@aroush.net] 
Sent: Monday, November 01, 2010 10:56 PM
To: lucene-net-user@lucene.apache.org; lucene-net-dev@incubator.apache.org
Subject: RE: Lucene.NET Community Status

Let me jump in here and offer some perspective about Lucene.Net (btw,  
it's not Lucene.NET :-) ).  This is based on my past involvement with  
the project -- since 2003 when it was on SourceForge.net and called  
dotLucene.

1) Up until early this year, I have been porting and supporting  
Lucene.Net since ver 1.4 (back in 2004 on SourceForge.net) to the  
current release on trunk ver. 2.9.2.  This is in NO WAY to say that  
others have not helped or contributed.  I'm just saying that I know  
the history and have the experience (I wrote and worked on search  
engines from 1998 to 2002).

2) Doing an initial port of a new Java Lucene release to C# Lucene is  
very hard; it's the most complex part of the port even using automated  
tools such as JLCA and my own customize scripts which I use pre-and  
port JLCA (you can search the listing on how I do the port).  What  
used to take me about 1 months with 90% of tests passing took me well  
over 4 months (for 2.9.x) with only 10% of tests passing.  This was no  
easy effort and won't be easier now since Java Lucene is using new  
Java language features that JLCA is not aware of (MS is not  
maintaining JLCA).  Put another way, porting is hard especially when  
you are dealing with > 5.6 GB source code consistent of > 610 source  
files.  You will know this ONLY if you have tried it out and  
maintained it -- this is why no one has stepped up to do an initial  
port otherwise there would be a port by now not only of Java Lucene  
but other projects too.

3) To simplify ports of new release, maintaining as small as possible  
delta between release is very important. This was a main pain point  
when I ported from 2.4 to 2.9.  The in-between ports were never done  
due to lack of time on my end.  See point #2.

4) Diverging away from Java Lucene, both API base and algorithm is  
risky and will just make point #2 more evidence.  Not only will you  
now need a deep knowledge of search engines to catch bugs, but also a  
deep knowledge of Lucene's internals.  Also, you risk compatibility as  
well as books and existing resources on the web that cover Lucene --  
hack, one can take any Java Lucene example and easily read it as a  
Lucene.Net code or use Luke to debug an index.  Keep in mind, the  
current port model that we have for Lucene.Net keeps the API  
one-to-one in sync with Java Lucene; just upper case method names.   
Yes, it's not fully .NET'es, but if you are looking for a search  
engine that is compatible with the open source search engine standard,  
and it is available in C#, Lucene.Net is it.

5) Beside making the port simpler, and per point #3 above, doing a  
line-per-line port, and maintaining API naming as well as the  
algorithm and file format of Java Lucene in C# Lucene means a Lucene  
index created by Java Lucene is usable, concurrently, by C# Lucene.  I  
have worked on one such project where a Java and C# code accessing the  
same index.  I'm not too interested in making Lucene.Net .NET'es and  
end up adding more risk to the project.

6) If anyone wants a different flavor of Lucene.Net, the code is on  
Apache, just fork it and start a new project.  Make it more .NET'es,  
use the latest that .NET has to offer, and all.  However, until when  
you have first hand experience with the port, and a good knowledge of  
Lucene and search engines, and the cycles to work on it, I really  
don't want to exercise this idea it will die as I know few folks have  
tried.

7) I can't speak for the other committers or those who contributed,  
but for me, I do this totally during my own time.  Each hour I spent  
on Lucene.Net is an hour away from my family or anything else.  I  
don't get paid, and I hardly get much off my Luene.Net work on the  
side.  As you may know, I was active with Lucene.Net till about early  
this year, (I had a family emergency).  I want to step up again, but  
we need more participation than just an offer to help or request  
divergence from the goal of the project, per the points that I made  
above.

I can go on, but the above are to clarify some of the issues and  
background of Lucene.Net.  Please keep those in mind when thinking  
about this project and how you can contribute -- especially comments  
about making Lucene.Net more .NET'es -- can't start that till when you  
first achieve commit-per-commit port of Java Lucene to C# Lucene.

If you agree with the above, and it makes sense to you, my suggestion  
is as follows:

1) Lucene.Net goes back into incubation and start all over again.
2) Start with cleaning up the webpage and make it more like other  
Apache project site.
3) Put together an official Lucene.Net 2.9.2 and get it released.
4) Start working on the next port.

#2, #3 can happen right away, and all that it takes to do them is  
coming up to speed on how-to using existing Apache documentation.  Who  
is up to this task?

#4 is a bit more complicated.  I don't want to go through the port  
pain that I had with 2.9.0 -- it was too much.  JLCA that comes with  
VS 2005 is out of date; I would love to try out a newer version from  
www.artinsoft.com, but it is $$.

I hope the above helps and I have not offended or discouraged anyone  
as it isn't my intention.  I just want to clarify few things about  
Lucene.Net

PS: One final point.  Look at CLucene, NLucene and few other variation  
of Java Lucene ports that were done at Lucene internal level with the  
goal of maintaining language look feature and look-and-fell, such as  
C++, those projects are either way out of date in terms of release  
version support or offer only partial support (index read only).  I  
don't want to use this to bad mouth another project, but to make a  
point that porting is hard if you diverge from the core.  As is,  
Lucene.Net is not dead, it's slow and needs contributors who will  
step-up.

Thanks,

-- George


RE: Lucene.NET Community Status

Posted by Digy <di...@gmail.com>.
We a still think of hosting the project in an another location.

DIGY

-----Original Message-----
From: George Aroush [mailto:george@aroush.net] 
Sent: Monday, November 01, 2010 10:56 PM
To: lucene-net-user@lucene.apache.org; lucene-net-dev@incubator.apache.org
Subject: RE: Lucene.NET Community Status

Let me jump in here and offer some perspective about Lucene.Net (btw,  
it's not Lucene.NET :-) ).  This is based on my past involvement with  
the project -- since 2003 when it was on SourceForge.net and called  
dotLucene.

1) Up until early this year, I have been porting and supporting  
Lucene.Net since ver 1.4 (back in 2004 on SourceForge.net) to the  
current release on trunk ver. 2.9.2.  This is in NO WAY to say that  
others have not helped or contributed.  I'm just saying that I know  
the history and have the experience (I wrote and worked on search  
engines from 1998 to 2002).

2) Doing an initial port of a new Java Lucene release to C# Lucene is  
very hard; it's the most complex part of the port even using automated  
tools such as JLCA and my own customize scripts which I use pre-and  
port JLCA (you can search the listing on how I do the port).  What  
used to take me about 1 months with 90% of tests passing took me well  
over 4 months (for 2.9.x) with only 10% of tests passing.  This was no  
easy effort and won't be easier now since Java Lucene is using new  
Java language features that JLCA is not aware of (MS is not  
maintaining JLCA).  Put another way, porting is hard especially when  
you are dealing with > 5.6 GB source code consistent of > 610 source  
files.  You will know this ONLY if you have tried it out and  
maintained it -- this is why no one has stepped up to do an initial  
port otherwise there would be a port by now not only of Java Lucene  
but other projects too.

3) To simplify ports of new release, maintaining as small as possible  
delta between release is very important. This was a main pain point  
when I ported from 2.4 to 2.9.  The in-between ports were never done  
due to lack of time on my end.  See point #2.

4) Diverging away from Java Lucene, both API base and algorithm is  
risky and will just make point #2 more evidence.  Not only will you  
now need a deep knowledge of search engines to catch bugs, but also a  
deep knowledge of Lucene's internals.  Also, you risk compatibility as  
well as books and existing resources on the web that cover Lucene --  
hack, one can take any Java Lucene example and easily read it as a  
Lucene.Net code or use Luke to debug an index.  Keep in mind, the  
current port model that we have for Lucene.Net keeps the API  
one-to-one in sync with Java Lucene; just upper case method names.   
Yes, it's not fully .NET'es, but if you are looking for a search  
engine that is compatible with the open source search engine standard,  
and it is available in C#, Lucene.Net is it.

5) Beside making the port simpler, and per point #3 above, doing a  
line-per-line port, and maintaining API naming as well as the  
algorithm and file format of Java Lucene in C# Lucene means a Lucene  
index created by Java Lucene is usable, concurrently, by C# Lucene.  I  
have worked on one such project where a Java and C# code accessing the  
same index.  I'm not too interested in making Lucene.Net .NET'es and  
end up adding more risk to the project.

6) If anyone wants a different flavor of Lucene.Net, the code is on  
Apache, just fork it and start a new project.  Make it more .NET'es,  
use the latest that .NET has to offer, and all.  However, until when  
you have first hand experience with the port, and a good knowledge of  
Lucene and search engines, and the cycles to work on it, I really  
don't want to exercise this idea it will die as I know few folks have  
tried.

7) I can't speak for the other committers or those who contributed,  
but for me, I do this totally during my own time.  Each hour I spent  
on Lucene.Net is an hour away from my family or anything else.  I  
don't get paid, and I hardly get much off my Luene.Net work on the  
side.  As you may know, I was active with Lucene.Net till about early  
this year, (I had a family emergency).  I want to step up again, but  
we need more participation than just an offer to help or request  
divergence from the goal of the project, per the points that I made  
above.

I can go on, but the above are to clarify some of the issues and  
background of Lucene.Net.  Please keep those in mind when thinking  
about this project and how you can contribute -- especially comments  
about making Lucene.Net more .NET'es -- can't start that till when you  
first achieve commit-per-commit port of Java Lucene to C# Lucene.

If you agree with the above, and it makes sense to you, my suggestion  
is as follows:

1) Lucene.Net goes back into incubation and start all over again.
2) Start with cleaning up the webpage and make it more like other  
Apache project site.
3) Put together an official Lucene.Net 2.9.2 and get it released.
4) Start working on the next port.

#2, #3 can happen right away, and all that it takes to do them is  
coming up to speed on how-to using existing Apache documentation.  Who  
is up to this task?

#4 is a bit more complicated.  I don't want to go through the port  
pain that I had with 2.9.0 -- it was too much.  JLCA that comes with  
VS 2005 is out of date; I would love to try out a newer version from  
www.artinsoft.com, but it is $$.

I hope the above helps and I have not offended or discouraged anyone  
as it isn't my intention.  I just want to clarify few things about  
Lucene.Net

PS: One final point.  Look at CLucene, NLucene and few other variation  
of Java Lucene ports that were done at Lucene internal level with the  
goal of maintaining language look feature and look-and-fell, such as  
C++, those projects are either way out of date in terms of release  
version support or offer only partial support (index read only).  I  
don't want to use this to bad mouth another project, but to make a  
point that porting is hard if you diverge from the core.  As is,  
Lucene.Net is not dead, it's slow and needs contributors who will  
step-up.

Thanks,

-- George


Re: Lucene.NET Community Status

Posted by Arne Claassen <ar...@mindtouch.com>.
On Nov 1, 2010, at 1:55 PM, George Aroush wrote:

> 3) Put together an official Lucene.Net 2.9.2 and get it released.


George,

What's involved in #3? Just build the binaries, strip out the source  
and zip up similar to the 2.0 release? I'd like to help get this out  
the door.

thanks,
Arne Claassen

MindTouch
San Diego, CA
http://twitter.com/sdether


RE: Lucene.NET Community Status

Posted by Phil Haack <ph...@microsoft.com>.
Hi George, 

Thanks for your input on this and setting the context straight. Quick question before I dive in.

How much is the www.artinsoft.com software that you would need? Maybe I can help look for sponsorship. I looked at the website and it's a bit of a mess. I don't know exactly which product you would need and what it would cost.

As for #2/#3 what is required to get started? Do you know if the documentation must be on the Apache site? Could we create a mirror CodePlex.com project that just houses releases and docs, but the official site would remain on Apache? The reason I ask is that CodePlex.com provides support for documentation using the MetaWeblog API (and thus Windows Live Writer) which is a fantastic tool for documentation. It lowers the bar to documentation and I've been using it very productively for my projects.

If it must stay on Apache, a little nudge in the right direction and I'll see what I can do to help.

Phil

-----Original Message-----
From: George Aroush [mailto:george@aroush.net] 
Sent: Monday, November 01, 2010 1:56 PM
To: lucene-net-user@lucene.apache.org; lucene-net-dev@incubator.apache.org
Subject: RE: Lucene.NET Community Status

Let me jump in here and offer some perspective about Lucene.Net (btw, it's not Lucene.NET :-) ).  This is based on my past involvement with the project -- since 2003 when it was on SourceForge.net and called dotLucene.

1) Up until early this year, I have been porting and supporting Lucene.Net since ver 1.4 (back in 2004 on SourceForge.net) to the current release on trunk ver. 2.9.2.  This is in NO WAY to say that others have not helped or contributed.  I'm just saying that I know the history and have the experience (I wrote and worked on search engines from 1998 to 2002).

2) Doing an initial port of a new Java Lucene release to C# Lucene is very hard; it's the most complex part of the port even using automated tools such as JLCA and my own customize scripts which I use pre-and port JLCA (you can search the listing on how I do the port).  What used to take me about 1 months with 90% of tests passing took me well over 4 months (for 2.9.x) with only 10% of tests passing.  This was no easy effort and won't be easier now since Java Lucene is using new Java language features that JLCA is not aware of (MS is not maintaining JLCA).  Put another way, porting is hard especially when you are dealing with > 5.6 GB source code consistent of > 610 source files.  You will know this ONLY if you have tried it out and maintained it -- this is why no one has stepped up to do an initial port otherwise there would be a port by now not only of Java Lucene but other projects too.

3) To simplify ports of new release, maintaining as small as possible delta between release is very important. This was a main pain point when I ported from 2.4 to 2.9.  The in-between ports were never done due to lack of time on my end.  See point #2.

4) Diverging away from Java Lucene, both API base and algorithm is risky and will just make point #2 more evidence.  Not only will you now need a deep knowledge of search engines to catch bugs, but also a deep knowledge of Lucene's internals.  Also, you risk compatibility as well as books and existing resources on the web that cover Lucene -- hack, one can take any Java Lucene example and easily read it as a Lucene.Net code or use Luke to debug an index.  Keep in mind, the current port model that we have for Lucene.Net keeps the API  
one-to-one in sync with Java Lucene; just upper case method names.   
Yes, it's not fully .NET'es, but if you are looking for a search engine that is compatible with the open source search engine standard, and it is available in C#, Lucene.Net is it.

5) Beside making the port simpler, and per point #3 above, doing a line-per-line port, and maintaining API naming as well as the algorithm and file format of Java Lucene in C# Lucene means a Lucene index created by Java Lucene is usable, concurrently, by C# Lucene.  I have worked on one such project where a Java and C# code accessing the same index.  I'm not too interested in making Lucene.Net .NET'es and end up adding more risk to the project.

6) If anyone wants a different flavor of Lucene.Net, the code is on Apache, just fork it and start a new project.  Make it more .NET'es, use the latest that .NET has to offer, and all.  However, until when you have first hand experience with the port, and a good knowledge of Lucene and search engines, and the cycles to work on it, I really don't want to exercise this idea it will die as I know few folks have tried.

7) I can't speak for the other committers or those who contributed, but for me, I do this totally during my own time.  Each hour I spent on Lucene.Net is an hour away from my family or anything else.  I don't get paid, and I hardly get much off my Luene.Net work on the side.  As you may know, I was active with Lucene.Net till about early this year, (I had a family emergency).  I want to step up again, but we need more participation than just an offer to help or request divergence from the goal of the project, per the points that I made above.

I can go on, but the above are to clarify some of the issues and background of Lucene.Net.  Please keep those in mind when thinking about this project and how you can contribute -- especially comments about making Lucene.Net more .NET'es -- can't start that till when you first achieve commit-per-commit port of Java Lucene to C# Lucene.

If you agree with the above, and it makes sense to you, my suggestion is as follows:

1) Lucene.Net goes back into incubation and start all over again.
2) Start with cleaning up the webpage and make it more like other Apache project site.
3) Put together an official Lucene.Net 2.9.2 and get it released.
4) Start working on the next port.

#2, #3 can happen right away, and all that it takes to do them is coming up to speed on how-to using existing Apache documentation.  Who is up to this task?

#4 is a bit more complicated.  I don't want to go through the port pain that I had with 2.9.0 -- it was too much.  JLCA that comes with VS 2005 is out of date; I would love to try out a newer version from www.artinsoft.com, but it is $$.

I hope the above helps and I have not offended or discouraged anyone as it isn't my intention.  I just want to clarify few things about Lucene.Net

PS: One final point.  Look at CLucene, NLucene and few other variation of Java Lucene ports that were done at Lucene internal level with the goal of maintaining language look feature and look-and-fell, such as  
C++, those projects are either way out of date in terms of release
version support or offer only partial support (index read only).  I don't want to use this to bad mouth another project, but to make a point that porting is hard if you diverge from the core.  As is, Lucene.Net is not dead, it's slow and needs contributors who will step-up.

Thanks,

-- George