[Report] Apache Lucene

[Grant Ingersoll <gs...@apache.org>]

=== Lucene Status Report: May 2011 ===

Background: Lucene has been asked by the Board to report on the state
of the community vis-a-vis the problems around the recent
commit/revert incident
(https://issues.apache.org/jira/browse/SOLR-2272 and the related
Lucene issue as well as several other mail threads)

The primary root of these problems arose from a disagreement about how
best to move forward with the two products the Lucene PMC ships:
Apache Lucene and Apache Solr.  A majority of the PMC/committership is
in favor of a more modular approach to Solr which essentially means
refactoring code that has lived in Solr for a long time into modules
that can be more easily consumed at the Java API level (as opposed to
the Solr REST API level.)  Others have resisted these changes,
sometimes for technical reasons and sometimes for what appear to be
business/political reasons.  Still others have a view that they should
be taken on a case by case basis.  These people are not against the
refactoring, but don't think is absolutely necessary that it must be
done in order to make other contributions to that particular code
base.  After long debate, we seem to have arrived at a consensus that
those who wish to do the refactoring should go ahead with it, but it
shouldn't require others to stop working in the areas that are of
refactoring interest.  For the record, the business/political
reasoning has been clearly repudiated by the rest of the PMC.

Other concerns have arisen about the use of IRC such that we have
started to use a logging client for IRC.  We have also reminded
everyone to keep all decisions on list and to allow proposed decisions
to "bake" before committing, at least when it comes to major issues/changes.

Some in the community have also raised concerns about Lucid
Imagination's role in development.  While Lucid does employ a good
number (but nowhere near the majority) of committers [1], (and which
is almost completely balanced by IBM's presence) the general consensus
seems to be that it is not a concern.  Furthermore, during the recent
debates, it is quite clear that Lucid employees are free to have
independent viewpoints on what to do.  Naturally, given a number of
committers in one company, it warrants the PMC keeping a watchful eye on
it.  Likewise, however, it should also be clear that every
PMC/committer involved in Lucene (with the exception of Andi Vajda) is
paid to work on Lucene/Solr and they all have financial interests and
are often in competition for the same clients.
All should recognize that this doesn't necessarily make for problems,
but can do so if people let it.

Beyond this, we have put forth a few other things that we can do to
help keep the community moving forward in a positive way.  These are
itemized below:

1. Obviously, with projects as big and widely used as Lucene and Solr,
it is hard to sometimes keep up with all the contributions that come
in.  Thus, we need to find a way to automate (similar to Hadoop's
patch checker) the basics of patch checking like having unit tests,
formatting, etc. such that contributors can get feedback sooner and so
that committers know that a patch is ready for review, thereby making
it easier to accept contributions and, hopefully, encourage newcomers.
We also need to more consistently promote contributors to committers
and committers to the PMC.  As with most of the ASF, our current
approach is dependent on remembering to make a nomination and we
should look for better ways to identify candidates (such a reporting
mechanism would likely benefit all the ASF, actually.)

2. We have added three new PMC Members: Doron Cohen, Shai Erera, Steve
Rowe

3. The Board should expect a resolution to change
the PMC Chair for the June Board Meeting.  We also plan on changing
the chair on a yearly basis.

4. Thanks to Greg's intervention, we all have been reminded as to
proper etiquette when it comes to commits/reverts such that the main
symptom of this disagreement should not happen again.

5. To some extent, we feel this has been overblown and many of us have
come to the conclusion that the simplest way to move forward is to get
back to writing code and improving Lucene and Solr and getting
releases out.  This is not intended to paper over the concerns, but
to note that the whole point of the project is to deliver open source
search software via the ASF guidelines.  To that end, we are working on 
releasing 3.2 of Lucene
and Solr as well as continuing development on 4.0.

[1] Current PMC Members/Committers and their employers

* means PMC
    Bill Au (billa@...)   -- CBS Interactive
    * Michael Busch (buschmi@...) -- Twitter
    * Doron Cohen (doronc@...) -- IBM
    * Shai Erera (shaie@...) -- IBM
    * Otis Gospodnetic (otis@...) -- Sematext
    * Erik Hatcher (ehatcher@...) -- Lucid
    * Chris Hostetter (hossman@...) -- Lucid
    * Grant Ingersoll (gsingers@...) -- Lucid
    * Mike McCandless (mikemccand@...) -- IBM
    * Ryan McKinley (ryan@...) -- Voyager GIS (Lucid advisor)
    * Mark Miller (markrmiller@...) -- Lucid
    * Robert Muir (rmuir@...) -- Lucid (recent)
    Noble Paul (noble@...) -- AOL
    * Steven Rowe (sarowe@...) -- Syracuse Univ.
    * Uwe Schindler (uschindler@...) -- SD Data Solutions
    Shalin Shekhar Mangar (shalin@...) -- AOL
    * Yonik Seeley (yonik@...) -- Lucid
    * Koji Sekiguchi (koji@...) -- Rondhuit
    Dawid Weiss (dweiss@...) -- CarrotSearch
    Stanislaw Osinski -- CarrotSearch
    * Simon Willnauer (simonw@...) -- JTeam/Independent
    Chris Male (chrism@...) -- JTeam
    Andi Vajda (vajda@...) -- Google
    *Scott Ganyo -- Actor
    * Mark Harwood -- Detica
    Adriano Crestani -- IBM