[lucy-dev] Extension directory layout

[Marvin Humphrey <ma...@rectangular.com>]

On Tue, Apr 10, 2012 at 12:35 PM, Nick Wellnhofer (Updated) (JIRA)
<ji...@apache.org> wrote:

> As I noted on lucy-dev I think we should make it easy for extension writers
> to work with the same "core", "lucy", "ruby" directory layout for their
> extensions. This includes copying "core" and possibly other files to the
> host language directory when building distributions. I added a
> C::C::P::B->cf_base_path methods to help with that.
>
> I'd also like to create a "dist" action in C::C::P::B based on ACTION_dist
> in Lucy::Build that helps with building extensions distros.

+1 with some comments.

I haven't really thought too hard about generalizing our directory layout
before now.  I suppose it solves certain problems and serves a decent starting
point for extensions.  Having the perl/ and ruby/ subdirectories structured
like CPAN and rubygems distributions respectively, and having them behave
adaptively based on whether certain elements can be found in the current
working directory or are present one level up... I think it would be hard to
improve on that design.  It makes sense to me to have a "dist" action based
around such assumptions.

Nevertheless, the current Lucy repository layout has some imperfections.

One problem is that we don't have a home for elements which are common across
all builds.  For example, right now charmonize.c lives under charmonizer/ but
it doesn't really belong there -- that specific charmonize.c is
custom-tailored for Lucy, while Charmonizer is designed as a suite of generic
modules[1].  We can't put charmonize.c under core/, though, because then it
will be treated as a Lucy core source file and linked into the Lucy shared
object.

To address this problem, perhaps it makes sense to introduce a new directory
named "common" as a home for files like charmonize.c.

    CHANGES
    CONTRIBUTING
    INSTALL
    LICENSE
    NOTICE
    README
    STATUS
    c/
    charmonizer/
    clownfish/
    common/     <------------
    core/
    devel/
    example-lang/
    lemon/
    modules/
    perl/
    ruby/
    tcl/

(Arguably, lemon also could live under "common").

Another possibility is to move everything currently under core/ one level down
into core/src.  If we want to do this, we should do it now -- before
extensions inherit the current shallower directory structure and we bake it
into a "dist" action.  However, my personal preference is to keep our core
source files as close to the surface as possible.  I'd prefer to stay away
from filepaths like this:

    http://svn.apache.org/repos/asf/lucene/dev/trunk/lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java

The other problem I can think of with generalizing our current repository
layout is that the layout within modules/ is definitely not generic.  IMO, the
best way to address that problem is to decide that modules/ is a Lucy-specific
implementation detail and to ignore it altogether in
Clownfish::CFC::Build::Perl.

One last thing: the directory name "core" conflicts with Unix core dumps.
Obviously we didn't care about that or it would have been named
something else.

Marvin Humphrey

[1] CFC ought to have its own custom charmonize.c which is distinct from
    Lucy's charmonize.c.