You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucy.apache.org by Peter Karman <pe...@peknet.com> on 2010/06/26 23:03:25 UTC
Re: [Lucy] Incubator or out?
Marvin Humphrey wrote on 6/26/10 12:57 PM:
>
> Why not declare victory and make our prototype our product?
+1
>
> Perhaps the best way to speed Lucy along is to submit an Incubator proposal
> which includes a collective software grant for KinoSearch, which we would
> subsequently rebadge as Lucy. In its recent review of Lucy's status, the
> Lucene PMC overwhelmingly identified community building as the most important
> task for us to focus on. The fastest way for us to get to the release that we
> need in order to attract new contributors and grow our community is to fill
> the remaining voids in the source tree via IP clearance work rather than
> software development.
agreed.
> Regardless of what happens, I think that Lucy's interests are best served by
> assimilating the KinoSearch code base and pursuing commuity development
> aggressively.
yes.
One thing that should attract more people is if we can get more than just Perl
bindings implemented using Lucy. I'm thinking C, PHP, Ruby, Python, etc. To my
knowledge, none of those languages have an IR library on the level of Lucy
(which is why Lucy started), and if we can get a stable core to 1.0, we can
attract capable developers from some of those communities. It might even be
worth seeing if we could leverage SWIG.
Maybe the above Goes Without Saying, but I'm saying it anyway. :)
In any case, I like this proposal, Marvin. Tell me how I can help.
--
Peter Karman . http://peknet.com/ . peter@peknet.com
Re: [Lucy] Incubator or out?
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Sat, Jun 26, 2010 at 10:17:03PM -0500, Peter Karman wrote:
> > Browse through the documentation at
> > <http://www.apache.org/dev/>, though if you're like me, it will take a while
> > to sink in.
> > And if you can spare a moment or two over the next week, let's
> > collaborate on an Incubator proposal via the Lucy wiki. (; I know this is a
> > busy time -- life happens! ;)
>
> I'll make some time.
OK, great.
We should leave filling in administrative details to you and/or Nate -- you
will learn more by completing those items, e.g. Nate reminded himself about
the CLA when he fleshed out the committer list.
I'm stubbing out outlines in no particular order to start off with. If you
want to flesh any of those out into real sentences, feel free. If not, I'll
get to them.
Expectations are not high -- unless you're subscribed to another Apache dev
list, you folks haven't witnessed a typical healthy Apache project in action.
;)
Marvin Humphrey
[Lucy] More bindings
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Sat, Jun 26, 2010 at 10:17:03PM -0500, Peter Karman wrote:
> I was actually gravitating toward C and PHP myself.
If you're interested in C bindings, I would like to collaborate with you on
that. (: No more than limited high-level discussions this week, though. :)
A C API is important not just for C users, but for users of dynamic language
hosts who have written rapid prototype extensions that need to be optimized
for high performance. C host bindings and a C API go hand in hand, though
they aren't exactly the same thing.
To simplify our task, I suggest that we initially target GCC/unixen only for
the C bindings. It's important that people be able to write cross-platform
extensions that work with the C API, but for the C host binding, I don't want
to deal with cross-platform build headaches right away.
> I was mostly thinking that it might be an interesting exercise to see if SWIG
> could get us partway toward any one (or more) language implementations. Best,
> IME, to truly make idiomatically-sensitive bindings though. It's gotta make new
> users feel more at home when they first take Lucy-in-HostLanguageX for a spin.
Also, a Clownfish binding empowers users to write subclasses entirely in the
host language. A SWIG binding wouldn't support that.
Marvin Humphrey
Re: [Lucy] Incubator or out?
Posted by Peter Karman <pe...@peknet.com>.
Marvin Humphrey wrote on 6/26/10 9:45 PM:
> On Sat, Jun 26, 2010 at 04:03:25PM -0500, Peter Karman wrote:
>> One thing that should attract more people is if we can get more than just Perl
>> bindings implemented using Lucy. I'm thinking C, PHP, Ruby, Python, etc. To my
>> knowledge, none of those languages have an IR library on the level of Lucy
>> (which is why Lucy started), and if we can get a stable core to 1.0, we can
>> attract capable developers from some of those communities.
>
> For what it's worth, writing bindings is a problem space I'm intensely
> interested in. If we can bring my other responsibilities down to a dull roar,
> I'll be addressing some of these. Ruby first.
Cool. I was actually gravitating toward C and PHP myself.
>
> Not that I would grouch if someone else volunteers!
heh.
>
>> It might even be worth seeing if we could leverage SWIG.
>
> The thing I don't like about SWIG for Perl at least is that it doesn't yield
> idiomatic interfaces. For Perl, I strongly prefer labeled parameters over
> positional arguments when there are two or more arguments to a method, and
> I've gone to considerable lengths to support labeled params with
> Clownfish::Binding::Perl.
agreed on all counts. Xapian is an example where they use SWIG for everything
*except* Perl bindings, for (what I assume are) similar reasons.
I was mostly thinking that it might be an interesting exercise to see if SWIG
could get us partway toward any one (or more) language implementations. Best,
IME, to truly make idiomatically-sensitive bindings though. It's gotta make new
users feel more at home when they first take Lucy-in-HostLanguageX for a spin.
> $lowerCamelCase =~ s/^(.)/lc($1)/e;
$lowerCamelCase = lcfirst( $lowerCamelCase );
# or even
$lowerCamelCase = "\l$lowerCamelCase";
but now I'm just showing off. ;)
>
> AFAIK, SWIG can't do that.
you're probably right.
>
> Nevertheless, it would be a Good Thing if someone starts hacking with SWIG to
> provide support for a language that otherwise wouldn't be there. "Progress
> not perfection", as Mike McCandless likes to say.
>
yes.
>
> If that interests you, it would be great if you would familiarize yourself
> with Apache institutions. Consider subscribing to general@incubator:
done.
> Browse through the documentation at
> <http://www.apache.org/dev/>, though if you're like me, it will take a while
> to sink in. And if you can spare a moment or two over the next week, let's
> collaborate on an Incubator proposal via the Lucy wiki. (; I know this is a
> busy time -- life happens! ;)
I'll make some time.
>
> I told the Lucene PMC that we would present a plan for Lucy's graduation to
> general@lucene by this upcoming Friday, July 2, based on the Incubator
> proposal template. Instead, I think we should provide them with a draft of an
> actual incubator proposal along with a short custom-tailored justification.
Sounds sane.
>
> The page is up at <http://wiki.apache.org/lucy/LucyIncubatorProposal>. The
> original template document is at
> <http://incubator.apache.org/guides/proposal.html>; it's a better read because
> it has links.
/me goes and reads...
--
Peter Karman . http://peknet.com/ . peter@peknet.com
Re: [Lucy] Incubator or out?
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Sat, Jun 26, 2010 at 04:03:25PM -0500, Peter Karman wrote:
> One thing that should attract more people is if we can get more than just Perl
> bindings implemented using Lucy. I'm thinking C, PHP, Ruby, Python, etc. To my
> knowledge, none of those languages have an IR library on the level of Lucy
> (which is why Lucy started), and if we can get a stable core to 1.0, we can
> attract capable developers from some of those communities.
For what it's worth, writing bindings is a problem space I'm intensely
interested in. If we can bring my other responsibilities down to a dull roar,
I'll be addressing some of these. Ruby first.
Not that I would grouch if someone else volunteers!
> It might even be worth seeing if we could leverage SWIG.
The thing I don't like about SWIG for Perl at least is that it doesn't yield
idiomatic interfaces. For Perl, I strongly prefer labeled parameters over
positional arguments when there are two or more arguments to a method, and
I've gone to considerable lengths to support labeled params with
Clownfish::Binding::Perl.
Also, different languages have different conventions for method naming: Java
likes lowerCamelCase, Perl prefers underscore_separation, and so on. One
reason that Clownfish uses method names with a combination of mixed
capitalization and underscores -- e.g. Open_InStream -- is that method names
of both varieties can be automatically derived:
my $clownfish_name = 'Open_InStream';
my $underscore_name = lc($clownfish_name);
my $lowerCamelCase = $clownfish_name;
$lowerCamelCase =~ s/_//g;
$lowerCamelCase =~ s/^(.)/lc($1)/e;
AFAIK, SWIG can't do that.
Nevertheless, it would be a Good Thing if someone starts hacking with SWIG to
provide support for a language that otherwise wouldn't be there. "Progress
not perfection", as Mike McCandless likes to say.
> In any case, I like this proposal, Marvin.
Excellent.
> Tell me how I can help.
I would consider it a major recruiting success if you end up on the Lucy PMC.
* You have extensive experience with open source software.
* You've been involved with KinoSearch and Lucy for several years.
* You have extensive experience with a different IR library -- SWISH --
which will enrich your perspective when voting on policy.
* I've trusted you to roll KinoSearch releases and they have all gone
perfectly. I expect that we can trust you to certify Lucy releases to
meet Apache's standards as well.
* Your written communication skills are top-notch.
* You have a good mix of impatience and diplomacy.
* Your project management skills are superior.
There are other things that you do well too, like portability hacking. But
the combination of traits above is not easy to find. So contribute whatever
you want -- you're a valuable member of the community regardless -- but in my
perfect world, you are on the PMC. We are going to need competent,
knowledgeable PMC members in order to govern ourselves effectively, and for
the project's health we need to ensure that I am dispensible on the PMC just
as I must be for the code base.
If that interests you, it would be great if you would familiarize yourself
with Apache institutions. Consider subscribing to general@incubator:
<http://incubator.apache.org/guides/lists.html> -- the debates there are very
informative! Browse through the documentation at
<http://www.apache.org/dev/>, though if you're like me, it will take a while
to sink in. And if you can spare a moment or two over the next week, let's
collaborate on an Incubator proposal via the Lucy wiki. (; I know this is a
busy time -- life happens! ;)
I told the Lucene PMC that we would present a plan for Lucy's graduation to
general@lucene by this upcoming Friday, July 2, based on the Incubator
proposal template. Instead, I think we should provide them with a draft of an
actual incubator proposal along with a short custom-tailored justification.
The page is up at <http://wiki.apache.org/lucy/LucyIncubatorProposal>. The
original template document is at
<http://incubator.apache.org/guides/proposal.html>; it's a better read because
it has links.
Marvin Humphrey