You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucy.apache.org by howard chen <ho...@gmail.com> on 2007/02/27 14:44:36 UTC
PHP blinding for Lucy?
Hello,
I know that Lucy planned to support Perl & Ruby but still in a very
early stage.
Anyway, If I am going to implmenet a blinding for PHP, what should I
know or need to prepare?
Is it still early to ask this question?
Thanks.
Re: PHP blinding for Lucy?
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Feb 27, 2007, at 6:24 PM, David Balmain wrote:
> How about just using doxygen. I don't have much experience with it but
> I'm pretty sure there would be a way to tag particular functions that
> are public so that when you generate the documentation you can
> generate only the public methods.
From the Doxygen manual:
21.124 PHP only commands
For PHP files there are a number of additional commands, that can be
used inside classes to make members public, private, or protected
even
though the language itself doesn’t support this notion.
To mark a single item use one of \private, \protected, \public. For
starting a section with a certain protection level use one of:
\privatesection, \protectedsection, \publicsection. The latter
commands
are similar to ”private:”, ”protected:”, and ”public:” in C++.
Too bad "package" isn't in that list. KS has three access levels:
public, private, and "distro", which I made up and means any file in
the KinoSearch distribution can use it -- the equivalent of Java
package if the whole distro was in one package. (I would have called
it "package" but that has a different meaning in Perl.)
I'll bet Doxygen allows you to define your own keywords... just
haven't found that yet.
Marvin Humphrey
Rectangular Research
http://www.rectangular.com/
Re: PHP blinding for Lucy?
Posted by David Balmain <db...@gmail.com>.
On 2/28/07, Marvin Humphrey <ma...@rectangular.com> wrote:
>
> On Feb 27, 2007, at 10:19 PM, David Balmain wrote:
>
> > On 2/28/07, Marvin Humphrey <ma...@rectangular.com> wrote:
> >> > Of course you could also have public and private include files.
> >>
> >> Hmm, can you elaborate? I'd basically given up hope that we'd be
> >> able to maintain tight control over symbol export, and was expecting
> >> to define the API via documentation only.
> >
> > I haven't tried this before and I'm not sure if other large C projects
> > do this but I was thinking you could have two sets of include files;
> > one that contains the public API and the other used to contain methods
> > used internally within Lucy. For example, an IndexWriter.h and an
> > IndexWriter_private.h.
> >
> > One way I am planning on cutting down on public methods is by keeping
> > the unit tests in the same file as the source code, surrounding them
> > with an #ifdef UNIT_TESTS. A lot of the methods in Ferret would be
> > static except that I needed to make them available for unit testing.
>
> I guess I just don't care a whole lot. Words to live by, courtesy of
> Larry Wall: "Perl doesn't have an infatuation with enforced privacy.
> It would prefer that you stayed out of its living room because you
> weren't invited, not because it has a shotgun."
Couldn't agree more. My motivation for doing this would be keeping the
public interface as minimal and simple as possible to avoid confusion
rather than to act as access control. And it allows me to keep my
static method names short but this would make no difference with
Lucy's coding standards so I guess it is a moot point.
> > I agree to a certain extent. However, when I released the first
> > version of Ferret, a lot of people complained the interface was too
> > Java-like. It can be difficult to find the happy medium between making
> > the interface easy for people who used Lucy in a different language
> > and people who their Ruby/Perl/PHP library to work in a certain way.
>
> Absolutely. What I'm suggesting wouldn't stand in the way of that.
> It would even lend superficial similarity to bindings which are
> otherwise implemented very differently.
Well as long as what you are talking about doesn't get in the way of
writing idiomatic bindings then it all sounds good to me. I look
forward to seeing what you come up with.
Cheers,
Dave
--
Dave Balmain
http://www.davebalmain.com/
Re: PHP blinding for Lucy?
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Feb 27, 2007, at 10:19 PM, David Balmain wrote:
> On 2/28/07, Marvin Humphrey <ma...@rectangular.com> wrote:
>> > Of course you could also have public and private include files.
>>
>> Hmm, can you elaborate? I'd basically given up hope that we'd be
>> able to maintain tight control over symbol export, and was expecting
>> to define the API via documentation only.
>
> I haven't tried this before and I'm not sure if other large C projects
> do this but I was thinking you could have two sets of include files;
> one that contains the public API and the other used to contain methods
> used internally within Lucy. For example, an IndexWriter.h and an
> IndexWriter_private.h.
>
> One way I am planning on cutting down on public methods is by keeping
> the unit tests in the same file as the source code, surrounding them
> with an #ifdef UNIT_TESTS. A lot of the methods in Ferret would be
> static except that I needed to make them available for unit testing.
I guess I just don't care a whole lot. Words to live by, courtesy of
Larry Wall: "Perl doesn't have an infatuation with enforced privacy.
It would prefer that you stayed out of its living room because you
weren't invited, not because it has a shotgun."
If it's easy to keep separate header files, cool -- but the way I
have KS and boilerplater set up right now, it wouldn't be, and I
don't think we should make any sacrifices in clarity or simplicity
for the sake of enforcing access control. If people want to hack,
they'll hack, and you can't stop 'em. Look at what all the people on
java-dev do. :)
> I agree to a certain extent. However, when I released the first
> version of Ferret, a lot of people complained the interface was too
> Java-like. It can be difficult to find the happy medium between making
> the interface easy for people who used Lucy in a different language
> and people who their Ruby/Perl/PHP library to work in a certain way.
Absolutely. What I'm suggesting wouldn't stand in the way of that.
It would even lend superficial similarity to bindings which are
otherwise implemented very differently.
> But if you wanted to take this route (making it possible for binding
> authors to generate some/most of the binding from some XML files), how
> about using SWIG?
For Perl, at least, XS is much more powerful, albeit complex. But
regardless, I wouldn't want to elevate consistency across bindings
over idiomatic implementation.
For instance, KS uses labeled params rather than ordered args for a
lot of things. Auto-generated bindings mirroring the C interface
wouldn't let you do this:
my $filter = KinoSearch::Search::RangeFilter->new(
field => 'product_number',
lower_term => '003',
upper_term => '060',
include_lower => 1,
include_upper => 1,
);
I assume you'll want to do something similar for Ruby. If we ever
prepare Java bindings, that won't fly. But the same method
description text could -- and should -- be used in the docs for each
of them.
Marvin Humphrey
Rectangular Research
http://www.rectangular.com/
Re: PHP blinding for Lucy?
Posted by David Balmain <db...@gmail.com>.
On 2/28/07, Marvin Humphrey <ma...@rectangular.com> wrote:
> > Of course you could also have public and private include files.
>
> Hmm, can you elaborate? I'd basically given up hope that we'd be
> able to maintain tight control over symbol export, and was expecting
> to define the API via documentation only.
I haven't tried this before and I'm not sure if other large C projects
do this but I was thinking you could have two sets of include files;
one that contains the public API and the other used to contain methods
used internally within Lucy. For example, an IndexWriter.h and an
IndexWriter_private.h.
One way I am planning on cutting down on public methods is by keeping
the unit tests in the same file as the source code, surrounding them
with an #ifdef UNIT_TESTS. A lot of the methods in Ferret would be
static except that I needed to make them available for unit testing.
><snip/>
> Now, returning to your point about Doxygen... With XML, we'd have to
> maintain separate files for the documentation, which would suck. So
> I'm all for using Doxygen, especially if we can rig things up so that
> the description can be isolated and parsed out reliably.
>
> I might go write an extractor tool which parses our header files and
> generates intermediate XML.
Well, I definitely think it might be a good idea to look at doxygen
and see if you can hook into its parser. I'm pretty sure it parses the
C code as well as the comments so it might do most of the work for
you.
> Then bindings authors could write their
> own final translation utilities in their language of choice, and use
> as much or as little as they wish.
>
> Hopefully they'd use more rather than less. It's to the user's
> benefit for various bindings to present reasonably consistent APIs
> while still being idiomatic, because it makes it easier to apply what
> you learned about one of them to another.
I agree to a certain extent. However, when I released the first
version of Ferret, a lot of people complained the interface was too
Java-like. It can be difficult to find the happy medium between making
the interface easy for people who used Lucy in a different language
and people who their Ruby/Perl/PHP library to work in a certain way.
But if you wanted to take this route (making it possible for binding
authors to generate some/most of the binding from some XML files), how
about using SWIG?
--
Dave Balmain
http://www.davebalmain.com/
Re: PHP blinding for Lucy?
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Feb 27, 2007, at 6:24 PM, David Balmain wrote:
> How about just using doxygen. I don't have much experience with it but
> I'm pretty sure there would be a way to tag particular functions that
> are public so that when you generate the documentation you can
> generate only the public methods.
I don't know it well either, but I'm sure you're right and it will
allow us to put in a public/non-public tag.
It would be even better if we could export at least some of the
documentation -- particularly method descriptions. I'd really like
to be able to synch up the Perl binding docs by running a script
rather than via copy-and-paste.
> Of course you could also have public and private include files.
Hmm, can you elaborate? I'd basically given up hope that we'd be
able to maintain tight control over symbol export, and was expecting
to define the API via documentation only.
>> I'm thinking we need shared
>> documentation. XML, maybe? Then each binding would require an
>> appropriate XML-to-whatever translation utility.
>
> I'm not entirely sure I'm on the same wavelength as you today. By
> 'whatever' do you mean the specific languages documentation format?
Yes, that was what I was thinking. But perhaps not quite so
ambitious as may have come across.
> If
> that is the case then I don't see this working as the ruby API for
> Lucy will probably be quite different to the PHP API.
If we're reasonably careful about how we word things, many method
descriptions could be reused across all bindings. And one of the
things about the naming convention we've settled on for method
invocations is that you can derive either lowerCamelCase or
separated_by_underscores method names with a simple transform:
Sim_Length_Norm => lengthNorm
Sim_Length_Norm => length_norm
If we tag every last thing, enough so that we could actually
generate, say, both POD and javadoc without intervention, then sure,
XML is wayyyy too verbose. Anything would be, really, because
language syntaxes are too distinct. But if we set our sights a
little lower, and just try to share method names, method
descriptions, and public/non-public access control, that's doable --
and it's a whole lot of savings. (Maybe parameter lists and return
values, too, but that's a little harder.)
<method>
<name>Sim_Length_Norm</name>
<acl>public</acl>
<description>
Computes the normalization value for a field given the total
number of
terms contained in a field. These values, together with field
boosts,
are stored in an index and multipled into scores for hits on
each field
by the search code.
Matches in longer fields are less precise, so implementations
of this
method usually return smaller values when numTokens is large,
and larger
values when numTokens is small.
That these values are computed under IxWriter_Add_Document and
stored
then using Sim_Encode_Norm. Thus they have limited precision, and
documents must be re-indexed if this method is altered.
</description>
</method>
Note the use of "IxWriter_Add_Document" and "Sim_Encode_Norm" within
the description. Those method names are identifiable patterns,
matchable with this regex:
# $1 is class nick, $2 is short method name
/([A-Z][A-Za-z]+)_([A-Z]\w+)/
It's easy to sub out IxWriter_Add_Document for this, which will
generate a nicely formatted link...
L<IndexWriter::add_document|Lucy::Index::IndexWriter/"add_document">
Now, returning to your point about Doxygen... With XML, we'd have to
maintain separate files for the documentation, which would suck. So
I'm all for using Doxygen, especially if we can rig things up so that
the description can be isolated and parsed out reliably.
I might go write an extractor tool which parses our header files and
generates intermediate XML. Then bindings authors could write their
own final translation utilities in their language of choice, and use
as much or as little as they wish.
Hopefully they'd use more rather than less. It's to the user's
benefit for various bindings to present reasonably consistent APIs
while still being idiomatic, because it makes it easier to apply what
you learned about one of them to another.
Marvin Humphrey
Rectangular Research
http://www.rectangular.com/
Re: PHP blinding for Lucy?
Posted by David Balmain <db...@gmail.com>.
On 2/28/07, Marvin Humphrey <ma...@rectangular.com> wrote:
>
> On Feb 27, 2007, at 5:44 AM, howard chen wrote:
>
> > Hello,
> >
> > I know that Lucy planned to support Perl & Ruby but still in a very
> > early stage.
> >
> > Anyway, If I am going to implmenet a blinding for PHP, what should I
> > know or need to prepare?
> >
> > Is it still early to ask this question?
>
> It's still a bit early, but stick around.
>
> A couple of days ago, I managed to get the first dev release of
> KinoSearch 0.20 (0.20_01) out the door. A lot of the work that has
> gone into it was informed by discussions here; the new version is
> considerably closer to Ferret and to the theoretical "Lucy" than the
> current KS release, 0.15.
>
> One thing we're going to need is a way of communicating to bindings
> authors what is and isn't public API.
How about just using doxygen. I don't have much experience with it but
I'm pretty sure there would be a way to tag particular functions that
are public so that when you generate the documentation you can
generate only the public methods.
Of course you could also have public and private include files.
> I'm thinking we need shared
> documentation. XML, maybe? Then each binding would require an
> appropriate XML-to-whatever translation utility.
I'm not entirely sure I'm on the same wavelength as you today. By
'whatever' do you mean the specific languages documentation format? If
that is the case then I don't see this working as the ruby API for
Lucy will probably be quite different to the PHP API. But, maybe
you're talking about something completely different.
Cheers,
Dave
--
Dave Balmain
http://www.davebalmain.com/
Re: PHP blinding for Lucy?
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Feb 27, 2007, at 5:44 AM, howard chen wrote:
> Hello,
>
> I know that Lucy planned to support Perl & Ruby but still in a very
> early stage.
>
> Anyway, If I am going to implmenet a blinding for PHP, what should I
> know or need to prepare?
>
> Is it still early to ask this question?
It's still a bit early, but stick around.
A couple of days ago, I managed to get the first dev release of
KinoSearch 0.20 (0.20_01) out the door. A lot of the work that has
gone into it was informed by discussions here; the new version is
considerably closer to Ferret and to the theoretical "Lucy" than the
current KS release, 0.15.
One thing we're going to need is a way of communicating to bindings
authors what is and isn't public API. I'm thinking we need shared
documentation. XML, maybe? Then each binding would require an
appropriate XML-to-whatever translation utility.
Marvin Humphrey
Rectangular Research
http://www.rectangular.com/