[lucy-dev] Argument processing model

[Marvin Humphrey <ma...@rectangular.com>]

Greets,

Since we'll be dealing with Lucy/Clownfish argument processing in the context
of a C host binding soon, now seems like a good time to review the high level
design.

Clownfish method declarations may specify optional parameters with default
values.  The default values must be constants. 

    public incremented Hits*
    Hits(Searcher *self, Obj *query, uint32_t offset = 0,
         uint32_t num_wanted = 10, SortSpec *sort_spec = NULL);

In this example, 'query' is a required parameter, while 'offset',
'num_wanted', and 'sort_spec' are all optional parameters with default values.
The invocant, 'self', while not a parameter per se, also has no default value
and is therefore required.

Default values are considered part of the method signature.  Any subclass
which overrides a method must specify exactly the same default values;
changing a default value is a compatibility break.  The Clownfish compiler
errors out if it detects a method signature conflict.

Since the C language doesn't support either default values or optional
parameters, those features must be implemented by an intermediate layer.  In
the case of Lucy's Perl interface, it's the autogenerated XS wrapper layer
which assumes this responsibility:

  * Ensure that all required parameters were supplied.
  * Detect which optional parameters were not supplied and assign defaults.
  * Validate that supplied arguments are of the proper type (for example
    confirming that sort_spec, if non-NULL, isa Lucy::Search::SortSpec
    object).
  * Invoke the implementing core function.

Because this intermediate layer does so much work, Lucy's core code doesn't
have to.  For instance, lucy_Searcher_hits() -- the implementing function for
the example above -- does not check whether 'self' isa Lucy::Search::Searcher,
nor verify that 'self' is non-NULL.  It assumes that it may depend on the
constraints imposed by the method signature.

When core code invokes other core code, we do not benefit from the protection
of the intermediate layer -- but we also do not pay its overhead, and in
general, compiler warnings provide us with enough protection.  In other words,
our C code doesn't get type safety a la Java, but this is C after all. :)

When we add new host bindings for dynamic languages such as Python, Ruby, and
Tcl, all of which provide built-in support for optional parameters and default
values, we will need to autogenerate binding code which accesses said
functionality.  Host bindings for these languages will probably less complex
than our Perl binding code, since Perl does not provide native support for
labeled parameters, optional parameters or default values and we have to hack
those in.

Implementing a C API will be conceptually more complicated, because we will
have to make some decisions and leave some features out.  We can't support
default values from C, for example.  I suspect that we will still want to
autogenerate an intermediate layer which performs some amount of type
checking, but we will just have to see how things play out.

Marvin Humphrey

Re: [lucy-dev] Argument processing model

[Joe Schaefer <jo...@yahoo.com>]

Don't be surprised if once a C API is available that other people write
dynamic bindings for that instead.  IOW I'm not sure this is such a
great idea for the goals of this project.

--- On Mon, 11/7/11, Marvin Humphrey <ma...@rectangular.com> wrote:

From: Marvin Humphrey <ma...@rectangular.com>
Subject: Re: [lucy-dev] Argument processing model
To: lucy-dev@incubator.apache.org
Date: Monday, November 7, 2011, 2:31 PM

On Sun, Nov 06, 2011 at 12:01:59PM -0800, webmasters@ctosonline.org wrote:
> 
> On Oct 31, 2011, at 8:57 PM, Marvin Humphrey wrote:
> 
> > Implementing a C API will be conceptually more complicated, because we will
> > have to make some decisions and leave some features out.  We can't support
> > default values from C, for example.  
> 
> Perl’s gv_fullname function became gv_fullname3 and then gv_fullname4 when
> more params were added.  You could conceivably do the same for default
> parameters, but it might lead to API explosion, which, come to think of it,
> renders this a very bad idea.

;)

The C API will just have to be less sophisticated because C itself is less
sophisticated.  No labeled params for C, either. :)

Lucy C API users will have to enter default argument values manually.  But we
can protect them somewhat with a policy of considering the changing of a
default value a backwards compatibility break.

Lucy is primarily targeted at high level dynamic languages.  It is not
optimized for C developers. 

Marvin Humphrey

Re: [lucy-dev] Argument processing model

[Marvin Humphrey <ma...@rectangular.com>]

On Sun, Nov 06, 2011 at 12:01:59PM -0800, webmasters@ctosonline.org wrote:
> 
> On Oct 31, 2011, at 8:57 PM, Marvin Humphrey wrote:
> 
> > Implementing a C API will be conceptually more complicated, because we will
> > have to make some decisions and leave some features out.  We can't support
> > default values from C, for example.  
> 
> Perl’s gv_fullname function became gv_fullname3 and then gv_fullname4 when
> more params were added.  You could conceivably do the same for default
> parameters, but it might lead to API explosion, which, come to think of it,
> renders this a very bad idea.

;)

The C API will just have to be less sophisticated because C itself is less
sophisticated.  No labeled params for C, either. :)

Lucy C API users will have to enter default argument values manually.  But we
can protect them somewhat with a policy of considering the changing of a
default value a backwards compatibility break.

Lucy is primarily targeted at high level dynamic languages.  It is not
optimized for C developers. 

Marvin Humphrey

Re: [lucy-dev] Argument processing model

[we...@ctosonline.org]

On Oct 31, 2011, at 8:57 PM, Marvin Humphrey wrote:

> Implementing a C API will be conceptually more complicated, because we will
> have to make some decisions and leave some features out.  We can't support
> default values from C, for example.  

Perl’s gv_fullname function became gv_fullname3 and then gv_fullname4 when more params were added.  You could conceivably do the same for default parameters, but it might lead to API explosion, which, come to think of it, renders this a very bad idea.

> I suspect that we will still want to
> autogenerate an intermediate layer which performs some amount of type
> checking, but we will just have to see how things play out.
> 
> Marvin Humphrey
> 
>