[lucy-dev] Go bindings

[Marvin Humphrey <ma...@rectangular.com>]

Lucifers,

I'm pleased to report that I've pushed some branches to the Clownfish and
Lucy ASF repos which should make for a solid start on Go (golang) bindings.

  http://s.apache.org/clownfish-go_bindings_1
  http://s.apache.org/lucy-go_bindings_1

At present there is only just enough Lucy functionality to support a sample
app, but that should be enough to allow for review.  Compared with Philip
Southam's "golucy" ( https://github.com/philipsoutham/golucy ) these
bindings are incomplete, but place greater emphasis on solving some
hard problems in Clownfish.

Design notes:

*   Clownfish parcels are mapped 1:1 to Go packages -- i.e. there is only
    `lucy`, while `index`, `search`, `analysis` and the like are not broken
    out into separate packages a la Lucene's Java packages.
*   The Go package dir, which by convention takes the name of the package,
    has been put one level down from `go` -- e.g. `$LUCYREPO/go/lucy`.  This
    allows us to put whatever junk we want into `go/`.
*   The build is powered by a Go "script" named "build.go".
*   Clownfish-powered code ends up linked statically into Go binaries --
    avoiding all the permissions, versioning and deployment complications of
    dynamic linking to shared libraries outside of $GOPATH.
*   Clownfish headers get installed into a dedicated subdir of $GOPATH/pkg.
*   Clownfish exception handling has been implemented using Go's
    panic/recover/defer.
*   Idiomatic Go error handling is achieved by trapping exceptions in
    user-level method wrappers and returning an `error` alongside any return
    value.
*   Clownfish abstract classes are mapped to Go interfaces.
*   Clownfish concrete classes are mapped to Go struct pointers.
*   Constructors adopt Go naming conventions, e.g. `lucy.NewSchema()`.
*   Lucy documents are organized as Go structs.  Field names are mapped
    case-insensitively and values are accessed using reflection techniques
    similar to those used by Go's `encoding/json` package.

Challenges:

*   Adapting Clownfish's subtyping for Go is hard, because Go does not
    support inheritance.  This is especially a problem with functionality
    which can only be unlocked by overriding methods on concrete classes
    like QueryParser, IndexManager and Schema.
*   Making Clownfish-powered packages "go-gettable" would be hard and the
    result would be inherently fragile.  I think we should accept for the
    time being that it won't be feasible for either Clownfish or Lucy. We
    aren't the only people that have this problem, though.
*   Go data cannot be passed to C (because that interferes with Go dev plans
    to implement compacting garbage collection).

Marvin Humphrey

Re: [lucy-dev] Go bindings

[Logan Bell <lo...@gmail.com>]

I am a Go n00b as well, but the more development we get going for
other bindings the all the more better for the project. Great work and
+1

Thanks,
Logan

On Tue, Nov 18, 2014 at 3:19 PM, Peter Karman <pe...@peknet.com> wrote:
> On 11/17/14, 3:12 PM, Marvin Humphrey wrote:
>> Lucifers,
>>
>> I'm pleased to report that I've pushed some branches to the Clownfish and
>> Lucy ASF repos which should make for a solid start on Go (golang) bindings.
>>
>>   http://s.apache.org/clownfish-go_bindings_1
>>   http://s.apache.org/lucy-go_bindings_1
>>
>> At present there is only just enough Lucy functionality to support a sample
>> app, but that should be enough to allow for review.  Compared with Philip
>> Southam's "golucy" ( https://github.com/philipsoutham/golucy ) these
>> bindings are incomplete, but place greater emphasis on solving some
>> hard problems in Clownfish.
>>
>
> I'm a Go n00b and so can't comment on any design specifics.
>
> On principle I think this is terrific and want to say thanks to Marvin
> for keeping the ball rolling forward on expanding Lucy's host language
> implementations.
>
> +1
>
> --
> Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Re: [lucy-dev] Go bindings

[Peter Karman <pe...@peknet.com>]

On 11/17/14, 3:12 PM, Marvin Humphrey wrote:
> Lucifers,
> 
> I'm pleased to report that I've pushed some branches to the Clownfish and
> Lucy ASF repos which should make for a solid start on Go (golang) bindings.
> 
>   http://s.apache.org/clownfish-go_bindings_1
>   http://s.apache.org/lucy-go_bindings_1
> 
> At present there is only just enough Lucy functionality to support a sample
> app, but that should be enough to allow for review.  Compared with Philip
> Southam's "golucy" ( https://github.com/philipsoutham/golucy ) these
> bindings are incomplete, but place greater emphasis on solving some
> hard problems in Clownfish.
> 

I'm a Go n00b and so can't comment on any design specifics.

On principle I think this is terrific and want to say thanks to Marvin
for keeping the ball rolling forward on expanding Lucy's host language
implementations.

+1

-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Re: [lucy-dev] Go bindings

[Marvin Humphrey <ma...@rectangular.com>]

On Sun, Nov 30, 2014 at 5:43 AM, Nick Wellnhofer <we...@aevum.de> wrote:
> On 30/11/2014 06:10, Marvin Humphrey wrote:

> OK, I can push all commits that change charmonizer.main to master anyway.

Thanks!

On a side note, all this rebasing is generating extra noise on the
commits mailing list.  I'm going to try using Github more for
proposing branches for a while and see how that goes.

> What do you have in mind regarding the build process? Based on Makefiles or
> a invoking the C compiler directly?

I would say invoking the C compiler directly (followed by the
archiver). For concurrency, maybe fork(), maybe pthreads.

Using Makefiles internally might be more reliable in the short term,
but it's a lot of churn to make that happen and I think we would
eventually want to move away from that.

Marvin Humphrey

Re: [lucy-dev] Go bindings

[Nick Wellnhofer <we...@aevum.de>]

On 30/11/2014 06:10, Marvin Humphrey wrote:
> I was able to rebase my changes on top of your perl_build_with_make
> branch without too much trouble, so I've discarded my own refactoring
> of charmonizer.main and pushed a new branch go_bindings_2.  All
> non-charmonizer changes were left intact, except for one trivial new
> commit modifying build.go.

OK, I can push all commits that change charmonizer.main to master anyway. They 
don't affect the Perl build yet. I'm not so sure about really making the 
switch for the Perl build system.  Maybe the Makefile-based build should be 
optional for now. On the other hand, if we want to go forward with letting CFC 
build the static libraries, similar changes have to be made anyway.

> We'll probably collide less often in build scripts once we move
> responsibility for building static archives from our charmonizer apps
> to CFC.

What do you have in mind regarding the build process? Based on Makefiles or a 
invoking the C compiler directly?

Nick

Re: [lucy-dev] Go bindings

[Marvin Humphrey <ma...@rectangular.com>]

On Sun, Nov 23, 2014 at 8:32 AM, Nick Wellnhofer <we...@aevum.de> wrote:

> I only noticed that some of the commits clash with my
> work on speeding up the Perl build by using Makefiles. You made some nice
> additions like the `--host` option that I also planned to implement. Just go
> ahead with the go_bindings branch and I'll merge my changes afterwards.

I was able to rebase my changes on top of your perl_build_with_make
branch without too much trouble, so I've discarded my own refactoring
of charmonizer.main and pushed a new branch go_bindings_2.  All
non-charmonizer changes were left intact, except for one trivial new
commit modifying build.go.

We'll probably collide less often in build scripts once we move
responsibility for building static archives from our charmonizer apps
to CFC.

Marvin Humphrey

Re: [lucy-dev] Go bindings

[Marvin Humphrey <ma...@rectangular.com>]

On Sun, Nov 23, 2014 at 8:32 AM, Nick Wellnhofer <we...@aevum.de> wrote:

> I can't really comment on most of the changes as I lack understanding of the
> Go <-> C interface.

The gory details of the binding implementation are important, but there are a
number of higher level design decisions being made as well.

I'd like some of the design rationales to go into the email record, so I'm
going to elaborate on some of my earlier bullet points in separate emails.

Marvin Humphrey

Re: [lucy-dev] Go bindings

[Nick Wellnhofer <we...@aevum.de>]

On 17/11/2014 22:12, Marvin Humphrey wrote:
> Lucifers,
>
> I'm pleased to report that I've pushed some branches to the Clownfish and
> Lucy ASF repos which should make for a solid start on Go (golang) bindings.

I can't really comment on most of the changes as I lack understanding of the 
Go <-> C interface. I only noticed that some of the commits clash with my work 
on speeding up the Perl build by using Makefiles. You made some nice additions 
like the `--host` option that I also planned to implement. Just go ahead with 
the go_bindings branch and I'll merge my changes afterwards.

Nick