[lucy-dev] Go bindings -- build.go

[Marvin Humphrey <ma...@rectangular.com>]

On Mon, Nov 17, 2014 at 1:12 PM, Marvin Humphrey <ma...@rectangular.com> wrote:
> *   The build is powered by a Go "script" named "build.go".

Go's package management infrastructure (the "go" tool) does not suffice to
build Lucy or any Clownfish-powered project, because it cannot be extended to
run arbitrary commands.

This blog post lays out our options:

    http://blog.mattbasta.com/things_that_make_me_sad_in_go.html#user-content-the-big-community-mess

We could have used a Makefile, but Make isn't portable.  So I chose to emulate
Syncthing's "build.go" example because that was in line with how we handle
other hosts.

> *   Making Clownfish-powered packages "go-gettable" would be hard and the
>     result would be inherently fragile.  I think we should accept for the
>     time being that it won't be feasible for either Clownfish or Lucy. We
>     aren't the only people that have this problem, though.

Go supports conditional inclusion of code using naming conventions:

    http://golang.org/pkg/go/build/#hdr-Build_Constraints

Ultimately, I found the approach Rob Pike lays out in this article
illuminating but unpersuasive:

    https://golang.org/doc/articles/go_command.html

Packaging systems have to provide hooks, in my opinion.

Here's what he recommends instead:

    Limitations

    As mentioned above, the go command is not a general-purpose build tool. In
    particular, it does not have any facility for generating Go source files
    during a build. Instead, if you want to use a tool like yacc or the
    protocol buffer compiler, you will need to write a makefile (or a
    configuration file for the build tool of your choice) to generate the Go
    files and then check those generated source files into your repository.
    This is more work for you, the package author, but it is significantly
    less work for your users, who can use "go get" without needing to obtain
    and build any additional tools.

So, to support "go get" we could...

*   Run CFC (and lemon and flex), outputting C code.
*   Dump all C code into the Go package directory.
*   Run charmonizer for several popular OS/architecture combinations and
    store the result in conditionally included files.
*   Check it all in.
*   Publish everything to a dedicated repo (because we sure ain't gonna regen
    all that with every commit and check it into the main repo.)

Possibly we might perform that ritual at release points -- in which case
publishing our Go code would be not unlike publishing to any other downstream
package manager.

However, things still wouldn't work even then because we're up against the
showstopper C symbol export issue.  And even if that gets resolved, the output
of Charmonizer is *compiler*-specific -- tying it to the OS/architecture is
inherently fragile.

Ergo, no go.  You can see why I threw up my hands and went with "build.go".

Marvin Humphrey