[lucy-dev] Derive CF search path from host

[Marvin Humphrey <ma...@rectangular.com>]

Greets,

It would be in the Clownfish "symbiotic" spirit to derive our search path of
include dirs from the host-specific search path: sys.path for Python, @INC for
Perl, and so on.  It seems to me that Clownfish search path behavior for the
current interface should be:

1.  Search each dir added via add_source_dir, most recently added first.
2.  Search each dir added via add_include_dir, most recently added first.
3.  Search dirs derived from host-specific search paths in the same order that
    the host would search for its own modules.

In theory, the patch below would facilitate tasks such as installing Lucy to
~/lib/perl5 (which we assume Perl only knows about from PERL5LIB), then
working on a Lucy extension.  Right now, CFC won't find files installed
somewhere like `~/lib/perl5/darwin-thread-multi-2level/Clownfish/_include`,
and this patch is supposed to fix that.  However, at the moment we get an
error like this:

    marvin@knut:/Users/Shared/projects/lucy/perl $ ./Build code
    Parsing Clownfish files...
    File Clownfish/ByteBuf.cfh already registered at
/Users/marvin/lib/perl5/darwin-thread-multi-2level/Clownfish/CFC/Perl/Build.pm
line 224.

Not being able to install CFC and the Clownfish runtime to an intermediate
directory and then build Lucy has been a minor inconvenience while prepping
for the 0.4.0 release.  I was hoping to address the problem with a simple fix,
but it looks like things are more complicated.  Not a release blocker,
though.

In the longer term, I'd like to get rid of add_source_dir() and specify source
files explicitly, which would make CFC's interface more like that of a
traditional compiler. That has implications for search path behavior as well
-- but the principle of deriving the Clownfish search path from the host
search path still holds.

Marvin Humphrey

diff --git a/compiler/perl/lib/Clownfish/CFC.pm
b/compiler/perl/lib/Clownfish/CFC.pm
index 8b80ea5..fea762b 100644
--- a/compiler/perl/lib/Clownfish/CFC.pm
+++ b/compiler/perl/lib/Clownfish/CFC.pm
@@ -244,6 +244,7 @@ BEGIN { XSLoader::load( 'Clownfish::CFC', '0.01' ) }
     BEGIN { push our @ISA, 'Clownfish::CFC::Base' }
     use Carp;
     use Clownfish::CFC::Util qw( verify_args );
+    use File::Spec::Functions qw( catdir );

     our %new_PARAMS = (
         dest => undef,
@@ -253,7 +254,12 @@ BEGIN { XSLoader::load( 'Clownfish::CFC', '0.01' ) }
         my ( $either, %args ) = @_;
         confess "no subclassing allowed" unless $either eq __PACKAGE__;
         verify_args( \%new_PARAMS, %args ) or confess $@;
-        return _new( @args{qw( dest )} );
+        my $self = _new( @args{qw( dest )} );
+        for my $dir (reverse(@INC)) {
+            my $include = catdir($dir, 'Clownfish', '_include');
+            $self->add_include_dir($include);
+        }
+        return $self;
     }

     # Recreate host language specific metadata of dependencies.
diff --git a/compiler/src/CFCHierarchy.c b/compiler/src/CFCHierarchy.c
index dddb121..4793a97 100644
--- a/compiler/src/CFCHierarchy.c
+++ b/compiler/src/CFCHierarchy.c
@@ -192,10 +192,13 @@ void
 CFCHierarchy_add_include_dir(CFCHierarchy *self, const char *include_dir) {
     size_t n = self->num_includes;
     size_t size = (n + 2) * sizeof(char*);
-    self->includes      = (char**)REALLOCATE(self->includes, size);
-    self->includes[n]   = CFCUtil_strdup(include_dir);
+    char **old_includes = self->includes;
+    self->includes      = (char**)MALLOCATE(size);
+    self->includes[0]   = CFCUtil_strdup(include_dir);
+    memcpy(self->includes + 1, old_includes, n * sizeof(char*));
     self->includes[n+1] = NULL;
     self->num_includes  = n + 1;
+    free(old_includes);
 }

 void

Re: [lucy-dev] Derive CF search path from host

[Peter Karman <pe...@peknet.com>]

Marvin Humphrey wrote on 7/19/14 3:31 PM:
> Greets,
> 
> It would be in the Clownfish "symbiotic" spirit to derive our search path of
> include dirs from the host-specific search path: sys.path for Python, @INC for
> Perl, and so on.

+1


-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Re: [lucy-dev] Derive CF search path from host

[Nick Wellnhofer <we...@aevum.de>]

On Jul 19, 2014, at 21:31 , Marvin Humphrey <ma...@rectangular.com> wrote:

> 1.  Search each dir added via add_source_dir, most recently added first.
> 2.  Search each dir added via add_include_dir, most recently added first.
> 3.  Search dirs derived from host-specific search paths in the same order that
>    the host would search for its own modules.

+1. With that change, we shouldn’t need the ‘cf_system_include_dirs’ method from Clownfish::CFC::Perl::Build.

> In the longer term, I'd like to get rid of add_source_dir() and specify source
> files explicitly, which would make CFC's interface more like that of a
> traditional compiler. That has implications for search path behavior as well
> -- but the principle of deriving the Clownfish search path from the host
> search path still holds.

This might work for the Perl bindings, but it wouldn’t be very convenient for the C bindings which use the ‘cfc’ command-line utility.

Nick