[lucy-dev] Maintaining manifest to avoid Module::Build warnings

[Marvin Humphrey <ma...@rectangular.com>]

On Fri, Dec 10, 2010 at 09:14:18AM -0600, Peter Karman wrote:
> If we always build a dist from a 'svn export' rather than a working dir.
> Or even a clean 'svn co', does this issue go away?

There's no problem with the CPAN dist tarball.  Since the Perl 'dist' action
involves creating a MANIFEST file, unpacking and building from the CPAN
tarball doesn't incur any warnings.

The problems occur when building from an SVN checkout, or building from the
(theoretical) official ASF tarball.  If you check out Lucy from SVN right now
and run Build.PL, you get output like this:

    marvin@smokey:~/projects/lucy/perl $ perl Build.PL 
    Creating new 'MYMETA.yml' with configuration results
    Can't find dist packages without a MANIFEST file
    Run 'Build manifest' to generate one

    WARNING: Possible missing or corrupt 'MANIFEST' file.
    Nothing to enter for 'provides' field in metafile.
    Creating new 'Build' script for 'Lucy' version '0.001000'
    marvin@smokey:~/projects/lucy/perl $ 

Since SVN is targeted only at developers, we can continue to live with the
warnings.  However, I don't think that it's kosher to show warnings like that
when building from the official ASF release.  To avoid them, we have to
provide a trunk/perl/MANIFEST file in the official dist, which leaves us with
two choices:

  1. Maintain trunk/perl/MANIFEST continuously.
  2. Create trunk/perl/MANIFEST as part of the ASF release process.

Per Hoss's recommendation, we're trying to keep the release generation process
to Jukka's ideal of "svn export | tar | gzip".  That suggests that we would go
to the trouble of maintaining trunk/perl/MANIFEST.  I'm OK with that -- we've
done it before -- but would like to make the process less labor intensive.

I think it will be possible to hack up something using a combination of svn
commands.  If "svn list -R" listed newly added files, then the following would
work great:

    svn list -R | sort > MANIFEST

Since that's not how "svn list -R" behaves, we'll have to come up with
something a little more involved, but it still will just be a short script we
can dump into trunk/devel/bin.

Marvin Humphrey

Re: [lucy-dev] Maintaining manifest to avoid Module::Build warnings

[Marvin Humphrey <ma...@rectangular.com>]

On Fri, Dec 10, 2010 at 12:13:01PM -0800, Chris Hostetter wrote:
> I think you are missing a third option, which is to have a top level 
> "pre-build" command (using make, or shell, or whatever) that does whatever 
> bootstraping is neccessary for all of the individual subdirs -- including 
> programaticly create whatever files/directories you don't wnat to keep in 
> svn because they should be automaticly generated.  

So IIUC, this would be the sequence necessary for building and testing the
Perl bindings from the ASF tarball?

  tar -zxf lucy-0.1.0-incubating.tar.gz
  cd lucy-0.1.0/
  make setup
  cd perl/
  perl Build.PL
  ./Build test

That seems kind of involved... maybe I'm not grokking?

It seems to me that building lucy-as-$LANGUAGE_X-extension should follow the
standard build procedure for $LANGUAGE_X as closely as possible.  In the case
of Perl, that would mean that "make setup" would be invoked when the user runs
"./Build", rather than manually.  Of course then it wouldn't solve our problem
with Build.PL and warnings...

> arguably, you should consider having a top level "build" command of some 
> kind just to handle looping over all of the differnet langauge specific 
> sub-builds anyway, and this could just be a part of that (or a pre-req for 
> it) -- run "make setup" first if you want to then peicemeal cd && build 
> individual subdirs yourself; or run "make all" if you want make to do them 
> all for you in sequence.

An "all" target would be handy, but it would be tricky to set up.  Different
host language targets may have been compiled using different compiler options
or even different compilers.  We'd have to ensure that there was no collisions
for any of the compiled objects or config files.  Right now, the .o files get
dropped right next to the corresponding .c files; that would have to
change.

We could still do a "test-all" target, though, which would build and test Lucy
for each host in sequence, cleaning up in between.  It wouldn't be practical
for debugging, but it would be helpful for voting.

Marvin Humphrey

Re: [lucy-dev] Maintaining manifest to avoid Module::Build warnings

[Chris Hostetter <ho...@fucit.org>]

: The problems occur when building from an SVN checkout, or building from the
: (theoretical) official ASF tarball.  If you check out Lucy from SVN right now
	...
:     WARNING: Possible missing or corrupt 'MANIFEST' file.
:     Nothing to enter for 'provides' field in metafile.
:     Creating new 'Build' script for 'Lucy' version '0.001000'
:     marvin@smokey:~/projects/lucy/perl $ 
	...
: when building from the official ASF release.  To avoid them, we have to
: provide a trunk/perl/MANIFEST file in the official dist, which leaves us with
: two choices:
: 
:   1. Maintain trunk/perl/MANIFEST continuously.
:   2. Create trunk/perl/MANIFEST as part of the ASF release process.

I think you are missing a third option, which is to have a top level 
"pre-build" command (using make, or shell, or whatever) that does whatever 
bootstraping is neccessary for all of the individual subdirs -- including 
programaticly create whatever files/directories you don't wnat to keep in 
svn because they should be automaticly generated.  

arguably, you should consider having a top level "build" command of some 
kind just to handle looping over all of the differnet langauge specific 
sub-builds anyway, and this could just be a part of that (or a pre-req for 
it) -- run "make setup" first if you want to then peicemeal cd && build 
individual subdirs yourself; or run "make all" if you want make to do them 
all for you in sequence.


-Hoss

Re: [lucy-dev] Maintaining manifest to avoid Module::Build warnings

[Peter Karman <pe...@peknet.com>]

Marvin Humphrey wrote on 12/10/10 12:36 PM:

> 
>   1. Maintain trunk/perl/MANIFEST continuously.

+1



> Since that's not how "svn list -R" behaves, we'll have to come up with
> something a little more involved, but it still will just be a short script we
> can dump into trunk/devel/bin.
> 

and worst case, if the packaging howto includes:

 ./Build distcheck

then at least omissions can be manually caught and fixed.


-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com