You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucy.apache.org by Marvin Humphrey <ma...@rectangular.com> on 2010/11/16 21:16:20 UTC
[lucy-dev] Slow migration to Makefiles
On Tue, Nov 16, 2010 at 09:39:20AM -0800, Mattmann, Chris A (388J) wrote:
> > Over time, we should expect to migrate a lot of the build structure to
> > Makefiles. I hate make, but it's the lowest common denominator.
>
> Is that strictly true? I mean, the reality is whatever you could do in make,
> could be done in e.g., Ant, right?
Sure -- but we could also do everything in Perl/Ruby/Python/etc. The primary
advantage that Make has over Ant and all of those is that it's already there
on every system.
The point of migrating to Makefiles would be to share build routines across
host bindings. Building Lucy for C or Python shouldn't require Perl, or Java,
or whatever. For now, we have to put up with a Perl dependency, but I would
like to eliminate that, at least for simple building of the library as a user
would. (Developers will continue to have to deal with the Perl dependency,
but after I rewrite Clownfish::Parser to be based on the Lemon parser
generator rather than Parse::RecDescent, they'll only need core Perl.)
We don't want to depend on Make exclusively for the build IMO -- the
contortions necessary for cross-platform compatibility when solving complex
problems aren't worth it. Instead, I think we should keep the Makefiles
simple, but use scripts to generate input for them. Probably such dev helper
scripts will continue to be written in Perl, like the update_snowstem.pl I
just added last week.
I don't think it's reasonable to expect Simon or Robert to fully grok a
sophisticated Module::Build subclass like trunk/perl/buildlib/Lucy/Build.pm.
However, I do think that it's reasonable to expect Lucy committers to
understand shared Makefiles, and I also think it's reasonable to expect them
to understand simple Perl scripts like update_snowstem.pl.
Marvin Humphrey
Re: [lucy-dev] Slow migration to Makefiles
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Marvin,
On Nov 16, 2010, at 12:16 PM, Marvin Humphrey wrote:
> On Tue, Nov 16, 2010 at 09:39:20AM -0800, Mattmann, Chris A (388J) wrote:
>
>>> Over time, we should expect to migrate a lot of the build structure to
>>> Makefiles. I hate make, but it's the lowest common denominator.
>>
>> Is that strictly true? I mean, the reality is whatever you could do in make,
>> could be done in e.g., Ant, right?
>
> Sure -- but we could also do everything in Perl/Ruby/Python/etc. The primary
> advantage that Make has over Ant and all of those is that it's already there
> on every system.
It seems to me that Ant is on most *nix systems (just like Make), it's just whether it's configured or not by default or enabled for an install package.
>
> The point of migrating to Makefiles would be to share build routines across
> host bindings. Building Lucy for C or Python shouldn't require Perl, or Java,
> or whatever. For now, we have to put up with a Perl dependency, but I would
> like to eliminate that, at least for simple building of the library as a user
> would. (Developers will continue to have to deal with the Perl dependency,
> but after I rewrite Clownfish::Parser to be based on the Lemon parser
> generator rather than Parse::RecDescent, they'll only need core Perl.)
Hmmm, possibly, though I wouldn't liken building Lucy to PLs -- that doesn't seem like a fair comparison. In reality, building Lucy likely requires some "build tool", be it Make, or Ant, or Python buildouts, or setuptools, or whatever, independent of whatever PL Lucy is implemented in.
>
> We don't want to depend on Make exclusively for the build IMO -- the
> contortions necessary for cross-platform compatibility when solving complex
> problems aren't worth it. Instead, I think we should keep the Makefiles
> simple, but use scripts to generate input for them. Probably such dev helper
> scripts will continue to be written in Perl, like the update_snowstem.pl I
> just added last week.
Build scripts generating other build scripts just seems like bloat to me. Of course, like i mentioned I haven't stepped up to rewrite the Lucy build system, so I'm just pontificating at this point. ;)
>
> I don't think it's reasonable to expect Simon or Robert to fully grok a
> sophisticated Module::Build subclass like trunk/perl/buildlib/Lucy/Build.pm.
> However, I do think that it's reasonable to expect Lucy committers to
> understand shared Makefiles, and I also think it's reasonable to expect them
> to understand simple Perl scripts like update_snowstem.pl.
Seems to me that the complexity of either one of your two examples (Module::Bulid or Make) aren't too far apart. Also, SImon *is* a Lucy committer, so we should definitely make sure he gets how to build the project, especially if we want to grow beyond you, Nate, or Peter being the only folks who know Lucy or what's going on with it...
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [lucy-dev] Slow migration to Makefiles
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Thanks for the feedback, Robert...
Cheers,
Chris
On Nov 17, 2010, at 11:22 AM, Robert Muir wrote:
> On Wed, Nov 17, 2010 at 2:08 PM, Nathan Kurz <na...@verse.com> wrote:
>> On Tue, Nov 16, 2010 at 12:31 PM, Mattmann, Chris A (388J)
>> <ch...@jpl.nasa.gov> wrote:
>>> Hmm, my 2 cents is that it's infinitely simpler to understand a build.xml file (or better yet a Maven pom.xml :) -- just my opinion people no tomatoes!) than it is to understand makefiles, or better yet, programs that generate makefiles on the fly, or that generate other build scripts on the fly etc etc.
>>
>> I much prefer Make to all alternatives. Lucy is at base a C project,
>> and Make is the standard for C. Certainly other things can work, but
>> most anything else causes me about the same amount of alarm as a
>> project that has only a README.doc in Word format.
>>
>
> +1. I've worked a lot with ant on the lucene-java project, and it
> sucks. in fact all build system suck. its just a matter of what 'sucks
> less' for what you are trying to do.
>
> For java, ant sucks less than Make. Especially since there are tools
> built around it (including ones distributed with ant) for things like
> junit test integration.
>
> I think Make is a better path for a project like Lucy that isn't
> java-oriented but more of the unix/C mentality. This is because there
> are tools built around Make (including yes, things like autoconf) for
> the C environment.
>
> For example: since we have been discussing snowball recently, I had to
> add several new languages to a customized snowball build and i found
> this to be completely painless with snowball's make build.
>
> I don't think we need to have a "build system war" discussion, since
> its like editors, everyone has their own opinions. But how many C
> projects do you see using make? Basically all of them.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [lucy-dev] Slow migration to Makefiles
Posted by "Henry C." <he...@cityweb.co.za>.
On Wed, November 17, 2010 21:22, Robert Muir wrote:
> But how many C projects do you see using make? Basically all of them.
or conversely, how many C projects do you see not using make? Basically none.
Re: [lucy-dev] Slow migration to Makefiles
Posted by Robert Muir <rc...@gmail.com>.
On Wed, Nov 17, 2010 at 2:08 PM, Nathan Kurz <na...@verse.com> wrote:
> On Tue, Nov 16, 2010 at 12:31 PM, Mattmann, Chris A (388J)
> <ch...@jpl.nasa.gov> wrote:
>> Hmm, my 2 cents is that it's infinitely simpler to understand a build.xml file (or better yet a Maven pom.xml :) -- just my opinion people no tomatoes!) than it is to understand makefiles, or better yet, programs that generate makefiles on the fly, or that generate other build scripts on the fly etc etc.
>
> I much prefer Make to all alternatives. Lucy is at base a C project,
> and Make is the standard for C. Certainly other things can work, but
> most anything else causes me about the same amount of alarm as a
> project that has only a README.doc in Word format.
>
+1. I've worked a lot with ant on the lucene-java project, and it
sucks. in fact all build system suck. its just a matter of what 'sucks
less' for what you are trying to do.
For java, ant sucks less than Make. Especially since there are tools
built around it (including ones distributed with ant) for things like
junit test integration.
I think Make is a better path for a project like Lucy that isn't
java-oriented but more of the unix/C mentality. This is because there
are tools built around Make (including yes, things like autoconf) for
the C environment.
For example: since we have been discussing snowball recently, I had to
add several new languages to a customized snowball build and i found
this to be completely painless with snowball's make build.
I don't think we need to have a "build system war" discussion, since
its like editors, everyone has their own opinions. But how many C
projects do you see using make? Basically all of them.
Re: [lucy-dev] Slow migration to Makefiles
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Thanks Nate, cool, alright, so it sounds like the 3 of you are fine with make and if it's a simple make ; make all ; make install (probably with a sudo in there somewhere ;) ) then even I could get it right (which is saying something!) :)
Would be nice to know what Simon thinks since he's a Lucy committer now too...
Cheers,
Chris
On Nov 17, 2010, at 11:08 AM, Nathan Kurz wrote:
> On Tue, Nov 16, 2010 at 12:31 PM, Mattmann, Chris A (388J)
> <ch...@jpl.nasa.gov> wrote:
>> Hmm, my 2 cents is that it's infinitely simpler to understand a build.xml file (or better yet a Maven pom.xml :) -- just my opinion people no tomatoes!) than it is to understand makefiles, or better yet, programs that generate makefiles on the fly, or that generate other build scripts on the fly etc etc.
>
> I much prefer Make to all alternatives. Lucy is at base a C project,
> and Make is the standard for C. Certainly other things can work, but
> most anything else causes me about the same amount of alarm as a
> project that has only a README.doc in Word format.
>
>> Ant is available on nearly every Linux distribution that I've come across in recent years (installed into /usr/bin/ant or some variant).
>
> I don't recall the details, but I recently tried to install Ant on my
> current desktop (Linux Slamd64) and gave up. I'll do it from source
> at some point, but think it's silly that I'm not able to make updates
> to the Lucy project page until then. My initial impressions of Ant
> are hence quite negative.
>
>> That said, these are just my preferences (as are Marvin's for Make/programs that generate makes and so forth :) ). What do others think? The key question to ask yourselves is:
>>
>> 1. will Marvin be the *only* RM that this project ever sees?
>
> Had to look up RM. No, presumably there will be other Release
> Managers so that Marvin can spend his time on areas more demanding of
> his particular expertise.
>
>> 2. will Marvin be the *only* person building this project, ever?
>
> No, I presume that some significant percentage of users will be
> building this. The bar should be pretty low, roughly equivalent to
> 'make config; make all; make install'.
>
>> 3. of the 2-3 existing Lucy developers, what are the preferences? I know Marvin's: what about Peter/Nate?
>
> Make without reliance on autoconf or other impenetrable junk. The
> general approach Marvin is currently using seems fine, although
> removing reliance on Perl seems good. I want something short that
> can be clearly understood in it's entirety.
>
> --nate
>
> ps. My feelings on Make are reasonably echoed here:
> http://blog.jgc.org/2010/11/things-make-got-right-and-how-to-make.html
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [lucy-dev] Slow migration to Makefiles
Posted by Nathan Kurz <na...@verse.com>.
On Tue, Nov 16, 2010 at 12:31 PM, Mattmann, Chris A (388J)
<ch...@jpl.nasa.gov> wrote:
> Hmm, my 2 cents is that it's infinitely simpler to understand a build.xml file (or better yet a Maven pom.xml :) -- just my opinion people no tomatoes!) than it is to understand makefiles, or better yet, programs that generate makefiles on the fly, or that generate other build scripts on the fly etc etc.
I much prefer Make to all alternatives. Lucy is at base a C project,
and Make is the standard for C. Certainly other things can work, but
most anything else causes me about the same amount of alarm as a
project that has only a README.doc in Word format.
> Ant is available on nearly every Linux distribution that I've come across in recent years (installed into /usr/bin/ant or some variant).
I don't recall the details, but I recently tried to install Ant on my
current desktop (Linux Slamd64) and gave up. I'll do it from source
at some point, but think it's silly that I'm not able to make updates
to the Lucy project page until then. My initial impressions of Ant
are hence quite negative.
> That said, these are just my preferences (as are Marvin's for Make/programs that generate makes and so forth :) ). What do others think? The key question to ask yourselves is:
>
> 1. will Marvin be the *only* RM that this project ever sees?
Had to look up RM. No, presumably there will be other Release
Managers so that Marvin can spend his time on areas more demanding of
his particular expertise.
> 2. will Marvin be the *only* person building this project, ever?
No, I presume that some significant percentage of users will be
building this. The bar should be pretty low, roughly equivalent to
'make config; make all; make install'.
> 3. of the 2-3 existing Lucy developers, what are the preferences? I know Marvin's: what about Peter/Nate?
Make without reliance on autoconf or other impenetrable junk. The
general approach Marvin is currently using seems fine, although
removing reliance on Perl seems good. I want something short that
can be clearly understood in it's entirety.
--nate
ps. My feelings on Make are reasonably echoed here:
http://blog.jgc.org/2010/11/things-make-got-right-and-how-to-make.html
Re: [lucy-dev] Slow migration to Makefiles
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Peter,
On Nov 16, 2010, at 2:01 PM, Peter Karman wrote:
> Mattmann, Chris A (388J) wrote on 11/16/2010 02:31 PM:
>> Hmm, my 2 cents is that it's infinitely simpler to understand a
>> build.xml file (or better yet a Maven pom.xml :) -- just my opinion
>> people no tomatoes!) than it is to understand makefiles, or better
>> yet, programs that generate makefiles on the fly, or that generate
>> other build scripts on the fly etc etc.
>>
>> Ant is available on nearly every Linux distribution that I've come
>> across in recent years (installed into /usr/bin/ant or some variant).
>>
>
> A quick check of the 2 CentOS dists I have available (5.3 and 5.4)
> reveals that neither have ant installed. Both have make. Ant is, of
> course, available as an installable package. But it's not part of the
> standard build tools, afaik.
Well, whether it's "part of the standard build tools" is up for debate. If it's available as a package that's classified in that category then I would say it's "part of the standard build tools" just not "installed by default". But that's just semantics. I build CentOS systems a lot (have way too many sitting in a closet at home :) ) and I select most of the installable packages including Ant.
>
>>
>> That said, these are just my preferences (as are Marvin's for
>> Make/programs that generate makes and so forth :) ). What do others
>> think? The key question to ask yourselves is:
>>
>> 1. will Marvin be the *only* RM that this project ever sees?
>
> No. I have done KS releases; I expect to do Lucy too.
Great.
>
> 2. will
>> Marvin be the *only* person building this project, ever?
>
> No.
+1
>
> 3. of the
>> 2-3 existing Lucy developers, what are the preferences? I know
>> Marvin's: what about Peter/Nate?
>
> Make is my pref. For the reasons Marvin states.
>
> I don't expect our Makefiles to be complicated. I expect them to
> delegate to more sophisticated, generated scripts, as Marvin suggests.
Okey dok, welp, so long as others are in agreement with this, I'm fine with it, +1...
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
Phone: +1 (818) 354-8810
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [lucy-dev] Slow migration to Makefiles
Posted by Peter Karman <pe...@peknet.com>.
Mattmann, Chris A (388J) wrote on 11/16/2010 02:31 PM:
> Hmm, my 2 cents is that it's infinitely simpler to understand a
> build.xml file (or better yet a Maven pom.xml :) -- just my opinion
> people no tomatoes!) than it is to understand makefiles, or better
> yet, programs that generate makefiles on the fly, or that generate
> other build scripts on the fly etc etc.
>
> Ant is available on nearly every Linux distribution that I've come
> across in recent years (installed into /usr/bin/ant or some variant).
>
A quick check of the 2 CentOS dists I have available (5.3 and 5.4)
reveals that neither have ant installed. Both have make. Ant is, of
course, available as an installable package. But it's not part of the
standard build tools, afaik.
>
> That said, these are just my preferences (as are Marvin's for
> Make/programs that generate makes and so forth :) ). What do others
> think? The key question to ask yourselves is:
>
> 1. will Marvin be the *only* RM that this project ever sees?
No. I have done KS releases; I expect to do Lucy too.
2. will
> Marvin be the *only* person building this project, ever?
No.
3. of the
> 2-3 existing Lucy developers, what are the preferences? I know
> Marvin's: what about Peter/Nate?
Make is my pref. For the reasons Marvin states.
I don't expect our Makefiles to be complicated. I expect them to
delegate to more sophisticated, generated scripts, as Marvin suggests.
--
Peter Karman . http://peknet.com/ . peter@peknet.com
RE: [lucy-dev] Slow migration to Makefiles
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hmm, my 2 cents is that it's infinitely simpler to understand a build.xml file (or better yet a Maven pom.xml :) -- just my opinion people no tomatoes!) than it is to understand makefiles, or better yet, programs that generate makefiles on the fly, or that generate other build scripts on the fly etc etc.
Ant is available on nearly every Linux distribution that I've come across in recent years (installed into /usr/bin/ant or some variant).
That said, these are just my preferences (as are Marvin's for Make/programs that generate makes and so forth :) ). What do others think? The key question to ask yourselves is:
1. will Marvin be the *only* RM that this project ever sees?
2. will Marvin be the *only* person building this project, ever?
3. of the 2-3 existing Lucy developers, what are the preferences? I know Marvin's: what about Peter/Nate?
4. of the 1 new Lucy committers as part of Apache Lucy in the Incubator (e.g., Simon?)
5. are the mentors ever going to build and use this system? Or scarier yet, maintain it? My answer on that is that at some point I'd like to build it myself and understand it, but sophisticated Makefiles are not my cup of tea.
The community right now is small so it will be very driven by whomever picks up the shovel and starts to dig the hole, but it would be nice if the tool used to dig that hole is something that not only Marvin can wield...
Cheers,
Chris
________________________________________
From: Marvin Humphrey [marvin@rectangular.com]
Sent: Tuesday, November 16, 2010 12:16 PM
To: lucy-dev@incubator.apache.org
Subject: [lucy-dev] Slow migration to Makefiles
On Tue, Nov 16, 2010 at 09:39:20AM -0800, Mattmann, Chris A (388J) wrote:
> > Over time, we should expect to migrate a lot of the build structure to
> > Makefiles. I hate make, but it's the lowest common denominator.
>
> Is that strictly true? I mean, the reality is whatever you could do in make,
> could be done in e.g., Ant, right?
Sure -- but we could also do everything in Perl/Ruby/Python/etc. The primary
advantage that Make has over Ant and all of those is that it's already there
on every system.
The point of migrating to Makefiles would be to share build routines across
host bindings. Building Lucy for C or Python shouldn't require Perl, or Java,
or whatever. For now, we have to put up with a Perl dependency, but I would
like to eliminate that, at least for simple building of the library as a user
would. (Developers will continue to have to deal with the Perl dependency,
but after I rewrite Clownfish::Parser to be based on the Lemon parser
generator rather than Parse::RecDescent, they'll only need core Perl.)
We don't want to depend on Make exclusively for the build IMO -- the
contortions necessary for cross-platform compatibility when solving complex
problems aren't worth it. Instead, I think we should keep the Makefiles
simple, but use scripts to generate input for them. Probably such dev helper
scripts will continue to be written in Perl, like the update_snowstem.pl I
just added last week.
I don't think it's reasonable to expect Simon or Robert to fully grok a
sophisticated Module::Build subclass like trunk/perl/buildlib/Lucy/Build.pm.
However, I do think that it's reasonable to expect Lucy committers to
understand shared Makefiles, and I also think it's reasonable to expect them
to understand simple Perl scripts like update_snowstem.pl.
Marvin Humphrey