You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Greg Stein <gs...@lyra.org> on 2001/05/25 03:51:37 UTC
Re: new build system
On Thu, May 24, 2001 at 09:11:43PM +0200, RADICS Peter wrote:
> Hello gstein,
>
> Could you explain some rationale behind this rewamping of the make
> process?
No.
Just kidding... :-) As Karl pointed out, the paper on "Recursive Make
Considered Harmful" is actually quite a valid paper. The new build system is
a lot faster than before. We also have just a single Makefile to maintain,
rather than two dozen scattered across the system.
There were some things that we wanted to do which were a giant pain when
dealing with those scattered Makefiles. For example: adding the --silent
flag to every libtool line. Or a rule for creating the .la-a files needs
when we do a static-link install of mod_dav_svn into an Apache source tree.
Automake also produces Makefiles that require GNU make. That is a big pain
for the FreeBSD users out there, and some other fringe/older OSes (I've got
to imagine that SunOS or AIX users will be happier).
The automake system also required GCC to be present to deal with dependency
generation. And you couldn't turn that off! I have no idea how people were
building SVN on AIX systems using the xlc compiler (maybe nobody tried?).
The new dependency generation is portable, and also much more finely tuned.
On that last bit: GCC's dependencies included every file included, not just
the SVN files. This *really* pissed me off. Whenever I would go and
configure APR, it would touch include/apr.h. BAM! My entire Subversion
project would recompile. I've been lazy about moving to my faster
development machine, so that is actually quite painful. I've gotten pretty
used to avoiding unnecessary updates to APR. That's just wrong. The tools
should work *for* me, rather than against me. We want dependencies on the
stuff that we change, and use the presumption that our building blocks
(Neon, APR, etc) only change their APIs in compatible ways.
Another thing about automake is its insane predilection for rebuilding the
./configure script and associated files. Any change to those files, and
everything goes to seed. I think it is an absolute horror for a Makefile to
ever run ./confiure, for any reason. That is a user operation, not something
to be left to a Makefile. Same thing for generating ./configure from the
configure.in file -- we have autogen.sh to do that; the Makefile shouldn't
ever try it. But you can't change that behavior in Automake.
Automake also has some weird dependencies on a ChangeLog file being present.
That is silly. We might *distribute* one, but in day-to-day development?
Screw that.
The underlying point is that Automake may be fine for FSF/GNU projects,
which all adhere to a very specific set of rules. But for your Average Joe
project, we have our own policies and desires. Automake isn't flexible
enough for that.
Then there is that whole recursion thing I mentioned. We get speed and
clarity with the new system. Removing automake and its bazillion
substitutions has also dramatically sped up autogen.sh and the last step of
./configure.
Lastly, we have a lot better control over all the details of building,
installation, cleaning, etc. In particular, the "make check" stuff is a big
win. We were replacing that in all of the Makefiles for the test
directories. But they were all inconsistent. And they didn't work together.
etc etc.
It also just feels simpler. Need to change something? Go and tweak
Makefile.in and it fixes the whole system. Or maybe adjust gen-make.py to
alter how it produces build-outputs.mk.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: new build system
Posted by "Chris D. Sloan" <cd...@cs.hmc.edu>.
On Fri, May 25, 2001 at 10:31:56AM -0400, Greg Hudson wrote:
> Moreover, having a single Makefile is less functional in that you
> can't run "make" in a subdirectory to build only targets in that
> subdirectory, as is often convenient. (And it's not just convenient
> to save runtime; it's convenient because (1) the world isn't an
> optimal place, and sometimes the build blows out in a directory you
> didn't care about, and it's easier to just build subcomponents than to
> fix the problem, and (2) you're often already in the subdir in a
> previously completed build tree, making some change, and don't want to
> jump back to the top level to test it.)
With recursive make, you can type make in a subdirectory and get just
that directory, and I agree that's useful, but when you are at the top
directory you cannot make specific targets in the recursive makefiles.
One makefile basically ignores directories, but recursive makefiles
artificially hide their contents. Personally, I have had much better
luck with a single makefile than with recursive makefiles, but a fair
amount probably depends on personal style.
Chris
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: new build system
Posted by Greg Stein <gs...@lyra.org>.
On Sat, Jun 02, 2001 at 10:35:17AM -0400, Greg Hudson wrote:
> Two more notes:
>
> >> Why doesn't apr's configure script avoid touching apr.h when there
> >> is no actual change?
>
> > AC_OUTPUT() doesn't do that, AFAIK.
>
> AC_CONFIG_HEADER does. From experience, I believe it would be
> possible to use AC_CONFIG_HEADER with apr.h without corrupting the
> namespace with autoconf symbols.
Hmm. I'll take a look at that. It would be nice not to touch that header on
a simple reconfig.
> (Even better would be to make apr.h
> not vary per platform and per compiler, but that's asking a lot for a
> portability layer.)
The various type mappings are in there, so it will have to change.
> > What I find is pretty neat is when you do a "make" now, when
> > everything is already up-to-date. We get a bazillion lines of output
> > from APR as it recurses all over the place, telling you "entering
> > directory X" and "leaving directory X". Then our portion prints
> > nothing. Nice and quiet :-)
>
> If you don't like those enter/exit messages from gnu make, just use
> make -S.
Ah. Cool. Dang... where do you find this stuff? :-) (and don't say RTFM)
I also realized that SVN users can do "make local-all" to just build the SVN
portions and skip APR/Neon. Once you build the two external projects, the
local-all trick can keep things much quieter.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: new build system
Posted by Greg Stein <gs...@lyra.org>.
On Fri, May 25, 2001 at 10:31:56AM -0400, Greg Hudson wrote:
>...
> > On that last bit: GCC's dependencies included every file included,
> > not just the SVN files.
>
> (Your statement is too expansive; the dependencies shouldn't include
> system headers.) How is it supposed to know that the APR files are
> part of a different project?
At the moment, we only include specific headers into dependencies. Namely,
those in the "current directory" and the headers listed in the [includes]
section of build.conf.
> How do you know that a change to apr.h
> doesn't affect the files which include it?
I don't :-) I took the position that we simply assume changes there will
not affect us. Given that APR is supposed to be a library (someday :-), then
it will take pains to avoid binary incompatibility from one version to the
next.
> Why doesn't apr's
> configure script avoid touching apr.h when there is no actual change?
AC_OUTPUT() doesn't do that, AFAIK.
>...
> > shouldn't ever try it. But you can't change that behavior in
> > Automake.
>
> Right with you up until the last line; there's an AM_MAINTAINER_MODE
> option to change that behavior.
Ah. Okee... Thanks.
> > There were some things that we wanted to do which were a giant pain
> > when dealing with those scattered Makefiles. For example: adding the
> > --silent flag to every libtool line.
>
> I'm not sure why Subversion should be different from pretty much all
> other libtool-using programs in regards to the --silent flag, but I'm
> also not sure what the problem is; we seemed to be doing this before
> the build system change via the definition of LIBTOOL.
We had to redefine LIBTOOL in every Makefile.am. I called that "painful" :-)
The --silent flag is handy, preventing a lot of output from spewing during
the build process. And I've never needed the stuff that libtool prints out
(i.e. who cares what it does under the covers; it is just a tool).
> > Or a rule for creating the .la-a files needs when we do a
> > static-link install of mod_dav_svn into an Apache source tree.
>
> This problem is over my head, but I can certainly see how changing the
> object generation rules would be a pain in automake. I'm not actually
> a big fan of automake and don't use it in my own projects, but I do
> use recursive make, and have never seen a compelling reason not to.
Recursive make would be an option, and we could use a system similar to
APR(UTIL) where every Makefile includes a "rules.mk" file. It actually works
reasonably well. But we still have that whole recursion thing going on.
What I find is pretty neat is when you do a "make" now, when everything is
already up-to-date. We get a bazillion lines of output from APR as it
recurses all over the place, telling you "entering directory X" and "leaving
directory X". Then our portion prints nothing. Nice and quiet :-)
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: new build system
Posted by Joe Orton <jo...@btconnect.com>.
On Fri, May 25, 2001 at 10:31:56AM -0400, Greg Hudson wrote:
...
> > There were some things that we wanted to do which were a giant pain
> > when dealing with those scattered Makefiles. For example: adding the
> > --silent flag to every libtool line.
>
> I'm not sure why Subversion should be different from pretty much all
> other libtool-using programs in regards to the --silent flag, but I'm
> also not sure what the problem is; we seemed to be doing this before
> the build system change via the definition of LIBTOOL.
This was my mistake - it turns out you can add --silent to the LIBTOOL
variable everywhere by doing LIBTOOL="$LIBTOOL --silent" in configure.in
after using the PROG_LIBTOOL macro, so adding it to every Makefile.am
was overkill. :(
joe
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: new build system
Posted by Greg Stein <gs...@lyra.org>.
On Sun, May 27, 2001 at 12:29:36AM -0400, Garance A Drosihn wrote:
>...
> >Then there is that whole recursion thing I mentioned. We get
> >speed and clarity with the new system. Removing automake and
> >its bazillion substitutions has also dramatically sped up
> >autogen.sh and the last step of ./configure.
>
> Speeding up the build process also sounds like a win. As long
> as it all works right, this seems like a worthwhile alternative
> to try out. If it works, then faster builds are always nice
> to have!
If it didn't work, I wouldn't have checked it in :-)
> (any idea how well this works with 'make -j'?)
Actually, make -j should work much better with the single makefile. It can
parallelize the entire build, not just the bits within a single directory.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: new build system
Posted by Garance A Drosihn <dr...@rpi.edu>.
At 8:51 PM -0700 5/24/01, Greg Stein wrote:
>Automake also produces Makefiles that require GNU make. That
>is a big pain for the FreeBSD users out there, and some other
>fringe/older OSes (I've got to imagine that SunOS or AIX
>users will be happier).
As I am primarily a freebsd/openbsd/MacOS-10 user, this sounds
good to me... :-)
>The automake system also required GCC to be present to deal
>with dependency generation. And you couldn't turn that off!
>I have no idea how people were building SVN on AIX systems
>using the xlc compiler (maybe nobody tried?).
Chances are that anyone who did try has been forced to build
gcc/gmake due to other automake'd packages on the same systems.
I haven't tried to build SVN anywhere yet, but I do know that
our AIX and IRIX boxes do have gcc and gmake built for them.
>Then there is that whole recursion thing I mentioned. We get
>speed and clarity with the new system. Removing automake and
>its bazillion substitutions has also dramatically sped up
>autogen.sh and the last step of ./configure.
Speeding up the build process also sounds like a win. As long
as it all works right, this seems like a worthwhile alternative
to try out. If it works, then faster builds are always nice
to have!
(any idea how well this works with 'make -j'?)
--
Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu
Senior Systems Programmer or gad@freebsd.org
Rensselaer Polytechnic Institute or drosih@rpi.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org