You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Alan Conway <ac...@redhat.com> on 2007/04/11 00:30:06 UTC
c++: why does parallel build fail for make check?
make -j<N> fails building the c++ tests with an "invalid Foo.lo" error.
Anyone have an idea what might be causing it?
Cheers,
Alan.
Re: c++: why does parallel build fail for make check?
Posted by Alan Conway <ac...@redhat.com>.
On Thu, 2007-04-12 at 04:19 +0200, Jim Meyering wrote:
> > The real key to this problem is that Automake generates the following
> > very strange rule:
> >
> > check-am: all-am
> > $(MAKE) $(AM_MAKEFLAGS) $(check_LTLIBRARIES)
> > $(MAKE) $(AM_MAKEFLAGS) check-TESTS
> >
> > With the result that there are *two* instances of make running
> > simultaneously when you make check with -j. As long as they are building
> > disjoint sets of targets it works, but I can't figure out why on earth
> > automake would do that instead of:
>
> Automake's rule ensures that the following are built in order:
> all-am
> check_LTLIBRARIES
> check-TESTS
>
> > check-am: check-TESTS
> > check-TESTS: $(TESTS) $(check_LTLIBRARIES)
>
> Doing it this way would mistakenly let some of the latter rules
> run before the all-am ones, when run via make -jN.
What's wrong with that? If some of the latter rules can be built in
parallel with all-am, so much the better in a -jN build.
If you did want to forcibly allow all-am to complete first it can
probably be done via dependencies also rather than sub-makes, something
like
$(check_LTLIBRARIES): all-am
$(check-TESTS): all-am
Cheers,
Alan.
Re: c++: why does parallel build fail for make check?
Posted by Jim Meyering <ji...@meyering.net>.
Alan Conway <ac...@redhat.com> wrote:
> On Wed, 2007-04-11 at 22:49 +0200, Jim Meyering wrote:
>> Alan Conway <ac...@redhat.com> wrote:
>> > make -j<N> fails building the c++ tests with an "invalid Foo.lo" error.
>> > Anyone have an idea what might be causing it?
>>
>> Hi Alan,
>>
>> The symptoms look suspiciously like those of the problem I fixed
>> back in December:
> [snip]
>> Since this is rather subtle, I think it's more maintainable
>> simply to turn off parallelism in the src/tests/ directory.
>
> I think I've found a better fix. The problem I was seeing involved
> double-building of check_LTLIBRARIES.
Right. That's what triggers the ltmain misbehavior I mentioned.
That script really should be immune to such abuse.
> This can be avoided simply by
> removing the dependency check: $(check_LTLIBRARIES) in Makefile.am
> With that change (just comitted) parallel builds work fine.
Great!
If more people build regularly with -j2 or more, (or even an autobuilder),
then any future regression will be exposed right away.
> The real key to this problem is that Automake generates the following
> very strange rule:
>
> check-am: all-am
> $(MAKE) $(AM_MAKEFLAGS) $(check_LTLIBRARIES)
> $(MAKE) $(AM_MAKEFLAGS) check-TESTS
>
> With the result that there are *two* instances of make running
> simultaneously when you make check with -j. As long as they are building
> disjoint sets of targets it works, but I can't figure out why on earth
> automake would do that instead of:
Automake's rule ensures that the following are built in order:
all-am
check_LTLIBRARIES
check-TESTS
> check-am: check-TESTS
> check-TESTS: $(TESTS) $(check_LTLIBRARIES)
Doing it this way would mistakenly let some of the latter rules
run before the all-am ones, when run via make -jN.
Re: c++: why does parallel build fail for make check?
Posted by Alan Conway <ac...@redhat.com>.
On Wed, 2007-04-11 at 22:49 +0200, Jim Meyering wrote:
> Alan Conway <ac...@redhat.com> wrote:
> > make -j<N> fails building the c++ tests with an "invalid Foo.lo" error.
> > Anyone have an idea what might be causing it?
>
> Hi Alan,
>
> The symptoms look suspiciously like those of the problem I fixed
> back in December:
[snip]
> Since this is rather subtle, I think it's more maintainable
> simply to turn off parallelism in the src/tests/ directory.
I think I've found a better fix. The problem I was seeing involved
double-building of check_LTLIBRARIES. This can be avoided simply by
removing the dependency check: $(check_LTLIBRARIES) in Makefile.am
With that change (just comitted) parallel builds work fine.
The real key to this problem is that Automake generates the following
very strange rule:
check-am: all-am
$(MAKE) $(AM_MAKEFLAGS) $(check_LTLIBRARIES)
$(MAKE) $(AM_MAKEFLAGS) check-TESTS
With the result that there are *two* instances of make running
simultaneously when you make check with -j. As long as they are building
disjoint sets of targets it works, but I can't figure out why on earth
automake would do that instead of:
check-am: check-TESTS
check-TESTS: $(TESTS) $(check_LTLIBRARIES)
This way the problem would not have arisen because make would have full
knowledge of the dependency tree and would resolve multiple declarations
of the same dependencies correctly.
Is there some reason for starting sub-makes like this that I'm missing?
If not can we fix automake to do it right?
In any case we can do parallel builds :)
Cheers,
Alan.
Re: c++: why does parallel build fail for make check?
Posted by Jim Meyering <ji...@meyering.net>.
Alan Conway <ac...@redhat.com> wrote:
> make -j<N> fails building the c++ tests with an "invalid Foo.lo" error.
> Anyone have an idea what might be causing it?
Hi Alan,
The symptoms look suspiciously like those of the problem I fixed
back in December:
Subject: [cpp] fix for parallel build failure
http://mail-archives.apache.org/mod_mbox/incubator-qpid-dev/200612.mbox/%3c8764cm832c.fsf@rho.meyering.net%3e
I'm taking a look now...
Yep. It's similar.
Since "make check" builds both $(TESTS) and $(noinst_PROGRAMS),
you end up with two invocations of libtool (ltmain) for e.g.,
client_test. Then, since the two invocations step on each others'
temporary files, one of them fails.
Since this is rather subtle, I think it's more maintainable
simply to turn off parallelism in the src/tests/ directory.
Here's the change I've just checked in:
2007-04-11 Jim Meyering <ji...@meyering.net>
* src/tests/Makefile.am: Use .NOTPARALLEL target to suppress
parallelism in this directory, even when make is invoked with
e.g., -j2.
Index: src/tests/Makefile.am
===================================================================
--- src/tests/Makefile.am (révision 527651)
+++ src/tests/Makefile.am (copie de travail)
@@ -121,3 +121,7 @@
.valgrindrc: .valgrindrc-default
cp $(srcdir)/.valgrindrc-default .valgrindrc
+# Tell GNU make not to build targets in this directory in parallel.
+# This is necessary because with two or more identical and simultaneous
+# ltmain invocations, one may corrupt the temporaries of the other.
+.NOTPARALLEL:
Re: c++: why does parallel build fail for make check?
Posted by Jim Meyering <ji...@meyering.net>.
FYI, I've also remove all trailing blanks in src/tests/Makefile.am.