You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Sergei Trofimovich <sl...@gmail.com> on 2022/06/23 07:49:36 UTC
'make -j install' fails for missing depends
Hi subversion developers!
I'm working on fixing parallel build (or install) faliures in upstream
packages used in NixOS linux distribution. Mostly to speed package builds
up on continuous build farm.
That usually means that
$ ./configure && make && make install
is changed to
$ ./configure && make -j$(nproc) && make -j$(nproc) install
To make the error more obvioud GNU make recently added --shuffle option
to better expose missing dependencies across Makefile targets:
https://savannah.gnu.org/bugs/index.php?62100
'subversion-1.14.2' came as one of the packages that sometimes fails on
'make install'
There is a build log snippet that exposed missing install dependencies:
$ ./configure --disable-static --prefix=/<<NIX>>/subversion-1.14.2 --bindir=/<<NIX>>/subversion-1.14.2/bin --sbindir=/<<NIX>>/subversion-1.14.2/sbin --includedir=/<<NIX>>/subversion-1.14.2-dev/include --oldincludedir=/<<NIX>>/subversion-1.14.2-dev/include --mandir=/<<NIX>>/subversion-1.14.2-man/share/man --infodir=/<<NIX>>/subversion-1.14.2/share/info --docdir=/<<NIX>>/subversion-1.14.2/share/doc/subversion --libdir=/<<NIX>>/subversion-1.14.2/lib --libexecdir=/<<NIX>>/subversion-1.14.2/libexec --localedir=/<<NIX>>/subversion-1.14.2/share/locale --with-berkeley-db --without-apxs --without-swig --without-sasl --with-serf=/<<NIX>>/serf-1.3.9 --with-zlib=/<<NIX>>/zlib-1.2.12-dev --with-sqlite=/<<NIX>>/sqlite-3.38.5-dev --build=x86_64-unknown-linux-gnu --host=x86_64-unknown-linux-gnu
...
$ make -j16 -l16 SHELL=/<<NIX>>/bash-5.1-p16/bin/bash APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules
...
$ make --shuffle APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules pkgconfigdir=/<<NIX>>/subversion-1.14.2-dev/lib/pkgconfig m4datadir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal aclocaldir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal install
test -d /<<NIX>>/subversion-1.14.2-dev/include/subversion-1 || \
/<<NIX>>/coreutils-9.1/bin/install -c -d /<<NIX>>/subversion-1.14.2-dev/include/subversion-1
(subversion/svnversion/svnversion . 2> /dev/null || \
svnversion . 2> /dev/null || \
echo "unknown"; \
) > /<<NIX>>/subversion-1.14.2-dev/include/subversion-1/svn-revision.txt
/<<NIX>>/coreutils-9.1/bin/install -c -d /<<NIX>>/subversion-1.14.2/lib /<<NIX>>/subversion-1.14.2/share/pkgconfig
cd subversion/libsvn_fs_base ; /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --mode=install /<<NIX>>/coreutils-9.1/bin/install -c libsvn_fs_base-1.la /<<NIX>>/subversion-1.14.2/lib/libsvn_fs_base-1.la
libtool: warning: relinking 'libsvn_fs_base-1.la'
libtool: install: (cd /build/subversion/subversion/libsvn_fs_base; /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2 -L/<<NIX>>/openssl-1.1.1o/lib -L/<<NIX>>/db-5.3.28/lib -L/<<NIX>>/expat-2.4.8/lib -L/<<NIX>>/serf-1.3.9/lib -rpath /<<NIX>>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo bdb/changes-table.lo bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo bdb/rev-table.lo bdb/strings-table.lo bdb/txn-table.lo bdb/uuids-table.lo dag.lo err.lo fs.lo id.lo key-gen.lo lock.lo node-rev.lo reps-strings.lo revs-txns.lo trail.lo tree.lo util/fs_skels.lo uuid.lo ../../subversion/libsvn_delta/libsvn_delta-1.la ../../subversion/libsvn_subr/libsvn_subr-1.la -L/<<NIX>>/apr-util-1.6.1/lib -laprutil-1 -L/<<NIX>>/apr-1.7.0/lib -lapr-1 -ldb-5.3 ../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
/<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No such file or directory
/<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_fs_util-1: No such file or directory
collect2: error: ld returned 1 exit status
libtool: error: error: relink 'libsvn_fs_base-1.la' with the above command before installing it
make: *** [build-outputs.mk:1422: install-bdb-lib] Error 1 shuffle=1656015674
The failure as I understand it:
'install-bdb-lib' install target assumes that '-lsvn_delta-1' and '-lsvn_fs_util-1'
are already installed into 'DESTDIR'. But they are not. 'libtool' fails to relink
'libsvn_fs_base' against 'DESTDIR'.
I think the dependencies usually happen to be present if 'install-fsmod-lib' target
was lucky to be executed first. But nothing in 'Makefile.in' or 'build-outputs.mk'
seems to have an equivalent of dependency like:
# hypothetical fix
install-bdb-lib : install-fsmod-lib
It looks like 'build.conf' does contain some build dependency information.
I also see special cases like fs-lib to handle some of install deps:
# manual
Makefile.in:SVN_FS_LIB_INSTALL_DEPS = @SVN_FS_LIB_INSTALL_DEPS@
# generated:
build-outputs.mk:install-ramod-lib: $(SVN_FS_LIB_INSTALL_DEPS)
Would it make sense to add 'install-bdb-lib : install-fsmod-lib' dependency explicitly?
Thank you!
--
Sergei
Re: 'make -j install' fails for missing depends
Posted by Branko Čibej <br...@apache.org>.
On 24.06.2022 22:21, Nathan Hartman wrote:
> On Thu, Jun 23, 2022 at 5:58 AM Sergei Trofimovich <sl...@gmail.com> wrote:
>> The failure as I understand it:
>>
>> 'install-bdb-lib' install target assumes that '-lsvn_delta-1' and '-lsvn_fs_util-1'
>> are already installed into 'DESTDIR'. But they are not. 'libtool' fails to relink
>> 'libsvn_fs_base' against 'DESTDIR'.
>>
>> I think the dependencies usually happen to be present if 'install-fsmod-lib' target
>> was lucky to be executed first. But nothing in 'Makefile.in' or 'build-outputs.mk'
>> seems to have an equivalent of dependency like:
>>
>> # hypothetical fix
>> install-bdb-lib : install-fsmod-lib
> I've filed this as issue #4901 in our bugtracker:
> https://issues.apache.org/jira/browse/SVN-4901
This needs to be fixed in build.conf and the build generator, not in
Makefile.in.
-- Brane
Re: 'make -j install' fails for missing depends
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Jun 23, 2022 at 5:58 AM Sergei Trofimovich <sl...@gmail.com> wrote:
> The failure as I understand it:
>
> 'install-bdb-lib' install target assumes that '-lsvn_delta-1' and '-lsvn_fs_util-1'
> are already installed into 'DESTDIR'. But they are not. 'libtool' fails to relink
> 'libsvn_fs_base' against 'DESTDIR'.
>
> I think the dependencies usually happen to be present if 'install-fsmod-lib' target
> was lucky to be executed first. But nothing in 'Makefile.in' or 'build-outputs.mk'
> seems to have an equivalent of dependency like:
>
> # hypothetical fix
> install-bdb-lib : install-fsmod-lib
I've filed this as issue #4901 in our bugtracker:
https://issues.apache.org/jira/browse/SVN-4901
Cheers,
Nathan
Re: 'make -j install' fails for missing depends
Posted by Daniel Sahlberg <da...@gmail.com>.
Den fre 24 juni 2022 kl 21:15 skrev Nathan Hartman <hartman.nathan@gmail.com
>:
> If so, I don't currently have a (convenient) setup to experiment with
> this at the moment; are you able to just hack your hypothetical fix
> into your SVN makefile, i.e., `install-bdb-lib : install-fsmod-lib`,
> and run make with shuffle=1656015674 and tell us if that appears to
> fix it? Then I can go digging to see where to add it correctly.
>
Reproduction, without --shuffle, running on Ubuntu 21.10 (under WSL, with
all available updates):
$ make distclean
$ ./autogen.sh
# Obviously, replace --prefix with something usable on your system!
$ ./configure --with-berkeley-db --prefix=/home/dsg/4901
$ make install-bdb-lib
# Output:
# /usr/bin/install -c -d /home/dsg/4901/lib /home/dsg/4901/share/pkgconfig
# cd subversion/libsvn_fs_base ; /bin/bash "/home/dsg/svn_4901/libtool"
--mode=install /usr/bin/install -c libsvn_fs_base-1.la /home/dsg/4901/lib/
libsvn_fs_base-1.la
# libtool: warning: relinking 'libsvn_fs_base-1.la'
# libtool: install: (cd /home/dsg/svn_4901/subversion/libsvn_fs_base;
/bin/bash "/home/dsg/svn_4901/libtool" --tag CC --silent --mode=relink gcc
-g -O2 -rpath /home/dsg/4901/lib -version-info 0 -Wl,--no-undefined -o
libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo bdb/changes-table.lo
bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo
bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo
bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo
bdb/rev-table.lo bdb/strings-table.lo bdb/txn-table.lo bdb/uuids-table.lo
dag.lo err.lo fs.lo id.lo key-gen.lo lock.lo node-rev.lo reps-strings.lo
revs-txns.lo trail.lo tree.lo util/fs_skels.lo uuid.lo
../../subversion/libsvn_delta/libsvn_delta-1.la
../../subversion/libsvn_subr/libsvn_subr-1.la -L/usr/lib/x86_64-linux-gnu
-laprutil-1 -L/usr/lib/x86_64-linux-gnu -lapr-1 -ldb-5.3
../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
# /usr/bin/ld: cannot find -lsvn_fs_util-1
# collect2: error: ld returned 1 exit status
# libtool: error: error: relink 'libsvn_fs_base-1.la' with the above
command before installing it
# make: *** [build-outputs.mk:1422: install-bdb-lib] Error 1
# Fix; run install-fsmod-lib first:
$ make install-fsmod-lib
$ make install-bdb-lib
# Installs correctly
The following change to build.conf seems to fix it by adding an install
dependency (similar to the ones already existing):
[[[
Index: build.conf
===================================================================
--- build.conf (revision 1902243)
+++ build.conf (working copy)
@@ -288,6 +288,7 @@
sources = *.c bdb/*.c util/*.c
install = bdb-lib
libs = libsvn_delta libsvn_subr aprutil apriconv apr bdb libsvn_fs_util
+add-install-deps = $(SVN_FS_LIB_INSTALL_DEPS)
msvc-export = ../libsvn_fs_base/fs_init.h
msvc-delayload = yes
]]]
I must admit I'm not really sure I understand the build system, so I'm not
committing this myself without review.
Kind regards,
Daniel
Re: 'make -j install' fails for missing depends
Posted by Daniel Sahlberg <da...@gmail.com>.
Den tors 30 juni 2022 kl 00:29 skrev Sergei Trofimovich <sl...@gmail.com>:
> On Wed, 29 Jun 2022 09:16:17 +0200
> Daniel Sahlberg <da...@gmail.com> wrote:
>
> > Den lör 25 juni 2022 kl 00:41 skrev Sergei Trofimovich <slyich@gmail.com
> >:
> >
> > > On Fri, 24 Jun 2022 15:14:23 -0400
> > > Nathan Hartman <ha...@gmail.com> wrote:
> > >
> > > > On Thu, Jun 23, 2022 at 5:58 AM Sergei Trofimovich <sl...@gmail.com>
>
> > > wrote:
> > > > >
> > > > > Hi subversion developers!
> > > > >
> > > > > I'm working on fixing parallel build (or install) faliures in
> upstream
> > > > > packages used in NixOS linux distribution. Mostly to speed
> package
> > > builds
> > > > > up on continuous build farm.
> > > > >
> > > > > That usually means that
> > > > > $ ./configure && make && make install
> > > > > is changed to
> > > > > $ ./configure && make -j$(nproc) && make -j$(nproc) install
> > > > >
> > > > > To make the error more obvioud GNU make recently added --shuffle
> option
> > > > > to better expose missing dependencies across Makefile targets:
> > > > > https://savannah.gnu.org/bugs/index.php?62100
> > > > >
> > > > > 'subversion-1.14.2' came as one of the packages that sometimes
> fails on
> > > > > 'make install'
> > > > >
> > > > > There is a build log snippet that exposed missing install
> dependencies:
> > > > >
> > > > > $ ./configure --disable-static
> --prefix=/<<NIX>>/subversion-1.14.2
> > > --bindir=/<<NIX>>/subversion-1.14.2/bin
> > > --sbindir=/<<NIX>>/subversion-1.14.2/sbin
> > > --includedir=/<<NIX>>/subversion-1.14.2-dev/include
> > > --oldincludedir=/<<NIX>>/subversion-1.14.2-dev/include
> > > --mandir=/<<NIX>>/subversion-1.14.2-man/share/man
> > > --infodir=/<<NIX>>/subversion-1.14.2/share/info
> > > --docdir=/<<NIX>>/subversion-1.14.2/share/doc/subversion
> > > --libdir=/<<NIX>>/subversion-1.14.2/lib
> > > --libexecdir=/<<NIX>>/subversion-1.14.2/libexec
> > > --localedir=/<<NIX>>/subversion-1.14.2/share/locale --with-berkeley-db
> > > --without-apxs --without-swig --without-sasl
> > > --with-serf=/<<NIX>>/serf-1.3.9 --with-zlib=/<<NIX>>/zlib-1.2.12-dev
> > > --with-sqlite=/<<NIX>>/sqlite-3.38.5-dev
> --build=x86_64-unknown-linux-gnu
> > > --host=x86_64-unknown-linux-gnu
> > > > > ...
> > > > > $ make -j16 -l16 SHELL=/<<NIX>>/bash-5.1-p16/bin/bash
> > > APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules
> > > > > ...
> > > > > $ make --shuffle
> > > APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules
> > > pkgconfigdir=/<<NIX>>/subversion-1.14.2-dev/lib/pkgconfig
> > > m4datadir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal
> > > aclocaldir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal install
> > > > > test -d /<<NIX>>/subversion-1.14.2-dev/include/subversion-1 ||
> \
> > > > > /<<NIX>>/coreutils-9.1/bin/install -c -d
> > > /<<NIX>>/subversion-1.14.2-dev/include/subversion-1
> > > > > (subversion/svnversion/svnversion . 2> /dev/null || \
> > > > > svnversion . 2> /dev/null || \
> > > > > echo "unknown";
> \
> > > > > ) >
> > > /<<NIX>>/subversion-1.14.2-dev/include/subversion-1/svn-revision.txt
> > > > > /<<NIX>>/coreutils-9.1/bin/install -c -d
> > > /<<NIX>>/subversion-1.14.2/lib
> /<<NIX>>/subversion-1.14.2/share/pkgconfig
> > > > > cd subversion/libsvn_fs_base ; /<<NIX>>/bash-5.1-p16/bin/bash
> > > "/build/subversion/libtool" --mode=install
> > > /<<NIX>>/coreutils-9.1/bin/install -c libsvn_fs_base-1.la
> > > /<<NIX>>/subversion-1.14.2/lib/libsvn_fs_base-1.la
> > > > > libtool: warning: relinking 'libsvn_fs_base-1.la'
> > > > > libtool: install: (cd
> /build/subversion/subversion/libsvn_fs_base;
> > > /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --tag CC
> > > --silent --mode=relink gcc -shared -g -O2 -g -O2
> > > -L/<<NIX>>/openssl-1.1.1o/lib -L/<<NIX>>/db-5.3.28/lib
> > > -L/<<NIX>>/expat-2.4.8/lib -L/<<NIX>>/serf-1.3.9/lib -rpath
> > > /<<NIX>>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o
> > > libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo
> bdb/changes-table.lo
> > > bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo
> > > bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo
> > > bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo
> > > bdb/rev-table.lo bdb/strings-table.lo bdb/txn-table.lo
> bdb/uuids-table.lo
> > > dag.lo err.lo fs.lo id.lo key-gen.lo lock.lo node-rev.lo
> reps-strings.lo
> > > revs-txns.lo trail.lo tree.lo util/fs_skels.lo uuid.lo
> > > ../../subversion/libsvn_delta/libsvn_delta-1.la
> > > ../../subversion/libsvn_subr/libsvn_subr-1.la
> > > -L/<<NIX>>/apr-util-1.6.1/lib -laprutil-1 -L/<<N
> > > IX>>/apr-1.7.0/lib -lapr-1 -ldb-5.3 ../../subversion/libsvn_fs_util/
> > > libsvn_fs_util-1.la )
> > > > > /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No
> such
> > > file or directory
> > > > > /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_fs_util-1:
> No
> > > such file or directory
> > > > > collect2: error: ld returned 1 exit status
> > > > > libtool: error: error: relink 'libsvn_fs_base-1.la' with
> the
> > > above command before installing it
> > > > > make: *** [build-outputs.mk:1422: install-bdb-lib] Error 1
> > > shuffle=1656015674
> > > > >
> > > > > The failure as I understand it:
> > > > >
> > > > > 'install-bdb-lib' install target assumes that '-lsvn_delta-1' and
> > > '-lsvn_fs_util-1'
> > > > > are already installed into 'DESTDIR'. But they are not. 'libtool'
> > > fails to relink
> > > > > 'libsvn_fs_base' against 'DESTDIR'.
> > > > >
> > > > > I think the dependencies usually happen to be present if
> > > 'install-fsmod-lib' target
> > > > > was lucky to be executed first. But nothing in 'Makefile.in' or '
> > > build-outputs.mk'
> > > > > seems to have an equivalent of dependency like:
> > > > >
> > > > > # hypothetical fix
> > > > > install-bdb-lib : install-fsmod-lib
> > > > >
> > > > > It looks like 'build.conf' does contain some build dependency
> > > information.
> > > > >
> > > > > I also see special cases like fs-lib to handle some of install
> deps:
> > > > >
> > > > > # manual
> > > > > Makefile.in:SVN_FS_LIB_INSTALL_DEPS = @SVN_FS_LIB_INSTALL_DEPS@
> > > > >
> > > > > # generated:
> > > > > build-outputs.mk:install-ramod-lib: $(SVN_FS_LIB_INSTALL_DEPS)
> > > > >
> > > > > Would it make sense to add 'install-bdb-lib : install-fsmod-lib'
> > > dependency explicitly?
> > > > >
> > > > > Thank you!
> > > > >
> > > > > --
> > > > >
> > > > > Sergei
> > > >
> > > >
> > > > Hi Sergei,
> > > >
> > > > This is a very useful feature for GNU Make to have! This is the first
> > > > I've heard of it and it should allow many, many projects, not just
> > > > Subversion, to fix these kinds of pesky issues in parallel builds.
> > > >
> > > > It seems like this landed in Make literally a few days ago so isn't
> in
> > > > any released version. Is that correct?
> > >
> > > That is correct: there are no releases or snapshots of GNU make with a
> > > feature yet. I'm using locally built make from tip of the git tree.
> > >
> > > > If so, I don't currently have a (convenient) setup to experiment with
> > > > this at the moment; are you able to just hack your hypothetical fix
> > > > into your SVN makefile, i.e., `install-bdb-lib : install-fsmod-lib`,
> > > > and run make with shuffle=1656015674 and tell us if that appears to
> > > > fix it? Then I can go digging to see where to add it correctly.
> > >
> > > Before the hack my installs were failing 60% of the time (6 of 10
> attmpts
> > > failed). After the change below 100 of 100 install succeeded.
> > >
> > > I had to add two depdendencies to get there:
> > >
> > > install-bdb-lib: install-fsmod-lib
> > > install-serf-lib: install-fsmod-lib
> > >
> > > The change itself (with linkage errors they fix):
> > >
> > > $ svn diff
> > > Index: Makefile.in
> > > ===================================================================
> > > --- Makefile.in (revision 1902174)
> > > +++ Makefile.in (working copy)
> > > @@ -431,6 +431,36 @@
> > >
> > > @INCLUDE_OUTPUTS@
> > >
> > > +# A hack to test https://issues.apache.org/jira/browse/SVN-4901:
> > > +# orkaround missing dependencies on installing libtool libraries
> > > +# that depend on other libraries:
> > > +#
> > > +## case 1: fs_base depend on fs_util and delta:
> > > +#
> > > +# libtool: install: (cd /build/subversion/subversion/libsvn_fs_base;
> bash
> > > "/build/subversion/libtool" \
> > > +# --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2
> > > -L...-openssl-1.1.1p/lib \
> > > +# -L...-db-5.3.28/lib -L...-expat-2.4.8/lib -L...-serf-1.3.9/lib
> > > -rpath ...-subversion-1.14.2/lib \
> > > +# -version-info 0 -Wl,--no-undefined \
> > > +# -o libsvn_fs_base-1.la bdb/bdb-err.lo \
> > > +# ... ../../subversion/libsvn_delta/libsvn_delta-1.la
> > > ../../subversion/libsvn_subr/libsvn_subr-1.la \
> > > +# -L...-apr-util-1.6.1/lib -laprutil-1 -L...-apr-1.7.0/lib
> -lapr-1
> > > -ldb-5.3 \
> > > +# ../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
> > > +# ld: cannot find -lsvn_delta-1: No such file or directory
> > > +# ld: cannot find -lsvn_fs_util-1: No such file or directory
> > > +install-bdb-lib: install-fsmod-lib
> > > +#
> > > +## case 2: ra-serf depends on delta:
> > > +#
> > > +# libtool: install: (cd /build/subversion/subversion/libsvn_ra_serf;
> bash
> > > "/build/subversion/libtool" \
> > > +# --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2
> > > -L...-openssl-1.1.1p/lib \
> > > +# -L...-db-5.3.28/lib -L...-expat-2.4.8/lib -L...-serf-1.3.9/lib
> > > -rpath ...-subversion-1.14.2/lib \
> > > +# -version-info 0 -Wl,--no-undefined
> > > +# -o libsvn_ra_serf-1.la blame.lo \
> > > +# ... xml.lo ../../subversion/libsvn_delta/libsvn_delta-1.la
> > > ../../subversion/libsvn_subr/libsvn_subr-1.la \
> > > +# -L...-apr-util-1.6.1/lib -laprutil-1 -L...-apr-1.7.0/lib
> -lapr-1
> > > -lserf-1 -L...-zlib-1.2.12-dev/lib -lz )
> > > +# ld: cannot find -lsvn_delta-1: No such file or directory
> > > +install-serf-lib: install-fsmod-lib
> > > +
> > > local-all: @BUILD_RULES@ @TRANSFORM_LIBTOOL_SCRIPTS@
> > >
> > > transform-libtool-scripts: @BUILD_RULES@
> > >
> >
> > I have committed a fix in r1902335 adding install-fsmod-lib as a
> depencency
> > for install-bdb-lib.
> >
> > It was tested by Nathan and I believe it is correct, but I must admit I
> > have almost no experience in the build system.
> >
> > Sergei, can you make some tests on /trunk?
>
> Sure! Ran a few builds against current trunk. I don't see bdb-related
> failures any more \o/.
>
> But I still do see libsvn_ra_serf related failures:
>
> /<<NIX>>/coreutils-9.1/bin/install -c -d /<<NIX>>/subversion-1.14.2/lib
> /<<NIX>>/subversion-1.14.2/share/pkgconfig
> cd subversion/libsvn_ra_serf ; /<<NIX>>/bash-5.1-p16/bin/bash
> "/build/subversion/libtool" --mode=install
> /<<NIX>>/coreutils-9.1/bin/install -c libsvn_ra_serf-1.la
> /<<NIX>>/subversion-1.14.2/lib/libsvn_ra_serf-1.la
> libtool: warning: relinking 'libsvn_ra_serf-1.la'
> libtool: install: (cd /build/subversion/subversion/libsvn_ra_serf;
> /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --tag CC
> --silent --mode=relink gcc -shared -g -O2 -g -O2
> -L/<<NIX>>/openssl-1.1.1p/lib -L/<<NIX>>/db-5.3.28/lib
> -L/<<NIX>>/expat-2.4.8/lib -L/<<NIX>>/serf-1.3.9/lib -rpath
> /<<NIX>>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o
> libsvn_ra_serf-1.la blame.lo blncache.lo commit.lo eagain_bucket.lo
> get_deleted_rev.lo get_file.lo get_lock.lo getdate.lo getlocations.lo
> getlocationsegments.lo getlocks.lo inherited_props.lo list.lo lock.lo
> log.lo merge.lo mergeinfo.lo multistatus.lo options.lo property.lo
> replay.lo request_body.lo sb_bucket.lo serf.lo stat.lo stream_bucket.lo
> update.lo util.lo util_error.lo xml.lo ../../subversion/libsvn_delta/
> libsvn_delta-1.la ../../subversion/libsvn_subr/libsvn_subr-1.la
> -L/<<NIX>>/apr-util-1.6.1/lib -laprutil-1 -L/<<NIX>>/apr-1.7.0/lib -lapr-1
> -lserf-1 -L/<<NIX>>/zlib-1.2.12-dev/lib -lz )
> /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No such file or
> directory
> collect2: error: ld returned 1 exit status
> libtool: error: error: relink 'libsvn_ra_serf-1.la' with the above
> command before installing it
>
>
> Adding 'install-serf-lib: install-fsmod-lib' seems to work it around.
> I also tried the fix below on top of trunk. It fixes install failures for
> me as well:
>
> --- build.conf (revision 1902359)
> +++ build.conf (working copy)
> @@ -340,6 +340,7 @@
> path = subversion/libsvn_ra_serf
> install = serf-lib
> libs = libsvn_delta libsvn_subr aprutil apriconv apr serf zlib
> +add-install-deps = $(SVN_FS_LIB_INSTALL_DEPS)
> msvc-static = yes
>
> # Accessing repositories via SVN
>
> --
>
> Sergei
>
Thanks Sergei, both for testing the original fix and for investigating this
new one. I've checked and, for the benefit of the list, adding my tests:
# Reproduction recipe:
$ make distclean
$ ./autogen.sh
$ ./configure --prefix=$PWD/../4901
$ make install-serf-lib
[...]
libtool: warning: relinking 'libsvn_ra_serf-1.la'
libtool: install: (cd /home/dsg/svn_trunk/subversion/libsvn_ra_serf;
/bin/bash "/home/dsg/svn_trunk/libtool" --tag CC --silent --mode=relink
gcc -g -O2 -rpath /home/dsg/4901/lib -version-info 0 -Wl,--no-undefined -o
libsvn_ra_serf-1.la blame.lo blncache.lo commit.lo eagain_bucket.lo
get_deleted_rev.lo get_file.lo get_lock.lo getdate.lo getlocations.lo
getlocationsegments.lo getlocks.lo inherited_props.lo list.lo lock.lo
log.lo merge.lo mergeinfo.lo multistatus.lo options.lo property.lo
replay.lo request_body.lo sb_bucket.lo serf.lo stat.lo stream_bucket.lo
update.lo util.lo util_error.lo xml.lo ../../subversion/libsvn_delta/
libsvn_delta-1.la ../../subversion/libsvn_subr/libsvn_subr-1.la
-L/usr/lib/x86_64-linux-gnu -laprutil-1 -L/usr/lib/x86_64-linux-gnu -lapr-1
-lserf-1 -lz )
/usr/bin/ld: cannot find -lsvn_delta-1
/usr/bin/ld: cannot find -lsvn_subr-1
[...]
# The following succeeds
$ make install-fsmod-lib
$ make install-serf-lib
I've checked and verified you patch and it seems correct. Committed as
r1902364.
Kind regards,
Daniel
Re: 'make -j install' fails for missing depends
Posted by Sergei Trofimovich <sl...@gmail.com>.
On Wed, 29 Jun 2022 09:16:17 +0200
Daniel Sahlberg <da...@gmail.com> wrote:
> Den lör 25 juni 2022 kl 00:41 skrev Sergei Trofimovich <sl...@gmail.com>:
>
> > On Fri, 24 Jun 2022 15:14:23 -0400
> > Nathan Hartman <ha...@gmail.com> wrote:
> >
> > > On Thu, Jun 23, 2022 at 5:58 AM Sergei Trofimovich <sl...@gmail.com>
> > wrote:
> > > >
> > > > Hi subversion developers!
> > > >
> > > > I'm working on fixing parallel build (or install) faliures in upstream
> > > > packages used in NixOS linux distribution. Mostly to speed package
> > builds
> > > > up on continuous build farm.
> > > >
> > > > That usually means that
> > > > $ ./configure && make && make install
> > > > is changed to
> > > > $ ./configure && make -j$(nproc) && make -j$(nproc) install
> > > >
> > > > To make the error more obvioud GNU make recently added --shuffle option
> > > > to better expose missing dependencies across Makefile targets:
> > > > https://savannah.gnu.org/bugs/index.php?62100
> > > >
> > > > 'subversion-1.14.2' came as one of the packages that sometimes fails on
> > > > 'make install'
> > > >
> > > > There is a build log snippet that exposed missing install dependencies:
> > > >
> > > > $ ./configure --disable-static --prefix=/<<NIX>>/subversion-1.14.2
> > --bindir=/<<NIX>>/subversion-1.14.2/bin
> > --sbindir=/<<NIX>>/subversion-1.14.2/sbin
> > --includedir=/<<NIX>>/subversion-1.14.2-dev/include
> > --oldincludedir=/<<NIX>>/subversion-1.14.2-dev/include
> > --mandir=/<<NIX>>/subversion-1.14.2-man/share/man
> > --infodir=/<<NIX>>/subversion-1.14.2/share/info
> > --docdir=/<<NIX>>/subversion-1.14.2/share/doc/subversion
> > --libdir=/<<NIX>>/subversion-1.14.2/lib
> > --libexecdir=/<<NIX>>/subversion-1.14.2/libexec
> > --localedir=/<<NIX>>/subversion-1.14.2/share/locale --with-berkeley-db
> > --without-apxs --without-swig --without-sasl
> > --with-serf=/<<NIX>>/serf-1.3.9 --with-zlib=/<<NIX>>/zlib-1.2.12-dev
> > --with-sqlite=/<<NIX>>/sqlite-3.38.5-dev --build=x86_64-unknown-linux-gnu
> > --host=x86_64-unknown-linux-gnu
> > > > ...
> > > > $ make -j16 -l16 SHELL=/<<NIX>>/bash-5.1-p16/bin/bash
> > APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules
> > > > ...
> > > > $ make --shuffle
> > APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules
> > pkgconfigdir=/<<NIX>>/subversion-1.14.2-dev/lib/pkgconfig
> > m4datadir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal
> > aclocaldir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal install
> > > > test -d /<<NIX>>/subversion-1.14.2-dev/include/subversion-1 || \
> > > > /<<NIX>>/coreutils-9.1/bin/install -c -d
> > /<<NIX>>/subversion-1.14.2-dev/include/subversion-1
> > > > (subversion/svnversion/svnversion . 2> /dev/null || \
> > > > svnversion . 2> /dev/null || \
> > > > echo "unknown"; \
> > > > ) >
> > /<<NIX>>/subversion-1.14.2-dev/include/subversion-1/svn-revision.txt
> > > > /<<NIX>>/coreutils-9.1/bin/install -c -d
> > /<<NIX>>/subversion-1.14.2/lib /<<NIX>>/subversion-1.14.2/share/pkgconfig
> > > > cd subversion/libsvn_fs_base ; /<<NIX>>/bash-5.1-p16/bin/bash
> > "/build/subversion/libtool" --mode=install
> > /<<NIX>>/coreutils-9.1/bin/install -c libsvn_fs_base-1.la
> > /<<NIX>>/subversion-1.14.2/lib/libsvn_fs_base-1.la
> > > > libtool: warning: relinking 'libsvn_fs_base-1.la'
> > > > libtool: install: (cd /build/subversion/subversion/libsvn_fs_base;
> > /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --tag CC
> > --silent --mode=relink gcc -shared -g -O2 -g -O2
> > -L/<<NIX>>/openssl-1.1.1o/lib -L/<<NIX>>/db-5.3.28/lib
> > -L/<<NIX>>/expat-2.4.8/lib -L/<<NIX>>/serf-1.3.9/lib -rpath
> > /<<NIX>>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o
> > libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo bdb/changes-table.lo
> > bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo
> > bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo
> > bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo
> > bdb/rev-table.lo bdb/strings-table.lo bdb/txn-table.lo bdb/uuids-table.lo
> > dag.lo err.lo fs.lo id.lo key-gen.lo lock.lo node-rev.lo reps-strings.lo
> > revs-txns.lo trail.lo tree.lo util/fs_skels.lo uuid.lo
> > ../../subversion/libsvn_delta/libsvn_delta-1.la
> > ../../subversion/libsvn_subr/libsvn_subr-1.la
> > -L/<<NIX>>/apr-util-1.6.1/lib -laprutil-1 -L/<<N
> > IX>>/apr-1.7.0/lib -lapr-1 -ldb-5.3 ../../subversion/libsvn_fs_util/
> > libsvn_fs_util-1.la )
> > > > /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No such
> > file or directory
> > > > /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_fs_util-1: No
> > such file or directory
> > > > collect2: error: ld returned 1 exit status
> > > > libtool: error: error: relink 'libsvn_fs_base-1.la' with the
> > above command before installing it
> > > > make: *** [build-outputs.mk:1422: install-bdb-lib] Error 1
> > shuffle=1656015674
> > > >
> > > > The failure as I understand it:
> > > >
> > > > 'install-bdb-lib' install target assumes that '-lsvn_delta-1' and
> > '-lsvn_fs_util-1'
> > > > are already installed into 'DESTDIR'. But they are not. 'libtool'
> > fails to relink
> > > > 'libsvn_fs_base' against 'DESTDIR'.
> > > >
> > > > I think the dependencies usually happen to be present if
> > 'install-fsmod-lib' target
> > > > was lucky to be executed first. But nothing in 'Makefile.in' or '
> > build-outputs.mk'
> > > > seems to have an equivalent of dependency like:
> > > >
> > > > # hypothetical fix
> > > > install-bdb-lib : install-fsmod-lib
> > > >
> > > > It looks like 'build.conf' does contain some build dependency
> > information.
> > > >
> > > > I also see special cases like fs-lib to handle some of install deps:
> > > >
> > > > # manual
> > > > Makefile.in:SVN_FS_LIB_INSTALL_DEPS = @SVN_FS_LIB_INSTALL_DEPS@
> > > >
> > > > # generated:
> > > > build-outputs.mk:install-ramod-lib: $(SVN_FS_LIB_INSTALL_DEPS)
> > > >
> > > > Would it make sense to add 'install-bdb-lib : install-fsmod-lib'
> > dependency explicitly?
> > > >
> > > > Thank you!
> > > >
> > > > --
> > > >
> > > > Sergei
> > >
> > >
> > > Hi Sergei,
> > >
> > > This is a very useful feature for GNU Make to have! This is the first
> > > I've heard of it and it should allow many, many projects, not just
> > > Subversion, to fix these kinds of pesky issues in parallel builds.
> > >
> > > It seems like this landed in Make literally a few days ago so isn't in
> > > any released version. Is that correct?
> >
> > That is correct: there are no releases or snapshots of GNU make with a
> > feature yet. I'm using locally built make from tip of the git tree.
> >
> > > If so, I don't currently have a (convenient) setup to experiment with
> > > this at the moment; are you able to just hack your hypothetical fix
> > > into your SVN makefile, i.e., `install-bdb-lib : install-fsmod-lib`,
> > > and run make with shuffle=1656015674 and tell us if that appears to
> > > fix it? Then I can go digging to see where to add it correctly.
> >
> > Before the hack my installs were failing 60% of the time (6 of 10 attmpts
> > failed). After the change below 100 of 100 install succeeded.
> >
> > I had to add two depdendencies to get there:
> >
> > install-bdb-lib: install-fsmod-lib
> > install-serf-lib: install-fsmod-lib
> >
> > The change itself (with linkage errors they fix):
> >
> > $ svn diff
> > Index: Makefile.in
> > ===================================================================
> > --- Makefile.in (revision 1902174)
> > +++ Makefile.in (working copy)
> > @@ -431,6 +431,36 @@
> >
> > @INCLUDE_OUTPUTS@
> >
> > +# A hack to test https://issues.apache.org/jira/browse/SVN-4901:
> > +# orkaround missing dependencies on installing libtool libraries
> > +# that depend on other libraries:
> > +#
> > +## case 1: fs_base depend on fs_util and delta:
> > +#
> > +# libtool: install: (cd /build/subversion/subversion/libsvn_fs_base; bash
> > "/build/subversion/libtool" \
> > +# --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2
> > -L...-openssl-1.1.1p/lib \
> > +# -L...-db-5.3.28/lib -L...-expat-2.4.8/lib -L...-serf-1.3.9/lib
> > -rpath ...-subversion-1.14.2/lib \
> > +# -version-info 0 -Wl,--no-undefined \
> > +# -o libsvn_fs_base-1.la bdb/bdb-err.lo \
> > +# ... ../../subversion/libsvn_delta/libsvn_delta-1.la
> > ../../subversion/libsvn_subr/libsvn_subr-1.la \
> > +# -L...-apr-util-1.6.1/lib -laprutil-1 -L...-apr-1.7.0/lib -lapr-1
> > -ldb-5.3 \
> > +# ../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
> > +# ld: cannot find -lsvn_delta-1: No such file or directory
> > +# ld: cannot find -lsvn_fs_util-1: No such file or directory
> > +install-bdb-lib: install-fsmod-lib
> > +#
> > +## case 2: ra-serf depends on delta:
> > +#
> > +# libtool: install: (cd /build/subversion/subversion/libsvn_ra_serf; bash
> > "/build/subversion/libtool" \
> > +# --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2
> > -L...-openssl-1.1.1p/lib \
> > +# -L...-db-5.3.28/lib -L...-expat-2.4.8/lib -L...-serf-1.3.9/lib
> > -rpath ...-subversion-1.14.2/lib \
> > +# -version-info 0 -Wl,--no-undefined
> > +# -o libsvn_ra_serf-1.la blame.lo \
> > +# ... xml.lo ../../subversion/libsvn_delta/libsvn_delta-1.la
> > ../../subversion/libsvn_subr/libsvn_subr-1.la \
> > +# -L...-apr-util-1.6.1/lib -laprutil-1 -L...-apr-1.7.0/lib -lapr-1
> > -lserf-1 -L...-zlib-1.2.12-dev/lib -lz )
> > +# ld: cannot find -lsvn_delta-1: No such file or directory
> > +install-serf-lib: install-fsmod-lib
> > +
> > local-all: @BUILD_RULES@ @TRANSFORM_LIBTOOL_SCRIPTS@
> >
> > transform-libtool-scripts: @BUILD_RULES@
> >
>
> I have committed a fix in r1902335 adding install-fsmod-lib as a depencency
> for install-bdb-lib.
>
> It was tested by Nathan and I believe it is correct, but I must admit I
> have almost no experience in the build system.
>
> Sergei, can you make some tests on /trunk?
Sure! Ran a few builds against current trunk. I don't see bdb-related
failures any more \o/.
But I still do see libsvn_ra_serf related failures:
/<<NIX>>/coreutils-9.1/bin/install -c -d /<<NIX>>/subversion-1.14.2/lib /<<NIX>>/subversion-1.14.2/share/pkgconfig
cd subversion/libsvn_ra_serf ; /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --mode=install /<<NIX>>/coreutils-9.1/bin/install -c libsvn_ra_serf-1.la /<<NIX>>/subversion-1.14.2/lib/libsvn_ra_serf-1.la
libtool: warning: relinking 'libsvn_ra_serf-1.la'
libtool: install: (cd /build/subversion/subversion/libsvn_ra_serf; /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2 -L/<<NIX>>/openssl-1.1.1p/lib -L/<<NIX>>/db-5.3.28/lib -L/<<NIX>>/expat-2.4.8/lib -L/<<NIX>>/serf-1.3.9/lib -rpath /<<NIX>>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o libsvn_ra_serf-1.la blame.lo blncache.lo commit.lo eagain_bucket.lo get_deleted_rev.lo get_file.lo get_lock.lo getdate.lo getlocations.lo getlocationsegments.lo getlocks.lo inherited_props.lo list.lo lock.lo log.lo merge.lo mergeinfo.lo multistatus.lo options.lo property.lo replay.lo request_body.lo sb_bucket.lo serf.lo stat.lo stream_bucket.lo update.lo util.lo util_error.lo xml.lo ../../subversion/libsvn_delta/libsvn_delta-1.la ../../subversion/libsvn_subr/libsvn_subr-1.la -L/<<NIX>>/apr-util-1.6.1/lib -laprutil-1 -L/<<NIX>>/apr-1.7.0/lib -lapr-1 -lserf-1 -L/<<NIX>>/zlib-1.2.12-dev/lib -lz )
/<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No such file or directory
collect2: error: ld returned 1 exit status
libtool: error: error: relink 'libsvn_ra_serf-1.la' with the above command before installing it
Adding 'install-serf-lib: install-fsmod-lib' seems to work it around.
I also tried the fix below on top of trunk. It fixes install failures for me as well:
--- build.conf (revision 1902359)
+++ build.conf (working copy)
@@ -340,6 +340,7 @@
path = subversion/libsvn_ra_serf
install = serf-lib
libs = libsvn_delta libsvn_subr aprutil apriconv apr serf zlib
+add-install-deps = $(SVN_FS_LIB_INSTALL_DEPS)
msvc-static = yes
# Accessing repositories via SVN
--
Sergei
Re: 'make -j install' fails for missing depends
Posted by Daniel Sahlberg <da...@gmail.com>.
Den lör 25 juni 2022 kl 00:41 skrev Sergei Trofimovich <sl...@gmail.com>:
> On Fri, 24 Jun 2022 15:14:23 -0400
> Nathan Hartman <ha...@gmail.com> wrote:
>
> > On Thu, Jun 23, 2022 at 5:58 AM Sergei Trofimovich <sl...@gmail.com>
> wrote:
> > >
> > > Hi subversion developers!
> > >
> > > I'm working on fixing parallel build (or install) faliures in upstream
> > > packages used in NixOS linux distribution. Mostly to speed package
> builds
> > > up on continuous build farm.
> > >
> > > That usually means that
> > > $ ./configure && make && make install
> > > is changed to
> > > $ ./configure && make -j$(nproc) && make -j$(nproc) install
> > >
> > > To make the error more obvioud GNU make recently added --shuffle option
> > > to better expose missing dependencies across Makefile targets:
> > > https://savannah.gnu.org/bugs/index.php?62100
> > >
> > > 'subversion-1.14.2' came as one of the packages that sometimes fails on
> > > 'make install'
> > >
> > > There is a build log snippet that exposed missing install dependencies:
> > >
> > > $ ./configure --disable-static --prefix=/<<NIX>>/subversion-1.14.2
> --bindir=/<<NIX>>/subversion-1.14.2/bin
> --sbindir=/<<NIX>>/subversion-1.14.2/sbin
> --includedir=/<<NIX>>/subversion-1.14.2-dev/include
> --oldincludedir=/<<NIX>>/subversion-1.14.2-dev/include
> --mandir=/<<NIX>>/subversion-1.14.2-man/share/man
> --infodir=/<<NIX>>/subversion-1.14.2/share/info
> --docdir=/<<NIX>>/subversion-1.14.2/share/doc/subversion
> --libdir=/<<NIX>>/subversion-1.14.2/lib
> --libexecdir=/<<NIX>>/subversion-1.14.2/libexec
> --localedir=/<<NIX>>/subversion-1.14.2/share/locale --with-berkeley-db
> --without-apxs --without-swig --without-sasl
> --with-serf=/<<NIX>>/serf-1.3.9 --with-zlib=/<<NIX>>/zlib-1.2.12-dev
> --with-sqlite=/<<NIX>>/sqlite-3.38.5-dev --build=x86_64-unknown-linux-gnu
> --host=x86_64-unknown-linux-gnu
> > > ...
> > > $ make -j16 -l16 SHELL=/<<NIX>>/bash-5.1-p16/bin/bash
> APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules
> > > ...
> > > $ make --shuffle
> APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules
> pkgconfigdir=/<<NIX>>/subversion-1.14.2-dev/lib/pkgconfig
> m4datadir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal
> aclocaldir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal install
> > > test -d /<<NIX>>/subversion-1.14.2-dev/include/subversion-1 || \
> > > /<<NIX>>/coreutils-9.1/bin/install -c -d
> /<<NIX>>/subversion-1.14.2-dev/include/subversion-1
> > > (subversion/svnversion/svnversion . 2> /dev/null || \
> > > svnversion . 2> /dev/null || \
> > > echo "unknown"; \
> > > ) >
> /<<NIX>>/subversion-1.14.2-dev/include/subversion-1/svn-revision.txt
> > > /<<NIX>>/coreutils-9.1/bin/install -c -d
> /<<NIX>>/subversion-1.14.2/lib /<<NIX>>/subversion-1.14.2/share/pkgconfig
> > > cd subversion/libsvn_fs_base ; /<<NIX>>/bash-5.1-p16/bin/bash
> "/build/subversion/libtool" --mode=install
> /<<NIX>>/coreutils-9.1/bin/install -c libsvn_fs_base-1.la
> /<<NIX>>/subversion-1.14.2/lib/libsvn_fs_base-1.la
> > > libtool: warning: relinking 'libsvn_fs_base-1.la'
> > > libtool: install: (cd /build/subversion/subversion/libsvn_fs_base;
> /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --tag CC
> --silent --mode=relink gcc -shared -g -O2 -g -O2
> -L/<<NIX>>/openssl-1.1.1o/lib -L/<<NIX>>/db-5.3.28/lib
> -L/<<NIX>>/expat-2.4.8/lib -L/<<NIX>>/serf-1.3.9/lib -rpath
> /<<NIX>>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o
> libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo bdb/changes-table.lo
> bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo
> bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo
> bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo
> bdb/rev-table.lo bdb/strings-table.lo bdb/txn-table.lo bdb/uuids-table.lo
> dag.lo err.lo fs.lo id.lo key-gen.lo lock.lo node-rev.lo reps-strings.lo
> revs-txns.lo trail.lo tree.lo util/fs_skels.lo uuid.lo
> ../../subversion/libsvn_delta/libsvn_delta-1.la
> ../../subversion/libsvn_subr/libsvn_subr-1.la
> -L/<<NIX>>/apr-util-1.6.1/lib -laprutil-1 -L/<<N
> IX>>/apr-1.7.0/lib -lapr-1 -ldb-5.3 ../../subversion/libsvn_fs_util/
> libsvn_fs_util-1.la )
> > > /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No such
> file or directory
> > > /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_fs_util-1: No
> such file or directory
> > > collect2: error: ld returned 1 exit status
> > > libtool: error: error: relink 'libsvn_fs_base-1.la' with the
> above command before installing it
> > > make: *** [build-outputs.mk:1422: install-bdb-lib] Error 1
> shuffle=1656015674
> > >
> > > The failure as I understand it:
> > >
> > > 'install-bdb-lib' install target assumes that '-lsvn_delta-1' and
> '-lsvn_fs_util-1'
> > > are already installed into 'DESTDIR'. But they are not. 'libtool'
> fails to relink
> > > 'libsvn_fs_base' against 'DESTDIR'.
> > >
> > > I think the dependencies usually happen to be present if
> 'install-fsmod-lib' target
> > > was lucky to be executed first. But nothing in 'Makefile.in' or '
> build-outputs.mk'
> > > seems to have an equivalent of dependency like:
> > >
> > > # hypothetical fix
> > > install-bdb-lib : install-fsmod-lib
> > >
> > > It looks like 'build.conf' does contain some build dependency
> information.
> > >
> > > I also see special cases like fs-lib to handle some of install deps:
> > >
> > > # manual
> > > Makefile.in:SVN_FS_LIB_INSTALL_DEPS = @SVN_FS_LIB_INSTALL_DEPS@
> > >
> > > # generated:
> > > build-outputs.mk:install-ramod-lib: $(SVN_FS_LIB_INSTALL_DEPS)
> > >
> > > Would it make sense to add 'install-bdb-lib : install-fsmod-lib'
> dependency explicitly?
> > >
> > > Thank you!
> > >
> > > --
> > >
> > > Sergei
> >
> >
> > Hi Sergei,
> >
> > This is a very useful feature for GNU Make to have! This is the first
> > I've heard of it and it should allow many, many projects, not just
> > Subversion, to fix these kinds of pesky issues in parallel builds.
> >
> > It seems like this landed in Make literally a few days ago so isn't in
> > any released version. Is that correct?
>
> That is correct: there are no releases or snapshots of GNU make with a
> feature yet. I'm using locally built make from tip of the git tree.
>
> > If so, I don't currently have a (convenient) setup to experiment with
> > this at the moment; are you able to just hack your hypothetical fix
> > into your SVN makefile, i.e., `install-bdb-lib : install-fsmod-lib`,
> > and run make with shuffle=1656015674 and tell us if that appears to
> > fix it? Then I can go digging to see where to add it correctly.
>
> Before the hack my installs were failing 60% of the time (6 of 10 attmpts
> failed). After the change below 100 of 100 install succeeded.
>
> I had to add two depdendencies to get there:
>
> install-bdb-lib: install-fsmod-lib
> install-serf-lib: install-fsmod-lib
>
> The change itself (with linkage errors they fix):
>
> $ svn diff
> Index: Makefile.in
> ===================================================================
> --- Makefile.in (revision 1902174)
> +++ Makefile.in (working copy)
> @@ -431,6 +431,36 @@
>
> @INCLUDE_OUTPUTS@
>
> +# A hack to test https://issues.apache.org/jira/browse/SVN-4901:
> +# orkaround missing dependencies on installing libtool libraries
> +# that depend on other libraries:
> +#
> +## case 1: fs_base depend on fs_util and delta:
> +#
> +# libtool: install: (cd /build/subversion/subversion/libsvn_fs_base; bash
> "/build/subversion/libtool" \
> +# --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2
> -L...-openssl-1.1.1p/lib \
> +# -L...-db-5.3.28/lib -L...-expat-2.4.8/lib -L...-serf-1.3.9/lib
> -rpath ...-subversion-1.14.2/lib \
> +# -version-info 0 -Wl,--no-undefined \
> +# -o libsvn_fs_base-1.la bdb/bdb-err.lo \
> +# ... ../../subversion/libsvn_delta/libsvn_delta-1.la
> ../../subversion/libsvn_subr/libsvn_subr-1.la \
> +# -L...-apr-util-1.6.1/lib -laprutil-1 -L...-apr-1.7.0/lib -lapr-1
> -ldb-5.3 \
> +# ../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
> +# ld: cannot find -lsvn_delta-1: No such file or directory
> +# ld: cannot find -lsvn_fs_util-1: No such file or directory
> +install-bdb-lib: install-fsmod-lib
> +#
> +## case 2: ra-serf depends on delta:
> +#
> +# libtool: install: (cd /build/subversion/subversion/libsvn_ra_serf; bash
> "/build/subversion/libtool" \
> +# --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2
> -L...-openssl-1.1.1p/lib \
> +# -L...-db-5.3.28/lib -L...-expat-2.4.8/lib -L...-serf-1.3.9/lib
> -rpath ...-subversion-1.14.2/lib \
> +# -version-info 0 -Wl,--no-undefined
> +# -o libsvn_ra_serf-1.la blame.lo \
> +# ... xml.lo ../../subversion/libsvn_delta/libsvn_delta-1.la
> ../../subversion/libsvn_subr/libsvn_subr-1.la \
> +# -L...-apr-util-1.6.1/lib -laprutil-1 -L...-apr-1.7.0/lib -lapr-1
> -lserf-1 -L...-zlib-1.2.12-dev/lib -lz )
> +# ld: cannot find -lsvn_delta-1: No such file or directory
> +install-serf-lib: install-fsmod-lib
> +
> local-all: @BUILD_RULES@ @TRANSFORM_LIBTOOL_SCRIPTS@
>
> transform-libtool-scripts: @BUILD_RULES@
>
I have committed a fix in r1902335 adding install-fsmod-lib as a depencency
for install-bdb-lib.
It was tested by Nathan and I believe it is correct, but I must admit I
have almost no experience in the build system.
Sergei, can you make some tests on /trunk?
Kind regards,
Daniel
Re: 'make -j install' fails for missing depends
Posted by Sergei Trofimovich <sl...@gmail.com>.
On Fri, 24 Jun 2022 15:14:23 -0400
Nathan Hartman <ha...@gmail.com> wrote:
> On Thu, Jun 23, 2022 at 5:58 AM Sergei Trofimovich <sl...@gmail.com> wrote:
> >
> > Hi subversion developers!
> >
> > I'm working on fixing parallel build (or install) faliures in upstream
> > packages used in NixOS linux distribution. Mostly to speed package builds
> > up on continuous build farm.
> >
> > That usually means that
> > $ ./configure && make && make install
> > is changed to
> > $ ./configure && make -j$(nproc) && make -j$(nproc) install
> >
> > To make the error more obvioud GNU make recently added --shuffle option
> > to better expose missing dependencies across Makefile targets:
> > https://savannah.gnu.org/bugs/index.php?62100
> >
> > 'subversion-1.14.2' came as one of the packages that sometimes fails on
> > 'make install'
> >
> > There is a build log snippet that exposed missing install dependencies:
> >
> > $ ./configure --disable-static --prefix=/<<NIX>>/subversion-1.14.2 --bindir=/<<NIX>>/subversion-1.14.2/bin --sbindir=/<<NIX>>/subversion-1.14.2/sbin --includedir=/<<NIX>>/subversion-1.14.2-dev/include --oldincludedir=/<<NIX>>/subversion-1.14.2-dev/include --mandir=/<<NIX>>/subversion-1.14.2-man/share/man --infodir=/<<NIX>>/subversion-1.14.2/share/info --docdir=/<<NIX>>/subversion-1.14.2/share/doc/subversion --libdir=/<<NIX>>/subversion-1.14.2/lib --libexecdir=/<<NIX>>/subversion-1.14.2/libexec --localedir=/<<NIX>>/subversion-1.14.2/share/locale --with-berkeley-db --without-apxs --without-swig --without-sasl --with-serf=/<<NIX>>/serf-1.3.9 --with-zlib=/<<NIX>>/zlib-1.2.12-dev --with-sqlite=/<<NIX>>/sqlite-3.38.5-dev --build=x86_64-unknown-linux-gnu --host=x86_64-unknown-linux-gnu
> > ...
> > $ make -j16 -l16 SHELL=/<<NIX>>/bash-5.1-p16/bin/bash APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules
> > ...
> > $ make --shuffle APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules pkgconfigdir=/<<NIX>>/subversion-1.14.2-dev/lib/pkgconfig m4datadir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal aclocaldir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal install
> > test -d /<<NIX>>/subversion-1.14.2-dev/include/subversion-1 || \
> > /<<NIX>>/coreutils-9.1/bin/install -c -d /<<NIX>>/subversion-1.14.2-dev/include/subversion-1
> > (subversion/svnversion/svnversion . 2> /dev/null || \
> > svnversion . 2> /dev/null || \
> > echo "unknown"; \
> > ) > /<<NIX>>/subversion-1.14.2-dev/include/subversion-1/svn-revision.txt
> > /<<NIX>>/coreutils-9.1/bin/install -c -d /<<NIX>>/subversion-1.14.2/lib /<<NIX>>/subversion-1.14.2/share/pkgconfig
> > cd subversion/libsvn_fs_base ; /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --mode=install /<<NIX>>/coreutils-9.1/bin/install -c libsvn_fs_base-1.la /<<NIX>>/subversion-1.14.2/lib/libsvn_fs_base-1.la
> > libtool: warning: relinking 'libsvn_fs_base-1.la'
> > libtool: install: (cd /build/subversion/subversion/libsvn_fs_base; /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2 -L/<<NIX>>/openssl-1.1.1o/lib -L/<<NIX>>/db-5.3.28/lib -L/<<NIX>>/expat-2.4.8/lib -L/<<NIX>>/serf-1.3.9/lib -rpath /<<NIX>>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo bdb/changes-table.lo bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo bdb/rev-table.lo bdb/strings-table.lo bdb/txn-table.lo bdb/uuids-table.lo dag.lo err.lo fs.lo id.lo key-gen.lo lock.lo node-rev.lo reps-strings.lo revs-txns.lo trail.lo tree.lo util/fs_skels.lo uuid.lo ../../subversion/libsvn_delta/libsvn_delta-1.la ../../subversion/libsvn_subr/libsvn_subr-1.la -L/<<NIX>>/apr-util-1.6.1/lib -laprutil-1 -L/<<N
IX>>/apr-1.7.0/lib -lapr-1 -ldb-5.3 ../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
> > /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No such file or directory
> > /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_fs_util-1: No such file or directory
> > collect2: error: ld returned 1 exit status
> > libtool: error: error: relink 'libsvn_fs_base-1.la' with the above command before installing it
> > make: *** [build-outputs.mk:1422: install-bdb-lib] Error 1 shuffle=1656015674
> >
> > The failure as I understand it:
> >
> > 'install-bdb-lib' install target assumes that '-lsvn_delta-1' and '-lsvn_fs_util-1'
> > are already installed into 'DESTDIR'. But they are not. 'libtool' fails to relink
> > 'libsvn_fs_base' against 'DESTDIR'.
> >
> > I think the dependencies usually happen to be present if 'install-fsmod-lib' target
> > was lucky to be executed first. But nothing in 'Makefile.in' or 'build-outputs.mk'
> > seems to have an equivalent of dependency like:
> >
> > # hypothetical fix
> > install-bdb-lib : install-fsmod-lib
> >
> > It looks like 'build.conf' does contain some build dependency information.
> >
> > I also see special cases like fs-lib to handle some of install deps:
> >
> > # manual
> > Makefile.in:SVN_FS_LIB_INSTALL_DEPS = @SVN_FS_LIB_INSTALL_DEPS@
> >
> > # generated:
> > build-outputs.mk:install-ramod-lib: $(SVN_FS_LIB_INSTALL_DEPS)
> >
> > Would it make sense to add 'install-bdb-lib : install-fsmod-lib' dependency explicitly?
> >
> > Thank you!
> >
> > --
> >
> > Sergei
>
>
> Hi Sergei,
>
> This is a very useful feature for GNU Make to have! This is the first
> I've heard of it and it should allow many, many projects, not just
> Subversion, to fix these kinds of pesky issues in parallel builds.
>
> It seems like this landed in Make literally a few days ago so isn't in
> any released version. Is that correct?
That is correct: there are no releases or snapshots of GNU make with a
feature yet. I'm using locally built make from tip of the git tree.
> If so, I don't currently have a (convenient) setup to experiment with
> this at the moment; are you able to just hack your hypothetical fix
> into your SVN makefile, i.e., `install-bdb-lib : install-fsmod-lib`,
> and run make with shuffle=1656015674 and tell us if that appears to
> fix it? Then I can go digging to see where to add it correctly.
Before the hack my installs were failing 60% of the time (6 of 10 attmpts
failed). After the change below 100 of 100 install succeeded.
I had to add two depdendencies to get there:
install-bdb-lib: install-fsmod-lib
install-serf-lib: install-fsmod-lib
The change itself (with linkage errors they fix):
$ svn diff
Index: Makefile.in
===================================================================
--- Makefile.in (revision 1902174)
+++ Makefile.in (working copy)
@@ -431,6 +431,36 @@
@INCLUDE_OUTPUTS@
+# A hack to test https://issues.apache.org/jira/browse/SVN-4901:
+# orkaround missing dependencies on installing libtool libraries
+# that depend on other libraries:
+#
+## case 1: fs_base depend on fs_util and delta:
+#
+# libtool: install: (cd /build/subversion/subversion/libsvn_fs_base; bash "/build/subversion/libtool" \
+# --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2 -L...-openssl-1.1.1p/lib \
+# -L...-db-5.3.28/lib -L...-expat-2.4.8/lib -L...-serf-1.3.9/lib -rpath ...-subversion-1.14.2/lib \
+# -version-info 0 -Wl,--no-undefined \
+# -o libsvn_fs_base-1.la bdb/bdb-err.lo \
+# ... ../../subversion/libsvn_delta/libsvn_delta-1.la ../../subversion/libsvn_subr/libsvn_subr-1.la \
+# -L...-apr-util-1.6.1/lib -laprutil-1 -L...-apr-1.7.0/lib -lapr-1 -ldb-5.3 \
+# ../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
+# ld: cannot find -lsvn_delta-1: No such file or directory
+# ld: cannot find -lsvn_fs_util-1: No such file or directory
+install-bdb-lib: install-fsmod-lib
+#
+## case 2: ra-serf depends on delta:
+#
+# libtool: install: (cd /build/subversion/subversion/libsvn_ra_serf; bash "/build/subversion/libtool" \
+# --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2 -L...-openssl-1.1.1p/lib \
+# -L...-db-5.3.28/lib -L...-expat-2.4.8/lib -L...-serf-1.3.9/lib -rpath ...-subversion-1.14.2/lib \
+# -version-info 0 -Wl,--no-undefined
+# -o libsvn_ra_serf-1.la blame.lo \
+# ... xml.lo ../../subversion/libsvn_delta/libsvn_delta-1.la ../../subversion/libsvn_subr/libsvn_subr-1.la \
+# -L...-apr-util-1.6.1/lib -laprutil-1 -L...-apr-1.7.0/lib -lapr-1 -lserf-1 -L...-zlib-1.2.12-dev/lib -lz )
+# ld: cannot find -lsvn_delta-1: No such file or directory
+install-serf-lib: install-fsmod-lib
+
local-all: @BUILD_RULES@ @TRANSFORM_LIBTOOL_SCRIPTS@
transform-libtool-scripts: @BUILD_RULES@
> Thanks for bringing this (and the new/upcoming GNU Make feature) to
> our attention. That's exciting news.
Glad you see it useful! Thank you!
--
Sergei
Re: 'make -j install' fails for missing depends
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Jun 23, 2022 at 5:58 AM Sergei Trofimovich <sl...@gmail.com> wrote:
>
> Hi subversion developers!
>
> I'm working on fixing parallel build (or install) faliures in upstream
> packages used in NixOS linux distribution. Mostly to speed package builds
> up on continuous build farm.
>
> That usually means that
> $ ./configure && make && make install
> is changed to
> $ ./configure && make -j$(nproc) && make -j$(nproc) install
>
> To make the error more obvioud GNU make recently added --shuffle option
> to better expose missing dependencies across Makefile targets:
> https://savannah.gnu.org/bugs/index.php?62100
>
> 'subversion-1.14.2' came as one of the packages that sometimes fails on
> 'make install'
>
> There is a build log snippet that exposed missing install dependencies:
>
> $ ./configure --disable-static --prefix=/<<NIX>>/subversion-1.14.2 --bindir=/<<NIX>>/subversion-1.14.2/bin --sbindir=/<<NIX>>/subversion-1.14.2/sbin --includedir=/<<NIX>>/subversion-1.14.2-dev/include --oldincludedir=/<<NIX>>/subversion-1.14.2-dev/include --mandir=/<<NIX>>/subversion-1.14.2-man/share/man --infodir=/<<NIX>>/subversion-1.14.2/share/info --docdir=/<<NIX>>/subversion-1.14.2/share/doc/subversion --libdir=/<<NIX>>/subversion-1.14.2/lib --libexecdir=/<<NIX>>/subversion-1.14.2/libexec --localedir=/<<NIX>>/subversion-1.14.2/share/locale --with-berkeley-db --without-apxs --without-swig --without-sasl --with-serf=/<<NIX>>/serf-1.3.9 --with-zlib=/<<NIX>>/zlib-1.2.12-dev --with-sqlite=/<<NIX>>/sqlite-3.38.5-dev --build=x86_64-unknown-linux-gnu --host=x86_64-unknown-linux-gnu
> ...
> $ make -j16 -l16 SHELL=/<<NIX>>/bash-5.1-p16/bin/bash APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules
> ...
> $ make --shuffle APACHE_LIBEXECDIR=/<<NIX>>/subversion-1.14.2/modules pkgconfigdir=/<<NIX>>/subversion-1.14.2-dev/lib/pkgconfig m4datadir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal aclocaldir=/<<NIX>>/subversion-1.14.2-dev/share/aclocal install
> test -d /<<NIX>>/subversion-1.14.2-dev/include/subversion-1 || \
> /<<NIX>>/coreutils-9.1/bin/install -c -d /<<NIX>>/subversion-1.14.2-dev/include/subversion-1
> (subversion/svnversion/svnversion . 2> /dev/null || \
> svnversion . 2> /dev/null || \
> echo "unknown"; \
> ) > /<<NIX>>/subversion-1.14.2-dev/include/subversion-1/svn-revision.txt
> /<<NIX>>/coreutils-9.1/bin/install -c -d /<<NIX>>/subversion-1.14.2/lib /<<NIX>>/subversion-1.14.2/share/pkgconfig
> cd subversion/libsvn_fs_base ; /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --mode=install /<<NIX>>/coreutils-9.1/bin/install -c libsvn_fs_base-1.la /<<NIX>>/subversion-1.14.2/lib/libsvn_fs_base-1.la
> libtool: warning: relinking 'libsvn_fs_base-1.la'
> libtool: install: (cd /build/subversion/subversion/libsvn_fs_base; /<<NIX>>/bash-5.1-p16/bin/bash "/build/subversion/libtool" --tag CC --silent --mode=relink gcc -shared -g -O2 -g -O2 -L/<<NIX>>/openssl-1.1.1o/lib -L/<<NIX>>/db-5.3.28/lib -L/<<NIX>>/expat-2.4.8/lib -L/<<NIX>>/serf-1.3.9/lib -rpath /<<NIX>>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo bdb/changes-table.lo bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo bdb/rev-table.lo bdb/strings-table.lo bdb/txn-table.lo bdb/uuids-table.lo dag.lo err.lo fs.lo id.lo key-gen.lo lock.lo node-rev.lo reps-strings.lo revs-txns.lo trail.lo tree.lo util/fs_skels.lo uuid.lo ../../subversion/libsvn_delta/libsvn_delta-1.la ../../subversion/libsvn_subr/libsvn_subr-1.la -L/<<NIX>>/apr-util-1.6.1/lib -laprutil-1 -L/<<NIX>>/apr-1.7.0/lib -lapr-1 -ldb-5.3 ../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
> /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No such file or directory
> /<<NIX>>/binutils-2.38/bin/ld: cannot find -lsvn_fs_util-1: No such file or directory
> collect2: error: ld returned 1 exit status
> libtool: error: error: relink 'libsvn_fs_base-1.la' with the above command before installing it
> make: *** [build-outputs.mk:1422: install-bdb-lib] Error 1 shuffle=1656015674
>
> The failure as I understand it:
>
> 'install-bdb-lib' install target assumes that '-lsvn_delta-1' and '-lsvn_fs_util-1'
> are already installed into 'DESTDIR'. But they are not. 'libtool' fails to relink
> 'libsvn_fs_base' against 'DESTDIR'.
>
> I think the dependencies usually happen to be present if 'install-fsmod-lib' target
> was lucky to be executed first. But nothing in 'Makefile.in' or 'build-outputs.mk'
> seems to have an equivalent of dependency like:
>
> # hypothetical fix
> install-bdb-lib : install-fsmod-lib
>
> It looks like 'build.conf' does contain some build dependency information.
>
> I also see special cases like fs-lib to handle some of install deps:
>
> # manual
> Makefile.in:SVN_FS_LIB_INSTALL_DEPS = @SVN_FS_LIB_INSTALL_DEPS@
>
> # generated:
> build-outputs.mk:install-ramod-lib: $(SVN_FS_LIB_INSTALL_DEPS)
>
> Would it make sense to add 'install-bdb-lib : install-fsmod-lib' dependency explicitly?
>
> Thank you!
>
> --
>
> Sergei
Hi Sergei,
This is a very useful feature for GNU Make to have! This is the first
I've heard of it and it should allow many, many projects, not just
Subversion, to fix these kinds of pesky issues in parallel builds.
It seems like this landed in Make literally a few days ago so isn't in
any released version. Is that correct?
If so, I don't currently have a (convenient) setup to experiment with
this at the moment; are you able to just hack your hypothetical fix
into your SVN makefile, i.e., `install-bdb-lib : install-fsmod-lib`,
and run make with shuffle=1656015674 and tell us if that appears to
fix it? Then I can go digging to see where to add it correctly.
Thanks for bringing this (and the new/upcoming GNU Make feature) to
our attention. That's exciting news.
Cheers,
Nathan