You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Daniel Shahaf <d....@daniel.shahaf.name> on 2012/04/15 07:33:48 UTC
Faster build/check times
Trying to enumerate ways to reduce build times.
Faster builds:
- export CONFIG_SHELL=/bin/sh (minimal sh preferred)
- out-of-tree build (i.e., run 'configure' in an empty dir), with the build tree in a tmpfs
- disable unneeded components (eg, swig, bdb) when they're not needed
- pass '-q' to configure, '-s' to make
- pass '-C' to configure
- pass '-j' or '-j[number]' to make
- use cpuset(1)/taskset(1) to bind make to N-1 CPUs
Faster checks:
- PARALLEL=1
- CLEANUP="" (and rm -rf svn-test-work/ before the build)
- put svn-test-work/ in a tmpfs
- SET_LOG_LEVEL=WARN (or ERROR)
What else?
Thanks,
Daniel
Re: Faster build/check times
Posted by Ashod Nakashian <as...@yahoo.com>.
----- Original Message -----
> From: Daniel Shahaf <d....@daniel.shahaf.name>
> To: Ashod Nakashian <as...@yahoo.com>; "dev@subversion.apache.org" <de...@subversion.apache.org>
> Cc:
> Sent: Sunday, April 15, 2012 1:28 PM
> Subject: Re: Faster build/check times
>
>
>
> On Sun, Apr 15, 2012, at 02:09, Ashod Nakashian wrote:
>>
>>
>>
>>
>> ----- Original Message -----
>> > From: Daniel Shahaf <d....@daniel.shahaf.name>
>> > To: Ashod Nakashian <as...@yahoo.com>;
> "dev@subversion.apache.org" <de...@subversion.apache.org>
>> > Cc:
>> > Sent: Sunday, April 15, 2012 12:16 PM
>> > Subject: Re: Faster build/check times
>> >
>> >
>> >
>> > On Sun, Apr 15, 2012, at 00:56, Ashod Nakashian wrote:
>> >> >________________________________
>> >> > From: Daniel Shahaf <d....@daniel.shahaf.name>
>> >> >To: dev@subversion.apache.org
>> >> >Sent: Sunday, April 15, 2012 9:33 AM
>> >> >Subject: Faster build/check times
>> >> >
>> >> >Trying to enumerate ways to reduce build times.
>> >>
>> >> Very timely! I'm afraid I don't have anything to add, but
> think
>> > it'd
>> >> be useful to know what's the *minimum* set of features
> required to
>> >> build the API + CLI frontend (i.e. libs + svn).
>> >
>> > All of the following components are optional: libsvn_fs_base(bdb),
>> > libsvn_ra_neon, libsvn_ra_serf, mod_dav_svn(httpd), swig, java,
> ctypes, kwallet,
>> > gnome-keyring. Building the 'tools' makefile target (or VS
> project) is
>> > done by default, but it can be harmlessly left out.
>>
>> Thanks! Here is what I came up with, in case it's useful to anyone:
>>
>> ./configure --disable-mod-activation --without-gssapi --without-apxs
> --without-berkeley-db --without-neon --without-serf --without-swig
> --without-ctypesgen --without-kwallet --without-gnome-keyring --disable-javahl
> --disable-keychain
>>
>> How can the tools be disabled without modifying makefiles?
>>
>
> 'make bin atomic-ra-revprop-change entries-dump' instead of
> 'make'.
>
> (The last two are in $(TEST_DEPS), and are needed to run certain py tests
> manually,
> but there is no test-deps convenience target to avoid listing them explicitly
> by.)
>
> And while I'm on the topic, another useful invocation:
> % make check TESTS="`echo
> subversion/tests/cmdline/{basic_tests.py,merge_tests.py}`"
>
Lovely! Thanks a bundle.
>> -Ash
>>
>> >
>> > That leaves a client that can talk svn:// and can talk file:// to FSFS
>
>> > repositories.
>> >
>> > SQLite, APR, APR-Util are mandatory dependencies.
>> >
>> >>
>> >> Alternatively, what features can be disabled and still get the
> bare minimum
>> > output?
>> >>
>> >> (Apologies if I'm being lazy for not playing with configure
> to figure
>> > it out myself - I think chances are high someone has something handy
> for pasting
>> > at an arm's length.)
>> >>
>> >> Cheers,
>> >> -Ash
>> >>
>> >> P.S. Thanks Daniel for this list!
>> >>
>> >> >
>> >> >Faster builds:
>> >> >- export CONFIG_SHELL=/bin/sh (minimal sh preferred)
>> >> >- out-of-tree build (i.e., run 'configure' in an
> empty dir),
>> > with the build tree in a tmpfs
>> >> >- disable unneeded components (eg, swig, bdb) when
> they're not
>> > needed
>> >> >- pass '-q' to configure, '-s' to make
>> >> >- pass '-C' to configure
>> >> >- pass '-j' or '-j[number]' to make
>> >> >- use cpuset(1)/taskset(1) to bind make to N-1 CPUs
>> >> >
>> >> >Faster checks:
>> >> >- PARALLEL=1
>> >> >- CLEANUP="" (and rm -rf svn-test-work/ before the
> build)
>> >> >- put svn-test-work/ in a tmpfs
>> >> >- SET_LOG_LEVEL=WARN (or ERROR)
>> >> >
>> >> >What else?
>> >> >
>> >> >
>> >> >Thanks,
>> >> >
>> >> >Daniel
>> >> >
>> >> >
>> >> >
>> >
>
Re: Faster build/check times
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
On Sun, Apr 15, 2012, at 02:09, Ashod Nakashian wrote:
>
>
>
>
> ----- Original Message -----
> > From: Daniel Shahaf <d....@daniel.shahaf.name>
> > To: Ashod Nakashian <as...@yahoo.com>; "dev@subversion.apache.org" <de...@subversion.apache.org>
> > Cc:
> > Sent: Sunday, April 15, 2012 12:16 PM
> > Subject: Re: Faster build/check times
> >
> >
> >
> > On Sun, Apr 15, 2012, at 00:56, Ashod Nakashian wrote:
> >> >________________________________
> >> > From: Daniel Shahaf <d....@daniel.shahaf.name>
> >> >To: dev@subversion.apache.org
> >> >Sent: Sunday, April 15, 2012 9:33 AM
> >> >Subject: Faster build/check times
> >> >
> >> >Trying to enumerate ways to reduce build times.
> >>
> >> Very timely! I'm afraid I don't have anything to add, but think
> > it'd
> >> be useful to know what's the *minimum* set of features required to
> >> build the API + CLI frontend (i.e. libs + svn).
> >
> > All of the following components are optional: libsvn_fs_base(bdb),
> > libsvn_ra_neon, libsvn_ra_serf, mod_dav_svn(httpd), swig, java, ctypes, kwallet,
> > gnome-keyring. Building the 'tools' makefile target (or VS project) is
> > done by default, but it can be harmlessly left out.
>
> Thanks! Here is what I came up with, in case it's useful to anyone:
>
> ./configure --disable-mod-activation --without-gssapi --without-apxs --without-berkeley-db --without-neon --without-serf --without-swig --without-ctypesgen --without-kwallet --without-gnome-keyring --disable-javahl --disable-keychain
>
> How can the tools be disabled without modifying makefiles?
>
'make bin atomic-ra-revprop-change entries-dump' instead of 'make'.
(The last two are in $(TEST_DEPS), and are needed to run certain py tests manually,
but there is no test-deps convenience target to avoid listing them explicitly by.)
And while I'm on the topic, another useful invocation:
% make check TESTS="`echo subversion/tests/cmdline/{basic_tests.py,merge_tests.py}`"
> -Ash
>
> >
> > That leaves a client that can talk svn:// and can talk file:// to FSFS
> > repositories.
> >
> > SQLite, APR, APR-Util are mandatory dependencies.
> >
> >>
> >> Alternatively, what features can be disabled and still get the bare minimum
> > output?
> >>
> >> (Apologies if I'm being lazy for not playing with configure to figure
> > it out myself - I think chances are high someone has something handy for pasting
> > at an arm's length.)
> >>
> >> Cheers,
> >> -Ash
> >>
> >> P.S. Thanks Daniel for this list!
> >>
> >> >
> >> >Faster builds:
> >> >- export CONFIG_SHELL=/bin/sh (minimal sh preferred)
> >> >- out-of-tree build (i.e., run 'configure' in an empty dir),
> > with the build tree in a tmpfs
> >> >- disable unneeded components (eg, swig, bdb) when they're not
> > needed
> >> >- pass '-q' to configure, '-s' to make
> >> >- pass '-C' to configure
> >> >- pass '-j' or '-j[number]' to make
> >> >- use cpuset(1)/taskset(1) to bind make to N-1 CPUs
> >> >
> >> >Faster checks:
> >> >- PARALLEL=1
> >> >- CLEANUP="" (and rm -rf svn-test-work/ before the build)
> >> >- put svn-test-work/ in a tmpfs
> >> >- SET_LOG_LEVEL=WARN (or ERROR)
> >> >
> >> >What else?
> >> >
> >> >
> >> >Thanks,
> >> >
> >> >Daniel
> >> >
> >> >
> >> >
> >
Re: Faster build/check times
Posted by Ashod Nakashian <as...@yahoo.com>.
----- Original Message -----
> From: Daniel Shahaf <d....@daniel.shahaf.name>
> To: Ashod Nakashian <as...@yahoo.com>; "dev@subversion.apache.org" <de...@subversion.apache.org>
> Cc:
> Sent: Sunday, April 15, 2012 12:16 PM
> Subject: Re: Faster build/check times
>
>
>
> On Sun, Apr 15, 2012, at 00:56, Ashod Nakashian wrote:
>> >________________________________
>> > From: Daniel Shahaf <d....@daniel.shahaf.name>
>> >To: dev@subversion.apache.org
>> >Sent: Sunday, April 15, 2012 9:33 AM
>> >Subject: Faster build/check times
>> >
>> >Trying to enumerate ways to reduce build times.
>>
>> Very timely! I'm afraid I don't have anything to add, but think
> it'd
>> be useful to know what's the *minimum* set of features required to
>> build the API + CLI frontend (i.e. libs + svn).
>
> All of the following components are optional: libsvn_fs_base(bdb),
> libsvn_ra_neon, libsvn_ra_serf, mod_dav_svn(httpd), swig, java, ctypes, kwallet,
> gnome-keyring. Building the 'tools' makefile target (or VS project) is
> done by default, but it can be harmlessly left out.
Thanks! Here is what I came up with, in case it's useful to anyone:
./configure --disable-mod-activation --without-gssapi --without-apxs --without-berkeley-db --without-neon --without-serf --without-swig --without-ctypesgen --without-kwallet --without-gnome-keyring --disable-javahl --disable-keychain
How can the tools be disabled without modifying makefiles?
-Ash
>
> That leaves a client that can talk svn:// and can talk file:// to FSFS
> repositories.
>
> SQLite, APR, APR-Util are mandatory dependencies.
>
>>
>> Alternatively, what features can be disabled and still get the bare minimum
> output?
>>
>> (Apologies if I'm being lazy for not playing with configure to figure
> it out myself - I think chances are high someone has something handy for pasting
> at an arm's length.)
>>
>> Cheers,
>> -Ash
>>
>> P.S. Thanks Daniel for this list!
>>
>> >
>> >Faster builds:
>> >- export CONFIG_SHELL=/bin/sh (minimal sh preferred)
>> >- out-of-tree build (i.e., run 'configure' in an empty dir),
> with the build tree in a tmpfs
>> >- disable unneeded components (eg, swig, bdb) when they're not
> needed
>> >- pass '-q' to configure, '-s' to make
>> >- pass '-C' to configure
>> >- pass '-j' or '-j[number]' to make
>> >- use cpuset(1)/taskset(1) to bind make to N-1 CPUs
>> >
>> >Faster checks:
>> >- PARALLEL=1
>> >- CLEANUP="" (and rm -rf svn-test-work/ before the build)
>> >- put svn-test-work/ in a tmpfs
>> >- SET_LOG_LEVEL=WARN (or ERROR)
>> >
>> >What else?
>> >
>> >
>> >Thanks,
>> >
>> >Daniel
>> >
>> >
>> >
>
Re: Faster build/check times
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
On Sun, Apr 15, 2012, at 00:56, Ashod Nakashian wrote:
> >________________________________
> > From: Daniel Shahaf <d....@daniel.shahaf.name>
> >To: dev@subversion.apache.org
> >Sent: Sunday, April 15, 2012 9:33 AM
> >Subject: Faster build/check times
> >
> >Trying to enumerate ways to reduce build times.
>
> Very timely! I'm afraid I don't have anything to add, but think it'd
> be useful to know what's the *minimum* set of features required to
> build the API + CLI frontend (i.e. libs + svn).
All of the following components are optional: libsvn_fs_base(bdb), libsvn_ra_neon, libsvn_ra_serf, mod_dav_svn(httpd), swig, java, ctypes, kwallet, gnome-keyring. Building the 'tools' makefile target (or VS project) is done by default, but it can be harmlessly left out.
That leaves a client that can talk svn:// and can talk file:// to FSFS repositories.
SQLite, APR, APR-Util are mandatory dependencies.
>
> Alternatively, what features can be disabled and still get the bare minimum output?
>
> (Apologies if I'm being lazy for not playing with configure to figure it out myself - I think chances are high someone has something handy for pasting at an arm's length.)
>
> Cheers,
> -Ash
>
> P.S. Thanks Daniel for this list!
>
> >
> >Faster builds:
> >- export CONFIG_SHELL=/bin/sh (minimal sh preferred)
> >- out-of-tree build (i.e., run 'configure' in an empty dir), with the build tree in a tmpfs
> >- disable unneeded components (eg, swig, bdb) when they're not needed
> >- pass '-q' to configure, '-s' to make
> >- pass '-C' to configure
> >- pass '-j' or '-j[number]' to make
> >- use cpuset(1)/taskset(1) to bind make to N-1 CPUs
> >
> >Faster checks:
> >- PARALLEL=1
> >- CLEANUP="" (and rm -rf svn-test-work/ before the build)
> >- put svn-test-work/ in a tmpfs
> >- SET_LOG_LEVEL=WARN (or ERROR)
> >
> >What else?
> >
> >
> >Thanks,
> >
> >Daniel
> >
> >
> >
Re: Faster build/check times
Posted by Ashod Nakashian <as...@yahoo.com>.
>________________________________
> From: Daniel Shahaf <d....@daniel.shahaf.name>
>To: dev@subversion.apache.org
>Sent: Sunday, April 15, 2012 9:33 AM
>Subject: Faster build/check times
>
>Trying to enumerate ways to reduce build times.
Very timely! I'm afraid I don't have anything to add, but think it'd be useful to know what's the *minimum* set of features required to build the API + CLI frontend (i.e. libs + svn).
Alternatively, what features can be disabled and still get the bare minimum output?
(Apologies if I'm being lazy for not playing with configure to figure it out myself - I think chances are high someone has something handy for pasting at an arm's length.)
Cheers,
-Ash
P.S. Thanks Daniel for this list!
>
>Faster builds:
>- export CONFIG_SHELL=/bin/sh (minimal sh preferred)
>- out-of-tree build (i.e., run 'configure' in an empty dir), with the build tree in a tmpfs
>- disable unneeded components (eg, swig, bdb) when they're not needed
>- pass '-q' to configure, '-s' to make
>- pass '-C' to configure
>- pass '-j' or '-j[number]' to make
>- use cpuset(1)/taskset(1) to bind make to N-1 CPUs
>
>Faster checks:
>- PARALLEL=1
>- CLEANUP="" (and rm -rf svn-test-work/ before the build)
>- put svn-test-work/ in a tmpfs
>- SET_LOG_LEVEL=WARN (or ERROR)
>
>What else?
>
>
>Thanks,
>
>Daniel
>
>
>
Re: Faster build/check times
Posted by Philip Martin <ph...@wandisco.com>.
Daniel Shahaf <d....@daniel.shahaf.name> writes:
> Trying to enumerate ways to reduce build times.
>
> Faster builds:
> - export CONFIG_SHELL=/bin/sh (minimal sh preferred)
> - out-of-tree build (i.e., run 'configure' in an empty dir), with the build tree in a tmpfs
> - disable unneeded components (eg, swig, bdb) when they're not needed
> - pass '-q' to configure, '-s' to make
> - pass '-C' to configure
> - pass '-j' or '-j[number]' to make
> - use cpuset(1)/taskset(1) to bind make to N-1 CPUs
- build against installed dependencies rather than in-tree dependencies
- don't build both static and shared libraries: pass either
--disable-static or --disable-shared to configure.
> Faster checks:
> - PARALLEL=1
> - CLEANUP="" (and rm -rf svn-test-work/ before the build)
> - put svn-test-work/ in a tmpfs
> - SET_LOG_LEVEL=WARN (or ERROR)
- set TMPDIR to the tmpfs filesystem if /tmp is not already a tmpfs
filesystem.
- don't enable SELinux at boot.
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com