You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Justin Erenkrantz <ju...@erenkrantz.com> on 2005/09/30 07:03:32 UTC
[PATCH] Use rsync to install manual
Sitting around waiting for another install to complete due to the large
size of our manual, I wrote this patch to switch install-man and
install-htdocs to use rsync if available. If it isn't available, it'll
fall back on the current cp and rm -rf .svn strategy. (Tested with and
without rsync and it appears okay in my testing.)
This greatly speeds up installation. =) So, any objections to
committing? -- justin
Index: Makefile.in
===================================================================
--- Makefile.in (revision 292616)
+++ Makefile.in (working copy)
@@ -122,8 +122,12 @@
else \
echo Installing HTML documents ; \
$(MKINSTALLDIRS) $(DESTDIR)$(htdocsdir) ; \
- test -d $(htdocs-srcdir) && (cd $(htdocs-srcdir) && cp -rp * $(DESTDIR)$(htdocsdir)) ; \
- cd $(DESTDIR)$(htdocsdir) && find . -name ".svn" -type d -print | xargs rm -rf 2>/dev/null || true; \
+ if test -d $(htdocs-srcdir) && test "x$(RSYNC)" != "x" && test -x $(RSYNC) ; then \
+ $(RSYNC) --exclude .svn -a $(htdocs-srcdir)/ $(DESTDIR)$(htdocsdir); \
+ else \
+ test -d $(htdocs-srcdir) && (cd $(htdocs-srcdir) && cp -rp * $(DESTDIR)$(htdocsdir)) ; \
+ cd $(DESTDIR)$(htdocsdir) && find . -name ".svn" -type d -print | xargs rm -rf 2>/dev/null || true; \
+ fi; \
fi
install-error:
@@ -201,8 +205,12 @@
@test -d $(DESTDIR)$(manualdir) || $(MKINSTALLDIRS) $(DESTDIR)$(manualdir)
@cp -p $(top_srcdir)/docs/man/*.1 $(DESTDIR)$(mandir)/man1
@cp -p $(top_srcdir)/docs/man/*.8 $(DESTDIR)$(mandir)/man8
- @(cd $(top_srcdir)/docs/manual && cp -rp * $(DESTDIR)$(manualdir))
- @(cd $(DESTDIR)$(manualdir) && find . -name ".svn" -type d -print | xargs rm -rf 2>/dev/null ) || true
+ @if test "x$(RSYNC)" != "x" && test -x $(RSYNC) ; then \
+ $(RSYNC) --exclude .svn -a $(top_srcdir)/docs/manual/ $(DESTDIR)$(manualdir); \
+ else \
+ cd $(top_srcdir)/docs/manual && cp -rp * $(DESTDIR)$(manualdir); \
+ cd $(DESTDIR)$(manualdir) && find . -name ".svn" -type d -print | xargs rm -rf 2>/dev/null; \
+ fi
install-suexec:
@if test -f $(builddir)/support/suexec; then \
Index: configure.in
===================================================================
--- configure.in (revision 292616)
+++ configure.in (working copy)
@@ -297,6 +297,7 @@
AC_PATH_PROG(RM, rm)
AC_PATH_PROG(PKGCONFIG, pkg-config)
+AC_PATH_PROG(RSYNC, rsync)
AC_PROG_AWK
AC_PROG_LN_S
AC_CHECK_TOOL(RANLIB, ranlib, true)
Index: acinclude.m4
===================================================================
--- acinclude.m4 (revision 292616)
+++ acinclude.m4 (working copy)
@@ -75,6 +75,7 @@
APACHE_SUBST(EXTRA_INCLUDES)
APACHE_SUBST(LIBTOOL)
APACHE_SUBST(SHELL)
+ APACHE_SUBST(RSYNC)
APACHE_SUBST(MODULE_DIRS)
APACHE_SUBST(MODULE_CLEANDIRS)
APACHE_SUBST(PORT)
Re: [PATCH] Use rsync to install manual
Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Fri, Sep 30, 2005 at 10:07:28AM +0100, Nick Kew wrote:
> Sounds sensible in principle.
>
> Excuse my autofoo ignorance, but how does it handle the case where,
> for example, rsync is available but fails due to a local firewall?
rsync won't use sockets, or any kind of networking when doing local
files only. It just uses stdout -> stdin between two rsync processes :)
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: [PATCH] Use rsync to install manual
Posted by Nick Kew <ni...@webthing.com>.
On Friday 30 September 2005 06:03, Justin Erenkrantz wrote:
> This greatly speeds up installation. =) So, any objections to
> committing? -- justin
Sounds sensible in principle.
Excuse my autofoo ignorance, but how does it handle the case where,
for example, rsync is available but fails due to a local firewall?
--
Nick Kew
Re: [PATCH] Use rsync to install manual
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On Fri, Sep 30, 2005 at 10:04:12AM +0100, Colm MacCarthaigh wrote:
> Please don't use the -a option for rsync. use -rlptDv instead; we
> shouldn't be trouncing the ownership of the destination files, cp
> doesn't. I'd also add the --numeric-ids option, which will also prevent
> rsync from doing the uid lookups and can greatly speed things up on
> systems with no (or slow) nscd's (or slow uid lookups in general).
Fair enough. I doubt we need D (devices) or v (verbose) though.
> Do (DESTDIR)$(htdocsdir) and (DESTDIR)$(manualdir) have a trailing "/"
> reliably? If not, we need to make the destinations be
> "(DESTDIR)$(htdocsdir)/" and (DESTDIR)$(manualdir)/, otherwise on some
> systems rsync will behave differently when those destinations are
> symlinks otherwise. (it will unlink the symlink and replace it with the
> directory heirarchy).
Hmm. Okay - will add the trailing slash to the dest dirs too.
Thanks! -- justin
Re: [PATCH] Use rsync to install manual
Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Fri, Sep 30, 2005 at 10:04:12AM +0100, Colm MacCarthaigh wrote:
> Please don't use the -a option for rsync. use -rlptDv instead;
Oops, force of habit; that shouldn't include the -v option.
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: [PATCH] Use rsync to install manual
Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Thu, Sep 29, 2005 at 10:03:32PM -0700, Justin Erenkrantz wrote:
> This greatly speeds up installation. =) So, any objections to
> committing? -- justin
Yes :)
Please don't use the -a option for rsync. use -rlptDv instead; we
shouldn't be trouncing the ownership of the destination files, cp
doesn't. I'd also add the --numeric-ids option, which will also prevent
rsync from doing the uid lookups and can greatly speed things up on
systems with no (or slow) nscd's (or slow uid lookups in general).
Do (DESTDIR)$(htdocsdir) and (DESTDIR)$(manualdir) have a trailing "/"
reliably? If not, we need to make the destinations be
"(DESTDIR)$(htdocsdir)/" and (DESTDIR)$(manualdir)/, otherwise on some
systems rsync will behave differently when those destinations are
symlinks otherwise. (it will unlink the symlink and replace it with the
directory heirarchy).
But apart from those nits, +1! It always annoys me how long that
takes too :-)
> Index: Makefile.in
> ===================================================================
> --- Makefile.in (revision 292616)
> +++ Makefile.in (working copy)
> @@ -122,8 +122,12 @@
> else \
> echo Installing HTML documents ; \
> $(MKINSTALLDIRS) $(DESTDIR)$(htdocsdir) ; \
> - test -d $(htdocs-srcdir) && (cd $(htdocs-srcdir) && cp -rp * $(DESTDIR)$(htdocsdir)) ; \
> - cd $(DESTDIR)$(htdocsdir) && find . -name ".svn" -type d -print | xargs rm -rf 2>/dev/null || true; \
> + if test -d $(htdocs-srcdir) && test "x$(RSYNC)" != "x" && test -x $(RSYNC) ; then \
> + $(RSYNC) --exclude .svn -a $(htdocs-srcdir)/ $(DESTDIR)$(htdocsdir); \
> + else \
> + test -d $(htdocs-srcdir) && (cd $(htdocs-srcdir) && cp -rp * $(DESTDIR)$(htdocsdir)) ; \
> + cd $(DESTDIR)$(htdocsdir) && find . -name ".svn" -type d -print | xargs rm -rf 2>/dev/null || true; \
> + fi; \
> fi
>
> install-error:
> @@ -201,8 +205,12 @@
> @test -d $(DESTDIR)$(manualdir) || $(MKINSTALLDIRS) $(DESTDIR)$(manualdir)
> @cp -p $(top_srcdir)/docs/man/*.1 $(DESTDIR)$(mandir)/man1
> @cp -p $(top_srcdir)/docs/man/*.8 $(DESTDIR)$(mandir)/man8
> - @(cd $(top_srcdir)/docs/manual && cp -rp * $(DESTDIR)$(manualdir))
> - @(cd $(DESTDIR)$(manualdir) && find . -name ".svn" -type d -print | xargs rm -rf 2>/dev/null ) || true
> + @if test "x$(RSYNC)" != "x" && test -x $(RSYNC) ; then \
> + $(RSYNC) --exclude .svn -a $(top_srcdir)/docs/manual/ $(DESTDIR)$(manualdir); \
> + else \
> + cd $(top_srcdir)/docs/manual && cp -rp * $(DESTDIR)$(manualdir); \
> + cd $(DESTDIR)$(manualdir) && find . -name ".svn" -type d -print | xargs rm -rf 2>/dev/null; \
> + fi
>
> install-suexec:
> @if test -f $(builddir)/support/suexec; then \
> Index: configure.in
> ===================================================================
> --- configure.in (revision 292616)
> +++ configure.in (working copy)
> @@ -297,6 +297,7 @@
>
> AC_PATH_PROG(RM, rm)
> AC_PATH_PROG(PKGCONFIG, pkg-config)
> +AC_PATH_PROG(RSYNC, rsync)
> AC_PROG_AWK
> AC_PROG_LN_S
> AC_CHECK_TOOL(RANLIB, ranlib, true)
> Index: acinclude.m4
> ===================================================================
> --- acinclude.m4 (revision 292616)
> +++ acinclude.m4 (working copy)
> @@ -75,6 +75,7 @@
> APACHE_SUBST(EXTRA_INCLUDES)
> APACHE_SUBST(LIBTOOL)
> APACHE_SUBST(SHELL)
> + APACHE_SUBST(RSYNC)
> APACHE_SUBST(MODULE_DIRS)
> APACHE_SUBST(MODULE_CLEANDIRS)
> APACHE_SUBST(PORT)
>
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: [PATCH] Use rsync to install manual
Posted by Jeff Trawick <tr...@gmail.com>.
On 9/30/05, Justin Erenkrantz <ju...@erenkrantz.com> wrote:
> Sitting around waiting for another install to complete due to the large
> size of our manual, I wrote this patch to switch install-man and
> install-htdocs to use rsync if available.
very cool!
+1 concept (leaving rsync switch discussion to others ;) )