You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mirrors@apache.org by "Henk P. Penning" <he...@cs.uu.nl> on 2000/04/03 09:18:39 UTC
error during rsync - web page
Hi,
the past few days I get an error rsync'ing the archive:
receiving file list ... opendir(bugs/shadow): Permission denied
done
IO error encountered - skipping file deletion
bugs/data/
skipping non-regular file dist/404.html
skipping non-regular file dist/binaries/macosxserver
skipping non-regular file dist/binaries/osf1
skipping non-regular file dist/binaries/sinix
skipping non-regular file docs/404.html
foundation/
...........
-- something is wrong with directory 'bugs/shadow'.
-- the 'skipping ...' lines have been there for a long time.
Why are they 'non-regular' ?
-- the mirror web page http://www.apache.org/info/how-to-mirror.html
still gives the old majordomo address related to this list.
-- How do I email the person/people taking care
of the archive, mirroring etc.
Is there an address other than this list ?
thanks, regards.
Henk Penning
Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \__/ \
Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/ \__/
Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \__/ \
News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/ \__/
Re: error during rsync - web page
Posted by Costas Tavernarakis <ta...@otenet.gr>.
On Tue, Apr 04, 2000 at 05:08:52PM -0500, Chase Phillips wrote:
> rsync's man page states that -a breaks down into -rlptg, which can be
> unsafe without some additional flags. unsafe of course only if the
> sender's machine has been compromised. Without knowing rsync's default
> behaviour, I do endorse changing the recommended flags to allow for
> symlinks; perhaps to:
>
> rsync -rtvzl --delete --safe-links --exclude incoming \
> dev.apache.org::apache-site /local/path/to/mirror
>
Well, the -p part of the -a expansion, is at least unnecessary for most
sites, and having a smaller command line is not a good enough reason to
promote its use IMHO.
Otherwise, the command line for most sites should most propably be
rsync -rtvzlHS --delete --safe-links --exclude /incoming \
dev.apache.org::apache-site/ /local/path/to/mirror
The -S is "no-harm" and may prove useful in a "just in case" way,
if a "core" file gets dropped somewhere in the tree.
The -H would help preserve hard links if they exist.
In the long run, the site could change the few soft links it
has to hard ones. That way, mirror sites could have some
improved security, since they wouldn't need the
"SymLinksIfOwnerMatch" Option in the Directory section
of httpd.conf.
I wonder, though, if -z does anything more than increase the
CPU utilization on both the sending and the receiving sides..
> > receiving file list ... opendir(bugs/shadow): Permission denied
> > done
> > IO error encountered - skipping file deletion
> >
> > Note the 'IO error - skipping file deletion'.
> >
>
> referring to rsync's man page, i see that once an IO error is encountered
> on the sending side, deletion is turned off at the receiving side for the
> rest of that session. is there a problem at dev.apache.org with a part of
> the filesystem? if not, of what is this symptomatic?
The I/O error IS the "permission denied" error on bugs/shadow .
I've had to exclude this path to get my mirror working right.
Costas Tavernarakis
apache.otenet.gr admin
Re: error during rsync - web page
Posted by Chase Phillips <sh...@ameth.org>.
> says: You might want to call it, ..., for example
>
> rsync -rtvz --delete --exclude incoming \
> dev.apache.org::apache-site /local/path/to/mirror
>
> I took it as a recommandation.
> Maybe the example should say: rsync -avz ... ...
rsync's man page states that -a breaks down into -rlptg, which can be
unsafe without some additional flags. unsafe of course only if the
sender's machine has been compromised. Without knowing rsync's default
behaviour, I do endorse changing the recommended flags to allow for
symlinks; perhaps to:
rsync -rtvzl --delete --safe-links --exclude incoming \
dev.apache.org::apache-site /local/path/to/mirror
if -a is endorsed, maybe a quick note about the presence of security
issues could be stated for those new to rsync?
> receiving file list ... opendir(bugs/shadow): Permission denied
> done
> IO error encountered - skipping file deletion
>
> Note the 'IO error - skipping file deletion'.
>
> I never see one without the other, so they could be related.
> I think 'skipping file deletion' is to be avoided.
referring to rsync's man page, i see that once an IO error is encountered
on the sending side, deletion is turned off at the receiving side for the
rest of that session. is there a problem at dev.apache.org with a part of
the filesystem? if not, of what is this symptomatic?
fyi, i run rsync as:
> /usr/local/bin/rsync -rltvzp --delete --safe-links --exclude incoming \
> dev.apache.org::apache-site /ftp/pub/mirrors/ftp.apache.org
the end of an rsync run earlier today gave:
> wrote 13583 bytes read 29446 bytes 3442.32 bytes/sec
> total size is 360113950 speedup is 8369.10
i'd love for the IO error to be a non-issue, but some deeper problem may
be waiting to be discovered. could someone at dev.apache.org check this
one out more thoroughly?
thanks for your time,
Chase Phillips
--
shepard at ameth.org ][ 01110010011000010110011101100101
http://www.ameth.org/ ][ 11001110100101100111100100110001
Re: [qmirror] Re: error during rsync - web page
Posted by jason andrade <ja...@dstc.edu.au>.
On Tue, 4 Apr 2000, Henk P. Penning wrote:
> receiving file list ... opendir(bugs/shadow): Permission denied
> done
> IO error encountered - skipping file deletion
> bugs/data/
> skipping non-regular file dist/404.html
> skipping non-regular file dist/binaries/macosxserver
> skipping non-regular file dist/binaries/osf1
> skipping non-regular file dist/binaries/sinix
> skipping non-regular file docs/404.html
> foundation/
>
> Note the 'IO error - skipping file deletion'.
try adding
--exclude="shadow/"
or maybe
--exclude="bugs/shadow/"
> I never see one without the other, so they could be related.
> I think 'skipping file deletion' is to be avoided.
yes - i once ended up an extra 5G of data on a disk because i didn't
realise rsync was _really_ not deleting files..
-jason
Re: error during rsync - web page
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Tue, 4 Apr 2000, Manoj Kasichainula wrote:
> Date: Tue, 4 Apr 2000 01:42:15 -0500
> From: Manoj Kasichainula <ma...@io.com>
> To: Pierpaolo Fumagalli <pi...@apache.org>
> Cc: mirrors@apache.org, Ryan Bloom <rb...@apache.org>,
> Henk P. Penning <he...@cs.uu.nl>
> Subject: Re: error during rsync - web page
> Sender: manojk@dillinger.io.com
>
> On Mon, Apr 03, 2000 at 02:44:02PM -0700, Pierpaolo Fumagalli wrote:
> > You guys (Ryan/Manoj) know why this is happening?
> >
> > "Henk P. Penning" <he...@cs.uu.nl> wrote:
> > > -- something is wrong with directory 'bugs/shadow'.
>
> /www/jakarta.apache.org/bugs/shadow is only readable by root and
> nobody. rsync --server seems to be running as brian (though there are
> other rsync processes running as nobody).
>
> But are the mirrors supposed to be trying to mirror the bugdb CGIs?
> I'd think that would be dangerous, because it would fragment the
> bugdb.
It sounds dangerous :-).
The problem is I get
receiving file list ... opendir(bugs/shadow): Permission denied
done
IO error encountered - skipping file deletion
bugs/data/
skipping non-regular file dist/404.html
skipping non-regular file dist/binaries/macosxserver
skipping non-regular file dist/binaries/osf1
skipping non-regular file dist/binaries/sinix
skipping non-regular file docs/404.html
foundation/
Note the 'IO error - skipping file deletion'.
I never see one without the other, so they could be related.
I think 'skipping file deletion' is to be avoided.
> > > -- the 'skipping ...' lines have been there for a long time.
> > > Why are they 'non-regular' ?
>
> They're symlinks. -l will allow rsyncing them over properly (though I
> use -a, which preserves lots of other attributes as well).
Ok, I see.
The apache mirror page
http://www.apache.org/info/how-to-mirror.html
says: You might want to call it, ..., for example
rsync -rtvz --delete --exclude incoming \
dev.apache.org::apache-site /local/path/to/mirror
I took it as a recommandation.
Maybe the example should say: rsync -avz ... ...
>
The mirror mail list address is still listed as 'majordomo@apache.org'.
Thanks for your efforts.
regards.
Henk Penning
Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \__/ \
Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/ \__/
Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \__/ \
News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/ \__/
Re: error during rsync - web page
Posted by Manoj Kasichainula <ma...@io.com>.
On Mon, Apr 03, 2000 at 02:44:02PM -0700, Pierpaolo Fumagalli wrote:
> You guys (Ryan/Manoj) know why this is happening?
>
> "Henk P. Penning" <he...@cs.uu.nl> wrote:
> > -- something is wrong with directory 'bugs/shadow'.
/www/jakarta.apache.org/bugs/shadow is only readable by root and
nobody. rsync --server seems to be running as brian (though there are
other rsync processes running as nobody).
But are the mirrors supposed to be trying to mirror the bugdb CGIs?
I'd think that would be dangerous, because it would fragment the
bugdb.
> > -- the 'skipping ...' lines have been there for a long time.
> > Why are they 'non-regular' ?
They're symlinks. -l will allow rsyncing them over properly (though I
use -a, which preserves lots of other attributes as well).
Re: error during rsync - web page
Posted by Pierpaolo Fumagalli <pi...@apache.org>.
You guys (Ryan/Manoj) know why this is happening?
Pier
"Henk P. Penning" <he...@cs.uu.nl> wrote:
>
> Hi,
>
> the past few days I get an error rsync'ing the archive:
>
> receiving file list ... opendir(bugs/shadow): Permission denied
> done
> IO error encountered - skipping file deletion
> bugs/data/
> skipping non-regular file dist/404.html
> skipping non-regular file dist/binaries/macosxserver
> skipping non-regular file dist/binaries/osf1
> skipping non-regular file dist/binaries/sinix
> skipping non-regular file docs/404.html
> foundation/
> ...........
>
> -- something is wrong with directory 'bugs/shadow'.
>
> -- the 'skipping ...' lines have been there for a long time.
> Why are they 'non-regular' ?
>
> -- the mirror web page http://www.apache.org/info/how-to-mirror.html
> still gives the old majordomo address related to this list.
>
> -- How do I email the person/people taking care
> of the archive, mirroring etc.
> Is there an address other than this list ?
>
> thanks, regards.
>
> Henk Penning
>
> Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \__/ \
> Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/ \__/
> Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \__/ \
> News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/ \__/
--
----------------------------------------------------------------------
pier: stable structure erected over water to allow docking of seacraft
<ma...@betaversion.org> <http://www.betaversion.org/~pier/>
----------------------------------------------------------------------