You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mirrors@apache.org by Gorm Jorgensen <Go...@Area51.DK> on 2001/09/29 12:58:36 UTC
Problems updating via rsync
I'm having problems updating ::apache-site via rsync from dev.apache.org
I have received the same error the last threee days, no matter what time I
try to update.
@ERROR: max connections (25) reached - try again later
Anyone else having problems ?
--
Gorm Jorgensen - Area51.DK
d+ s: a-- C+++ UB++++ P+ L++ E--- W+++ N+ w--- O- M-- V-- PGP++ tv- G--
Re: Problems updating via rsync
Posted by jason andrade <ja...@dstc.edu.au>.
On Mon, 1 Oct 2001, Henk P. Penning wrote:
> I've tried to keep my mouth shut for a long time; I agreed
> with the effort that were made to improve the situation;
> even tried to calm down those who were pushing too hard;
> but now I've had it.
shrug. i lost it a while ago, but there's only so much effort
i am willing to put into a project now, before i give up.
> Well, actually your wish is granted. The new crud is all in
> 'dist/jakarta/'. So let's consider that issue decided.
not really. these are the dist files (which users tell me they
can't even find/understand on our mirror and they just go back
to apache.org. the dist/jakarta doesn't appear to be anything
like the jakarta.apache.org web pages. i want it consistent
as a mirror, otherwise what's the point.
> I would consider it unacceptable if a mirror would have to set
> up 10 to 15 virtual hosts for just one project.
i used to, but my frustation level with this means i don't really
care - i'll now do whatever to get a working full apache mirror.
obviously needing to setup a separate vhost for each apache subproject
is less than sensible, but unless there is a publish system putting
them all under one tree, i don't see how to avoid that.
> I would prefer a top division of
>
> -- dist/ stuff for distribution
> -- hist/ historical stuff
> -- devel/ stuff in development, beta
>
> It would make mirroring so easy. If the projects do not agree,
> I would like to see a dist/hist/devel split per project;
> The mirrors should 'exclude' 'hist' and 'devel' to avoid
> problems with bad mirrors.
there should be no "projects do not agree". it's a template and
people work within it.
> Works fine for CPAN (perl stuff).
coincidentally, ask is a CPAN developer, so i assume it's the same
system in use at both cpan and apache.org. but for some reason at
apache, picking the closest mirror usually gets me mexico or somewhere
else (i'm in australia and maintain two local apache mirrors.. go figure)
> Just my thoughts. I think I like the idea of a mirror template box.
> Let apache.org try to mirror themselves; they'll find out
> soon enough how set up things properly.
that's the whole point of setting up a master site - so that every other
site is a mirror, including www.apache.org and it can be rotated through
RR or have a www1.apache.org -> wwwX.apache.org.
> PS mirror status: I've been busy all day to get my mirror into sync.
> It can't be done. and it's useless because of errors in 'index.html'.
i started doing large amounts of search and replace with the html files
in the apache mirror to try and point things at actually locally mirrored
references rather than back to apache.org all the time. i've given up -
there is no way to keep up with changes.
-jason
Re: Problems updating via rsync
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Mon, 1 Oct 2001, jason andrade wrote:
> Date: Mon, 1 Oct 2001 07:16:32 +1000 (EST)
> From: jason andrade <ja...@dstc.edu.au>
> To: mirrors@apache.org
> Subject: Re: Problems updating via rsync
> Sender: mirrors-return-371-henkp=cs.uu.nl@apache.org
>
> On Sun, 30 Sep 2001, Henk P. Penning wrote:
>
> > > Ohh no, Apache guys is there attention on this problem ?
> >
> > -- The 'dist' archive is a big mess now: current stuff, old stuff,
> > devel stuff; it's all there. It appears, disappears,
> > appears again.
> >
> > -- there is not a byte of guidance or documentation
> >
> > -- the server/connection is very, very slow.
> >
> > Isn't there someone with an ounce of brains to organise this ?
>
>
> a few of us have been asking this for more than a year now - perhaps
> not quite like that. andrew kenna has come on board recently and
> started to help - i think that the apache group (well, brian and ask
> who seem to reply more often than most on this list) just don't have
> enough time to address this. there appear to only be around half
> a dozen `active' mirrors in terms of commenting on the list. does
> that mean the consensus is that the majority think the current setup
> is fine and the rest of us are wasting time debating it.
I've tried to keep my mouth shut for a long time; I agreed
with the effort that were made to improve the situation;
even tried to calm down those who were pushing too hard;
but now I've had it.
The stupid errors in de new index page did it for me;
clearly those in charge are clueless as to the needs
of mirror sites.
All the mirrors have been completely useless for years;
now someone improves the index pages and fucks up (again).
> i'd personally like to see:
>
>
> o apache web site which allows mirroring of *all* sub projects either
> as apache.localmirror.foo/project/ or project.apache.localmirror.foo.
>
> i've given up on really wanting the former - i'll even setup the 10 or 15
> virtual servers needed for the latter now :-/. it's more work/less
> automatic than how apache previously used to work, but if that is what
> is needed, so be it.
Well, actually your wish is granted. The new crud is all in
'dist/jakarta/'. So let's consider that issue decided.
Besides, if the projects want something else, how hard would it
be to set up one authorative template mirror site? I mean,
set up a box, collect the right stuff in the right place and
mirror everything from there! I would give the job to the same
guys that run 'http://www.apache.org/' and ask them to make
it work (look ma, no politics).
I would consider it unacceptable if a mirror would have to set
up 10 to 15 virtual hosts for just one project.
> o decide if apache is going to stay as a ftp and web site all in one or
> separate the functions out. make the ftp site a pure distribution
> system (no html files..) and have a separate web site that can be
> mirrored in a much more coherent fashion.
I would prefer a top division of
-- dist/ stuff for distribution
-- hist/ historical stuff
-- devel/ stuff in development, beta
It would make mirroring so easy. If the projects do not agree,
I would like to see a dist/hist/devel split per project;
The mirrors should 'exclude' 'hist' and 'devel' to avoid
problems with bad mirrors.
I see no reason why a proper index.html file per 'dist' (sub) directory
would be so hard to maintain.
> o similar to other projects, have a tiered mirroring system. have a master
> apache site that is only contactable by tier-1 mirrors and then use those
> to propgate out regionally.
Works fine for CPAN (perl stuff).
> at the moment apache is a approximately a 4G archive. it's mirrorable
> by a lot of sites, but keeping track of all of this looks to have become
> a nightmare.
As I've said before, I would't mind if only full mirrors were allowed.
I assume that a clean 'dist/' tree would take up much less.
Moreover, 4 GB costs nothing, and an rsync check is quick enough.
A template mirror box wold be cheap, easy to admin and can easily
handle 200+ rsynced mirrors a day.
Just my thoughts. I think I like the idea of a mirror template box.
Let apache.org try to mirror themselves; they'll find out
soon enough how set up things properly.
> -jason
regards.
Henk Penning
PS mirror status: I've been busy all day to get my mirror into sync.
It can't be done. and it's useless because of errors in 'index.html'.
Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \
Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/
Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \
News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/
Re: Total incompetence (was Problems updating via rsync)
Posted by The Flying Hamster <ha...@vom.tm>.
On Mon, Oct 01, 2001 at 05:49:07PM +0200, Bert Vermeulen wrote:
[...]
> > Most importantly: HAVE SOMEONE RESPONSIBLE FOR THE MIRROR, who is on list
> > list.
>
> That's the most important point, allright. For an example of how it should
> be done, just look at the Proftpd project. The person responsible for this
> has done a wonderful job, and has managed to get some 200 mirrors working
> perfectly. Something wrong with your mirror? You get a mail.
Scripts and automation are your friends in the wilderness, I've a
stack of other plans for making the work required reduce to close to
zero but as always time is the problem. If anyone wants the scripts
and bits which help me maintain the mirror list and DNS for
www|ftp.*.proftpd.org just shout, they're available for the asking.
BTW thanks for the ego boost, though it's actually only about 140
mirrors (that's counting both www & ftp mirrors) ;)
Mark
--
The Flying Hamster <ha...@suespammers.org> http://hamster.wibble.org/
I don't suffer from insanity, I enjoy every minute of it.
Re: Total incompetence (was Problems updating via rsync)
Posted by Bert Vermeulen <be...@be.easynet.net>.
On Mon, 1 Oct 2001, Tornoci Laszlo wrote:
> Lots of people don't comment, because the problem is a long standing one,
> and we haven't seen any real reaction from the apache people to the
> complaints raised here. It is beyond my understanding, how a very good and
> successful open source project with the backing of firms like IBM can show
> such a total incompetence in dealing with a mirror structure.
> Apache guys! PLEASE DO SOMETHING WITH THE MIRROR STRUCTURE! Don't
> start lengthy discussions about what would be the 'optimal' way of
> mirroring! Almost anything is better than the current situation!
I wouldn't perhaps put it in quite those words, but you're absolutely right.
> Most importantly: HAVE SOMEONE RESPONSIBLE FOR THE MIRROR, who is on list
> list.
That's the most important point, allright. For an example of how it should
be done, just look at the Proftpd project. The person responsible for this
has done a wonderful job, and has managed to get some 200 mirrors working
perfectly. Something wrong with your mirror? You get a mail.
The comparison with the Apache project mirroring is just painful.
Bert Vermeulen
bert.vermeulen@be.easynet.net
Total incompetence (was Problems updating via rsync)
Posted by Tornoci Laszlo <to...@xenia.sote.hu>.
On Mon, 1 Oct 2001, jason andrade wrote:
> On Sun, 30 Sep 2001, Henk P. Penning wrote:
> >
> > Isn't there someone with an ounce of brains to organise this ?
>
>
> a few of us have been asking this for more than a year now - perhaps
> not quite like that. andrew kenna has come on board recently and
> started to help - i think that the apache group (well, brian and ask
> who seem to reply more often than most on this list) just don't have
> enough time to address this. there appear to only be around half
> a dozen `active' mirrors in terms of commenting on the list. does
> that mean the consensus is that the majority think the current setup
> is fine and the rest of us are wasting time debating it.
Lots of people don't comment, because the problem is a long standing one,
and we haven't seen any real reaction from the apache people to the
complaints raised here. It is beyond my understanding, how a very good and
successful open source project with the backing of firms like IBM can show
such a total incompetence in dealing with a mirror structure.
Apache guys! PLEASE DO SOMETHING WITH THE MIRROR STRUCTURE! Don't
start lengthy discussions about what would be the 'optimal' way of
mirroring! Almost anything is better than the current situation!
Most importantly: HAVE SOMEONE RESPONSIBLE FOR THE MIRROR, who is on list
list.
Yours: Laszlo
----------------------------------------------------------------------
Laszlo L Tornoci Inst Pathophysiology
E-mail: torlasz@xenia.sote.hu Semmelweis Univ Med School
torlasz@net.sote.hu Nagyvarad ter 4.
fax: (36-1)-210-4409 Budapest, H-1089, Hungary
----------------------------------------------------------------------
Re: Problems updating via rsync
Posted by jason andrade <ja...@dstc.edu.au>.
On Sun, 30 Sep 2001, Henk P. Penning wrote:
> > Ohh no, Apache guys is there attention on this problem ?
>
> -- The 'dist' archive is a big mess now: current stuff, old stuff,
> devel stuff; it's all there. It appears, disappears,
> appears again.
>
> -- there is not a byte of guidance or documentation
>
> -- the server/connection is very, very slow.
>
> Isn't there someone with an ounce of brains to organise this ?
a few of us have been asking this for more than a year now - perhaps
not quite like that. andrew kenna has come on board recently and
started to help - i think that the apache group (well, brian and ask
who seem to reply more often than most on this list) just don't have
enough time to address this. there appear to only be around half
a dozen `active' mirrors in terms of commenting on the list. does
that mean the consensus is that the majority think the current setup
is fine and the rest of us are wasting time debating it.
i am not sure how we get this addressed. offering bandwidth and
time only goes so far - there needs to be an interest in the apache
group itself to want to organize the site and projects in some
way that actually scales for the next 2 years and i'm sure there are
probably politics involved.
i'd personally like to see:
o apache web site which allows mirroring of *all* sub projects either
as apache.localmirror.foo/project/ or project.apache.localmirror.foo.
i've given up on really wanting the former - i'll even setup the 10 or 15
virtual servers needed for the latter now :-/. it's more work/less
automatic than how apache previously used to work, but if that is what
is needed, so be it.
o decide if apache is going to stay as a ftp and web site all in one or
separate the functions out. make the ftp site a pure distribution
system (no html files..) and have a separate web site that can be
mirrored in a much more coherent fashion.
o similar to other projects, have a tiered mirroring system. have a master
apache site that is only contactable by tier-1 mirrors and then use those
to propgate out regionally.
at the moment apache is a approximately a 4G archive. it's mirrorable
by a lot of sites, but keeping track of all of this looks to have become
a nightmare.
-jason
Re: Problems updating via rsync
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Sun, 30 Sep 2001, Gorm Jorgensen wrote:
> Date: Sun, 30 Sep 2001 21:37:48 +0200
> From: Gorm Jorgensen <Go...@Area51.DK>
> To: mirrors@apache.org
> Subject: Re: Problems updating via rsync
> Sender: mirrors-return-369-henkp=cs.uu.nl@apache.org
>
> Quoting Dmitry Morozovsky (marck@rinet.ru):
> >
> > Moreover, I've stopped rsync after noticing that it's mirroring the whole
> > structure (about 2G) two times yesterday.
> >
> Ohh no, Apache guys is there attention on this problem ?
-- The 'dist' archive is a big mess now: current stuff, old stuff,
devel stuff; it's all there. It appears, disappears,
appears again.
-- there is not a byte of guidance or documentation
-- the server/connection is very, very slow.
Isn't there someone with an ounce of brains to organise this ?
> Gorm Jorgensen - Area51.DK
regards.
Henk Penning
Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \
Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/
Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \
News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/
Re: Problems updating via rsync
Posted by Gorm Jorgensen <Go...@Area51.DK>.
Quoting Dmitry Morozovsky (marck@rinet.ru):
>
> Moreover, I've stopped rsync after noticing that it's mirroring the whole
> structure (about 2G) two times yesterday.
>
Ohh no, Apache guys is there attention on this problem ?
--
Gorm Jorgensen - Area51.DK
d+ s: a-- C+++ UB++++ P+ L++ E--- W+++ N+ w--- O- M-- V-- PGP++ tv- G--
Re: Problems updating via rsync
Posted by Dmitry Morozovsky <ma...@rinet.ru>.
On Sat, 29 Sep 2001, Paul Schuh wrote:
Moreover, I've stopped rsync after noticing that it's mirroring the whole
structure (about 2G) two times yesterday.
PS> I'm having the same problem. Last two days.
PS>
PS> -- Paul
PS>
PS> On Sat, 29 Sep 2001, Gorm Jorgensen wrote:
PS>
PS> >
PS> > I'm having problems updating ::apache-site via rsync from dev.apache.org
PS> > I have received the same error the last threee days, no matter what time I
PS> > try to update.
PS> >
PS> > @ERROR: max connections (25) reached - try again later
PS> >
PS> > Anyone else having problems ?
PS> >
PS> >
PS>
PS> --
PS> -=* Paul Schuh, CISSP http://lime.ece.cornell.edu/
PS> -=* Network Engineer
PS> -=* School of Electrical and Computer Engineering, Cornell University
PS>
Sincerely,
D.Marck [DM5020, DM268-RIPE, DM3-RIPN]
------------------------------------------------------------------------
*** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru ***
------------------------------------------------------------------------
Re: Problems updating via rsync
Posted by Paul Schuh <sc...@ece.cornell.edu>.
I'm having the same problem. Last two days.
-- Paul
On Sat, 29 Sep 2001, Gorm Jorgensen wrote:
>
> I'm having problems updating ::apache-site via rsync from dev.apache.org
> I have received the same error the last threee days, no matter what time I
> try to update.
>
> @ERROR: max connections (25) reached - try again later
>
> Anyone else having problems ?
>
>
--
-=* Paul Schuh, CISSP http://lime.ece.cornell.edu/
-=* Network Engineer
-=* School of Electrical and Computer Engineering, Cornell University