You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mirrors@apache.org by jason andrade <ja...@rtfmconsult.com> on 2003/02/22 04:26:05 UTC
Re: Apache.org mirrors: how-to-mirror update
On Mon, 20 Jan 2003, Joshua Slive wrote:
Hi Joshua,
It looks like the apache master mirror is full again and needs to
be kicked (and/or people idling on it need to be kicked..)
@ERROR: max connections (25) reached - try again later
rsync: connection unexpectedly closed (68 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(151)
Since we can't connect to update we are obviously going to go stale :-/
regards,
-jason
Re: Apache.org mirrors: how-to-mirror update
Posted by jason andrade <ja...@rtfmconsult.com>.
On Sat, 22 Feb 2003, Joshua Slive wrote:
> We can check the server, but we really have no control over the clients.
> We consider the rsync repository to be a public resource, so it is
> difficult to control client access.
which is fair enough but i wasn't suggesting enforcing this - i was
just asking to get a message sent out to mirrors to please upgrade
and make it something for new mirrors to check.
also to add something to the motd (please make sure you are using
the latest version of rsync 2.5.5 blah..)
> I don't think increasing the limit will do any good. When a screwed up
> client comes along, it will eat up any number of available connections.
> Increasing the connections would just increase the damage that the bad
> client does to the server.
i figured that - sometimes when you have more slots (and a lower timeout)
it helps alleviate the pain.
> Having other rsync locations is certainly possible. The only reason I
> haven't personally pushed this is that it increases the complexity of the
> mirroring system, and I like simple things. It would also mean denying
> general rsync access to www.apache.org. But this may be the way to go.
i would limit rsync access to www.apache.org to mirrors only - but i was
more suggesting that you have two sets of mirror shares - one which
any mirror can access (and has a certain user limit) and another which
only some mirrors can access (host acl list).
regards,
-jason
Re: Apache.org mirrors: how-to-mirror update
Posted by Joshua Slive <jo...@slive.ca>.
On Sun, 23 Feb 2003, jason andrade wrote:
> get all the people using rsync to update to the latest version, e.g rsync
> 2.5.5 and make sure that is also running on the server.
We can check the server, but we really have no control over the clients.
We consider the rsync repository to be a public resource, so it is
difficult to control client access.
> also consider lowering the timeout, perhaps to 600
I'll pass that suggestion on to the admin.
> lastly, please consider making two different sharesets available - one
> for "general" mirrors and the other for 'tier1' mirrors who can also
> onfeed to others in their region.
>
> lastly, there are a huge number of apache mirrors - much more than
> most other projects that i know of so the default limit might be
> quite low if all of these are syncing from a single site (the apache
> master site)
I don't think increasing the limit will do any good. When a screwed up
client comes along, it will eat up any number of available connections.
Increasing the connections would just increase the damage that the bad
client does to the server.
Having other rsync locations is certainly possible. The only reason I
haven't personally pushed this is that it increases the complexity of the
mirroring system, and I like simple things. It would also mean denying
general rsync access to www.apache.org. But this may be the way to go.
Thanks for your ideas.
Joshua.
Re: Apache.org mirrors: how-to-mirror update
Posted by jason andrade <ja...@rtfmconsult.com>.
On Sat, 22 Feb 2003, Joshua Slive wrote:
> We cleared this earlier in the day and blocked the offending party, then
> another one came in and blocked everything up. It's getting pretty
> annoying.
it is indeed.
> If anyone has any ideas on what is causing this or how to avoid it, they
> would be welcome.
get all the people using rsync to update to the latest version, e.g rsync
2.5.5 and make sure that is also running on the server.
also consider lowering the timeout, perhaps to 600
lastly, please consider making two different sharesets available - one
for "general" mirrors and the other for 'tier1' mirrors who can also
onfeed to others in their region.
lastly, there are a huge number of apache mirrors - much more than
most other projects that i know of so the default limit might be
quite low if all of these are syncing from a single site (the apache
master site)
regards,
-jason
Re: Apache.org mirrors: how-to-mirror update
Posted by Joshua Slive <jo...@slive.ca>.
On Sun, 23 Feb 2003, Yusmar Yahaya wrote:
> i think it probably relates back to apache.org's "how to mirror apache"
> guide page, where it states that you have to pull the repository between
> 8am GMT till noon GMT.
>
> from the error messages i've got lately, it was during that time frame.
> perhaps if each region were given a time period to rsync, then you can
> roughly predict the traffic size. e.g. each region should rsync between
> 8am-noon at their own timezone, not 8am-noon GMT.
No, but perhaps I haven't been clear about the problem. This isn't a
question of www.apache.org being over capacity. This is a question of a
single client coming in and making multiple rsync connections until it
fills up all sockets. I have no idea why the clients are doing this, but
it probably is some very badly programmed script. The weird thing is that
is happening over and over again from multiple clients all over the world.
Joshua.
Re: Mirror not added?
Posted by Joshua Slive <jo...@slive.ca>.
On Sun, 23 Feb 2003, Neo Wee Teck wrote:
> Hi
>
> I haven seen my mirror in the official list. Please check :)
>
> http://apache.en.com.sg
>
> Singapore Mirror
>
> Maintained by : Neo Wee Teck
> Update : 6hrs interval daily
>
> Email : slash@en.com.sg
Got it. Thanks.
Joshua.
Re: I can't update
Posted by Joshua Slive <jo...@slive.ca>.
On Thu, 27 Feb 2003, Ivan Canizares wrote:
> > Hi
> >
> > I can;t update the packages from apache.org, with this command
> >
> > rsync -rtlzv --delete --exclude incoming www.apache.org::apache-dist
> > /ftp/pub/apache/dist
> >
> > the crontab say this:
> > produced the following output:
> >
> > rsync: failed to connect to www.apache.org: Connection timed out
> > rsync error: error in socket IO (code 10) at clientserver.c(83)
I'm not sure how I can help you here. Other mirror do not seem to be
having this problem. Could it be something on your local network?
Joshua.
Re: I can't update
Posted by Joshua Slive <jo...@slive.ca>.
This is getting irritating. You've sent this message at least three
times, and received at least two replies. What's going on?
On Thu, 27 Feb 2003, Ivan Canizares wrote:
> > Hi
> >
> > I can;t update the packages from apache.org, with this command
> >
> > rsync -rtlzv --delete --exclude incoming www.apache.org::apache-dist
> > /ftp/pub/apache/dist
> >
> > the crontab say this:
> > produced the following output:
> >
> > rsync: failed to connect to www.apache.org: Connection timed out
> > rsync error: error in socket IO (code 10) at clientserver.c(83)
> >
> > Can you help me
> > Bye
> > Ivan Canizares
> >
>
>
Re: I can't update
Posted by Ivan Canizares <ti...@ula.ve>.
> Hi
>
> I can;t update the packages from apache.org, with this command
>
> rsync -rtlzv --delete --exclude incoming www.apache.org::apache-dist
> /ftp/pub/apache/dist
>
> the crontab say this:
> produced the following output:
>
> rsync: failed to connect to www.apache.org: Connection timed out
> rsync error: error in socket IO (code 10) at clientserver.c(83)
>
> Can you help me
> Bye
> Ivan Canizares
>
Re: I can't update
Posted by jason andrade <ja...@rtfmconsult.com>.
On Tue, 25 Feb 2003, Ivan Canizares wrote:
> Hi
>
> I can;t update the packages from apache.org, with this command
>
> rsync -rtlzv --delete --exclude incoming www.apache.org::apache-dist
> /ftp/pub/apache/dist
>
> the crontab say this:
> produced the following output:
>
> failed to connect to www.apache.org - Connection timed out
for a start, turn off compression. instead try
rsync -rltvH --stats --delete --max-delete=1000 --exclude="incoming/" www.apache.org::apache-dist/ /ftp/pub/apache/dist/
i would also recommend you pick a closer mirror if possible, to sync from
other than www.apache.org (but perhaps there are no other mirrors in
.ve)
regards,
-jason
I can't update
Posted by Ivan Canizares <ti...@ula.ve>.
Hi
I can;t update the packages from apache.org, with this command
rsync -rtlzv --delete --exclude incoming www.apache.org::apache-dist
/ftp/pub/apache/dist
the crontab say this:
produced the following output:
failed to connect to www.apache.org - Connection timed out
Can you help me
Bye
Ivan Canizares
Re: Mirror not added?
Posted by Joshua Slive <jo...@slive.ca>.
On Tue, 25 Feb 2003, Yusmar Yahaya wrote:
> loc : Kuala Lumpur, Malaysia (+8GMT)
>
> URL : http://mirror.leafbug.org/apache-dist
> (rsync www.apache.org::apache-dist)
>
> freq. : twice daily at 8am GMT and 8pm GMT (4:30am MYT & 4:30pm MYT)
>
> link : relocated from 1.5bps SDSL to 2mbps dedicated leasedline
> yesterday. UPS backed, mirrored storage disk.
>
> svc date : since 14 Feb 2003.
Got it. Thanks.
Joshua.
Re: Mirror not added?
Posted by Yusmar Yahaya <yu...@boolean.pc.my>.
On Mon, 24 Feb 2003, Joshua Slive wrote:
>
> On Sun, 23 Feb 2003, Neo Wee Teck wrote:
> > Hi
> > I haven seen my mirror in the official list. Please check :)
> > http://apache.en.com.sg
>
> If nobody beats me to it, I'll try to get to this soon.
>
And when you're updating the list, I hope you could please update mine
into that as well? thanks, as my server is not yet updated.
BTW, here's the info again.
loc : Kuala Lumpur, Malaysia (+8GMT)
URL : http://mirror.leafbug.org/apache-dist
(rsync www.apache.org::apache-dist)
freq. : twice daily at 8am GMT and 8pm GMT (4:30am MYT & 4:30pm MYT)
link : relocated from 1.5bps SDSL to 2mbps dedicated leasedline
yesterday. UPS backed, mirrored storage disk.
svc date : since 14 Feb 2003.
Thank you.
-- _______________________
~Yusmar Yahaya yus@boolean.pc.my
http://leafbug.org/~yus
Re: Mirror not added?
Posted by Joshua Slive <jo...@slive.ca>.
On Sun, 23 Feb 2003, Neo Wee Teck wrote:
> Hi
>
> I haven seen my mirror in the official list. Please check :)
>
> http://apache.en.com.sg
>
If nobody beats me to it, I'll try to get to this soon.
Joshua.
Mirror not added?
Posted by Neo Wee Teck <sl...@en.com.sg>.
Hi
I haven seen my mirror in the official list. Please check :)
http://apache.en.com.sg
Singapore Mirror
Maintained by : Neo Wee Teck
Update : 6hrs interval daily
Email : slash@en.com.sg
Re: Apache.org mirrors: how-to-mirror update
Posted by Yusmar Yahaya <yu...@boolean.pc.my>.
On Sat, 22 Feb 2003, Joshua Slive wrote:
> We cleared this earlier in the day and blocked the offending party, then
> another one came in and blocked everything up. It's getting pretty
> annoying.
>
> If anyone has any ideas on what is causing this or how to avoid it, they
> would be welcome.
>
i think it probably relates back to apache.org's "how to mirror apache"
guide page, where it states that you have to pull the repository between
8am GMT till noon GMT.
from the error messages i've got lately, it was during that time frame.
perhaps if each region were given a time period to rsync, then you can
roughly predict the traffic size. e.g. each region should rsync between
8am-noon at their own timezone, not 8am-noon GMT.
my 2cents.
-- _______________________
~Yusmar Yahaya yus@boolean.pc.my
http://leafbug.org/~yus
Re: Apache.org mirrors: how-to-mirror update
Posted by Joshua Slive <jo...@slive.ca>.
We cleared this earlier in the day and blocked the offending party, then
another one came in and blocked everything up. It's getting pretty
annoying.
If anyone has any ideas on what is causing this or how to avoid it, they
would be welcome.
Anyway, I passed this on to the admins and they will deal with it.
Joshua.
On Sat, 22 Feb 2003, Haesu wrote:
> I am noticing the same issue right now...
>
> We are unable to retrieve the latest updates for apache projects.. We are
> going stale.. :-/ I am getting connection full from the rsync server..
>
> Best regards,
> -hc
>
>
> On Sat, 22 Feb 2003, jason andrade wrote:
>
> > On Mon, 20 Jan 2003, Joshua Slive wrote:
> >
> > Hi Joshua,
> >
> > It looks like the apache master mirror is full again and needs to
> > be kicked (and/or people idling on it need to be kicked..)
> >
> > @ERROR: max connections (25) reached - try again later
> > rsync: connection unexpectedly closed (68 bytes read so far)
> > rsync error: error in rsync protocol data stream (code 12) at io.c(151)
> >
> >
> > Since we can't connect to update we are obviously going to go stale :-/
> >
> > regards,
> >
> > -jason
> >
>
Re: Apache.org mirrors: how-to-mirror update
Posted by Haesu <ha...@towardex.com>.
I am noticing the same issue right now...
We are unable to retrieve the latest updates for apache projects.. We are
going stale.. :-/ I am getting connection full from the rsync server..
Best regards,
-hc
On Sat, 22 Feb 2003, jason andrade wrote:
> On Mon, 20 Jan 2003, Joshua Slive wrote:
>
> Hi Joshua,
>
> It looks like the apache master mirror is full again and needs to
> be kicked (and/or people idling on it need to be kicked..)
>
> @ERROR: max connections (25) reached - try again later
> rsync: connection unexpectedly closed (68 bytes read so far)
> rsync error: error in rsync protocol data stream (code 12) at io.c(151)
>
>
> Since we can't connect to update we are obviously going to go stale :-/
>
> regards,
>
> -jason
>