You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mirrors@apache.org by Joshua Slive <jo...@slive.ca> on 2002/08/11 19:11:57 UTC
Re: Mirror created
Amish K. Munshi wrote:
> hi,
>
> I have created a mirror of the FTP site at
> http://mirror.linux-site.net/mirror/apache
> I did mail to the requested email address before 2 days, The
> instructions page asked to be patient, I would like to know how long
> does it usually take to add the URL to their site.
> Is there a way to tell weather there no problem in the mailing.
The mirror request address doesn't seem to be well looked-after at the
moment, but I can add the mirror. However, it appears that your mirror
is not very up-to-date at the moment. In particular, the httpd
directory seems both out-of-date, and seems to be missing files even
from the date it was mirrored.
If you can fix that, and also let me know in what country your mirror is
based, I will add it to the list.
Joshua.
Re: Mirror created
Posted by Ravi Simhambhatla <ra...@simhams.com>.
Hi,
I had send an email some time ago to the list regarding the apache mirror I
created. However, I have not received any response. I was wondering if an
admin could look into the mirror to ensure it's integrity. Details:
Server: http://apache.simhams.com
Httpd version: 1.3.22
Os: Redhat Linux 7.2
Mirroring Technique: rsync
Refresh period: daily
Refresh time: 4:00am Pacific
Please let me know when you have added this mirror to your list.
Cheers,
Ravi
> On Mon, 12 Aug 2002, Joshua Slive wrote:
>
> > Date: Mon, 12 Aug 2002 13:46:15 -0400
> > From: Joshua Slive <jo...@slive.ca>
> > To: "mirrors@apache.org" <mi...@apache.org>
> > Subject: Re: Mirror created
> >
> > Aaron Bannert wrote:
>
> > > We really need to do something about this. I stopped running the
> > > jakarta and xml site mirrors to my machine because the nightly
> > > deltas were so huge. It would be great if we could get the
> > > dist directories to only be official project distributions,
> > > so we could mirror just that instead of things like automated
> > > gump builds.
> >
> > Yes, certainly. When I have discussed this with the jakarta people, we
> > have been clear that nightly builds cannot go out to the mirrors. The
> > other question, however, is whether all the old release (or pre-release)
> > builds should stay up on the mirrors (as they are for httpd) or if we
> > should prune down to just current releases. I have mixed feelings about
> > that.
>
> Hi,
>
> this was discussed here a year ago; is there an archive of this
> list ?
>
> I would strongly favor that the projects split their archive in
>
> -- hist/ historical stuff
> -- dist/ current stuff
> -- devel/ development stuff
>
> Mirrors should only carry the 'dist' stuff.
>
> -- For mirror users, all the 'hist' and 'devel' stuff is very confusing.
> -- Those interested in 'hist' and 'devel' would go to the source
> anyway, and not trust any mirror to "have everything, always".
>
> I have no idea how feasible/realistic this is, but I think
> this is what the mirrors and average users want.
>
> Note that I don't think diskspace or bandwidth are the problem,
> although a year ago it took more than 2 days continuous mirroring
> to get to 75% sync with the jakarta archive (due to performance
> problems on the jakarta end).
>
> > Joshua.
>
> regards.
>
> Henk Penning
>
> Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \
> Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/
> Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \
> News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/
Re: Mirror created
Posted by Pier Fumagalli <pi...@apache.org>.
"Aaron Bannert" <aa...@clove.org> wrote:
> On Mon, Aug 12, 2002 at 06:55:49PM +0100, Pier Fumagalli wrote:
>> How can I figure out what the delta is? I'm mirroring world on Nagoya, and
>> I have the ability to make something happen on jakarta-site...
>
> I just read the output of my rsync scripts every day (cron job email).
Ah... :) I don't send out the output if no errors occur! :) Let's see if
without "--quiet" it tells me some stats! :)
Pier
Re: Mirror created
Posted by Aaron Bannert <aa...@clove.org>.
On Mon, Aug 12, 2002 at 06:55:49PM +0100, Pier Fumagalli wrote:
> How can I figure out what the delta is? I'm mirroring world on Nagoya, and
> I have the ability to make something happen on jakarta-site...
I just read the output of my rsync scripts every day (cron job email).
-aaron
Re: Mirror reorganization
Posted by Aaron Bannert <aa...@clove.org>.
On Tue, Aug 13, 2002 at 02:05:24PM -0400, Cliff Woolley wrote:
> Urm, ouch. I have to claim responsibility for that one. I knew the
> mirrors would have to resync but I didn't think it was that big (and
> didn't bother to count MB's). :-(
This isn't your fault, it's the fault of the way we organize our files
and directories. The fact that we ever move files around after they
become "old" means that we are doubling the amount of sync bandwidth
over time. We eat this all at once (like last night) or slowly, but it
still eats unnecessary bandwidth.
Regardless, let's kill 6 (+/- 2) birds with one stone and fix it all
at once. :) Besides, coming up with a consistent scheme makes it easier
for everyone, and if we can solve daedalus' bandwidth problems at
the same time let's do it.
-aaron
Re: Mirror reorganization
Posted by "Karsten W. Rohrbach" <ka...@rohrbach.de>.
Brian Behlendorf(brian@collab.net)@2002.08.16 20:55:19 +0000:
> > On another note, how are other applications handling this sort of
> > mirroring? I know for one I use Debian and they actually place a
> > time-stamp on the primary server that is then propagated down to the
> > mirrors. Your new closer.cgi could easily query this file to see if its out
> > of date or not and get people _the best_ mirror possible ... just a
> > thought. Automate it and forget it.
>
> I love this.
#!/bin/sh
# cron script to update mirror source distribution timestamps
# should run on BSD boxen
TSFILE="/where/ever/pub/.timestamp-`/bin/hostname`"
/bin/date -u "+%s" > ${TSFILE}
#eof
this will be propagated to all mirrors when they sync/cvsup, so it
should be updated all 5 minutes or so from cron. if the apache.org box
doing these updates is secure, it might be an interesting feature to
generate a gnupg key for the update job and create a detached signature,
allowing to discover timestamp tampering. :-/ now that's overkill, okay
;-)
in turn, it will be very easy, then, to fetch these files from a script
grokking the mirror urls and analysing their staleness. therefor +%s
will display the time relative to UTC (-u) in seconds epoch, which is
easier to grok in scripts, IMVHO.
regards,
/k
--
> 130 Jahre Staubsauger.
> Vorher hatte der Hamster quasi keine natuerlichen Feinde...
WebMonster Community Project -- Reliable and quick since 1998 -- All on BSD
http://www.webmonster.de/ - ftp://ftp.webmonster.de/ - http://www.rohrbach.de/
GnuPG: 0xDEC948A6 D/E BF11 83E8 84A1 F996 68B4 A113 B393 6BF4 DEC9 48A6
REVOKED: 0x2964BF46 D/E 42F9 9FFF 50D4 2F38 DBEE DF22 3340 4F4E 2964 BF46
REVOKED: 0x4C44DA59 RSA F9 A0 DF 91 74 07 6A 1C 5F 0B E0 6B 4D CD 8C 44
My mail is GnuPG signed -- Unsigned ones are bogus -- http://www.gnupg.org/
Please do not remove my address from To: and Cc: fields in mailing lists. 10x
Re: Mirror reorganization
Posted by Brian Behlendorf <br...@collab.net>.
On Tue, 13 Aug 2002, Scott Kveton wrote:
> Bandwidth is cheap and disk is cheaper ... I say keep the old revisions
> around.
Heh, for someone with an academic budget and access to I2, sure. :)
This is easy to solve:
a) an rsync module called "dist", containing what's in
http://www.apache.org/dist/ (e.g. http://dist.apache.org if we move to
that), and we keep releases around the greater of 1 year or 2 previous
releases.
b) another rsync module called "dist-complete", which contains a
complete record of all releases. Dunno what the URL for this would be,
but I wouldn't put it under dist.apache.org, so that HTTP mirrors/spiders
who assume dist.apache.org is lightweight don't get penalized.
> On another note, how are other applications handling this sort of
> mirroring? I know for one I use Debian and they actually place a
> time-stamp on the primary server that is then propagated down to the
> mirrors. Your new closer.cgi could easily query this file to see if its out
> of date or not and get people _the best_ mirror possible ... just a
> thought. Automate it and forget it.
I love this.
Brian
Re: Mirror reorganization
Posted by Scott Kveton <kv...@oregonstate.edu>.
> You bring up a good point. What are the issues behind not wanting to
> mirror the historical distributions?
Bandwidth is cheap and disk is cheaper ... I say keep the old revisions
around.
> As a mirror admin I wouldn't mind mirroring all source/binary
> distributions. As an ASF member I'd like to have our mirrors do whatever
> they can to improve reliability and reduce the bandwidth and hosting
> costs of the apache.org sites.
Agreed. That is my only goal in being a mirror; help out ASF in
whatever way I can. I use Apache everyday and feel like I have to give
something back ... this is the best way I know how.
On another note, how are other applications handling this sort of
mirroring? I know for one I use Debian and they actually place a
time-stamp on the primary server that is then propagated down to the
mirrors. Your new closer.cgi could easily query this file to see if its out
of date or not and get people _the best_ mirror possible ... just a
thought. Automate it and forget it.
Although this site is down at the moment,
http://www.de.debian.org/dmc/today/ gives a clear picture of who is
up-to-date as of when and they rank them ... I'm not saying you have to
get that crazy about it but if you're reworking things you could easily
do the timestamp ... just a thoguht.
> This brings up another secondary point, which probably deserves a
> post of its own: Would those who have donated mirror resources
> like to have a page that basicly says "thanks to these groups for
> donating their time, bandwidth and machines to our mirror network: ..."?
> I think it would be nice to recognise this contribution. :)
I don't get a pat on the back from any of the other sites I mirror ... no
reason for you all to start ... :-) Besides, somebody has to use all of
this idle I2 bandwidth ... might as well be for a good cause ... :-)
Scott :-)
Re: Mirror reorganization
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Tue, 13 Aug 2002, Aaron Bannert wrote:
> Date: Tue, 13 Aug 2002 12:35:47 -0700
> From: Aaron Bannert <aa...@clove.org>
> To: Henk P. Penning <he...@cs.uu.nl>
> Cc: "mirrors@apache.org" <mi...@apache.org>,
> "infrastructure@apache.org" <in...@apache.org>
> Subject: Re: Mirror reorganization
>
> On Tue, Aug 13, 2002 at 08:44:09PM +0200, Henk P. Penning wrote:
> > your problem and fine proposal assume that mirrors carry
> > the 'historical' stuff. If mirrors only carried the 'current'
> > stuff the problem would go away, and some maintenance would
> > become less critical (the index.html's you propose are great,
> > but leave ftp users 'in the cold').
>
> I wasn't aware that the ASF did any FTP at all, let alone though mirrors.
See http://www.apache.org/mirrors/ ; ftp/http ~ 50/50
> FTP users can navigate to the right directory and look for the highest
> numbered version of a particular distribution, the index.html just
> makes it easier (besides, that's what HTML was designed for :).
Hm, if http users need index.html files,
then ftp users need README files.
> > As I said earlier, there are reasons for the mirrors NOT to carry
> > the 'historical' stuff : 1) confusing and 2) nobody uses them.
>
> I disagree. I've used my own mirror for retrieving older distributions
> on many occasions. It also serves another significant purpose: backups.
You trust your own mirror, of course :-). I think that when Joe
Average, on occasion, is looking for something 'odd', he/she
would go to www.apache.org and not to a mirror. Who trusts
all the mirrors to have everything all of the time ? Nobody.
That's reason 1 why mirrors shouldn't carry 'old' stuff.
... and mirrors is not the way to organise backups.
> > So, isn't it wise to FIRST decide if mirrors should carry
> > the old stuff, at all ? I'm against it, but I'm flexible :-).
>
> You bring up a good point. What are the issues behind not wanting to
> mirror the historical distributions?
reason 1 : see above.
reason 2 : For Joe Average al these versions are very confusing.
Less than 1% of the jakarta stuff is interesting for
99% of the users.
> As a mirror admin I wouldn't mind mirroring all source/binary
> distributions.
It's not that I mind; it's because I think it is
useless (for experts) and confusing (for non-experts).
> As an ASF member I'd like to have our mirrors do whatever
> they can to improve reliability and reduce the bandwidth and hosting
> costs of the apache.org sites.
I agree; So what's better, regarding the historical stuff
1. ( serve all the mirrors the historical stuff
AND serve all the experts the historical stuff
)
2. ONLY serve all the experts the historical stuff
I think 2 is better. Lean, mean mirrors who don't bother
the source too much.
> -aaron
regards.
Henk Penning
Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \
Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/
Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \
News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/
Re: Mirror reorganization
Posted by Aaron Bannert <aa...@clove.org>.
On Tue, Aug 13, 2002 at 08:44:09PM +0200, Henk P. Penning wrote:
> your problem and fine proposal assume that mirrors carry
> the 'historical' stuff. If mirrors only carried the 'current'
> stuff the problem would go away, and some maintenance would
> become less critical (the index.html's you propose are great,
> but leave ftp users 'in the cold').
I wasn't aware that the ASF did any FTP at all, let alone though mirrors.
FTP users can navigate to the right directory and look for the highest
numbered version of a particular distribution, the index.html just
makes it easier (besides, that's what HTML was designed for :).
> As I said earlier, there are reasons for the mirrors NOT to carry
> the 'historical' stuff : 1) confusing and 2) nobody uses them.
I disagree. I've used my own mirror for retrieving older distributions
on many occasions. It also serves another significant purpose: backups.
> So, isn't it wise to FIRST decide if mirrors should carry
> the old stuff, at all ? I'm against it, but I'm flexible :-).
You bring up a good point. What are the issues behind not wanting to
mirror the historical distributions?
As a mirror admin I wouldn't mind mirroring all source/binary
distributions. As an ASF member I'd like to have our mirrors do whatever
they can to improve reliability and reduce the bandwidth and hosting
costs of the apache.org sites.
This brings up another secondary point, which probably deserves a
post of its own: Would those who have donated mirror resources
like to have a page that basicly says "thanks to these groups for
donating their time, bandwidth and machines to our mirror network: ..."?
I think it would be nice to recognise this contribution. :)
-aaron
Re: Mirror reorganization
Posted by Jason van Zyl <ja...@zenplex.com>.
On Tue, 2002-08-13 at 13:53, Aaron Bannert wrote:
> On Mon, Aug 12, 2002 at 11:07:44AM -0700, Aaron Bannert wrote:
> > If having to download older distributions presents a bandwidth problem
> > to our mirrors, then maybe we should think about a naming scheme that
> > doesn't cause our mirros to re-download the same files once they move
> > from the main distribution directory to the old/ directories, since that
> > effectively doubles the amount of bytes downloaded to the mirrors. Maybe
> > a "current" symlink to a directory of the correct version.
>
> Just to reiterate how much of a problem I think this is, last night
> in just the httpd-site directory, my mirror downloaded around 250MB
> of data, merely because the older binary distributions were moved to
> a new directory. This negates much of the efficiency of rsync.
>
> Therefore I propose that we restructure the directory layout to avoid the
> need to move files once they are on the site, use web pages to provide
> the proper amount of browsability and indirection, and at the same time
> come up with a structure that allows for efficient load-balancing to
> the mirrors:
How encapsulating all the artifacts for a project under it's project id?
This is what I've done with the Maven repository that's being mirrored @
ibiblio:
http://www.ibiblio.org/maven/
I think this makes it easier for people to find project artifacts and
then within that directory sort the artifacts into directories by type.
So far in the Maven we have jars and distributions. I think the
everything grouped by project id is easier to navigate and manage. This
has been my experience so far trying to deal with the Maven repository.
> 1) each project places source and binary distributions under
> "dist/projectname/"
>
> 2) The "projectname" directory contains an index.html listing
> the current source and binary distributions, links for
> any subproject, and links to the main project pages. Release
> notes are also appropriate for this page. The main current
> distribution is also prominently linked from this page.
>
> 3) Source files are placed in a src/ directory, with the following
> naming convention [only a proposal at this point], and are not
> moved once in place:
> src/projectname-version-src.tar.gz
> or src/projectname-version-src.zip
>
> 4) Binary distributions are placed in a binaries/ directory, with
> each distribution having its own subdirectory. Once files are
> in place, they are not moved. For example:
> binaries/linux/httpd-2.0.40-i686-pc-linux-gnu.tar.gz
> or binaries/win32/httpd-2.0.40-win32-x86.msi
>
> 5) All source and binary distributions are accompanied by a signature,
> preferably made by someone on that project's PMC. The file shall
> exist in the same location as the signed file, and have the exact
> same name, with ".asc" appended:
> src/httpd-2.0.40-src.tar.gz.asc
>
> [optional]
> 6) All source and binary distributions are accompanied by an MD5 checksum.
> The file shall exist in the same location as the checksummed file,
> and have the same name with ".md5" appended:
> src/httpd-2.0.40-src.tar.gz.md5
>
>
> Key points to this proposal:
>
> - Files are not moved once they are in place. This saves unnecessary bandwidth
> consumption by our mirrors when they are syncing to the main site.
> - Web pages are constructed to identify the most current distribution,
> as well as other information pertinent to the project.
> - The structure allows us to construct intelligent load-balancing scripts
> to distribute the load away from our central resources (daedalus).
> - The index.html files can be easily generated, so they would require
> little or no intervention to maintain.
>
> -aaron
--
jvz.
Jason van Zyl
jason@apache.org
http://tambora.zenplex.org
In short, man creates for himself a new religion of a rational
and technical order to justify his work and to be justified in it.
-- Jacques Ellul, The Technological Society
Re: Mirror reorganization
Posted by Cliff Woolley <jw...@virginia.edu>.
On Tue, 13 Aug 2002, Aaron Bannert wrote:
> Just to reiterate how much of a problem I think this is, last night
> in just the httpd-site directory, my mirror downloaded around 250MB
> of data, merely because the older binary distributions were moved to
> a new directory. This negates much of the efficiency of rsync.
Urm, ouch. I have to claim responsibility for that one. I knew the
mirrors would have to resync but I didn't think it was that big (and
didn't bother to count MB's). :-(
Sorry!
--Cliff
Re: Mirror reorganization
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Tue, 13 Aug 2002, Aaron Bannert wrote:
> Date: Tue, 13 Aug 2002 10:53:33 -0700
> From: Aaron Bannert <aa...@clove.org>
> To: "mirrors@apache.org" <mi...@apache.org>
> Cc: "infrastructure@apache.org" <in...@apache.org>
> Subject: Mirror reorganization
> Just to reiterate how much of a problem I think this is, last night
> in just the httpd-site directory, my mirror downloaded around 250MB
> of data, merely because the older binary distributions were moved to
> a new directory. This negates much of the efficiency of rsync.
>
> Therefore I propose that we restructure the directory layout to avoid the
> need to move files once they are on the site, use web pages to provide
> the proper amount of browsability and indirection, and at the same time
> come up with a structure that allows for efficient load-balancing to
> the mirrors:
Hi,
your problem and fine proposal assume that mirrors carry
the 'historical' stuff. If mirrors only carried the 'current'
stuff the problem would go away, and some maintenance would
become less critical (the index.html's you propose are great,
but leave ftp users 'in the cold').
As I said earlier, there are reasons for the mirrors NOT to carry
the 'historical' stuff : 1) confusing and 2) nobody uses them.
So, isn't it wise to FIRST decide if mirrors should carry
the old stuff, at all ? I'm against it, but I'm flexible :-).
> -aaron
regards.
Henk Penning
Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \
Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/
Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \
News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/
Mirror reorganization
Posted by Aaron Bannert <aa...@clove.org>.
On Mon, Aug 12, 2002 at 11:07:44AM -0700, Aaron Bannert wrote:
> If having to download older distributions presents a bandwidth problem
> to our mirrors, then maybe we should think about a naming scheme that
> doesn't cause our mirros to re-download the same files once they move
> from the main distribution directory to the old/ directories, since that
> effectively doubles the amount of bytes downloaded to the mirrors. Maybe
> a "current" symlink to a directory of the correct version.
Just to reiterate how much of a problem I think this is, last night
in just the httpd-site directory, my mirror downloaded around 250MB
of data, merely because the older binary distributions were moved to
a new directory. This negates much of the efficiency of rsync.
Therefore I propose that we restructure the directory layout to avoid the
need to move files once they are on the site, use web pages to provide
the proper amount of browsability and indirection, and at the same time
come up with a structure that allows for efficient load-balancing to
the mirrors:
1) each project places source and binary distributions under
"dist/projectname/"
2) The "projectname" directory contains an index.html listing
the current source and binary distributions, links for
any subproject, and links to the main project pages. Release
notes are also appropriate for this page. The main current
distribution is also prominently linked from this page.
3) Source files are placed in a src/ directory, with the following
naming convention [only a proposal at this point], and are not
moved once in place:
src/projectname-version-src.tar.gz
or src/projectname-version-src.zip
4) Binary distributions are placed in a binaries/ directory, with
each distribution having its own subdirectory. Once files are
in place, they are not moved. For example:
binaries/linux/httpd-2.0.40-i686-pc-linux-gnu.tar.gz
or binaries/win32/httpd-2.0.40-win32-x86.msi
5) All source and binary distributions are accompanied by a signature,
preferably made by someone on that project's PMC. The file shall
exist in the same location as the signed file, and have the exact
same name, with ".asc" appended:
src/httpd-2.0.40-src.tar.gz.asc
[optional]
6) All source and binary distributions are accompanied by an MD5 checksum.
The file shall exist in the same location as the checksummed file,
and have the same name with ".md5" appended:
src/httpd-2.0.40-src.tar.gz.md5
Key points to this proposal:
- Files are not moved once they are in place. This saves unnecessary bandwidth
consumption by our mirrors when they are syncing to the main site.
- Web pages are constructed to identify the most current distribution,
as well as other information pertinent to the project.
- The structure allows us to construct intelligent load-balancing scripts
to distribute the load away from our central resources (daedalus).
- The index.html files can be easily generated, so they would require
little or no intervention to maintain.
-aaron
Re: Mirror created
Posted by Aaron Bannert <aa...@clove.org>.
On Mon, Aug 12, 2002 at 01:46:15PM -0400, Joshua Slive wrote:
> Yes, certainly. When I have discussed this with the jakarta people, we
> have been clear that nightly builds cannot go out to the mirrors. The
> other question, however, is whether all the old release (or pre-release)
> builds should stay up on the mirrors (as they are for httpd) or if we
> should prune down to just current releases. I have mixed feelings about
> that.
I tend to think that having them available in the mirrors serves two
good purposes:
1) automatic backups of legacy code/binaries
2) it's an O(1) operation -- synch it once and you won't have to
ever again.
If having to download older distributions presents a bandwidth problem
to our mirrors, then maybe we should think about a naming scheme that
doesn't cause our mirros to re-download the same files once they move
from the main distribution directory to the old/ directories, since that
effectively doubles the amount of bytes downloaded to the mirrors. Maybe
a "current" symlink to a directory of the correct version.
-aaron
Re: Mirror created
Posted by Pier Fumagalli <pi...@apache.org>.
"Joshua Slive" <jo...@slive.ca> wrote:
> Pier Fumagalli wrote:
>>
>> How can I figure out what the delta is? I'm mirroring world on Nagoya, and
>> I have the ability to make something happen on jakarta-site...
>
> Someone on jakarta (I've forgotten the name :-( started working on this
> a while ago, but it seems to have stalled. What needs to happen:
>
> 1. Move all the nightly build directories out of
> jakarta.apache.org/builds/ (perhaps to jakarta.apache.org/nightlies/
>
> 2. Move jakarta.apache.org/builds/ to www.apache.org/dist/jakarta/ and
> simultaneously setup a redirect to cover old links.
>
> 3. Change jakarta download links to point to
> http://www.apache.org/dyn/closer.cgi/jakarta/ so that people will
> download from the mirrors.
>
> If you can get that rocking, that would be excellent. I don't think it
> will be all that hard to accomplish. A note to this list just before
> you do step 2 would be wise, since there will be a sudden crush of new
> content going out to the mirrors.
Will see what I can do over the weekend. When Jon (I believe) started, he
hit the bottom when everyone else failed updating the site in the way in
which they were supposed to...
Pier
Re: Mirror created
Posted by Joshua Slive <jo...@slive.ca>.
Pier Fumagalli wrote:
>
> How can I figure out what the delta is? I'm mirroring world on Nagoya, and
> I have the ability to make something happen on jakarta-site...
Someone on jakarta (I've forgotten the name :-( started working on this
a while ago, but it seems to have stalled. What needs to happen:
1. Move all the nightly build directories out of
jakarta.apache.org/builds/ (perhaps to jakarta.apache.org/nightlies/
2. Move jakarta.apache.org/builds/ to www.apache.org/dist/jakarta/ and
simultaneously setup a redirect to cover old links.
3. Change jakarta download links to point to
http://www.apache.org/dyn/closer.cgi/jakarta/ so that people will
download from the mirrors.
If you can get that rocking, that would be excellent. I don't think it
will be all that hard to accomplish. A note to this list just before
you do step 2 would be wise, since there will be a sudden crush of new
content going out to the mirrors.
Joshua.
Re: Mirror created
Posted by Pier Fumagalli <pi...@apache.org>.
"Joshua Slive" <jo...@slive.ca> wrote:
> Aaron Bannert wrote:
>> On Sun, Aug 11, 2002 at 10:46:35PM -0400, Joshua Slive wrote:
>>
>>> A quick check shows 1040829k. We hope to increase it considerably some
>>> time soon by including the jakarta and xml software distributions. We
>>> are therefore asking mirrors to have 10GB available. You should also be
>>> relatively well-connected. If you don't have a fat pipe to the
>>> Internet, there is not much point in being a mirror.
>>
>>
>> We really need to do something about this. I stopped running the
>> jakarta and xml site mirrors to my machine because the nightly
>> deltas were so huge. It would be great if we could get the
>> dist directories to only be official project distributions,
>> so we could mirror just that instead of things like automated
>> gump builds.
>
> Yes, certainly. When I have discussed this with the jakarta people, we
> have been clear that nightly builds cannot go out to the mirrors. The
> other question, however, is whether all the old release (or pre-release)
> builds should stay up on the mirrors (as they are for httpd) or if we
> should prune down to just current releases. I have mixed feelings about
> that.
How can I figure out what the delta is? I'm mirroring world on Nagoya, and
I have the ability to make something happen on jakarta-site...
Pier
Re: Mirror created
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Mon, 12 Aug 2002, Joshua Slive wrote:
> Date: Mon, 12 Aug 2002 13:46:15 -0400
> From: Joshua Slive <jo...@slive.ca>
> To: "mirrors@apache.org" <mi...@apache.org>
> Subject: Re: Mirror created
>
> Aaron Bannert wrote:
> > We really need to do something about this. I stopped running the
> > jakarta and xml site mirrors to my machine because the nightly
> > deltas were so huge. It would be great if we could get the
> > dist directories to only be official project distributions,
> > so we could mirror just that instead of things like automated
> > gump builds.
>
> Yes, certainly. When I have discussed this with the jakarta people, we
> have been clear that nightly builds cannot go out to the mirrors. The
> other question, however, is whether all the old release (or pre-release)
> builds should stay up on the mirrors (as they are for httpd) or if we
> should prune down to just current releases. I have mixed feelings about
> that.
Hi,
this was discussed here a year ago; is there an archive of this list ?
I would strongly favor that the projects split their archive in
-- hist/ historical stuff
-- dist/ current stuff
-- devel/ development stuff
Mirrors should only carry the 'dist' stuff.
-- For mirror users, all the 'hist' and 'devel' stuff is very confusing.
-- Those interested in 'hist' and 'devel' would go to the source anyway,
and not trust any mirror to "have everything, always".
I have no idea how feasible/realistic this is, but I think
this is what the mirrors and average users want.
Note that I don't think diskspace or bandwidth are the problem,
although a year ago it took more than 2 days continuous mirroring
to get to 75% sync with the jakarta archive (due to performance
problems on the jakarta end).
> Joshua.
regards.
Henk Penning
Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \
Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/
Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \
News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/
Re: Mirror created
Posted by Joshua Slive <jo...@slive.ca>.
Aaron Bannert wrote:
> On Sun, Aug 11, 2002 at 10:46:35PM -0400, Joshua Slive wrote:
>
>>A quick check shows 1040829k. We hope to increase it considerably some
>>time soon by including the jakarta and xml software distributions. We
>>are therefore asking mirrors to have 10GB available. You should also be
>>relatively well-connected. If you don't have a fat pipe to the
>>Internet, there is not much point in being a mirror.
>
>
> We really need to do something about this. I stopped running the
> jakarta and xml site mirrors to my machine because the nightly
> deltas were so huge. It would be great if we could get the
> dist directories to only be official project distributions,
> so we could mirror just that instead of things like automated
> gump builds.
Yes, certainly. When I have discussed this with the jakarta people, we
have been clear that nightly builds cannot go out to the mirrors. The
other question, however, is whether all the old release (or pre-release)
builds should stay up on the mirrors (as they are for httpd) or if we
should prune down to just current releases. I have mixed feelings about
that.
Joshua.
Re: Mirror created
Posted by Aaron Bannert <aa...@clove.org>.
On Sun, Aug 11, 2002 at 10:46:35PM -0400, Joshua Slive wrote:
> A quick check shows 1040829k. We hope to increase it considerably some
> time soon by including the jakarta and xml software distributions. We
> are therefore asking mirrors to have 10GB available. You should also be
> relatively well-connected. If you don't have a fat pipe to the
> Internet, there is not much point in being a mirror.
We really need to do something about this. I stopped running the
jakarta and xml site mirrors to my machine because the nightly
deltas were so huge. It would be great if we could get the
dist directories to only be official project distributions,
so we could mirror just that instead of things like automated
gump builds.
-aaron
Re: Mirror created
Posted by John Donagher <jo...@webmeta.com>.
Joshua-
My mirror (apache.webmeta.com) seems to no longer be on the list. I'm not sure
why this would be, as it seems to be up to date. I've been an apache mirror for
at least a year.
Thanks
John
On Sun, 11 Aug 2002, Joshua Slive wrote:
> Amish K. Munshi wrote:
> > Hi,
> >
> > What is the total size of the mirror? Mine is presently 898 Mb
> > and
> > rsync is still downloading something.
>
> A quick check shows 1040829k. We hope to increase it considerably some
> time soon by including the jakarta and xml software distributions. We
> are therefore asking mirrors to have 10GB available. You should also be
> relatively well-connected. If you don't have a fat pipe to the
> Internet, there is not much point in being a mirror.
>
> Joshua.
>
>
>
>
--
John Donagher
Public key available off http://www.keyserver.net
Key fingerprint = 4024 DF50 56EE 19A3 258A D628 22DE AD56 EEBE 8DDD
Re: Mirror created
Posted by Joshua Slive <jo...@slive.ca>.
Amish K. Munshi wrote:
> Hi,
>
> What is the total size of the mirror? Mine is presently 898 Mb
> and
> rsync is still downloading something.
A quick check shows 1040829k. We hope to increase it considerably some
time soon by including the jakarta and xml software distributions. We
are therefore asking mirrors to have 10GB available. You should also be
relatively well-connected. If you don't have a fat pipe to the
Internet, there is not much point in being a mirror.
Joshua.
Re: Mirror created
Posted by "Amish K. Munshi" <li...@munshi.d2g.com>.
Hi,
What is the total size of the mirror? Mine is presently 898 Mb and
rsync is still downloading something.
Bye.
On Sun, 2002-08-11 at 11:11, Joshua Slive wrote:
> Amish K. Munshi wrote:
> > hi,
> >
> > I have created a mirror of the FTP site at
> > http://mirror.linux-site.net/mirror/apache
> > I did mail to the requested email address before 2 days, The
> > instructions page asked to be patient, I would like to know how long
> > does it usually take to add the URL to their site.
> > Is there a way to tell weather there no problem in the mailing.
>
> The mirror request address doesn't seem to be well looked-after at the
> moment, but I can add the mirror. However, it appears that your mirror
> is not very up-to-date at the moment. In particular, the httpd
> directory seems both out-of-date, and seems to be missing files even
> from the date it was mirrored.
>
> If you can fix that, and also let me know in what country your mirror is
> based, I will add it to the list.
>
> Joshua.
>
--
7:30am up 1 day, 7 min, 0 users, load average: 0.58, 0.54, 0.27