You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mirrors@apache.org by Andrew Kenna <an...@stamina.com.au> on 2001/06/06 04:40:20 UTC
RE: /dist URL
I haven't mirrored the apache site for a couple of months due to bandwidth
problems but i'm only going under the assumption that the /dist/ is part of
the apache-site rsync module
Am I right in assuming this or am I way off the mark anyone ?
Please note I am not a member of the apache group, just someone who
maintains the mirror list and points out possible problems with sites to
their administrators.
Andrew
-----Original Message-----
From: Devendra Narayan [mailto:narayan@cc.kagu.sut.ac.jp]
Sent: Wednesday, June 06, 2001 12:47 PM
To: mirrors@apache.org
Subject: Re: /dist URL
Andrew Kenna wrote:
>
> the /dist/ is appended to all apache mirrors listed on the web site. Are
you
> guys mirroring the /dist/ portion of the apache web site ?
According to the 'how-to-mirror' page (
http://www.apache.org/info/how-to-mirror.html )
and the list of rsync modules at dev.apache.org :
apache-site Apache WWW Site <--- 1
apache-dist Apache software distribution <--- 2
I have, for many years now, placed the WWW site ( 1 above ) under my
htdocs tree ( http://sunsite.sut.ac.jp/pub/apache/ ) and the software
distribution ( 2 above )
under my ftp tree at ( ftp://sunsite.sut.ac.jp/pub/archives/www/apache/ ).
And these are
the URLs that are listed on the mirrors page. The ftp URL is correct but the
http URL
displays http://sunsite.sut.ac.jp/pub/apache/ on the mirrors page but
'internally' points
to http://sunsite.sut.ac.jp/pub/apache/dist/
I fail to see what am I doing wrong !! Someone on the Apache group
has to
correct this 'inconsistency' as well as provide the right files via rsync
:-(
[ FYI : SunSITE Japan is one of the oldest ftp/web servers - mirroring NCSA
httpd and
later Apache since 1993 ]
--------------------------------------------------------
Devendra Narayan
Senior Engineer, Information Systems Division,
Science University of Tokyo,
"Computers can never replace human stupidity"
---------------------------------------------------------
Re: Apache mirror system - comments
Posted by Devendra Narayan <na...@cc.kagu.sut.ac.jp>.
> well, you did ask for comments. i'm just voicing my views - in particular based
> on what i think might actually produce a workable scaleable mirror system for
> a project. it's not gospel by any means.
Of course, your views are important ( I hope the Apache group members also
can find time to read these mails. ).
> the apache project at this stage is approximately 600M of disk space. if you think lack
> of diskspace is an issue, try and deal with the freebsd archive (50G ?) or redhat (80G ?
> 100G?).
Yes, I agree with you - ( I hadn't considered the actual size of the project when I wrote
my last message ). You are right that we can, probably most of us, set aside a couple of
Gigabytes for Apache.
And if everyone agrees to this, let the Apache group state the actual sizes ( and
perhaps anticipated sizes ) of the various modules on the 'how to mirror page'
and make the whole archive simpler to mirror. That should solve many of the
problems.
I appreciate all the mails that Jason has sent in the past to improve the Apache
mirror system and Andrew's efforts to make things right. Keep up the good work,
Guys ( and gals - if there are any - if not - why not ? ).
-------
Devendra Narayan
Re: Apache mirror system - comments
Posted by jason andrade <ja...@dstc.edu.au>.
On Wed, 6 Jun 2001, Henk P. Penning wrote:
> I agree. It is better (for users and organizers) to have
> fewer complete mirrors than more partial mirrors.
>
> On the other hand, I can't think of a practical reason
> why each project (apache, jakarta, etc) can't have its
> own, self contained, easy-to-mirror, sub-site.
scale - i think that'd require more effort than having one
set of standards. one look and feel, one set of templates,
one HOWTO on mirroring.. minimal number of people to organize
it..
i'd prefer to go back to having apache.org/httpd/ apache.org/xml/
apache.org/jakarta/ and having that replicated on mirror sites.
than the current system where every subproject splits out.
-jason
Re: Apache mirror system - comments
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Wed, 6 Jun 2001, jason andrade wrote:
> Date: Wed, 6 Jun 2001 17:48:33 +1000 (EST)
> From: jason andrade <ja...@dstc.edu.au>
> To: mirrors@apache.org
> Subject: Re: Apache mirror system - comments
> Sender: mirrors-return-301-henkp=cs.uu.nl@apache.org
> i would happily allocate 2G of disk to the apache project for 12 months
> if that guaranteed me a mirroring system that was consistent and
> therefore well used. i appreciate you have disk shortages (i have disk
> shortages myself!) but given the above, some perspective is needed.
I agree. It is better (for users and organizers) to have
fewer complete mirrors than more partial mirrors.
On the other hand, I can't think of a practical reason
why each project (apache, jakarta, etc) can't have its
own, self contained, easy-to-mirror, sub-site.
> -jason
regards
Henk Penning
Henk P. Penning, Dept of Computer Science, Utrecht University \__/ \
Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands. \__/
Telephone: +31-30-2534106, fax: 2513791, NIC-handle: HPP1 _/ \__/ \
News.answers http://www.cs.uu.nl/cgi-bin/faqwais \__/ \__/ \__/
Re: Apache mirror system - comments
Posted by jason andrade <ja...@dstc.edu.au>.
On Wed, 6 Jun 2001, Devendra Narayan wrote:
> I request you to look at it with more consideration and a historical perspective.
> There probably are many sites ( including ours ) which started mirroring NCSA htttpd
> and later Apache many years ago ( Apache since it's inception ). Now, these sites
> have been 'official' mirror sites for many years.
well, you did ask for comments. i'm just voicing my views - in particular based
on what i think might actually produce a workable scaleable mirror system for
a project. it's not gospel by any means.
> In the meantime, however, Apache group has grown to develop a lot of
> things besides the httpd server ( XML, Jakarta etc ). Just because the previously
> 'official' mirror sites failed to mirror these new projects shouldn't be a reason to
> 'demote' them from their official status. There could be many reasons why they are
> not mirroring the new projects - lack of disk space, user interest etc.
i respectfully disagree with this. i would love to have a situation where historical
perspective is a factor (i'd qualify for a few archives on that basis..) but the reality
is based more around having a mirror system that works - i.e users can always find
things they're looking for at the mirror (e.g from your previous comments about having
all the links relative etc), and that it requires minimal work to maintain on the part
of the software project.
the apache project at this stage is approximately 600M of disk space. if you think lack
of diskspace is an issue, try and deal with the freebsd archive (50G ?) or redhat (80G ?
100G?).
i would happily allocate 2G of disk to the apache project for 12 months if that guaranteed
me a mirroring system that was consistent and therefore well used. i appreciate you
have disk shortages (i have disk shortages myself!) but given the above, some perspective
is needed.
> Yes, if the facilities that they have permit, they should mirror the whole thing
> but, those that have supported the distribution of the Apache httpd for so long, shouldn't
> be punished for not being able to keep up with the rapid expansion of the Apache group's
> projects.
i don't wish to denigrate anyone's archive or efforts - please don't see it as such. i'd
just like a workable mirror system. tomorrow would be great, thanks :-)
IMHO, it won't work to have say 250 sites mirroring apache, with varying levels of up to
dateness and completeness, i'd rather see 50 sites each with a complete archive. that
doesn't stop the other 200 sites mirroring it and making it available to users. once
again, that's just my opinion.
regards,
-jason
Re: Apache mirror system - comments
Posted by Devendra Narayan <na...@cc.kagu.sut.ac.jp>.
jason andrade wrote:
> > 1. Those who have enough disk space to mirror everything that Apache group
> > has to offer - web documents, distributions, XML-Apache, Jakarta
> > etc.
>
> i think these would/should be the only official mirrors for the apache
> project.
>
> > 2. People like me, who are short on disk space, may just want to mirror the web
> > documents and one of the dists ( httpd, Jakarta etc. ). So the web docs
> > module ( apache-site ) shouldn't contain any of the dists which should
> > all be available separately.
>
> you should be free to do this, but not expect to be considered an authoritative
> source of apache software.
>
> > 3. Others, who are even shorter on disk space, may want to mirror just one dist.
>
> as per above.
Jason,
I request you to look at it with more consideration and a historical perspective.
There probably are many sites ( including ours ) which started mirroring NCSA htttpd
and later Apache many years ago ( Apache since it's inception ). Now, these sites
have been 'official' mirror sites for many years.
In the meantime, however, Apache group has grown to develop a lot of
things besides the httpd server ( XML, Jakarta etc ). Just because the previously
'official' mirror sites failed to mirror these new projects shouldn't be a reason to
'demote' them from their official status. There could be many reasons why they are
not mirroring the new projects - lack of disk space, user interest etc.
Yes, if the facilities that they have permit, they should mirror the whole thing
but, those that have supported the distribution of the Apache httpd for so long, shouldn't
be punished for not being able to keep up with the rapid expansion of the Apache group's
projects.
What is the opinion of others on this ?
--------------------------------------------------------
Devendra Narayan
Senior Engineer, Information Systems Division,
Science University of Tokyo.
"Sex is not the answer. Sex is the question. Yes is the answer."
---------------------------------------------------------
Re: Apache mirror system - comments
Posted by John Donagher <jo...@webmeta.com>.
On Wed, 6 Jun 2001, jason andrade wrote:
> On Wed, 6 Jun 2001, Devendra Narayan wrote:
>
> >
> > With all these discussions about Apache mirrors, I just have this to
> > say :
> > We can guess that there would be many types of mirror sites :
> >
> > 1. Those who have enough disk space to mirror everything that Apache group
> > has to offer - web documents, distributions, XML-Apache, Jakarta
> > etc.
>
> i think these would/should be the only official mirrors for the apache
> project.
>
I disagree. People with excess upstream bandwidth but limited disk space can
still provide a valuable service. I think it's fair to assume that Apache is
still by far the most downloaded component of the apache project. Making it
known which mirrors mirror which packages would be good though. See the RedHat
mirrors list for a possible way of presenting this data to the user.
John
--
John Donagher
Application Engineer
Intacct Corp. - Powerful Accounting on the Web
408-395-0989
720 University Ave.
Los Gatos CA 95032
www.intacct.com
Public key available off http://www.keyserver.net
Key fingerprint = 4024 DF50 56EE 19A3 258A D628 22DE AD56 EEBE 8DDD
Re: Apache mirror system - comments
Posted by jason andrade <ja...@dstc.edu.au>.
On Wed, 6 Jun 2001, Devendra Narayan wrote:
>
> With all these discussions about Apache mirrors, I just have this to
> say :
> We can guess that there would be many types of mirror sites :
>
> 1. Those who have enough disk space to mirror everything that Apache group
> has to offer - web documents, distributions, XML-Apache, Jakarta
> etc.
i think these would/should be the only official mirrors for the apache
project.
> 2. People like me, who are short on disk space, may just want to mirror the web
> documents and one of the dists ( httpd, Jakarta etc. ). So the web docs
> module ( apache-site ) shouldn't contain any of the dists which should
> all be available separately.
you should be free to do this, but not expect to be considered an authoritative
source of apache software.
> 3. Others, who are even shorter on disk space, may want to mirror just one dist.
as per above.
> Wouldn't it be great if most/all such mirror administrators could be satisfied ? Let's just devise
> a system ( using rsync ) where most/all the above needs can be satisfied and provide
> easy to use instructions for the mirror admins. In any case, all the packages/ modules
> offered for mirroring should be self contained ( with relative URLs / pointers etc. ).
i agree with the above to a certain point, but i think the line should be drawn somewhere
in terms of maintenance - for a site to be considered an official apache mirror and to be
listed as such, it has to provide a complete mirror (like the CPAN archive) of the whole
site. mirroring for personal or semi public use (e.g company, or university or where
there is no apache mirror) is fine, but isn't in the same category.
> Comments or suggestions, please ?
>
> One question - how to instruct rsync to ignore a local file/dir/link when mirroring ?
--exclude="filename" or --exclude="directoryname/"
regards,
-jason
Apache mirror system - comments
Posted by Devendra Narayan <na...@cc.kagu.sut.ac.jp>.
With all these discussions about Apache mirrors, I just have this to
say :
We can guess that there would be many types of mirror sites :
1. Those who have enough disk space to mirror everything that Apache group
has to offer - web documents, distributions, XML-Apache, Jakarta
etc.
2. People like me, who are short on disk space, may just want to mirror the web
documents and one of the dists ( httpd, Jakarta etc. ). So the web docs
module ( apache-site ) shouldn't contain any of the dists which should
all be available separately.
3. Others, who are even shorter on disk space, may want to mirror just one dist.
.... Possible other types of mirror sites
Wouldn't it be great if most/all such mirror administrators could be satisfied ? Let's just devise
a system ( using rsync ) where most/all the above needs can be satisfied and provide
easy to use instructions for the mirror admins. In any case, all the packages/ modules
offered for mirroring should be self contained ( with relative URLs / pointers etc. ).
Comments or suggestions, please ?
One question - how to instruct rsync to ignore a local file/dir/link when mirroring ?
--------------------------------------------------------
Devendra Narayan
Senior Engineer, Information Systems Division,
Science University of Tokyo,
"When it comes to thought, some people stop at nothing."
---------------------------------------------------------
Re: /dist URL
Posted by Devendra Narayan <na...@cc.kagu.sut.ac.jp>.
jason andrade wrote:
> > Yes, it is. But, in cases, like ours, where we provide both http and ftp data
> > it would be a waste to copy the dist twice ( once from apache-site and
> > once from apache-dist ). Thus, for pure ftp users, we only provide the
> > data mirrored from the apache-dist module. And I exclude dist when
> > mirroring the apache-site module.
> ..........
> or you can mirror the whole apache-site area and
> exclude the dist area, which you can separatly mirror to the right
> area, ........
I think that's what I said I was doing :-)
--
Devendra Narayan
Re: /dist URL
Posted by jason andrade <ja...@dstc.edu.au>.
On Wed, 6 Jun 2001, Devendra Narayan wrote:
> Yes, it is. But, in cases, like ours, where we provide both http and ftp data
> it would be a waste to copy the dist twice ( once from apache-site and
> once from apache-dist ). Thus, for pure ftp users, we only provide the
> data mirrored from the apache-dist module. And I exclude dist when
> mirroring the apache-site module.
you don't need to - the apache-site includes the dist area.
so you can either create one area which is visible to both ftp and http
(which we do). or you can mirror the whole apache-site area and
exclude the dist area, which you can separatly mirror to the right
area, or if you have enough disk space (though i wouldn't do this),
you can just use rsync locally to keep a separate ftp area of the
dist area.
> In any case, the inconsistency in the URLs displayed ( and real ) on the
> Apache mirrors page still remains !!
this, i agree with.
regards,
-jason
Re: /dist URL
Posted by Devendra Narayan <na...@cc.kagu.sut.ac.jp>.
Andrew Kenna wrote:
>
> I haven't mirrored the apache site for a couple of months due to bandwidth
> problems but i'm only going under the assumption that the /dist/ is part of
> the apache-site rsync module
>
> Am I right in assuming this or am I way off the mark anyone ?
>
> Please note I am not a member of the apache group, just someone who
> maintains the mirror list and points out possible problems with sites to
> their administrators.
Yes, it is. But, in cases, like ours, where we provide both http and ftp data
it would be a waste to copy the dist twice ( once from apache-site and
once from apache-dist ). Thus, for pure ftp users, we only provide the
data mirrored from the apache-dist module. And I exclude dist when
mirroring the apache-site module.
In any case, the inconsistency in the URLs displayed ( and real ) on the
Apache mirrors page still remains !!
Yes, I do know that you are not a member of Apache group - but wish
you were - perhaps that would make it more 'active' :-)
--------------------------------------------------------
Devendra Narayan
Senior Engineer, Information Systems Division,
Science University of Tokyo,
"Treat each day as your last, one day you will be right."
---------------------------------------------------------