closer (was Re: TODOs)

["Adam R. B. Jack" <aj...@trysybase.com>]

"Mark R. Diggory" <md...@latte.harvard.edu> wrote:

> ASF Repository:
>
> About this time what I'm maintaining is the following two repository
> directories:

Thanks for this write-up.

> I haven't yet explored the idea of getting a "repository.apache.org"
> virtual host going. Up to this point I wanted to see the reuse of the
> existing mirroring structure to get these artifacts out to multiple
> hosts. I know that Henri and others have been working on some new
> download pages and scripts for redirecting to mirrors for downloads. I'd
> be really interested in finding out how we can combine this sort of
> Metadata with a download client to get clients downloading "load
> balanced" across the mirrors. I suspect this would be a server side
> redirect mechanism of some sort.

In Depot Update we've tinkered with picking one based off this URL's
contents.

http://www.apache.org/dyn/closer.cgi/java-repository/

Are you talking about closer.cgi, or a newer script?

regards

Adam

Re: closer (was Re: TODOs)

["Mark R. Diggory" <md...@latte.harvard.edu>]

Adam R. B. Jack wrote:

>"Mark R. Diggory" <md...@latte.harvard.edu> wrote:
>
>  
>
>>ASF Repository:
>>
>>About this time what I'm maintaining is the following two repository
>>directories:
>>    
>>
>
>Thanks for this write-up.
>
>  
>
>>I haven't yet explored the idea of getting a "repository.apache.org"
>>virtual host going. Up to this point I wanted to see the reuse of the
>>existing mirroring structure to get these artifacts out to multiple
>>hosts. I know that Henri and others have been working on some new
>>download pages and scripts for redirecting to mirrors for downloads. I'd
>>be really interested in finding out how we can combine this sort of
>>Metadata with a download client to get clients downloading "load
>>balanced" across the mirrors. I suspect this would be a server side
>>redirect mechanism of some sort.
>>    
>>
>
>In Depot Update we've tinkered with picking one based off this URL's
>contents.
>
>http://www.apache.org/dyn/closer.cgi/java-repository/
>
>Are you talking about closer.cgi, or a newer script?
>
>regards
>
>Adam
>  
>
Hmm, I was reviewing the emails and most of it was about building proper 
links to closer.cgi from the various download pages in Jakarta Commons. 
Not neccessarily improving upon closer.cgi.

http://www.mail-archive.com/commons-dev@jakarta.apache.org/msg43827.html

I think what really needs to happen is that theres a "machine" readable 
or parametized version of this script which returns either a redirect or 
a machine parsable list of locations (for instance in XML/RDF) instead 
of the human readable html page. The current script doesn't offer this, 
but a "template" could written for it to meet this need. At first I 
thought it would be fine to just have a serverside script that returns a 
redirect. But after some thought I realized the client in this situation 
would probibly just be a simple Http client or ftp client and if the 
connection failed, there would be a need for some sort of mechanism for 
recovery and retry at a new location, in which case getting and parsing 
a list would give the client multiple locations to choose from. I think 
this could really be accomplished by extending the script as such that 
it could detect user-agent or other request header/parameter and return 
the appropriate formatted content instead. Then the client would parse 
the content and iterate over the list until it retrieved a successfull 
download.

-Mark