TODOs

["Adam R. B. Jack" <aj...@trysybase.com>]

Ok, time to re-gen a set of todos since we have a few folks who are able to
scratch this itch right now. Here are some things that come to my mind.

1) ASF Repository

Find out more about what has been implemented in the name of the ASF
Repository, and report to this list. I know something called this exists
somewhere (although I don't think it follows an ASF repo layout, but a Maven
one) I'd like to poke it, prod it, test it.

2) Repository/Artifact (Avalon interface)

This re-work was very close, needs to be completed (and the Ant Type needs
to be re-tested over it). One for me, since I started it...

3) Downloading

Create a separate DownloadManager that does downloads for us, and does
things like check MD5s, and allows user-set listeners for doing things like
'.' (dots) for downloaded blocks, or percentages in GUIs, etc.

4) API

Keep refactoring/documenting to generate a simple API. A 'hands on'
interface, for those wanting to manually work w/ Repository/Artifacts
(Avaloners) and an Updater one (for querying repositories and downloading
latest best fit).

5) Monitor the Gump errors and fix what needs fixing.

regards,

Adam
--
Experience the Unwired Enterprise:
http://www.sybase.com/unwiredenterprise
Try Sybase: http://www.try.sybase.com

Re: closer (was Re: TODOs)

["Mark R. Diggory" <md...@latte.harvard.edu>]

Adam R. B. Jack wrote:

>"Mark R. Diggory" <md...@latte.harvard.edu> wrote:
>
>  
>
>>ASF Repository:
>>
>>About this time what I'm maintaining is the following two repository
>>directories:
>>    
>>
>
>Thanks for this write-up.
>
>  
>
>>I haven't yet explored the idea of getting a "repository.apache.org"
>>virtual host going. Up to this point I wanted to see the reuse of the
>>existing mirroring structure to get these artifacts out to multiple
>>hosts. I know that Henri and others have been working on some new
>>download pages and scripts for redirecting to mirrors for downloads. I'd
>>be really interested in finding out how we can combine this sort of
>>Metadata with a download client to get clients downloading "load
>>balanced" across the mirrors. I suspect this would be a server side
>>redirect mechanism of some sort.
>>    
>>
>
>In Depot Update we've tinkered with picking one based off this URL's
>contents.
>
>http://www.apache.org/dyn/closer.cgi/java-repository/
>
>Are you talking about closer.cgi, or a newer script?
>
>regards
>
>Adam
>  
>
Hmm, I was reviewing the emails and most of it was about building proper 
links to closer.cgi from the various download pages in Jakarta Commons. 
Not neccessarily improving upon closer.cgi.

http://www.mail-archive.com/commons-dev@jakarta.apache.org/msg43827.html

I think what really needs to happen is that theres a "machine" readable 
or parametized version of this script which returns either a redirect or 
a machine parsable list of locations (for instance in XML/RDF) instead 
of the human readable html page. The current script doesn't offer this, 
but a "template" could written for it to meet this need. At first I 
thought it would be fine to just have a serverside script that returns a 
redirect. But after some thought I realized the client in this situation 
would probibly just be a simple Http client or ftp client and if the 
connection failed, there would be a need for some sort of mechanism for 
recovery and retry at a new location, in which case getting and parsing 
a list would give the client multiple locations to choose from. I think 
this could really be accomplished by extending the script as such that 
it could detect user-agent or other request header/parameter and return 
the appropriate formatted content instead. Then the client would parse 
the content and iterate over the list until it retrieved a successfull 
download.

-Mark

closer (was Re: TODOs)

["Adam R. B. Jack" <aj...@trysybase.com>]

"Mark R. Diggory" <md...@latte.harvard.edu> wrote:

> ASF Repository:
>
> About this time what I'm maintaining is the following two repository
> directories:

Thanks for this write-up.

> I haven't yet explored the idea of getting a "repository.apache.org"
> virtual host going. Up to this point I wanted to see the reuse of the
> existing mirroring structure to get these artifacts out to multiple
> hosts. I know that Henri and others have been working on some new
> download pages and scripts for redirecting to mirrors for downloads. I'd
> be really interested in finding out how we can combine this sort of
> Metadata with a download client to get clients downloading "load
> balanced" across the mirrors. I suspect this would be a server side
> redirect mechanism of some sort.

In Depot Update we've tinkered with picking one based off this URL's
contents.

http://www.apache.org/dyn/closer.cgi/java-repository/

Are you talking about closer.cgi, or a newer script?

regards

Adam

Re: TODOs

["Mark R. Diggory" <md...@latte.harvard.edu>]

Adam R. B. Jack wrote:

>Ok, time to re-gen a set of todos since we have a few folks who are able to
>scratch this itch right now. Here are some things that come to my mind.
>
>1) ASF Repository
>
>Find out more about what has been implemented in the name of the ASF
>Repository, and report to this list. I know something called this exists
>somewhere (although I don't think it follows an ASF repo layout, but a Maven
>one) I'd like to poke it, prod it, test it.
>
>  
>
ASF Repository:

About this time what I'm maintaining is the following two repository 
directories:

Production Release Repository: This repository is located in dist and 
mirrored to all Apache Mirrors. This is a mission critical repository 
for Apache, we maintain all Apache artifacts being released for public 
consumption here and it is rsynced to the www.ibiblio.org/maven 
directory every 4 hours overwriting anything updated. This allows us to 
manage the Apache content in the Maven repository locally which is more 
acceptable because we have fully control over when it is released into 
dist at Apache.
/www/www.apache.org/dist/java-repository
http://www.apache.org/dist/java-repository

Interm/Nightly/Beta Repository: This is a Maven repository outside the 
dist directory for artifacts we do not want to see distributed to 
mirrors or anywhere else for that matter. This is a repository for 
developers to test deployment into and to "release" developmental builds 
to the "Apache Developer Community".
/www/cvs.apache.org/repository
http://cvs.apache.org/repository

I haven't yet explored the idea of getting a "repository.apache.org" 
virtual host going. Up to this point I wanted to see the reuse of the 
existing mirroring structure to get these artifacts out to multiple 
hosts. I know that Henri and others have been working on some new 
download pages and scripts for redirecting to mirrors for downloads. I'd 
be really interested in finding out how we can combine this sort of 
Metadata with a download client to get clients downloading "load 
balanced" across the mirrors. I suspect this would be a server side 
redirect mechanism of some sort.

-Mark

>2) Repository/Artifact (Avalon interface)
>
>This re-work was very close, needs to be completed (and the Ant Type needs
>to be re-tested over it). One for me, since I started it...
>
>3) Downloading
>
>Create a separate DownloadManager that does downloads for us, and does
>things like check MD5s, and allows user-set listeners for doing things like
>'.' (dots) for downloaded blocks, or percentages in GUIs, etc.
>
>4) API
>
>Keep refactoring/documenting to generate a simple API. A 'hands on'
>interface, for those wanting to manually work w/ Repository/Artifacts
>(Avaloners) and an Updater one (for querying repositories and downloading
>latest best fit).
>
>5) Monitor the Gump errors and fix what needs fixing.
>
>regards,
>
>Adam
>--
>Experience the Unwired Enterprise:
>http://www.sybase.com/unwiredenterprise
>Try Sybase: http://www.try.sybase.com
>
>  
>