catalog for optional feature and layout discovery

[ao...@bellsouth.net]

Hello,

After reading a few of the conversations on the list I have come up with some ideas.

First Jason mentioned the use of decorations to embellish the functionality of the repository.  After some thought I will have to agree even though I did not at first.  Keeping the repository simple is a good thing.

Now there were discussions with Tim regarding the layout of the repository and how the URL was to be designed.  Then Steve talked about using meta data (the notion of associating attributes/properties with artifacts).  I think this can be used as meta data but that's one application of it.  I think these attributes can be used for just about any application.

So how do we enable a flexible repository that allows clients to discover the optional services/features implemented as decoration on the repository?  Also how do we enable any kind of regular repository structure so we do not have to restrict the layout of the repository?

If the repository published a simple manifest or catalog of sorts in a fixed location so client APIs (in any language) can access it, then we can use this to store the following:

1). List of optional decorations offered by the repository
2). List of directory structure and naming rules
3). Miscellaneous seed information required to negotiate with the repo

3 is just filler cuz I did not want to have just two points ;-).

Ok the list of decorative services, features or what have you describe the optional functionality offered by the repository.  One such feature may be the use of attributes for associating relational information with a repo artifact.  The potential features are limited only by the imagination.  Just the other day I was using xdelta (the binary diff tool) and it occured to me that the repository could offer xdelta generation on the fly for applications that want to upgrade but use diffs instead of pulling down massive binaries.  This is just an idea and its here to show how pleasantly carried away we can get with decorations so don't interpret me as trying to add it to the repository ;-).

The point is with this catalog we can publish these features - the means to do that can be debates but we should not spend time on debating what features to add.  At the end of the day this is left to the discretion of the user.  We just need to allow for all the possibilities.

Now in terms of #2 we can make the structure of the repository and the naming conventions used user definable by including naming and structure rules for the directory layout of the repository.  These rules tell clients how to navigate the repository to get at the artifacts they may be interested in.  And the published decorations tell the client how it can take advantage of optional services offered by the repository.

I'm also trying to concieve of how client API's can selectively leverage these feature using plugable repository feature extensions.  Also trying to get a feel for what this might look like and the use cases.  

These are just some ideas that have sparked in my head as a consequence of the recent activity on the list.  Thought I'd put them out to be discussed.  Thoughts? Comments?

Alex

RE: catalog for optional feature and layout discovery

[Tim Anderson <tm...@netspace.net.au>]

See inline

> From: aok123@bellsouth.net [mailto:aok123@bellsouth.net]
> Sent: Monday, 17 November 2003 12:19 PM
> To: repository@apache.org
> Subject: catalog for optional feature and layout discovery
> 
> 
> Hello,
> 
> After reading a few of the conversations on the list I have come 
> up with some ideas.
> 
> First Jason mentioned the use of decorations to embellish the 
> functionality of the repository.  After some thought I will have 
> to agree even though I did not at first.  Keeping the repository 
> simple is a good thing.
> 
> Now there were discussions with Tim regarding the layout of the 
> repository and how the URL was to be designed.  Then Steve talked 
> about using meta data (the notion of associating 
> attributes/properties with artifacts).  I think this can be used 
> as meta data but that's one application of it.  I think these 
> attributes can be used for just about any application.
> 

Steve's ideas tie in neatly with WebDAV's support for resource 
attribution. WebDAV is out of scope tho, I think.


> So how do we enable a flexible repository that allows clients to 
> discover the optional services/features implemented as decoration 
> on the repository?  Also how do we enable any kind of regular 
> repository structure so we do not have to restrict the layout of 
> the repository?

I don't think restricting the repository structure is a bad thing. It:
. enables artifacts to be located in one lookup
  Any discovery mechanism is going to require multiple lookups.

. enables users browing the repository to locate artifacts
  without examining a catalogue to determine where they are
  located.

. can still support any kind of meta-data (or decoration)
  Tools which do not understand the meta-data simply ignore it.

. leads to simpler tool implementation

> 
> If the repository published a simple manifest or catalog of sorts 
> in a fixed location so client APIs (in any language) can access 
> it, then we can use this to store the following:
> 
> 1). List of optional decorations offered by the repository
> 2). List of directory structure and naming rules
> 3). Miscellaneous seed information required to negotiate with the repo
> 
> 3 is just filler cuz I did not want to have just two points ;-).
> 
> Ok the list of decorative services, features or what have you 
> describe the optional functionality offered by the repository.  
> One such feature may be the use of attributes for associating 
> relational information with a repo artifact.  The potential 
> features are limited only by the imagination.  Just the other day 
> I was using xdelta (the binary diff tool) and it occured to me 
> that the repository could offer xdelta generation on the fly for 
> applications that want to upgrade but use diffs instead of 
> pulling down massive binaries.  This is just an idea and its here 
> to show how pleasantly carried away we can get with decorations 
> so don't interpret me as trying to add it to the repository ;-).
> 
> The point is with this catalog we can publish these features - 
> the means to do that can be debates but we should not spend time 
> on debating what features to add.  At the end of the day this is 
> left to the discretion of the user.  We just need to allow for 
> all the possibilities.
> 
> Now in terms of #2 we can make the structure of the repository 
> and the naming conventions used user definable by including 
> naming and structure rules for the directory layout of the 
> repository.  These rules tell clients how to navigate the 
> repository to get at the artifacts they may be interested in.  
> And the published decorations tell the client how it can take 
> advantage of optional services offered by the repository.
> 

The only advantages I can see for supporting user defined
naming conventions and structure are:
. unrestricted artifact URIs
. the possibility to retrofit ASF repository onto an existing 
  repository.

This is at the cost of:
. predictable repository structure
. simple tool implementation

and potentially:
. ease of repository navigation

> I'm also trying to concieve of how client API's can selectively 
> leverage these feature using plugable repository feature 
> extensions.  Also trying to get a feel for what this might look 
> like and the use cases.  
> 
> These are just some ideas that have sparked in my head as a 
> consequence of the recent activity on the list.  Thought I'd put 
> them out to be discussed.  Thoughts? Comments?
> 
> Alex
> 

I'm not against pluggable repository features, but I can't think
of too many uses for it. The only one which springs to mind is
a search facility, but it needn't be pluggable.

Also note that WebDAV and DeltaV provide limited search facilities.

- Tim