Repository discussion (was GroupId for DayTrader needed)

[Matt Hogstrom <ma...@hogstrom.org>]

Looks interesting.  It does solve some of the issues we're having.  One thing that keeps resonating 
in my ears from users is Tomcat is so easy.

The goal of the repository is to store artifacts needed by the server for both its own internal 
componentry as well as deployed applications.  At this point (correct me where I'm wrong) but we 
have the core repo and in place deployment which allows users to point to their application.  Not to 
mention hot-deploy.

I haven't thought through this all but do people think users will require multiple repo strategies? 
  I like the idea of a flat file structure so people can manipulate the dir on their own without a 
server active.

Perhaps we need a repo contest :)

Jason Dillon wrote:
> FYI, I have an initial impl of a JDBM-based hybrid repository here:
> 
>     
> http://svn.apache.org/repos/asf/geronimo/sandbox/svkmerge/geronimo-repository/geronimo-repository-providers/geronimo-repository-jdbm/ 
> 
> 
> This uses files to store the content to be compatible wit the existing 
> API, and JDBM for the metadata.
> 
> Repository fs looks like:
> 
>     metadata.db
>     metadata.lg
>     data / hash (of groupId, artifactId, version) / artifactId-version.type
> 
> If/when when the repo api is abstracted away from files, thrn it would 
> be easy to put all content into another artifacts.db to reduce any 
> chance of long filenames (except for the most insane cases).
> 
> I added a RepositoryCopier util too, so it would be trivial to make a 
> command-line version to turn a JDBM-repo into an m2-repo for folks that 
> don't use windows and prefer to see the files as is.
> 
> Have not done it yet, but would be easy to write a tool to add a new 
> file to and repo.  Can either specify the artifact or if built with 
> maven 2, then the META-INF/maven/* bits can be used to specify the right 
> groupId/artifactId/version/type to install with.  This tool can also 
> complain if not enough information was given... so it solves the problem 
> of people copying random artifacts into the repository.
> 
>  * * *
> 
> Also, this 
> http://svn.apache.org/repos/asf/geronimo/sandbox/svkmerge/geronimo-repository 
> is the new structure I am going to recommend for future reorg for 
> maven2-based systems, so you might want to take a peek at the overall 
> tree too.  This tree provides encapsulation of all of the repository 
> components (though currently missing the impls for m1 and m2 as well as 
> the admin portlets, but will eventually have them).
> 
> NOTE: This is just a POC, no plans yet to move this to trunk in any 
> way... though eventually I would like to do that.
> 
> --jason
> 
> 
> On Jul 21, 2006, at 4:34 PM, Dain Sundstrom wrote:
> 
>> On Jul 20, 2006, at 4:01 PM, Jason Dillon wrote:
>>
>>> It is not just how we use the m2-style repo inside of G, but also how 
>>> we build G using m2 that will cause problems for windows peeps.
>>>
>>> The later is going to be more trouble to resolve I think.
>>>
>>> I'm still thinking it would be a good idea to have some sort of 
>>> file-system abstraction for our m2-style repo... in the same way that 
>>> SVN has an abstraction... that could use BDB or a regular file system 
>>> (ie. FSFS) for actual storage.  WIth something like this, we could 
>>> have the default distro use a BDB-ish filesystem and completely 
>>> resolve the windows file name limitations.  With a set of cli tools 
>>> we can easily allow the BDB-ish to be converted to a FSFS.
>>
>> The repo is an interface.  Well it is really three interfaces 
>> depending how much you want to implement:
>>
>> public interface Repository {
>>     boolean contains(Artifact artifact);
>>     File getLocation(Artifact artifact);
>>     LinkedHashSet getDependencies(Artifact artifact);
>> }
>>
>> public interface WriteableRepository extends Repository {
>>     void copyToRepository(File source, Artifact destination, 
>> FileWriteMonitor monitor) throws IOException;
>>     void copyToRepository(InputStream source, int size, Artifact 
>> destination, FileWriteMonitor monitor) throws IOException;
>> }
>>
>> public interface ListableRepository extends Repository {
>>     SortedSet list();
>>     SortedSet list(Artifact query);
>> }
>>
>> We have an M1 and M2 implementation of these, so if you want another 
>> one you have some good base code to start with.
>>
>> -dain
> 
> 
> 
> 

Re: Repository discussion (was GroupId for DayTrader needed)

[Jason Dillon <ja...@planet57.com>]

I think we need to fix up how the repo is used before we can do  
anything fancier.  Specifically, we need to abstract the usage of  
java.io.File.  There is already too much going outside of the repo  
API to access files directly in the repository.

After that... then we could easily support an aggregated repo view  
over a set of repositories.

--jason


On Jul 25, 2006, at 11:45 AM, Matt Hogstrom wrote:

> Looks interesting.  It does solve some of the issues we're having.   
> One thing that keeps resonating in my ears from users is Tomcat is  
> so easy.
>
> The goal of the repository is to store artifacts needed by the  
> server for both its own internal componentry as well as deployed  
> applications.  At this point (correct me where I'm wrong) but we  
> have the core repo and in place deployment which allows users to  
> point to their application.  Not to mention hot-deploy.
>
> I haven't thought through this all but do people think users will  
> require multiple repo strategies?  I like the idea of a flat file  
> structure so people can manipulate the dir on their own without a  
> server active.
>
> Perhaps we need a repo contest :)
>
> Jason Dillon wrote:
>> FYI, I have an initial impl of a JDBM-based hybrid repository here:
>>     http://svn.apache.org/repos/asf/geronimo/sandbox/svkmerge/ 
>> geronimo-repository/geronimo-repository-providers/geronimo- 
>> repository-jdbm/ This uses files to store the content to be  
>> compatible wit the existing API, and JDBM for the metadata.
>> Repository fs looks like:
>>     metadata.db
>>     metadata.lg
>>     data / hash (of groupId, artifactId, version) / artifactId- 
>> version.type
>> If/when when the repo api is abstracted away from files, thrn it  
>> would be easy to put all content into another artifacts.db to  
>> reduce any chance of long filenames (except for the most insane  
>> cases).
>> I added a RepositoryCopier util too, so it would be trivial to  
>> make a command-line version to turn a JDBM-repo into an m2-repo  
>> for folks that don't use windows and prefer to see the files as is.
>> Have not done it yet, but would be easy to write a tool to add a  
>> new file to and repo.  Can either specify the artifact or if built  
>> with maven 2, then the META-INF/maven/* bits can be used to  
>> specify the right groupId/artifactId/version/type to install  
>> with.  This tool can also complain if not enough information was  
>> given... so it solves the problem of people copying random  
>> artifacts into the repository.
>>  * * *
>> Also, this http://svn.apache.org/repos/asf/geronimo/sandbox/ 
>> svkmerge/geronimo-repository is the new structure I am going to  
>> recommend for future reorg for maven2-based systems, so you might  
>> want to take a peek at the overall tree too.  This tree provides  
>> encapsulation of all of the repository components (though  
>> currently missing the impls for m1 and m2 as well as the admin  
>> portlets, but will eventually have them).
>> NOTE: This is just a POC, no plans yet to move this to trunk in  
>> any way... though eventually I would like to do that.
>> --jason
>> On Jul 21, 2006, at 4:34 PM, Dain Sundstrom wrote:
>>> On Jul 20, 2006, at 4:01 PM, Jason Dillon wrote:
>>>
>>>> It is not just how we use the m2-style repo inside of G, but  
>>>> also how we build G using m2 that will cause problems for  
>>>> windows peeps.
>>>>
>>>> The later is going to be more trouble to resolve I think.
>>>>
>>>> I'm still thinking it would be a good idea to have some sort of  
>>>> file-system abstraction for our m2-style repo... in the same way  
>>>> that SVN has an abstraction... that could use BDB or a regular  
>>>> file system (ie. FSFS) for actual storage.  WIth something like  
>>>> this, we could have the default distro use a BDB-ish filesystem  
>>>> and completely resolve the windows file name limitations.  With  
>>>> a set of cli tools we can easily allow the BDB-ish to be  
>>>> converted to a FSFS.
>>>
>>> The repo is an interface.  Well it is really three interfaces  
>>> depending how much you want to implement:
>>>
>>> public interface Repository {
>>>     boolean contains(Artifact artifact);
>>>     File getLocation(Artifact artifact);
>>>     LinkedHashSet getDependencies(Artifact artifact);
>>> }
>>>
>>> public interface WriteableRepository extends Repository {
>>>     void copyToRepository(File source, Artifact destination,  
>>> FileWriteMonitor monitor) throws IOException;
>>>     void copyToRepository(InputStream source, int size, Artifact  
>>> destination, FileWriteMonitor monitor) throws IOException;
>>> }
>>>
>>> public interface ListableRepository extends Repository {
>>>     SortedSet list();
>>>     SortedSet list(Artifact query);
>>> }
>>>
>>> We have an M1 and M2 implementation of these, so if you want  
>>> another one you have some good base code to start with.
>>>
>>> -dain