You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@archiva.apache.org by Deng Ching <oc...@apache.org> on 2010/06/17 07:07:56 UTC
A few notes on new repository API
A couple of things I noted down for the new repository API while working on
the generic metadata:
1. further improvements on generic metadata:
- metadata can be displayed by category (hierarchical) for easier
viewing
- implement a new plugin for the rating instead of including this in the
generic metadata (which is shown as key-value pairs) as suggested by Brett
2. make the location of the metadata repository (currently stored in the
file system) configurable
- one potential problem with having the default location in the root
directory of the actual repository is that the metadata repo can get
included if you run merge the Archiva repo to another repo (ex., running mvn
stage:copy in a staging repo)
Thanks,
Deng
Re: A few notes on new repository API
Posted by Brett Porter <br...@apache.org>.
On 18/06/2010, at 1:34 AM, Wendy Smoak wrote:
>
> Separately, would having two Archiva instances trying to update the
> metadata repository at the same time be problematic? Before with the
> database we would certainly have never tried pointing two instances at
> the same database...
For the current File implementation - there's no caching, but also no FS-level locking. There's potential for contention in writes if it were shared. Because everything is done on the fly, you could give them separate metadata stores - but that wouldn't work for storing custom metadata like Deng has implemented.
So moving forward with this we need a better implementation. This is why I'm leaning to JCR - it has all the mechanics for replication and locking built it (or you can back it onto a database and use whatever is natural there).
It very much depends on the use case and we might want to partition the content repository into parts that are 'generated' and that which is intended to be permanent.
- Brett
--
Brett Porter
brett@apache.org
http://brettporter.wordpress.com/
Re: A few notes on new repository API
Posted by Wendy Smoak <ws...@gmail.com>.
On Thu, Jun 17, 2010 at 10:29 AM, Deng Ching <oc...@apache.org> wrote:
> For the staging repos in vmbuild, the location of the index is configured
> outside of the repo so it doesn't get included when the repo is merged.
> Currently, I don't think the path to the metadata repository can be
> configured so it's always in the repo..
So it sounds like the merging feature needs to exclude the indexes and
the metadata if they are found within the repository.
Separately, would having two Archiva instances trying to update the
metadata repository at the same time be problematic? Before with the
database we would certainly have never tried pointing two instances at
the same database...
--
Wendy
Re: A few notes on new repository API
Posted by Deng Ching <oc...@apache.org>.
On Thu, Jun 17, 2010 at 9:18 PM, Wendy Smoak <ws...@gmail.com> wrote:
> On Thu, Jun 17, 2010 at 1:07 AM, Deng Ching <oc...@apache.org> wrote:
> > - one potential problem with having the default location in the root
> > directory of the actual repository is that the metadata repo can get
> > included if you run merge the Archiva repo to another repo (ex., running
> mvn
> > stage:copy in a staging repo)
>
> How do you avoid copying the indexes over in a merge? By default
> they're right in the repository also.
>
> Is the location of the metadata configurable, like it is for the indexes?
>
>
For the staging repos in vmbuild, the location of the index is configured
outside of the repo so it doesn't get included when the repo is merged.
Currently, I don't think the path to the metadata repository can be
configured so it's always in the repo..
Thanks,
Deng
Re: A few notes on new repository API
Posted by Wendy Smoak <ws...@gmail.com>.
On Thu, Jun 17, 2010 at 1:07 AM, Deng Ching <oc...@apache.org> wrote:
> - one potential problem with having the default location in the root
> directory of the actual repository is that the metadata repo can get
> included if you run merge the Archiva repo to another repo (ex., running mvn
> stage:copy in a staging repo)
How do you avoid copying the indexes over in a merge? By default
they're right in the repository also.
Is the location of the metadata configurable, like it is for the indexes?
We have two Archiva instances pointed at the same repository
filesystem and a VIP that switches over if one goes down. We'd like
to load balance across the two, but haven't tried that yet.
--
Wendy
Re: A few notes on new repository API
Posted by Deng Ching <oc...@apache.org>.
On Fri, Jun 18, 2010 at 12:22 PM, Brett Porter <br...@apache.org> wrote:
>
> On 17/06/2010, at 3:07 PM, Deng Ching wrote:
>
> > A couple of things I noted down for the new repository API while working
> on
> > the generic metadata:
> > 1. further improvements on generic metadata:
> > - metadata can be displayed by category (hierarchical) for easier
> > viewing
>
> This would be if you want to view all the metadata, instead of the generic
> one (which is not hierarchical, unless you change the way it is handled).
>
> > - implement a new plugin for the rating instead of including this in
> the
> > generic metadata (which is shown as key-value pairs) as suggested by
> Brett
>
> What I meant here is that if you want to do something on repeated
> instances, it'd be better to have a plugin (and standard namespace), that
> can validate the data and do more complicated handling, rather than
> requiring some additional naming convention. These are pretty simple pieces
> of code after all - but the generic one is still useful for those that want
> to just use a particular annotation internally without any new code.
>
> > 2. make the location of the metadata repository (currently stored in the
> > file system) configurable
>
> The file-based implementation is still only a proof of concept anyway - if
> we decide to harden that and use it I agree. But I personally would like to
> try and make JCR work and see how well it performs.
>
> > - one potential problem with having the default location in the root
> > directory of the actual repository is that the metadata repo can get
> > included if you run merge the Archiva repo to another repo (ex., running
> mvn
> > stage:copy in a staging repo)
>
>
> I think this is a fault of that goal more than Archiva :) That shouldn't
> impact the merge implementation happening now. We ignore all those in the
> scanning.
>
Sorry, I didn't mean the repo merge feature in Archiva but rather when you
run mvn stage:copy (like when we're releasing Archiva :)
Thanks,
Deng
Re: A few notes on new repository API
Posted by Brett Porter <br...@apache.org>.
On 17/06/2010, at 3:07 PM, Deng Ching wrote:
> A couple of things I noted down for the new repository API while working on
> the generic metadata:
> 1. further improvements on generic metadata:
> - metadata can be displayed by category (hierarchical) for easier
> viewing
This would be if you want to view all the metadata, instead of the generic one (which is not hierarchical, unless you change the way it is handled).
> - implement a new plugin for the rating instead of including this in the
> generic metadata (which is shown as key-value pairs) as suggested by Brett
What I meant here is that if you want to do something on repeated instances, it'd be better to have a plugin (and standard namespace), that can validate the data and do more complicated handling, rather than requiring some additional naming convention. These are pretty simple pieces of code after all - but the generic one is still useful for those that want to just use a particular annotation internally without any new code.
> 2. make the location of the metadata repository (currently stored in the
> file system) configurable
The file-based implementation is still only a proof of concept anyway - if we decide to harden that and use it I agree. But I personally would like to try and make JCR work and see how well it performs.
> - one potential problem with having the default location in the root
> directory of the actual repository is that the metadata repo can get
> included if you run merge the Archiva repo to another repo (ex., running mvn
> stage:copy in a staging repo)
I think this is a fault of that goal more than Archiva :) That shouldn't impact the merge implementation happening now. We ignore all those in the scanning.
- Brett
--
Brett Porter
brett@apache.org
http://brettporter.wordpress.com/