how to deal with the repository metadata

[Brett Porter <br...@apache.org>]

Ok, there was a bit of talk about this while I was away so I wanted to 
sum up the thoughts I've had to date about how we can deal with the 
metadata we have and work towards improving its quality.

Firstly, let's add a source to the distributionManagement in the POM 
which is rewritten by the repository tool:
"none" - there is no information about the POM's confidence level (the 
default)
"converted" - converted from a Maven 1.x POM, so we can be sure the 
format is valid but the data within it may be incomplete
"partner" - synced in directly from a partner site (and was a Maven2 
POM, current partners will be converted instead)
"deployed" - deployed to the repository directly using deploy:deploy
"verified" - hand verified the information in the POM

I think this is a sliding scale of confidence in the data. I think each 
should be able to have an interval attached to it to check for metadata 
updates (but not updates to the JAR itself - this is just about 
redownloading the POM). By default, I would check none and converted 
daily and the rest never. Once again, a CLI switch could check them all 
again. Your releases could requires a certain level of confidence - if 
you accept anything less than verified, you might risk a reproducibility 
problem in the future. One change that might be needed is to get 
maven-proxy to recognise this.

There have been more than one instance of a jar getting corrupted in the 
repository too. Because once compromised this might be propogated to 
multiple levels we do need a way to do integrity checks of local and 
internal repositories against the main one by checking that the sha1's 
match up and match what is local. This can be something added at a later 
date, just wanted to keep it in mind.

I don't think we should do a deprecation of artifacts, as of the chicken 
and egg problem, so lets just aim for the supersedes functionality in a 
later version. We can, however, do pointers which are similar and I 
think what is intended in the discussion so far: a POM is moved and 
replaced with a POM that points to the new one with a message. This 
makes historical builds work with a warning (from a clean repository).

Suggested syntax:
<distributionManagement>
  <relocation>
    <groupId>...</groupId>
    <artifactId>...</artifactId>
    <version>...</version>
    <message>...</message>
  </relocation>
</distributionManagement>

Version would match by default.

I think this covers all the outstanding issues and gives us a way to 
progressively improve the metadata without getting into a problematic state.

Any thoughts? Anything missing?

- Brett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: how to deal with the repository metadata

[Carlos Sanchez <ca...@gmail.com>]

Hi,

About deprecation, I agree that it's more a matter of "relocation",
the option you mention sounds good to me.

About the levels of confidence you mean that after "verified" nobody
can change that pom, so only the original authors of that artifact
will be allowed to mark it as verified, with the compromise that if
they find some problem they'll have to release a new version of both
pom and jar (although it doesn't change). In this case checksums will
help to avoid changing it by mistake, but it won't be enough. The
install and deploy goals will need to check if the remote pom exists
and prevent deployment if it has been already verified. And in the
case of our current setup, with syncs from different repos, we have to
tighten the security.

Regards.

On 7/17/05, Brett Porter <br...@apache.org> wrote:
> Ok, there was a bit of talk about this while I was away so I wanted to
> sum up the thoughts I've had to date about how we can deal with the
> metadata we have and work towards improving its quality.
> 
> Firstly, let's add a source to the distributionManagement in the POM
> which is rewritten by the repository tool:
> "none" - there is no information about the POM's confidence level (the
> default)
> "converted" - converted from a Maven 1.x POM, so we can be sure the
> format is valid but the data within it may be incomplete
> "partner" - synced in directly from a partner site (and was a Maven2
> POM, current partners will be converted instead)
> "deployed" - deployed to the repository directly using deploy:deploy
> "verified" - hand verified the information in the POM
> 
> I think this is a sliding scale of confidence in the data. I think each
> should be able to have an interval attached to it to check for metadata
> updates (but not updates to the JAR itself - this is just about
> redownloading the POM). By default, I would check none and converted
> daily and the rest never. Once again, a CLI switch could check them all
> again. Your releases could requires a certain level of confidence - if
> you accept anything less than verified, you might risk a reproducibility
> problem in the future. One change that might be needed is to get
> maven-proxy to recognise this.
> 
> There have been more than one instance of a jar getting corrupted in the
> repository too. Because once compromised this might be propogated to
> multiple levels we do need a way to do integrity checks of local and
> internal repositories against the main one by checking that the sha1's
> match up and match what is local. This can be something added at a later
> date, just wanted to keep it in mind.
> 
> I don't think we should do a deprecation of artifacts, as of the chicken
> and egg problem, so lets just aim for the supersedes functionality in a
> later version. We can, however, do pointers which are similar and I
> think what is intended in the discussion so far: a POM is moved and
> replaced with a POM that points to the new one with a message. This
> makes historical builds work with a warning (from a clean repository).
> 
> Suggested syntax:
> <distributionManagement>
>   <relocation>
>     <groupId>...</groupId>
>     <artifactId>...</artifactId>
>     <version>...</version>
>     <message>...</message>
>   </relocation>
> </distributionManagement>
> 
> Version would match by default.
> 
> I think this covers all the outstanding issues and gives us a way to
> progressively improve the metadata without getting into a problematic state.
> 
> Any thoughts? Anything missing?
> 
> - Brett
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
> 
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org