[PROPOSAL] maven2 and james repository short term solution

[Stefano Bagnara <ap...@bago.org>]

Hi all,

The problem: we have products (jspf and mime4j) that use maven2 as build
system. We now rely on "SNAPSHOT" projects (james-project and the james
maven-skin) and on a temporary repository I setup in my minotaur home
directory.

Of course this is no good because we can't publish official releases
including references to my home or including references to SNAPSHOT
projects.

Here is my proposal: please note that I don't think this is the "final"
solution for the "james/maven relationship" but I think that it is the
smaller concrete step we have to do and I think we can afford this task now.

1) "james-project" (1.0-SNAPSHOT) and "maven-skin" (1.0-SNAPSHOT)
artifacts are now in james/sandbox. We should find a better place for them.
My proposal is to move them to the root james folder and to rename
james-project to "project":
/james
|-project
|-maven2-skin
|-server
|-jspf
|-mime4j
'-....
To keep consistency in the repository we should create
branches/trunk/tags subfolder.

As a sidenote I would prefer a structure where we have
trunk/branches/tags at the top and having a clean structure inside
trunk. But I don't want to discuss this now: let's move by small steps.

2) "james-project" currently declare "apache:apache:3" as its parent
pom. I think we should keep james-project as parent (and remove
apache:apache:3) until apache will find a common way to publish a
"controlled" repository for third party libraries. So I would copy
useful things from apache:apache:3 to our james-project root pom.

3) we should introduce a james project third party library repository
where we'll include every "non-apache" library used by our releases.
Fortunately now that we don't include junit in our releases and that sun
released javamail/activation under CDDL we can safely put all our needed
libraries in that repository (we already put that library in different
folders of our repository).

I propose to create a "repository" (or "repositories" or "repo") folder
in our "james" repository and create a "third-party-m1" subfolder where
I would commit at least the "org.bouncycastle" stuff from
http://people.apache.org/~bago/maven/dist-m1/ .
We could also commit there every non-apache library our maven2 releases
will use: dnsjava for jspf.

Both jspf and mime4j have junit as a "test" dependency: imho we could
leave with this "problem" now and don't include junit in our repository.
 People can build and use our releases without running tests if they
don't want to take risks downloading junit from ibiblio.

4) we should prepare non-snapshots releases for our "project" and
"maven-skin" and maybe we should make the same thing for jspf (0.9) and
mime4j (0.3).

5) Vote the release and publish them to the m2-ibiblio-rsync-repository
as they will start being official james artifacts so I can remove any
reference to my home on minotaur.

6) Please note that our "jspf" released artifact pom will include a
reference to
svn.apache.org/repo/asf/james/repository/third-party-m1/dnsjava/dnsjava-2.0.1.jar
and people downloading the source distribution for jspf and running
maven will automatically download it from our (ASF) svn server.

This is indeed BAD to do for packages that may have a big user base so I
think that we can safely do this only for jspf/mime4j/postage/jsieve,
but this is not a solution for james. Btw we'll have much more time
before we decide to move the official build for james from ant to maven2
and I really hope that the maven and repository teams will have found a
solution to 3rd party and security by that time.

My "optional" proposal is to export the
"james/repository/third-party-m1" under a subfolder of our website
(http://james.apache.org/repo/third-party-m1-repository/) and use that
url in our poms. This would let us to avoid "hard references" to
subversion and let us using the apache mirroring system already in place
for the website infrastructure. I think we could also add both urls
(website/svn) in the pom but declaring the svn one as a snapshot
repository so that it will not be used by final releases.


So, what do you think? Alternative proposals are welcome: keep in mind
that the goal is being able to make official jspf and mime4j releases
soon with a realistic roadmap.

Stefano

Re: [PROPOSAL] maven2 and james repository short term solution

[Vincenzo Gianferrari Pini <vi...@praxis.it>]

I think that - for non crucial changes, based on personal judgement 
;-)  - if after a few days (2-3) there are no -1, the change can be done 
(lazy consensus). If it is something urgent the time can shorten.

A vote should be called for decisions that we know/feel that are more 
arguable.

The point is: if someone is not reading, perhaps because he is 
traveling, by definition he will not answer, so it does not make sense 
wait for his +0.

Anyway I'm open to change my mind :-) .

Vincenzo

Stefano Bagnara wrote:

> Should I better call Votes instead of proposal for this things?
> Imho we should try to be more agile than now.. I can't think that for 
> every single change I have to write the proposal, wait a week to 
> receive no answers, start a vote, wait at least 3 days..
>
> Stefano
>
> Vincenzo Gianferrari Pini wrote:
>
>> Stefano,
>>
>> we all read the lists, and when I answer I take it seriously.
>>
>> When I do not answer is because I don't have anything to say and I 
>> consider it like a +0. I would answer with a -1, -0 or +1 if it were 
>> the case. I would answer in any case if it were a vote.
>>
>> I think that others do the same.
>>
>> Vincenzo
>>
>> Stefano Bagnara wrote:
>>
>>> Stefano Bagnara wrote:
>>>
>>>> So, what do you think? Alternative proposals are welcome: keep in mind
>>>> that the goal is being able to make official jspf and mime4j releases
>>>> soon with a realistic roadmap.
>>>
>>>
>>>
>>> 4 days, 1 answer :-(
>>> What's the problem? Even a +/- 0 would be useful to know that you 
>>> read this and you don't have an opinion on this.
>>>
>>> Noel: can you please change the svn notification so that it works on 
>>> the full james repository? If you have specific options it would be 
>>> good if server commits had the server-dev reply-to and the other had 
>>> the general  reply-to.
>>>
>>> Stefano
>>
>
>
>

Re: [PROPOSAL] maven2 and james repository short term solution

[Stefano Bagnara <ap...@bago.org>]

Should I better call Votes instead of proposal for this things?
Imho we should try to be more agile than now.. I can't think that for 
every single change I have to write the proposal, wait a week to receive 
no answers, start a vote, wait at least 3 days..

Stefano

Vincenzo Gianferrari Pini wrote:
> Stefano,
> 
> we all read the lists, and when I answer I take it seriously.
> 
> When I do not answer is because I don't have anything to say and I 
> consider it like a +0. I would answer with a -1, -0 or +1 if it were the 
> case. I would answer in any case if it were a vote.
> 
> I think that others do the same.
> 
> Vincenzo
> 
> Stefano Bagnara wrote:
> 
>> Stefano Bagnara wrote:
>>
>>> So, what do you think? Alternative proposals are welcome: keep in mind
>>> that the goal is being able to make official jspf and mime4j releases
>>> soon with a realistic roadmap.
>>
>>
>> 4 days, 1 answer :-(
>> What's the problem? Even a +/- 0 would be useful to know that you read 
>> this and you don't have an opinion on this.
>>
>> Noel: can you please change the svn notification so that it works on 
>> the full james repository? If you have specific options it would be 
>> good if server commits had the server-dev reply-to and the other had 
>> the general  reply-to.
>>
>> Stefano

Re: [PROPOSAL] maven2 and james repository short term solution

[Vincenzo Gianferrari Pini <vi...@praxis.it>]

Stefano,

we all read the lists, and when I answer I take it seriously.

When I do not answer is because I don't have anything to say and I 
consider it like a +0. I would answer with a -1, -0 or +1 if it were the 
case. I would answer in any case if it were a vote.

I think that others do the same.

Vincenzo

Stefano Bagnara wrote:

> Stefano Bagnara wrote:
>
>> So, what do you think? Alternative proposals are welcome: keep in mind
>> that the goal is being able to make official jspf and mime4j releases
>> soon with a realistic roadmap.
>
>
> 4 days, 1 answer :-(
> What's the problem? Even a +/- 0 would be useful to know that you read 
> this and you don't have an opinion on this.
>
> Noel: can you please change the svn notification so that it works on 
> the full james repository? If you have specific options it would be 
> good if server commits had the server-dev reply-to and the other had 
> the general  reply-to.
>
> Stefano
>
>

Re: [PROPOSAL] maven2 and james repository short term solution

[Norman Maurer <nm...@byteaction.de>]

Well so what todo now ? I cast a +1 for that. Noone cast -1.
So  it should ok for Stefano to start with it.

bye
Norman

Vincenzo Gianferrari Pini schrieb:
> Stefano,
>
> we all read the lists, and when I answer I take it seriously.
>
> When I do not answer is because I don't have anything to say and I 
> consider it like a +0. I would answer with a -1, -0 or +1 if it were 
> the case. I would answer in any case if it were a vote.
>
> I think that others do the same.
>
> Vincenzo
>
> Stefano Bagnara wrote:
>
>> Stefano Bagnara wrote:
>>
>>> So, what do you think? Alternative proposals are welcome: keep in mind
>>> that the goal is being able to make official jspf and mime4j releases
>>> soon with a realistic roadmap.
>>
>>
>> 4 days, 1 answer :-(
>> What's the problem? Even a +/- 0 would be useful to know that you 
>> read this and you don't have an opinion on this.
>>
>> Noel: can you please change the svn notification so that it works on 
>> the full james repository? If you have specific options it would be 
>> good if server commits had the server-dev reply-to and the other had 
>> the general  reply-to.
>>
>> Stefano
>>
>>
>
> !EXCUBATOR:1,4509347445119299167376!

Re: [PROPOSAL] maven2 and james repository short term solution

[Stefano Bagnara <ap...@bago.org>]

Noel J. Bergman wrote:
>> 4 days, 1 answer :-(
> 
> Try to be more clear about what you are proposing.

Sorry, I don't know how to make it more clear: I just reread it and I 
don't know what to add. Is there any specific point you don't understand?

>> Noel: can you please change the svn notification so that it works on the
>> full james repository? If you have specific options it would be good if
>> server commits had the server-dev reply-to and the other had the general
>> reply-to.
> 
> That does not seem appropriate.
> 
> It appears that you are proposing a Maven repository.  Aside from my
> continued opposition to supporting Maven repositories until Maven
> authenticates their downloads, such repositories are supposed to be
> coordinated ASF-wide via the repository@ mailing list.  I'm not at all sure
> that what you propose is according to ASF policy.

Can you point us to the policy that block us from creating a folder in 
the repository and put there the libraries our projects need to be 
build? I think we already do that in the lib folder of james/server, 
don't we?

This is a proposal to be able to make a jspf and mime4j release.
If you have an alternative proposal you're welcome: I don't think that 
the alternative proposal could be "don't release jspf/mime4j", so I 
would be happy to understand what can we do.

I believe other ASF projects have releases based on maven2 and uses the 
ibiblio repository for this. My proposal has been studied and done 
because of your concern. I did my homework. Stop.

>> we should introduce a james project third party library repository
>> where we'll include every "non-apache" library used by our releases.
> 
> We should NOT do so except in conjunction with such an effort on an ASF-wide
> basis.  We already have enough problems with abuse of the ASF
> infrastructure.  Once again, the appropriate place to discuss repositories
> is repository@apache.org.

This discussions are already in progress. The final solution will 
probably need a new maven version, and much more discussions.. We're 
probably few (or many) months far away from an ASF-wide solution, and in 
the mean time I believe we have to find an interim solution.

>> As a sidenote I would prefer a structure where we have
>> trunk/branches/tags at the top
> 
> Subversion 101: A {ttb} structure is associated with each independently
> releasable artifact.

You should better tell us what you prefer.
I don't see why "Subversion 101" should be taken as "the truth" ;-)

Btw this is a sidenote and not part of this proposal. So let's ignore it 
  by now.

Stefano

Re: [PROPOSAL] maven2 and james repository short term solution

[Stefano Bagnara <ap...@bago.org>]

Noel J. Bergman wrote:
>> Noel: can you please change the svn notification so that it works on the
>> full james repository? If you have specific options it would be good if
>> server commits had the server-dev reply-to and the other had the general
>> reply-to.
> 
> That does not seem appropriate.

Please add the notifications: we need them even without the maven 
repository. I already committed things, and they won't be notified if 
you don't add the notification.

Independently from the result of this proposal we need the notifications 
working in all the folders of the repository or we are unable to keep 
under control what it happens. Currenlty if someone (even by mistake) 
commit something in a folder that has no notification we don't see this 
and this is not good.

Why do you think notifications I'm requesting are not "appropriate"?

> It appears that you are proposing a Maven repository.  Aside from my
> continued opposition to supporting Maven repositories until Maven
> authenticates their downloads, such repositories are supposed to be
> coordinated ASF-wide via the repository@ mailing list.  I'm not at all sure
> that what you propose is according to ASF policy.

See above.

Stefano

RE: [PROPOSAL] maven2 and james repository short term solution

["Noel J. Bergman" <no...@devtech.com>]

> 4 days, 1 answer :-(

Try to be more clear about what you are proposing.

> Noel: can you please change the svn notification so that it works on the
> full james repository? If you have specific options it would be good if
> server commits had the server-dev reply-to and the other had the general
> reply-to.

That does not seem appropriate.

It appears that you are proposing a Maven repository.  Aside from my
continued opposition to supporting Maven repositories until Maven
authenticates their downloads, such repositories are supposed to be
coordinated ASF-wide via the repository@ mailing list.  I'm not at all sure
that what you propose is according to ASF policy.

> we should introduce a james project third party library repository
> where we'll include every "non-apache" library used by our releases.

We should NOT do so except in conjunction with such an effort on an ASF-wide
basis.  We already have enough problems with abuse of the ASF
infrastructure.  Once again, the appropriate place to discuss repositories
is repository@apache.org.

> As a sidenote I would prefer a structure where we have
> trunk/branches/tags at the top

Subversion 101: A {ttb} structure is associated with each independently
releasable artifact.

	--- Noel

Re: [PROPOSAL] maven2 and james repository short term solution

[Stefano Bagnara <ap...@bago.org>]

Stefano Bagnara wrote:
> So, what do you think? Alternative proposals are welcome: keep in mind
> that the goal is being able to make official jspf and mime4j releases
> soon with a realistic roadmap.

4 days, 1 answer :-(
What's the problem? Even a +/- 0 would be useful to know that you read 
this and you don't have an opinion on this.

Noel: can you please change the svn notification so that it works on the 
full james repository? If you have specific options it would be good if 
server commits had the server-dev reply-to and the other had the general 
  reply-to.

Stefano

Re: [PROPOSAL] maven2 and james repository short term solution

[Stefano Bagnara <ap...@bago.org>]

Bernd Fondermann wrote:
> Hi,
> 
> this is all new stuff for me... please be aware stupid questions 
> follow... and sorry for being late.
> 
> Below some clarification questions, short answers are welcome.
> I am +0 with the rest, because I still have to and want to get familiar 
> with it.
> 
> Stefano Bagnara wrote:
>> Hi all,
>>
>> The problem: we have products (jspf and mime4j) that use maven2 as build
>> system. We now rely on "SNAPSHOT" projects (james-project and the james
>> maven-skin) and on a temporary repository I setup in my minotaur home
>> directory.
>>
>> 1) "james-project" (1.0-SNAPSHOT) and "maven-skin" (1.0-SNAPSHOT)
>> artifacts are now in james/sandbox. We should find a better place for 
>> them.
>> My proposal is to move them to the root james folder and to rename
>> james-project to "project":
>> /james
>> |-project
> 
> "project" is only website-related?
> if yes, please let's find a better name, for example "site-general" or 
> "project-site"
> if no, what is it for?

No, "james-project" would be the root artifact for every product 
released under the apache james umbrella.
It contains the commons informations for the James project (website, 
repositories, developers, licenses).
Think at it as the AbstractPOM for our POMs (m2 has many Object Oriented 
things in its architecture)

>> 3) we should introduce a james project third party library repository
>> where we'll include every "non-apache" library used by our releases.
>> Fortunately now that we don't include junit in our releases and that sun
>> released javamail/activation under CDDL we can safely put all our needed
>> libraries in that repository (we already put that library in different
>> folders of our repository).
> 
> I guess the goal is to remove all the multiple library-copies in the 
> projects and load them from the central repo?

The goal is being able to create a release for our m2 based products 
(mime4j and jspf need a release soon) and avoid to use jars from non-asf 
repositories because Noel had some security concerns with external 
repositories security.

> will the repo also contain james server jars, like "james-<version>.jar"?

Maybe, I don't know now. I would like to solve the mime4j/jspf problem 
now.. Then we can understand better and discuss if we need our own 
releases to be published there, also (I think we don't need this, 
because ASF releases can be published in official ASF-wide repositories).

>> 4) we should prepare non-snapshots releases for our "project" and
>> "maven-skin" and maybe we should make the same thing for jspf (0.9) and
>> mime4j (0.3).
> 
> you mean, official releases?

Yes, official releases. Snapshots can already been published in the 
ASF-wide m2 snapshot repository.

>> 5) Vote the release and publish them to the m2-ibiblio-rsync-repository
>> as they will start being official james artifacts so I can remove any
>> reference to my home on minotaur.
> 
> but if we do not trust ibiblio, can we really switch to it?

I didn't say ibiblio repository, I said m2-ibiblio-rsync-repository: 
this is an ASF repository with the paculiarity that is mirrored by 
ibiblio. We *trust* the content of m2-ibiblio-rsync-repository because 
ASF has the control over its content (or at least I understood this from 
past Noel sentences).

Stefano

Re: [PROPOSAL] maven2 and james repository short term solution

[Bernd Fondermann <bf...@brainlounge.de>]

Hi,

this is all new stuff for me... please be aware stupid questions 
follow... and sorry for being late.

Below some clarification questions, short answers are welcome.
I am +0 with the rest, because I still have to and want to get familiar 
with it.

Stefano Bagnara wrote:
> Hi all,
> 
> The problem: we have products (jspf and mime4j) that use maven2 as build
> system. We now rely on "SNAPSHOT" projects (james-project and the james
> maven-skin) and on a temporary repository I setup in my minotaur home
> directory.
> 
> 1) "james-project" (1.0-SNAPSHOT) and "maven-skin" (1.0-SNAPSHOT)
> artifacts are now in james/sandbox. We should find a better place for them.
> My proposal is to move them to the root james folder and to rename
> james-project to "project":
> /james
> |-project

"project" is only website-related?
if yes, please let's find a better name, for example "site-general" or 
"project-site"
if no, what is it for?

> |-maven2-skin
> |-server
> |-jspf
> |-mime4j
> '-....
> To keep consistency in the repository we should create
> branches/trunk/tags subfolder.
> 
> 3) we should introduce a james project third party library repository
> where we'll include every "non-apache" library used by our releases.
> Fortunately now that we don't include junit in our releases and that sun
> released javamail/activation under CDDL we can safely put all our needed
> libraries in that repository (we already put that library in different
> folders of our repository).

I guess the goal is to remove all the multiple library-copies in the 
projects and load them from the central repo?

will the repo also contain james server jars, like "james-<version>.jar"?

> 4) we should prepare non-snapshots releases for our "project" and
> "maven-skin" and maybe we should make the same thing for jspf (0.9) and
> mime4j (0.3).

you mean, official releases?

> 5) Vote the release and publish them to the m2-ibiblio-rsync-repository
> as they will start being official james artifacts so I can remove any
> reference to my home on minotaur.

but if we do not trust ibiblio, can we really switch to it?

   Bernd