Paolo's mvn problems

[Benson Margulies <bi...@gmail.com>]

Is there a page that says exactly how many 'trunks' to check out and
build to get to the point where Paolo's having problems with lib-src?

Re: Paolo's mvn problems

[Paolo Castagna <ca...@googlemail.com>]

Hi Benson

Benson Margulies wrote:
>> By the way, do you know any good and well managed example of a multi-module
>> (Apache) project with different versions/life cycles for each module?
> 
> Not, I think, in the sense that you mean.
> 
> All my experience concerns two categories of project: monoliths, like
> CXF, and ant farms, like Maven.
> 
> You can model CXF as one big build tree with many modules that are
> released on a completely locked lifecycle.
> 
> Maven, on the other hand, has 27 independently-release items. They are
> only and always released independently. No one ever makes a release of
> maven-and-its-plugins. The Maven PMC makes releases of all the
> different items, each on their own schedule. So no one is ever trying
> to make a single artifact with all the sources of all of them, it
> wouldn't make sense.

The Maven example is interesting. Even if it has independently released
items, it also has an "apache-maven" module which aggregate some of those
items in order to build the Apache Maven distribution:

 - https://svn.apache.org/repos/asf/maven/maven-3/trunk/apache-maven/
 - http://www.apache.org/dist/maven/source/

This is somehow similar to what I was trying to do with JenaDist.

Interestingly, they use <dependencies> (in their pom.xml file) +
<dependencySet> (in their assembly.xml file) (as my first attempt)
rather than <modules> + <moduleSets> (as my second attempt).
I am still unclear to what are pros/cons of these two different
approaches.

Andy seemed to be in favor of the first approach, therefore I will
rework JenaDist to use that instead.

> 
> It seems to me that Jena wants to be more maven-like than CXF-like.

Probably (but, I don't read minds).

I have not strong opinions on this, I just want to find the solution which
maximizes automation, minimizes process overhead and satisfy the need of
Jena users and developers as well as committers.

Sometimes it is difficult to understand what different people want.

The JenaDist was an attempt to provide Jena users a single .zip file to
download with all the necessary .jar in a lib/ directory (as we used to
do for Jena + ARQ in the past) and all the command line scripts.

Jena as a single jar has been mentioned in the past (and I want to
explore possible solutions to do that as well). This will be another
interesting experiment.

Paolo

Re: Paolo's mvn problems

[Paolo Castagna <ca...@googlemail.com>]

Hi Andy

Andy Seaborne wrote:
> On 22/11/11 02:27, Benson Margulies wrote:
>>> By the way, do you know any good and well managed example of a
>>> multi-module
>>> (Apache) project with different versions/life cycles for each module?
>>
>> Not, I think, in the sense that you mean.
>>
>> All my experience concerns two categories of project: monoliths, like
>> CXF, and ant farms, like Maven.
>>
>> You can model CXF as one big build tree with many modules that are
>> released on a completely locked lifecycle.
>>
>> Maven, on the other hand, has 27 independently-release items. They are
>> only and always released independently. No one ever makes a release of
>> maven-and-its-plugins. The Maven PMC makes releases of all the
>> different items, each on their own schedule. So no one is ever trying
>> to make a single artifact with all the sources of all of them, it
>> wouldn't make sense.
>>
>> It seems to me that Jena wants to be more maven-like than CXF-like.
> 
> "Jena wants" to be CXF-like in so far there is a single zip non-maven
> users can download and use locally. 

An attempt to produce this "single zip non-maven users can download and use locally":
https://repository.apache.org/content/repositories/snapshots/org/apache/jena/jena-dist/2.6.5-incubating-SNAPSHOT/jena-dist-2.6.5-incubating-20111122.000215-1-bin.zip
Name needs to change from jena-dist-* to apache-jena-* (done, testing right now).

> I'd been working to a model of one
> release per sub-system (hence worrying about incubator-general@
> bandwidth).  That seems to have gone by the wayside at the last minute.

No.

JenaDist does not substitute what we have, it complements it. For example,
say you want to do a new release of ARQ. JenaDist will not be involved in
doing that. You check out ARQ and do as usual.

Only if you want to distribute the new ARQ to non-maven users via a
single zip (as the one shown above) you will need to (eventually) use
JenaDist or other ways to produce that zip.

> 
> 
> Both first-time users and experienced users do not use maven either
> because they want a simpler experience or because they manage code
> locally to insulate themselves from maven.
> 
> The single zip is just want a collection of jars (and
> ideally-but-not-necessarily associated source code).  

Adding a lib-src/ directory with the source .jar for the jena-* modules
would be good. So far, I failed to do that.

> It does not need to be buildable locally; it does not include the parallel "test" stuff.
> 
> 
> JenaDest, as I understood it, builds the zip (it's still missing various
> files). It does not (can not?) produce the "source-release" very easily.

The source release for JenaDist is (could be) this:
https://repository.apache.org/content/repositories/snapshots/org/apache/jena/jena-dist/2.6.5-incubating-SNAPSHOT/jena-dist-2.6.5-incubating-20111122.000215-1-src.zip
The name needs to change (done, testing right now).

Paolo

>  We can do that separately.
> 
>     Andy
> 

Re: Paolo's mvn problems

[Andy Seaborne <an...@apache.org>]

On 22/11/11 02:27, Benson Margulies wrote:
>> By the way, do you know any good and well managed example of a multi-module
>> (Apache) project with different versions/life cycles for each module?
>
> Not, I think, in the sense that you mean.
>
> All my experience concerns two categories of project: monoliths, like
> CXF, and ant farms, like Maven.
>
> You can model CXF as one big build tree with many modules that are
> released on a completely locked lifecycle.
>
> Maven, on the other hand, has 27 independently-release items. They are
> only and always released independently. No one ever makes a release of
> maven-and-its-plugins. The Maven PMC makes releases of all the
> different items, each on their own schedule. So no one is ever trying
> to make a single artifact with all the sources of all of them, it
> wouldn't make sense.
>
> It seems to me that Jena wants to be more maven-like than CXF-like.

"Jena wants" to be CXF-like in so far there is a single zip non-maven 
users can download and use locally.  I'd been working to a model of one 
release per sub-system (hence worrying about incubator-general@ 
bandwidth).  That seems to have gone by the wayside at the last minute.


Both first-time users and experienced users do not use maven either 
because they want a simpler experience or because they manage code 
locally to insulate themselves from maven.

The single zip is just want a collection of jars (and 
ideally-but-not-necessarily associated source code).  It does not need 
to be buildable locally; it does not include the parallel "test" stuff.


JenaDest, as I understood it, builds the zip (it's still missing various 
files). It does not (can not?) produce the "source-release" very easily. 
  We can do that separately.

	Andy

Re: Paolo's mvn problems

[Benson Margulies <bi...@gmail.com>]

> By the way, do you know any good and well managed example of a multi-module
> (Apache) project with different versions/life cycles for each module?

Not, I think, in the sense that you mean.

All my experience concerns two categories of project: monoliths, like
CXF, and ant farms, like Maven.

You can model CXF as one big build tree with many modules that are
released on a completely locked lifecycle.

Maven, on the other hand, has 27 independently-release items. They are
only and always released independently. No one ever makes a release of
maven-and-its-plugins. The Maven PMC makes releases of all the
different items, each on their own schedule. So no one is ever trying
to make a single artifact with all the sources of all of them, it
wouldn't make sense.

It seems to me that Jena wants to be more maven-like than CXF-like.

Re: Paolo's mvn problems

[Benson Margulies <bi...@gmail.com>]

I'm not an expert on the 'core maven release'; I labor in the vinyards
of the plugins.

On Tue, Nov 22, 2011 at 5:52 AM, Bertrand Delacretaz
<bd...@apache.org> wrote:
> On Tue, Nov 22, 2011 at 2:23 AM, Paolo Castagna
> <ca...@googlemail.com> wrote:
>> ...By the way, do you know any good and well managed example of a multi-module
>> (Apache) project with different versions/life cycles for each module?...
>
> I think we're doing reasonably well in Sling, our release instructions
> are at http://sling.apache.org/site/release-management.html
>
> -Bertrand
>

Re: Paolo's mvn problems

[Paolo Castagna <ca...@googlemail.com>]

Bertrand Delacretaz wrote:
> On Tue, Nov 22, 2011 at 12:22 PM, Paolo Castagna
> <ca...@googlemail.com> wrote:
>> ... Sources here: https://svn.apache.org/repos/asf/sling/trunk/
>> Distribution here: http://www.apache.org/dist/sling/
>> Maven artifacts here: http://repo1.maven.org/maven2/org/apache/sling/
> 
> And tags here: https://svn.apache.org/repos/asf/sling/tags/
> Releases of individual modules get their own tag, I think that's important.

Yep.

We do that already (and we will continue to do so)m for example:
https://svn.apache.org/repos/asf/incubator/jena/Jena2/ARQ/tags/
https://svn.apache.org/repos/asf/incubator/jena/Jena2/TDB/tags/
...

If we use the Maven release plugin (via mvn -Papache-release
release:prepare and release:perform) the tags in SVN are done
automatically (which is one of the nice benefits of using the
Maven release plugin).

Thanks,
Paolo

> 
> -Bertrand

Re: Paolo's mvn problems

[Bertrand Delacretaz <bd...@apache.org>]

On Tue, Nov 22, 2011 at 12:22 PM, Paolo Castagna
<ca...@googlemail.com> wrote:
>... Sources here: https://svn.apache.org/repos/asf/sling/trunk/
> Distribution here: http://www.apache.org/dist/sling/
> Maven artifacts here: http://repo1.maven.org/maven2/org/apache/sling/

And tags here: https://svn.apache.org/repos/asf/sling/tags/
Releases of individual modules get their own tag, I think that's important.

-Bertrand

Re: Paolo's mvn problems

[Paolo Castagna <ca...@googlemail.com>]

Bertrand Delacretaz wrote:
> On Tue, Nov 22, 2011 at 2:23 AM, Paolo Castagna
> <ca...@googlemail.com> wrote:
>> ...By the way, do you know any good and well managed example of a multi-module
>> (Apache) project with different versions/life cycles for each module?...
> 
> I think we're doing reasonably well in Sling, our release instructions
> are at http://sling.apache.org/site/release-management.html

Thanks Bertrand,
I'll have a look.

Sources here: https://svn.apache.org/repos/asf/sling/trunk/
Distribution here: http://www.apache.org/dist/sling/
Maven artifacts here: http://repo1.maven.org/maven2/org/apache/sling/

Paolo

> 
> -Bertrand

Re: Paolo's mvn problems

[Bertrand Delacretaz <bd...@apache.org>]

On Tue, Nov 22, 2011 at 2:23 AM, Paolo Castagna
<ca...@googlemail.com> wrote:
> ...By the way, do you know any good and well managed example of a multi-module
> (Apache) project with different versions/life cycles for each module?...

I think we're doing reasonably well in Sling, our release instructions
are at http://sling.apache.org/site/release-management.html

-Bertrand

Re: Paolo's mvn problems

[Paolo Castagna <ca...@googlemail.com>]

Paolo Castagna wrote:
> Hi Benson,
> my question on users@maven.apache.org is for a new (and still work in progress)
> JenaDist module. It was in a Scratch area until today and there is still not
> consensus on what to do. Also, Andy was confused by it, which is not good.
> I might move it back to a Scratch area until it's clearer what we want.

By the way, do you know any good and well managed example of a multi-module
(Apache) project with different versions/life cycles for each module?

Cheers,
Paolo

Re: Paolo's mvn problems

[Paolo Castagna <ca...@googlemail.com>]

Paolo Castagna wrote:
> The file apache-jena-2.6.5-incubating-SNAPSHOT-bin.zip in target/ should have
> a lib-src/ with the -sources.jar files of the Jena artifacts. How do I do this?
> Even if we decide not to use this for Jena, this is something I am interested
> in since I might use it elsewhere.

Another source of confusion is caused by the presence of a file named:
apache-jena-2.6.5-incubating-SNAPSHOT-source-release.zip in the target/
directory.

This is the source-release of the JenaDist module which is clearly empty
and it does not contain what you find in the file named:
apache-jena-2.6.5-incubating-SNAPSHOT-src.zip
JenaDist, since it aggregates multiple modules, does not behave as all
the other single artifact modules.

Do you know how can I avoid, only for the JenaDist module, to have the
apache-jena-2.6.5-incubating-SNAPSHOT-source-release.zip created?

Please, note that the JenaDist's pom.xml file uses as a parent pom
JenaTop which uses Apache's one [1]. It is this one which generates
the -source-release.zip via the apache-release profile.

I tried to override the <profile>, but it does not seem to work.

Paolo

 [1] https://repository.apache.org/content/repositories/releases/org/apache/apache/

Re: Paolo's mvn problems

[Paolo Castagna <ca...@googlemail.com>]

Hi Benson,
my question on users@maven.apache.org is for a new (and still work in progress)
JenaDist module. It was in a Scratch area until today and there is still not
consensus on what to do. Also, Andy was confused by it, which is not good.
I might move it back to a Scratch area until it's clearer what we want.

However, if you want to try it, here is what you can do:

  mkdir /tmp/jena
  cd /tmp/jena
  svn co https://svn.apache.org/repos/asf/incubator/jena/Jena2/ARQ/trunk ARQ
  svn co https://svn.apache.org/repos/asf/incubator/jena/Jena2/LARQ/trunk jena-larq
  svn co https://svn.apache.org/repos/asf/incubator/jena/Jena2/jena/trunk Jena2
  svn co https://svn.apache.org/repos/asf/incubator/jena/Jena2/TDB/trunk TDB
  svn co https://svn.apache.org/repos/asf/incubator/jena/Jena2/IRI/trunk IRI
  svn co https://svn.apache.org/repos/asf/incubator/jena/Jena2/JenaTop/trunk JenaTop
  svn co https://svn.apache.org/repos/asf/incubator/jena/Jena2/JenaDist/trunk JenaDist
  cd JenaDist
  mvn -DskipTests=true -Papache-release package
  ls -la target/

The file apache-jena-2.6.5-incubating-SNAPSHOT-bin.zip in target/ should have
a lib-src/ with the -sources.jar files of the Jena artifacts. How do I do this?
Even if we decide not to use this for Jena, this is something I am interested
in since I might use it elsewhere.

I know... names are not ideal/consistent and the layout with multiple trunks does
not make it easy to checkout everything in one shot. But each Jena module has a
different version number (and they used to have different life cycles/speeds...
and they might continue to do so).

Paolo

Benson Margulies wrote:
> Is there a page that says exactly how many 'trunks' to check out and
> build to get to the point where Paolo's having problems with lib-src?