You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Paolo Castagna <ca...@googlemail.com> on 2011/09/05 15:58:45 UTC
We should start publishing SNAPSHOTs to the Apache Maven repository
Hi,
I propose these changes (see patch attached) to the ARQ's pom.xml file
and, if that is ok, similar changes to for Jena and the other Jena modules.
We will be using this <groupId>org.apache.jena</groupId> as groupId for
all the artifacts produced by the Jena project.
We can start publishing SNAPSHOTs to the Apache Maven Repository [1] and
receive feedback on those as well as the technical details of the releases
in Apache.
Paolo
[1] https://repository.apache.org/
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Andy Seaborne <an...@epimorphics.com>.
On 06/09/11 11:22, Paolo Castagna wrote:
> Hi Andy
>
> Andy Seaborne wrote:
>>
>>
>> On 05/09/11 14:58, Paolo Castagna wrote:
>>> Hi,
>>> I propose these changes (see patch attached) to the ARQ's pom.xml file
>>> and, if that is ok, similar changes to for Jena and the other Jena
>>> modules.
>>>
>>> We will be using this<groupId>org.apache.jena</groupId> as groupId for
>>> all the artifacts produced by the Jena project.
>>>
>>> We can start publishing SNAPSHOTs to the Apache Maven Repository [1] and
>>> receive feedback on those as well as the technical details of the
>>> releases
>>> in Apache.
>>>
>>> Paolo
>>>
>>> [1] https://repository.apache.org/
>>
>> There are various things we need to do to get the build into a state
>> where we can release on Apache. We also need to sort out the build
>> system (more in another email).
>>
>> I'd prefer that the changes as seen by our users are made in a single
>> step, or as few steps as possible. Advanced users do pickup snapshots,
>> as do builds of the other components that depend on ARQ. Fuseki and
>> TxTDB are picking up ARQ snapshots currently.
>
> I do agree: a single step change is better.
>
> I included the changes for ARQ's pom.xml file only, as an example of what
> I am proposing, but I also proposed to do same changes for all the other
> Jena modules: Eyeball, Fuseki, IRI, Joseki, LARQ, SDB, TDB and, last but
> not least, jena.
>
> We can change the<groupId> to "org.apache.jena" for all the modules in
> one step.
By "one step" I meant more than changes to all the POMs. If we change
the POMs, we can't release as we currently do so there is a gap. The
one step I'd ideally like to see includes any jar renaming.
I'd rather get the build sorted out, and start snapshots from something
close to an Apache compliant release process even for a Jena2 release.
Andy
> This way we can publish all the SNAPSHOTs on the Apache Maven Repository
> and test the technicalities of releasing artifacts and "dist" files into
> Apache.
>
> At release time, we will need to release on Apache infrastructure as well
> (i.e. next release for each module will be an Apache one).
>
> Notice: there are no SNAPSHOTs with "non Apache" groupIds here:
> https://repository.apache.org/content/repositories/snapshots/
>
> Also:
>
> "It is best to use the groupId and artifactId that will be used upon
> graduation. The version should include incubating (or incubator) to
> ensure that the artifacts created comply with Incubator release policy."
> -- http://incubator.apache.org/guides/releasemanagement.html#best-practice-maven
>
> We have an example with the LARQ module, but I had no chance to test
> the final step of the release process (i.e. there are SNAPSHOTs published:
> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/larq/
> the only way I can think of to test it end-to-end on Apache infrastructure
> is doing it.
>
> Also, maybe one of our mentors familiar with Maven and Maven in Apache
> can also help here or give feedback on what we are doing, for example by inspecting:
> https://svn.apache.org/repos/asf/incubator/jena/Jena2/LARQ/trunk/pom.xml
> and signal problems, missing things, etc. Any volunteer?
>
>> Just moving the non-Apache releaseable snapshots to Apache causes users
>> (and other modules) to have to do something in order to keep picking up
>> the latest from the new location. Just making this change and then
>> using it would force changes things so it has a cost.
>
> Yes, it has a cost.
>
>> Let's at least try to do this with a minimal number of change points,
>> ideally one.
>
> Ok.
>
>> Let's discuss where we want to get to, then see the way to
>> get there before making incremental steps that cause a number of
>> external changes.
>
> Here https://repository.apache.org/content/repositories/releases/org/apache/jena/
> and out of the incubator :-)
>
> Paolo
>
>>
>> Andy
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Paolo Castagna <ca...@googlemail.com>.
Hi Ian
Ian Dickinson wrote:
> Paolo wrote:
>> However, it might be handy to publish Javadocs, etc.
> There is already a mechanism in the Apache CMS for adding Javadocs. It
> just needs the build process, or some scripted process, to generate a
> .jar of the JavaDocs, and that to be committed to the staging site svn.
> A CMS script then unpacks the jar when staging is published, and
> positions the unpacked documentation on the external site. This saves
> having a bazillion separate Javadoc files in the SVN, but keeps us
> having a single documentation process.
>
> That's not to say that some aspects of mvn site might be useful, if we
> wanted to, say, publish test coverage results. But that's a step beyond
> what we do at the moment.
Yes, this is what I was thinking in addition to the Javadocs:
- coverage results
- firebugs reports
- release notes and/or changelog
- ...
However, it is still not clear to me if it is possible to point Maven
to a subdirectory and use a mixed approach where Apache CMS is used for
all the website + javadocs and Maven is used for automatically generated
reports about the project.
However, now that we have Jenkins this is less of a need since it is
sufficiently easy to publish some of those reports there, for example:
https://builds.apache.org/view/G-L/view/Jena/job/Jena_LARQ/
Thanks Ian.
Paolo
>
> Ian
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Benson Margulies <bi...@gmail.com>.
On Mon, Sep 12, 2011 at 4:36 AM, Ian Dickinson <ia...@epimorphics.com> wrote:
> Paolo wrote:
>> However, it might be handy to publish Javadocs, etc.
> There is already a mechanism in the Apache CMS for adding Javadocs. It
> just needs the build process, or some scripted process, to generate a
> .jar of the JavaDocs, and that to be committed to the staging site svn.
> A CMS script then unpacks the jar when staging is published, and
> positions the unpacked documentation on the external site. This saves
> having a bazillion separate Javadoc files in the SVN, but keeps us
> having a single documentation process.
So you want to run javadoc:javadoc as a plain plugin, not as a reportSet.
>
> That's not to say that some aspects of mvn site might be useful, if we
> wanted to, say, publish test coverage results. But that's a step beyond
> what we do at the moment.
>
> Ian
>
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Ian Dickinson <ia...@epimorphics.com>.
Paolo wrote:
> However, it might be handy to publish Javadocs, etc.
There is already a mechanism in the Apache CMS for adding Javadocs. It
just needs the build process, or some scripted process, to generate a
.jar of the JavaDocs, and that to be committed to the staging site svn.
A CMS script then unpacks the jar when staging is published, and
positions the unpacked documentation on the external site. This saves
having a bazillion separate Javadoc files in the SVN, but keeps us
having a single documentation process.
That's not to say that some aspects of mvn site might be useful, if we
wanted to, say, publish test coverage results. But that's a step beyond
what we do at the moment.
Ian
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Paolo Castagna <ca...@googlemail.com>.
Hi Benson
Benson Margulies wrote:
> On Mon, Sep 12, 2011 at 3:07 AM, Paolo Castagna
> <ca...@googlemail.com> wrote:
>> Hi Benson,
>> first of all, thank you for your reply. I followed all of your suggestions [1].
>>
>> A few comments inline.
>
> Ditto.
>> Benson Margulies wrote:
>>> the <scm/> element is wrong, it lacks 'trunk'.
>>>
>>> No credentials in the maven-release-plugin, use settings.xml instead.
>> I did not put the credentials in the maven-release-plugin.
>> I was just pointing to two environment variables: ${env.SF_USERNAME}
>> and ${env.SF_PASSWORD}. I removed them.
>
> Sorry, I was being brief. I understood what you did, I was just
> passing along the usual alternative.
>>> Why the special repo for non-snapshots? Aren't you planning to release
>>> at Apache, too?
>> The special repo for non-snapshots is just for testing.
>> Yes, we are planning to release at Apache!
>> Planning... I'd like to test the maven-release-plugin on the Apache
>> infrastructure.
>
> The way NexusPro works, you can do this without a custom repo. You
> just go ahead and run the release plugin, and you drop the resulting
> staging repo.
Good. I was not sure about this. Now it's clear.
This way we can test everything is working with the Apache infrastructure
without the need of a vote. When everything is fine, we can call a vote
on the artifacts in the staging repo.
>
>> It is still not clear to me if a multi-module project such as Jena
>> (which currently has Jena2, ARQ, LARQ, TDB, SDB, etc. modules) can
>> release just one single module publishing artifacts to the Apache
>> Maven repository.
>
> You don't aggregate them. You release them one-at-a-time. If you
> really want to release them all-at-once, give them a single TTB
> structure and aggregate them. Or have one extra thing to release which
> builds the release package out of all the others.
Ack
>
>
>
>> The binaries and sources to be released will probably contains all
>> the modules.
>>
>> Modules might have different life-cycles.
>>
>> Can I release the LARQ module, to test if everything works as expected
>> and we will release the rest of the modules and the Jena distribution
>> when they are ready?
>
> See above. Plan to vote to release them one at a time, and then vote
> to release a combined package. Or turn them into one branch structure.
>
>>> I think you could delete the distributionManagement element altogether.
>> Done.
>>
>>> You shouldn't put that redundant outputDirectory in reporting.
>> Removed.
>>
>>> If you are going to use the maven-site-plugin, use version 3.0 and
>>> specify the deployment location for it.
>> I don't think there are currently plans to use the maven-site-plugin.
>> Not for the site anyway.
>>
>> However, it might be handy to publish Javadocs, etc.
>>
>> Is it possible to use the maven-side-plugin to publish stuff into a
>> just a subdirectory of the website (the Jena project is using the Apache
>> CMS to manage the website)?
>>
>> If you know a project using this "mixed" approach, could you please
>> point me at it?
>
> I'd like to know the answer to this myself.
>
>>
>> Thanks again.
>>
>> Paolo
>>
>> [1] http://svn.apache.org/viewvc/incubator/jena/Jena2/LARQ/trunk/pom.xml?r1=1169616&r2=1169615&pathrev=1169616
>>
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Benson Margulies <bi...@gmail.com>.
On Mon, Sep 12, 2011 at 3:07 AM, Paolo Castagna
<ca...@googlemail.com> wrote:
> Hi Benson,
> first of all, thank you for your reply. I followed all of your suggestions [1].
>
> A few comments inline.
Ditto.
>
> Benson Margulies wrote:
>> the <scm/> element is wrong, it lacks 'trunk'.
>>
>> No credentials in the maven-release-plugin, use settings.xml instead.
>
> I did not put the credentials in the maven-release-plugin.
> I was just pointing to two environment variables: ${env.SF_USERNAME}
> and ${env.SF_PASSWORD}. I removed them.
Sorry, I was being brief. I understood what you did, I was just
passing along the usual alternative.
>
>> Why the special repo for non-snapshots? Aren't you planning to release
>> at Apache, too?
>
> The special repo for non-snapshots is just for testing.
> Yes, we are planning to release at Apache!
> Planning... I'd like to test the maven-release-plugin on the Apache
> infrastructure.
The way NexusPro works, you can do this without a custom repo. You
just go ahead and run the release plugin, and you drop the resulting
staging repo.
>
> It is still not clear to me if a multi-module project such as Jena
> (which currently has Jena2, ARQ, LARQ, TDB, SDB, etc. modules) can
> release just one single module publishing artifacts to the Apache
> Maven repository.
You don't aggregate them. You release them one-at-a-time. If you
really want to release them all-at-once, give them a single TTB
structure and aggregate them. Or have one extra thing to release which
builds the release package out of all the others.
>
> The binaries and sources to be released will probably contains all
> the modules.
>
> Modules might have different life-cycles.
>
> Can I release the LARQ module, to test if everything works as expected
> and we will release the rest of the modules and the Jena distribution
> when they are ready?
See above. Plan to vote to release them one at a time, and then vote
to release a combined package. Or turn them into one branch structure.
>
>> I think you could delete the distributionManagement element altogether.
>
> Done.
>
>> You shouldn't put that redundant outputDirectory in reporting.
>
> Removed.
>
>> If you are going to use the maven-site-plugin, use version 3.0 and
>> specify the deployment location for it.
>
> I don't think there are currently plans to use the maven-site-plugin.
> Not for the site anyway.
>
> However, it might be handy to publish Javadocs, etc.
>
> Is it possible to use the maven-side-plugin to publish stuff into a
> just a subdirectory of the website (the Jena project is using the Apache
> CMS to manage the website)?
>
> If you know a project using this "mixed" approach, could you please
> point me at it?
I'd like to know the answer to this myself.
>
>
> Thanks again.
>
> Paolo
>
> [1] http://svn.apache.org/viewvc/incubator/jena/Jena2/LARQ/trunk/pom.xml?r1=1169616&r2=1169615&pathrev=1169616
>
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Paolo Castagna <ca...@googlemail.com>.
Hi Benson,
first of all, thank you for your reply. I followed all of your suggestions [1].
A few comments inline.
Benson Margulies wrote:
> the <scm/> element is wrong, it lacks 'trunk'.
>
> No credentials in the maven-release-plugin, use settings.xml instead.
I did not put the credentials in the maven-release-plugin.
I was just pointing to two environment variables: ${env.SF_USERNAME}
and ${env.SF_PASSWORD}. I removed them.
> Why the special repo for non-snapshots? Aren't you planning to release
> at Apache, too?
The special repo for non-snapshots is just for testing.
Yes, we are planning to release at Apache!
Planning... I'd like to test the maven-release-plugin on the Apache
infrastructure.
It is still not clear to me if a multi-module project such as Jena
(which currently has Jena2, ARQ, LARQ, TDB, SDB, etc. modules) can
release just one single module publishing artifacts to the Apache
Maven repository.
The binaries and sources to be released will probably contains all
the modules.
Modules might have different life-cycles.
Can I release the LARQ module, to test if everything works as expected
and we will release the rest of the modules and the Jena distribution
when they are ready?
> I think you could delete the distributionManagement element altogether.
Done.
> You shouldn't put that redundant outputDirectory in reporting.
Removed.
> If you are going to use the maven-site-plugin, use version 3.0 and
> specify the deployment location for it.
I don't think there are currently plans to use the maven-site-plugin.
Not for the site anyway.
However, it might be handy to publish Javadocs, etc.
Is it possible to use the maven-side-plugin to publish stuff into a
just a subdirectory of the website (the Jena project is using the Apache
CMS to manage the website)?
If you know a project using this "mixed" approach, could you please
point me at it?
Thanks again.
Paolo
[1] http://svn.apache.org/viewvc/incubator/jena/Jena2/LARQ/trunk/pom.xml?r1=1169616&r2=1169615&pathrev=1169616
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Benson Margulies <bi...@gmail.com>.
the <scm/> element is wrong, it lacks 'trunk'.
No credentials in the maven-release-plugin, use settings.xml instead.
Why the special repo for non-snapshots? Aren't you planning to release
at Apache, too? I think you could delete the distributionManagement
element altogether.
You shouldn't put that redundant outputDirectory in reporting.
If you are going to use the maven-site-plugin, use version 3.0 and
specify the deployment location for it.
On Sun, Sep 11, 2011 at 6:03 PM, Ross Gardler
<rg...@opendirective.com> wrote:
> https://svn.apache.org/repos/asf/incubator/jena/Jena2/LARQ/trunk
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Ross Gardler <rg...@opendirective.com>.
On Sep 6, 2011 11:23 AM, "Paolo Castagna" <ca...@googlemail.com>
wrote
...
> Also, maybe one of our mentors familiar with Maven and Maven in Apache
> can also help here or give feedback on what we are doing, for example by
inspecting:
> https://svn.apache.org/repos/asf/incubator/jena/Jena2/LARQ/trunk/pom.xml
> and signal problems, missing things, etc. Any volunteer?
Just responding in the hope of drawing attention to it (I don't have maven
experience)
Ross
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Paolo Castagna <ca...@googlemail.com>.
Hi Andy
Andy Seaborne wrote:
>
>
> On 05/09/11 14:58, Paolo Castagna wrote:
>> Hi,
>> I propose these changes (see patch attached) to the ARQ's pom.xml file
>> and, if that is ok, similar changes to for Jena and the other Jena
>> modules.
>>
>> We will be using this<groupId>org.apache.jena</groupId> as groupId for
>> all the artifacts produced by the Jena project.
>>
>> We can start publishing SNAPSHOTs to the Apache Maven Repository [1] and
>> receive feedback on those as well as the technical details of the
>> releases
>> in Apache.
>>
>> Paolo
>>
>> [1] https://repository.apache.org/
>
> There are various things we need to do to get the build into a state
> where we can release on Apache. We also need to sort out the build
> system (more in another email).
>
> I'd prefer that the changes as seen by our users are made in a single
> step, or as few steps as possible. Advanced users do pickup snapshots,
> as do builds of the other components that depend on ARQ. Fuseki and
> TxTDB are picking up ARQ snapshots currently.
I do agree: a single step change is better.
I included the changes for ARQ's pom.xml file only, as an example of what
I am proposing, but I also proposed to do same changes for all the other
Jena modules: Eyeball, Fuseki, IRI, Joseki, LARQ, SDB, TDB and, last but
not least, jena.
We can change the <groupId> to "org.apache.jena" for all the modules in
one step.
This way we can publish all the SNAPSHOTs on the Apache Maven Repository
and test the technicalities of releasing artifacts and "dist" files into
Apache.
At release time, we will need to release on Apache infrastructure as well
(i.e. next release for each module will be an Apache one).
Notice: there are no SNAPSHOTs with "non Apache" groupIds here:
https://repository.apache.org/content/repositories/snapshots/
Also:
"It is best to use the groupId and artifactId that will be used upon
graduation. The version should include incubating (or incubator) to
ensure that the artifacts created comply with Incubator release policy."
-- http://incubator.apache.org/guides/releasemanagement.html#best-practice-maven
We have an example with the LARQ module, but I had no chance to test
the final step of the release process (i.e. there are SNAPSHOTs published:
https://repository.apache.org/content/repositories/snapshots/org/apache/jena/larq/
the only way I can think of to test it end-to-end on Apache infrastructure
is doing it.
Also, maybe one of our mentors familiar with Maven and Maven in Apache
can also help here or give feedback on what we are doing, for example by inspecting:
https://svn.apache.org/repos/asf/incubator/jena/Jena2/LARQ/trunk/pom.xml
and signal problems, missing things, etc. Any volunteer?
> Just moving the non-Apache releaseable snapshots to Apache causes users
> (and other modules) to have to do something in order to keep picking up
> the latest from the new location. Just making this change and then
> using it would force changes things so it has a cost.
Yes, it has a cost.
> Let's at least try to do this with a minimal number of change points,
> ideally one.
Ok.
> Let's discuss where we want to get to, then see the way to
> get there before making incremental steps that cause a number of
> external changes.
Here https://repository.apache.org/content/repositories/releases/org/apache/jena/
and out of the incubator :-)
Paolo
>
> Andy
Re: We should start publishing SNAPSHOTs to the Apache Maven repository
Posted by Andy Seaborne <an...@epimorphics.com>.
On 05/09/11 14:58, Paolo Castagna wrote:
> Hi,
> I propose these changes (see patch attached) to the ARQ's pom.xml file
> and, if that is ok, similar changes to for Jena and the other Jena modules.
>
> We will be using this<groupId>org.apache.jena</groupId> as groupId for
> all the artifacts produced by the Jena project.
>
> We can start publishing SNAPSHOTs to the Apache Maven Repository [1] and
> receive feedback on those as well as the technical details of the releases
> in Apache.
>
> Paolo
>
> [1] https://repository.apache.org/
There are various things we need to do to get the build into a state
where we can release on Apache. We also need to sort out the build
system (more in another email).
I'd prefer that the changes as seen by our users are made in a single
step, or as few steps as possible. Advanced users do pickup snapshots,
as do builds of the other components that depend on ARQ. Fuseki and
TxTDB are picking up ARQ snapshots currently.
Just moving the non-Apache releaseable snapshots to Apache causes users
(and other modules) to have to do something in order to keep picking up
the latest from the new location. Just making this change and then
using it would force changes things so it has a cost.
Let's at least try to do this with a minimal number of change points,
ideally one. Let's discuss where we want to get to, then see the way to
get there before making incremental steps that cause a number of
external changes.
Andy