You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Marshall Schor <ms...@schor.com> on 2010/10/18 16:22:29 UTC
reducing the number of "releases" of build tooling
Each "release" of the build tooling takes time and effort, from several people,
for doing the release, preparing test versions of our code referring to the new
release, and testing the build tooling, and then doing an svn "merge" of the
updated parent-pom references to pick up the new versions, back into the trunk.
I would like to reduce this :-). Here's a proposal:
1) Cancel the pending [VOTE] on the current build tooling, and revert the trunk
back to the previous SNAPSHOT.
2) Update the current projects in UIMA that use these build tools, to depend on
that previous SNAPSHOT, instead of depending on "released" versions.
3) Step 2) will create a "problem" when releasing the projects that use these:
projects, when released, cannot depend on snapshots. To overcome this, at the
time we release, say uimaj-sdk project, do one of the following:
3a) Change the build tooling for parent-pom-dist to include a <modules><module>
that specifies the uimaj-distr project, and then do the release from the
parent-pom-top. This will release the build tooling, plus all the items in
uimaj-distr, together.
Downside of this: it introduces a cross SVN-checkout directory dependency,
because the <module> must have some kind of relative reference from a "build"
checkout (working directory A, say), and the uimaj SDK checkout (working
directory B, for instance). It also requires that, after releasing, we update
parent-pom-distr to remove the <modules> section.
3b) When it's time to release, release the build tooling, like we do now, and
before closing the Nexus staging repo, also release the uimaj-distr projects.
Release will complain that there are "still" snapshot poms, but according to
http://jira.codehaus.org/browse/MRELEASE-583 (which is included in the set of
things included in the 2.1 version, which we're now using) the release plugin
will let you change these to release levels, as part of the release:prepare process.
With this approach, we can fiddle the build tooling as much as we need to while
getting a release to "go", and then release things.
So, I'm thinking 3b) is the way to go. Other opinions?
-Marshall
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
> On 10/22/2010 19:15, Marshall Schor wrote:
>>> On 10/22/2010 18:06, Marshall Schor wrote:
>>>>> On 10/22/2010 17:02, Marshall Schor wrote:
> [...]
>>>> Note also that you don't need to fiddle before building - this is more like a
>>>> one-time setup that tells your maven installation that it can reference the
>>>> Apache Maven Snapshot repository for artifacts - and that is where unreleased
>>>> things are.
>>> It really makes no difference. I expect to be able
>>> to extract the source code from svn onto a clean
>>> machine and build it without any extra setup. Every
>>> other open source project that I know works that
>>> way, and I see no good reason why we should be
>>> different.
>>
>> We could do this, but it would impose the requirement that the build tooling be
>> in a particular, fixed relative location relative to the other projects. I was
>> trying to decouple these.
>
> How so? When I built last night, I did a "mvn install"
> on the build tools. My hazy understanding of the process
> tells me that puts them in my local repo somehow, and
> that makes the location of the source irrelevant, doesn't
> it?
Yes, exactly right. My comment was for the case where the user *didn't* first
checkout and "mvn install" the build tooling first.
>
> I'm fine with having to explicitly build the build tools
> first, if that's what you mean.
OK. I've added that "style" to the one-time setup on our web site as the 1st of
2 alternatives.
-Marshall
>
> --Thilo
>
>
Re: reducing the number of "releases" of build tooling
Posted by Thilo Goetz <tw...@gmx.de>.
On 10/22/2010 19:15, Marshall Schor wrote:
>> On 10/22/2010 18:06, Marshall Schor wrote:
>>>> On 10/22/2010 17:02, Marshall Schor wrote:
[...]
>>> Note also that you don't need to fiddle before building - this is more like a
>>> one-time setup that tells your maven installation that it can reference the
>>> Apache Maven Snapshot repository for artifacts - and that is where unreleased
>>> things are.
>> It really makes no difference. I expect to be able
>> to extract the source code from svn onto a clean
>> machine and build it without any extra setup. Every
>> other open source project that I know works that
>> way, and I see no good reason why we should be
>> different.
>
> We could do this, but it would impose the requirement that the build tooling be
> in a particular, fixed relative location relative to the other projects. I was
> trying to decouple these.
How so? When I built last night, I did a "mvn install"
on the build tools. My hazy understanding of the process
tells me that puts them in my local repo somehow, and
that makes the location of the source irrelevant, doesn't
it?
I'm fine with having to explicitly build the build tools
first, if that's what you mean.
--Thilo
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
> On 10/22/2010 18:06, Marshall Schor wrote:
>>> On 10/22/2010 17:02, Marshall Schor wrote:
>>>> Hi -
>>>>
>>>> Let's do some debugging...
>>>>
>>>>
>>>>
>>>> On 10/22/2010 9:44 AM, Thilo Götz wrote:
>>>>> I got the "build" projects from svn. There are no build
>>>>> instructions in there, so I cd'ed to each subdir and did
>>>>> a mvn install. Some of those dirs contain various poms,
>>>>> and I'm obviously not supposed to mvn install them, as
>>>>> maven informs me. Anyway. Success otherwise.
>>>> This step (above) should be unnecessary, but it's ok to do.
>>>>
>>>>> On to uimaj-core/aggregate-uimaj. Doing a mvn install there
>>>>> gives me:
>>>> ...
>>>>> [ERROR] Non-resolvable parent POM: Could not find artifact
>>>>> org.apache.uima:parent-pom-top:pom:7
>>>> ...
>>>>> What am I doing wrong?
>>>> Nothing that I can see at the moment.
>>>> Here's some guesses:
>>>>
>>>> 1) Maybe you didn't build the parent-pom-top project in build?
>>> Apparently I didn't. Do I have to maven install every
>>> single on of those poms?
>> No, you don't need to "mvn install" every project. All you have to do is one:
>> mvn install - in the parent-pom-top project. It will build/install everything
>> else that's needed.
>> I'll add a README somewhere, that says this.
>>
>> The build tools are set up so a mvn install builds the particular parent-pom and
>> all children of that pom.
> Ah, that's good then. Just need to know which one's the
> magic pom then.
>
>>> Do you know on how many different machines
>>> I build at varying times (about 4-5 atm)?
>> Wow...
>>> On the bright side, this is the first time in about half a year
>>> that I've been able to build without having to manipulate some
>>> xml first. Even the sandbox built without a hitch.
>> :-)
>>> So what are we going to do about installing those poms? Nothing
>>> an Ant or even shell script couldn't handle, but can maven do this
>>> more elegantly? If not, I volunteer to write a script, if you
>>> will make sure it stays up to date with your maven adventures.
>>>
>> There's 2 cases: (1) Building from trunk, and (2) building from a "released"
>> source-release.
>>
>> In case 2, you don't have to do any "fiddling" with the build tooling (because
>> released source depends on released versions of our build tooling, which is in
>> maven central).
>>
>> In case 1, there is a bootstrapping issue: you have to either download/build the
>> build tooling, (one mvn install) or have a maven settings entry that points to
>> the Apache maven snapshot repository - which will enable the UIMA projects to
>> find their parent poms.
> I'd much rather build the build tools, I have no issue
> with that whatsoever. It just needs to be described
> somewhere how to do that.
OK, will do.
>>>> 2) Maybe your Maven .m2/settings.xml file isn't set up to refer to the Apache
>>>> maven snapshot repository?
>>> It most certainly isn't. I absolutely draw the line at fiddling
>>> with maven settings files before I build. Particularly given
>>> that I like to nuke my maven repo every now and then.
>>>
>> Note that you can nuke your maven repo without nuking your maven settings file.
>>
>> Note also that you don't need to fiddle before building - this is more like a
>> one-time setup that tells your maven installation that it can reference the
>> Apache Maven Snapshot repository for artifacts - and that is where unreleased
>> things are.
> It really makes no difference. I expect to be able
> to extract the source code from svn onto a clean
> machine and build it without any extra setup. Every
> other open source project that I know works that
> way, and I see no good reason why we should be
> different.
We could do this, but it would impose the requirement that the build tooling be
in a particular, fixed relative location relative to the other projects. I was
trying to decouple these.
> But we don't need to argue about that as long as
> I can just build the build tools myself.
>
>> You'll also need the maven settings file if you plan to do "deploying" or
>> "releasing" - it contains info on where the target servers are for uploading,
>> together with info on how password access to those is being managed.
> I have no plans to go there :-)
>
>>
>>> I don't do any of the other maven setup described on our website
>>> either, btw, I've never found it necessary.
>> When I try and build the docbooks or uimaj-core without the -Xmx800m setting it
>> fails. Our maven setup on our web site says to get around this by adding the
>> MAVEN_OPTS environment variable. I guess you do this some other way?
> No, I just don't do it, and it still works. May depend on what
> JVM you use.
>
True. Maybe the JVM has a higher default memory size limit.
>> That (plus the settings.xml) is really the only setup mentioned on our website,
>> other than downloading Maven 3, unless I missed something.
> There's the -XX:MaxPermSize=256m, whatever that is. Why the heck
> do we need that?
>
I don't recall, it another memory limit parameter, I think in Sun JVMs it a
place where interned strings, for instance, go, etc. Again, this may depend on
the particular JVM / platform you're building on.
>> Note: I've just updated the website for this section to add the info about maven
>> settings.xml file, under the category of it's optional, but needed if you're
>> building from trunk, not from released-sources.
> I'd rather you documented how to build from scratch on a naked
> machine without modifying the settings.xml.
OK, I'll add that.
-Marshall
>> -Marshall
>>> --Thilo
>>>
>>>
>>>
>
Re: reducing the number of "releases" of build tooling
Posted by Thilo Götz <tw...@gmx.de>.
On 10/22/2010 18:06, Marshall Schor wrote:
>> On 10/22/2010 17:02, Marshall Schor wrote:
>>> Hi -
>>>
>>> Let's do some debugging...
>>>
>>>
>>>
>>> On 10/22/2010 9:44 AM, Thilo Götz wrote:
>>>> I got the "build" projects from svn. There are no build
>>>> instructions in there, so I cd'ed to each subdir and did
>>>> a mvn install. Some of those dirs contain various poms,
>>>> and I'm obviously not supposed to mvn install them, as
>>>> maven informs me. Anyway. Success otherwise.
>>> This step (above) should be unnecessary, but it's ok to do.
>>>
>>>> On to uimaj-core/aggregate-uimaj. Doing a mvn install there
>>>> gives me:
>>> ...
>>>> [ERROR] Non-resolvable parent POM: Could not find artifact
>>>> org.apache.uima:parent-pom-top:pom:7
>>> ...
>>>> What am I doing wrong?
>>> Nothing that I can see at the moment.
>>> Here's some guesses:
>>>
>>> 1) Maybe you didn't build the parent-pom-top project in build?
>> Apparently I didn't. Do I have to maven install every
>> single on of those poms?
>
> No, you don't need to "mvn install" every project. All you have to do is one:
> mvn install - in the parent-pom-top project. It will build/install everything
> else that's needed.
> I'll add a README somewhere, that says this.
>
> The build tools are set up so a mvn install builds the particular parent-pom and
> all children of that pom.
Ah, that's good then. Just need to know which one's the
magic pom then.
>
>> Do you know on how many different machines
>> I build at varying times (about 4-5 atm)?
>
> Wow...
>> On the bright side, this is the first time in about half a year
>> that I've been able to build without having to manipulate some
>> xml first. Even the sandbox built without a hitch.
>
> :-)
>> So what are we going to do about installing those poms? Nothing
>> an Ant or even shell script couldn't handle, but can maven do this
>> more elegantly? If not, I volunteer to write a script, if you
>> will make sure it stays up to date with your maven adventures.
>>
>
> There's 2 cases: (1) Building from trunk, and (2) building from a "released"
> source-release.
>
> In case 2, you don't have to do any "fiddling" with the build tooling (because
> released source depends on released versions of our build tooling, which is in
> maven central).
>
> In case 1, there is a bootstrapping issue: you have to either download/build the
> build tooling, (one mvn install) or have a maven settings entry that points to
> the Apache maven snapshot repository - which will enable the UIMA projects to
> find their parent poms.
I'd much rather build the build tools, I have no issue
with that whatsoever. It just needs to be described
somewhere how to do that.
>
>>> 2) Maybe your Maven .m2/settings.xml file isn't set up to refer to the Apache
>>> maven snapshot repository?
>> It most certainly isn't. I absolutely draw the line at fiddling
>> with maven settings files before I build. Particularly given
>> that I like to nuke my maven repo every now and then.
>>
>
> Note that you can nuke your maven repo without nuking your maven settings file.
>
> Note also that you don't need to fiddle before building - this is more like a
> one-time setup that tells your maven installation that it can reference the
> Apache Maven Snapshot repository for artifacts - and that is where unreleased
> things are.
It really makes no difference. I expect to be able
to extract the source code from svn onto a clean
machine and build it without any extra setup. Every
other open source project that I know works that
way, and I see no good reason why we should be
different.
But we don't need to argue about that as long as
I can just build the build tools myself.
>
> You'll also need the maven settings file if you plan to do "deploying" or
> "releasing" - it contains info on where the target servers are for uploading,
> together with info on how password access to those is being managed.
I have no plans to go there :-)
>
>
>> I don't do any of the other maven setup described on our website
>> either, btw, I've never found it necessary.
>
> When I try and build the docbooks or uimaj-core without the -Xmx800m setting it
> fails. Our maven setup on our web site says to get around this by adding the
> MAVEN_OPTS environment variable. I guess you do this some other way?
No, I just don't do it, and it still works. May depend on what
JVM you use.
>
> That (plus the settings.xml) is really the only setup mentioned on our website,
> other than downloading Maven 3, unless I missed something.
There's the -XX:MaxPermSize=256m, whatever that is. Why the heck
do we need that?
>
> Note: I've just updated the website for this section to add the info about maven
> settings.xml file, under the category of it's optional, but needed if you're
> building from trunk, not from released-sources.
I'd rather you documented how to build from scratch on a naked
machine without modifying the settings.xml.
>
> -Marshall
>> --Thilo
>>
>>
>>
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
> On 10/22/2010 17:02, Marshall Schor wrote:
>> Hi -
>>
>> Let's do some debugging...
>>
>>
>>
>> On 10/22/2010 9:44 AM, Thilo Götz wrote:
>>> I got the "build" projects from svn. There are no build
>>> instructions in there, so I cd'ed to each subdir and did
>>> a mvn install. Some of those dirs contain various poms,
>>> and I'm obviously not supposed to mvn install them, as
>>> maven informs me. Anyway. Success otherwise.
>> This step (above) should be unnecessary, but it's ok to do.
>>
>>> On to uimaj-core/aggregate-uimaj. Doing a mvn install there
>>> gives me:
>> ...
>>> [ERROR] Non-resolvable parent POM: Could not find artifact
>>> org.apache.uima:parent-pom-top:pom:7
>> ...
>>> What am I doing wrong?
>> Nothing that I can see at the moment.
>> Here's some guesses:
>>
>> 1) Maybe you didn't build the parent-pom-top project in build?
> Apparently I didn't. Do I have to maven install every
> single on of those poms?
No, you don't need to "mvn install" every project. All you have to do is one:
mvn install - in the parent-pom-top project. It will build/install everything
else that's needed.
I'll add a README somewhere, that says this.
The build tools are set up so a mvn install builds the particular parent-pom and
all children of that pom.
> Do you know on how many different machines
> I build at varying times (about 4-5 atm)?
Wow...
> On the bright side, this is the first time in about half a year
> that I've been able to build without having to manipulate some
> xml first. Even the sandbox built without a hitch.
:-)
> So what are we going to do about installing those poms? Nothing
> an Ant or even shell script couldn't handle, but can maven do this
> more elegantly? If not, I volunteer to write a script, if you
> will make sure it stays up to date with your maven adventures.
>
There's 2 cases: (1) Building from trunk, and (2) building from a "released"
source-release.
In case 2, you don't have to do any "fiddling" with the build tooling (because
released source depends on released versions of our build tooling, which is in
maven central).
In case 1, there is a bootstrapping issue: you have to either download/build the
build tooling, (one mvn install) or have a maven settings entry that points to
the Apache maven snapshot repository - which will enable the UIMA projects to
find their parent poms.
>> 2) Maybe your Maven .m2/settings.xml file isn't set up to refer to the Apache
>> maven snapshot repository?
> It most certainly isn't. I absolutely draw the line at fiddling
> with maven settings files before I build. Particularly given
> that I like to nuke my maven repo every now and then.
>
Note that you can nuke your maven repo without nuking your maven settings file.
Note also that you don't need to fiddle before building - this is more like a
one-time setup that tells your maven installation that it can reference the
Apache Maven Snapshot repository for artifacts - and that is where unreleased
things are.
You'll also need the maven settings file if you plan to do "deploying" or
"releasing" - it contains info on where the target servers are for uploading,
together with info on how password access to those is being managed.
> I don't do any of the other maven setup described on our website
> either, btw, I've never found it necessary.
When I try and build the docbooks or uimaj-core without the -Xmx800m setting it
fails. Our maven setup on our web site says to get around this by adding the
MAVEN_OPTS environment variable. I guess you do this some other way?
That (plus the settings.xml) is really the only setup mentioned on our website,
other than downloading Maven 3, unless I missed something.
Note: I've just updated the website for this section to add the info about maven
settings.xml file, under the category of it's optional, but needed if you're
building from trunk, not from released-sources.
-Marshall
> --Thilo
>
>
>
Re: reducing the number of "releases" of build tooling
Posted by Thilo Götz <tw...@gmx.de>.
On 10/22/2010 17:02, Marshall Schor wrote:
> Hi -
>
> Let's do some debugging...
>
>
>
> On 10/22/2010 9:44 AM, Thilo Götz wrote:
>> I got the "build" projects from svn. There are no build
>> instructions in there, so I cd'ed to each subdir and did
>> a mvn install. Some of those dirs contain various poms,
>> and I'm obviously not supposed to mvn install them, as
>> maven informs me. Anyway. Success otherwise.
>
> This step (above) should be unnecessary, but it's ok to do.
>
>>
>> On to uimaj-core/aggregate-uimaj. Doing a mvn install there
>> gives me:
> ...
>> [ERROR] Non-resolvable parent POM: Could not find artifact
>> org.apache.uima:parent-pom-top:pom:7
> ...
>> What am I doing wrong?
>
> Nothing that I can see at the moment.
> Here's some guesses:
>
> 1) Maybe you didn't build the parent-pom-top project in build?
Apparently I didn't. Do I have to maven install every
single on of those poms? Do you know on how many different machines
I build at varying times (about 4-5 atm)?
On the bright side, this is the first time in about half a year
that I've been able to build without having to manipulate some
xml first. Even the sandbox built without a hitch.
So what are we going to do about installing those poms? Nothing
an Ant or even shell script couldn't handle, but can maven do this
more elegantly? If not, I volunteer to write a script, if you
will make sure it stays up to date with your maven adventures.
> 2) Maybe your Maven .m2/settings.xml file isn't set up to refer to the Apache
> maven snapshot repository?
It most certainly isn't. I absolutely draw the line at fiddling
with maven settings files before I build. Particularly given
that I like to nuke my maven repo every now and then.
I don't do any of the other maven setup described on our website
either, btw, I've never found it necessary.
--Thilo
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
Hi -
Let's do some debugging...
On 10/22/2010 9:44 AM, Thilo Götz wrote:
> I got the "build" projects from svn. There are no build
> instructions in there, so I cd'ed to each subdir and did
> a mvn install. Some of those dirs contain various poms,
> and I'm obviously not supposed to mvn install them, as
> maven informs me. Anyway. Success otherwise.
This step (above) should be unnecessary, but it's ok to do.
>
> On to uimaj-core/aggregate-uimaj. Doing a mvn install there
> gives me:
...
> [ERROR] Non-resolvable parent POM: Could not find artifact
> org.apache.uima:parent-pom-top:pom:7
...
> What am I doing wrong?
Nothing that I can see at the moment.
Here's some guesses:
1) Maybe you didn't build the parent-pom-top project in build?
2) Maybe your Maven .m2/settings.xml file isn't set up to refer to the Apache
maven snapshot repository?
(For # 2, I realize that the instructions for doing this are missing from the
one-time-setup - I'll fix that).
Your settings.xml file needs to have these entries to find the snapshots:
<repositories>
<repository>
<id>apache-nexus-snapshot</id>
<url>https://repository.apache.org/content/groups/snapshots-group/</url>
</repository>
</repositories>
You can avoid this if you have the parent-pom-top "mvn install"-ed into your
local repo, because it contains a ref to the Apache-wide common "POM", which
contains this repo info. But, I can see there's a "bootstrapping" issue here.
If you go the route of adding this to your settings file, I think it has to go
inside a <profile> element, something like this:
<profile>
<id>local-build-configuration</id>
<repositories>
... etc, insert above ...
</repositories>
</profile>
and then you need this to "activate" that profile, in your settings.xml:
<activeProfiles>
<activeProfile>local-build-configuration</activeProfile>
</activeProfiles>
Do either of these things help? If not, let's do some more debugging ...
-Marshall
Re: reducing the number of "releases" of build tooling
Posted by Thilo Götz <tw...@gmx.de>.
On 10/22/2010 15:07, Marshall Schor wrote:
> On 10/22/2010 8:40 AM, Thilo Götz wrote:
>> On 10/22/2010 14:02, Marshall Schor wrote:
>>>
>>> On 10/21/2010 12:26 PM, Jukka Zitting wrote:
>>>> Hi,
>>>>
>>>> On Mon, Oct 18, 2010 at 4:22 PM, Marshall Schor <ms...@schor.com> wrote:
>>>>> So, I'm thinking 3b) is the way to go. Other opinions?
>>>> I don't quite understand where all this complexity comes from.
>>>>
>>>> The only thing you need for a release is a source archive with
>>>> instructions on how to build it. If the release build depends on
>>>> previously unreleased tools, simply include them in the release
>>>> archive and make sure that the build instructions cover also how to
>>>> build the tools.
>>> Hi Jukka, nice to hear from you :-)
>>>
>>> I'm thinking out-loud how this might work. Consider building using this source
>>> archive, before the release has happened (while it's being voted on).
>>> - You get the source archive
>>> - You unzip it
>>> - You look at the README and see instructions:
>>> -- cd to this build tools dir, and do "mvn install"
>>> -- cd to this uima dir, and do "mvn install"
>>>
>>> After release, the first "mvn install" would no longer be needed, though
>>> (because the build tooling would have been updated to mvn central).
>>>
>>> Comparing that approach to 3b):
>>>
>>> The build tooling is separately released to the "staging" repository. This
>>> requires at release time, an additional "mvn release:prepare release:perform"
>>> operation. And the source release for UIMA is smaller in that it doesn't
>>> contain the maven release tooling.
>>>
>>> The advantage, I think, of this approach is that the UIMA distribution without
>>> the build tooling is buildable from source using just the one mvn install; Maven
>>> obtains the build tooling needed from maven central, via Maven's remote artifact
>>> fetching mechanism, just like it would obtain the common Apache POM. It more
>>> clearly isolates the Maven build tooling from the rest of the project, and
>>> promotes reuse of the build tooling among different releases (we're planning to
>>> release several things - the base UIMA-SDK, the UIMA-AS add-on, and the various
>>> "sandbox" projects). Also, by using this common build tooling approach, when we
>>> find problems and fix them, then they are fixed for all subsequent project releases.
>>>
>>> So, the additional complexity is really only in releasing the build tooling that
>>> changes with a *separate* mvn release:prepare/perform.
>>>
>>> I do see one small issue: Our current source README refers to the UIMA website
>>> for instructions on how to build, using maven. But over time, change will
>>> happen :-) - so the source bundle ought to be more self-contained, and have
>>> those instructions (on how to build) directly in the README, instead of via a
>>> reference to the website. They are quite tiny, something like:
>>>
>>> a) download/install Maven 3
>>> b) unzip the source distribution
>>> c) cd to a directory (depending on what it is you want to build) and do mvn install
>> This is after release. What about the current version in trunk?
>> How do I build that? Since the build tooling has not been released,
>> I just need to "mvn install" it and I'm ready to go?
>
> Yes, I think, so, and here's how it works.
Marshall, your background information is appreciated, but
what I need most of all is precise instructions on what
to do.
I got the "build" projects from svn. There are no build
instructions in there, so I cd'ed to each subdir and did
a mvn install. Some of those dirs contain various poms,
and I'm obviously not supposed to mvn install them, as
maven informs me. Anyway. Success otherwise.
On to uimaj-core/aggregate-uimaj. Doing a mvn install there
gives me:
~/code/uima-svn/core/aggregate-uimaj$ mvn install
[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project org.apache.uima:aggregate-uimaj:2.3.1-SNAPSHOT
(/home/tgoetz/code/uima-svn/core/aggregate-uimaj/pom.xml) has 1 error
[ERROR] Non-resolvable parent POM: Could not find artifact
org.apache.uima:parent-pom-top:pom:7-SNAPSHOT and 'parent.relativePath' points
at wrong local POM @ line 25, column 11 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2]
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
What am I doing wrong?
>
> The trunk has parent pom references to the current "trunk" version of the build
> tools, so these are at versions like 6-SNAPSHOT, etc.
>
> If you've downloaded and built the build tooling, from trunk, these are in your
> local repo.
>
> If you haven't done this, these are obtained from the Apache "snapshot"
> repository. This last is dependent on having the snapshot repository actually
> have these artifacts. They currently do, but that's only because someone
> "deployed" the lastest build of the build tooling at this level, to the snapshot
> repository.
>
> If and when we get around to setting up some kind of continuous integration,
> this snapshot deploy will happen automatically. That is, most CI systems (like
> Hudson, for instance), watch for SVN commits to your project, and when those
> happen, it schedules a "build" plus a deploy to the snapshot repo of the result
> (if the build is successful, of course).
>
> Note that if you already have the 6-SNAPSHOT version in your repository, if
> someone updates that build tool, you will (by default) soon get it updated in
> your local repository, via this path:
>
> 1) The CI system will notice the SVN change, and rebuild the artifact, and
> deploy it to the snapshot dir
> 2) (If there is no CI system, the person who updates the build tool, after
> checking the update into SVN and verify it's working, do a mvn deploy to update
> it to the snapshot repository).
> 3) When you run mvn install, the snapshots in your local .m2 repo are updated (I
> think once daily) from the snapshot repositories, so in about a day, you would
> see this update. (You can control this in your maven settings file, and you can
> request that things be updated when running the mvn command - just use the "-U"
> option to force an update).
>
> -Marshall
>>> Please let us know if we've overlooked something. Does this explanation make sense?
>>>
>>> -Marshall
>>>> BR,
>>>>
>>>> Jukka Zitting
>>>>
>>>>
>>
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
On 10/22/2010 8:40 AM, Thilo Götz wrote:
> On 10/22/2010 14:02, Marshall Schor wrote:
>>
>> On 10/21/2010 12:26 PM, Jukka Zitting wrote:
>>> Hi,
>>>
>>> On Mon, Oct 18, 2010 at 4:22 PM, Marshall Schor <ms...@schor.com> wrote:
>>>> So, I'm thinking 3b) is the way to go. Other opinions?
>>> I don't quite understand where all this complexity comes from.
>>>
>>> The only thing you need for a release is a source archive with
>>> instructions on how to build it. If the release build depends on
>>> previously unreleased tools, simply include them in the release
>>> archive and make sure that the build instructions cover also how to
>>> build the tools.
>> Hi Jukka, nice to hear from you :-)
>>
>> I'm thinking out-loud how this might work. Consider building using this source
>> archive, before the release has happened (while it's being voted on).
>> - You get the source archive
>> - You unzip it
>> - You look at the README and see instructions:
>> -- cd to this build tools dir, and do "mvn install"
>> -- cd to this uima dir, and do "mvn install"
>>
>> After release, the first "mvn install" would no longer be needed, though
>> (because the build tooling would have been updated to mvn central).
>>
>> Comparing that approach to 3b):
>>
>> The build tooling is separately released to the "staging" repository. This
>> requires at release time, an additional "mvn release:prepare release:perform"
>> operation. And the source release for UIMA is smaller in that it doesn't
>> contain the maven release tooling.
>>
>> The advantage, I think, of this approach is that the UIMA distribution without
>> the build tooling is buildable from source using just the one mvn install; Maven
>> obtains the build tooling needed from maven central, via Maven's remote artifact
>> fetching mechanism, just like it would obtain the common Apache POM. It more
>> clearly isolates the Maven build tooling from the rest of the project, and
>> promotes reuse of the build tooling among different releases (we're planning to
>> release several things - the base UIMA-SDK, the UIMA-AS add-on, and the various
>> "sandbox" projects). Also, by using this common build tooling approach, when we
>> find problems and fix them, then they are fixed for all subsequent project releases.
>>
>> So, the additional complexity is really only in releasing the build tooling that
>> changes with a *separate* mvn release:prepare/perform.
>>
>> I do see one small issue: Our current source README refers to the UIMA website
>> for instructions on how to build, using maven. But over time, change will
>> happen :-) - so the source bundle ought to be more self-contained, and have
>> those instructions (on how to build) directly in the README, instead of via a
>> reference to the website. They are quite tiny, something like:
>>
>> a) download/install Maven 3
>> b) unzip the source distribution
>> c) cd to a directory (depending on what it is you want to build) and do mvn install
> This is after release. What about the current version in trunk?
> How do I build that? Since the build tooling has not been released,
> I just need to "mvn install" it and I'm ready to go?
Yes, I think, so, and here's how it works.
The trunk has parent pom references to the current "trunk" version of the build
tools, so these are at versions like 6-SNAPSHOT, etc.
If you've downloaded and built the build tooling, from trunk, these are in your
local repo.
If you haven't done this, these are obtained from the Apache "snapshot"
repository. This last is dependent on having the snapshot repository actually
have these artifacts. They currently do, but that's only because someone
"deployed" the lastest build of the build tooling at this level, to the snapshot
repository.
If and when we get around to setting up some kind of continuous integration,
this snapshot deploy will happen automatically. That is, most CI systems (like
Hudson, for instance), watch for SVN commits to your project, and when those
happen, it schedules a "build" plus a deploy to the snapshot repo of the result
(if the build is successful, of course).
Note that if you already have the 6-SNAPSHOT version in your repository, if
someone updates that build tool, you will (by default) soon get it updated in
your local repository, via this path:
1) The CI system will notice the SVN change, and rebuild the artifact, and
deploy it to the snapshot dir
2) (If there is no CI system, the person who updates the build tool, after
checking the update into SVN and verify it's working, do a mvn deploy to update
it to the snapshot repository).
3) When you run mvn install, the snapshots in your local .m2 repo are updated (I
think once daily) from the snapshot repositories, so in about a day, you would
see this update. (You can control this in your maven settings file, and you can
request that things be updated when running the mvn command - just use the "-U"
option to force an update).
-Marshall
>> Please let us know if we've overlooked something. Does this explanation make sense?
>>
>> -Marshall
>>> BR,
>>>
>>> Jukka Zitting
>>>
>>>
>
Re: reducing the number of "releases" of build tooling
Posted by Thilo Götz <tw...@gmx.de>.
On 10/22/2010 14:02, Marshall Schor wrote:
>
>
> On 10/21/2010 12:26 PM, Jukka Zitting wrote:
>> Hi,
>>
>> On Mon, Oct 18, 2010 at 4:22 PM, Marshall Schor <ms...@schor.com> wrote:
>>> So, I'm thinking 3b) is the way to go. Other opinions?
>> I don't quite understand where all this complexity comes from.
>>
>> The only thing you need for a release is a source archive with
>> instructions on how to build it. If the release build depends on
>> previously unreleased tools, simply include them in the release
>> archive and make sure that the build instructions cover also how to
>> build the tools.
> Hi Jukka, nice to hear from you :-)
>
> I'm thinking out-loud how this might work. Consider building using this source
> archive, before the release has happened (while it's being voted on).
> - You get the source archive
> - You unzip it
> - You look at the README and see instructions:
> -- cd to this build tools dir, and do "mvn install"
> -- cd to this uima dir, and do "mvn install"
>
> After release, the first "mvn install" would no longer be needed, though
> (because the build tooling would have been updated to mvn central).
>
> Comparing that approach to 3b):
>
> The build tooling is separately released to the "staging" repository. This
> requires at release time, an additional "mvn release:prepare release:perform"
> operation. And the source release for UIMA is smaller in that it doesn't
> contain the maven release tooling.
>
> The advantage, I think, of this approach is that the UIMA distribution without
> the build tooling is buildable from source using just the one mvn install; Maven
> obtains the build tooling needed from maven central, via Maven's remote artifact
> fetching mechanism, just like it would obtain the common Apache POM. It more
> clearly isolates the Maven build tooling from the rest of the project, and
> promotes reuse of the build tooling among different releases (we're planning to
> release several things - the base UIMA-SDK, the UIMA-AS add-on, and the various
> "sandbox" projects). Also, by using this common build tooling approach, when we
> find problems and fix them, then they are fixed for all subsequent project releases.
>
> So, the additional complexity is really only in releasing the build tooling that
> changes with a *separate* mvn release:prepare/perform.
>
> I do see one small issue: Our current source README refers to the UIMA website
> for instructions on how to build, using maven. But over time, change will
> happen :-) - so the source bundle ought to be more self-contained, and have
> those instructions (on how to build) directly in the README, instead of via a
> reference to the website. They are quite tiny, something like:
>
> a) download/install Maven 3
> b) unzip the source distribution
> c) cd to a directory (depending on what it is you want to build) and do mvn install
This is after release. What about the current version in trunk?
How do I build that? Since the build tooling has not been released,
I just need to "mvn install" it and I'm ready to go?
>
> Please let us know if we've overlooked something. Does this explanation make sense?
>
> -Marshall
>> BR,
>>
>> Jukka Zitting
>>
>>
Re: reducing the number of "releases" of build tooling
Posted by Thilo Götz <tw...@gmx.de>.
On 10/26/2010 17:27, Thilo Götz wrote:
> On 10/26/2010 17:14, Marshall Schor wrote:
>>
>>
>> On 10/26/2010 8:02 AM, Jukka Zitting wrote:
>>> Hi,
>>>
>>> On Tue, Oct 26, 2010 at 2:10 PM, Marshall Schor <ms...@schor.com> wrote:
>>>> So, well, this isn't quite correct. Things placed here get copied into the top
>>>> of the top project, not the top level of the archive.
>>> Couldn't the top project simply be at the top level? Then this
>>> wouldn't be a problem. Convention over configuration, as they say.
>>>
>>
>> Good point.
>>
>> We're currently using a "flat hierarchy" organization for our multi-module
>> project, due to our Eclipse IDE legacy.
>>
>> Archive
>> top-dir-of-archive
>> top-project
>> README
>> LICENSE
>> NOTICE
>> RELEASE_NOTES etc.
>> other project1
>> other project2
>>
>> I was trying for:
>>
>> Archive
>> top-dir-of-archive
>> top-project
>> other project1
>> other project2
>> README
>> LICENSE
>> NOTICE
>> RELEASE_NOTES etc.
>>
>> I agree that we could restructure to eliminate the top-dir-of-archive, and
>> restructure things to have the top-project be that directory. That might break
>> some Eclipse things related to multi-module projects, so would need to be tried
>> out and tested.
>
> Would it be a problem if the README etc. don't show up
> in Eclipse? You don't need more than a text editor for
> those, and you can check in a couple of files from the
> command line. (Or maybe I'm misunderstood)
Strike the 'm
>
> --Thilo
>
>>
>> -Marshall
>>
>>> Jukka Zitting
>>>
>>>
Re: reducing the number of "releases" of build tooling
Posted by Thilo Götz <tw...@gmx.de>.
On 10/26/2010 17:14, Marshall Schor wrote:
>
>
> On 10/26/2010 8:02 AM, Jukka Zitting wrote:
>> Hi,
>>
>> On Tue, Oct 26, 2010 at 2:10 PM, Marshall Schor <ms...@schor.com> wrote:
>>> So, well, this isn't quite correct. Things placed here get copied into the top
>>> of the top project, not the top level of the archive.
>> Couldn't the top project simply be at the top level? Then this
>> wouldn't be a problem. Convention over configuration, as they say.
>>
>
> Good point.
>
> We're currently using a "flat hierarchy" organization for our multi-module
> project, due to our Eclipse IDE legacy.
>
> Archive
> top-dir-of-archive
> top-project
> README
> LICENSE
> NOTICE
> RELEASE_NOTES etc.
> other project1
> other project2
>
> I was trying for:
>
> Archive
> top-dir-of-archive
> top-project
> other project1
> other project2
> README
> LICENSE
> NOTICE
> RELEASE_NOTES etc.
>
> I agree that we could restructure to eliminate the top-dir-of-archive, and
> restructure things to have the top-project be that directory. That might break
> some Eclipse things related to multi-module projects, so would need to be tried
> out and tested.
Would it be a problem if the README etc. don't show up
in Eclipse? You don't need more than a text editor for
those, and you can check in a couple of files from the
command line. (Or maybe I'm misunderstood)
--Thilo
>
> -Marshall
>
>> Jukka Zitting
>>
>>
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
On 10/26/2010 8:02 AM, Jukka Zitting wrote:
> Hi,
>
> On Tue, Oct 26, 2010 at 2:10 PM, Marshall Schor <ms...@schor.com> wrote:
>> So, well, this isn't quite correct. Things placed here get copied into the top
>> of the top project, not the top level of the archive.
> Couldn't the top project simply be at the top level? Then this
> wouldn't be a problem. Convention over configuration, as they say.
>
Good point.
We're currently using a "flat hierarchy" organization for our multi-module
project, due to our Eclipse IDE legacy.
Archive
top-dir-of-archive
top-project
README
LICENSE
NOTICE
RELEASE_NOTES etc.
other project1
other project2
I was trying for:
Archive
top-dir-of-archive
top-project
other project1
other project2
README
LICENSE
NOTICE
RELEASE_NOTES etc.
I agree that we could restructure to eliminate the top-dir-of-archive, and
restructure things to have the top-project be that directory. That might break
some Eclipse things related to multi-module projects, so would need to be tried
out and tested.
-Marshall
> Jukka Zitting
>
>
Re: reducing the number of "releases" of build tooling
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Tue, Oct 26, 2010 at 2:10 PM, Marshall Schor <ms...@schor.com> wrote:
> So, well, this isn't quite correct. Things placed here get copied into the top
> of the top project, not the top level of the archive.
Couldn't the top project simply be at the top level? Then this
wouldn't be a problem. Convention over configuration, as they say.
BR,
Jukka Zitting
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
On 10/25/2010 2:06 PM, Marshall Schor wrote:
>
> On 10/22/2010 3:21 PM, Marshall Schor wrote:
>>> Hi,
>>>
>>> On Fri, Oct 22, 2010 at 7:44 PM, Marshall Schor <ms...@schor.com> wrote:
>>>> We could copy the build tooling stuff into each project we release (say, the
>>>> uimaj SDK, the uima-as add-on, and our Add-ons). Then each would be
>>>> independent. But we would have 3 (maybe more) copies of common build tooling to
>>>> keep in sync.
>>> Yeah, that's obviously not a good idea. One way to make this work
>>> would be to use svn:externals settings, but that's a bit hacky.
>>>
>>>> Or, we could keep the build tooling separate and shared, like it is now, but
>>>> impose a requirement that it be checked out and put in a particular relative
>>>> path with respect to the other projects' checkout, so we could use Maven's
>>>> <relativePath> element to help find the parent poms, etc. We had this approach
>>>> before, but it caused problems because of the necessity to have these checkouts
>>>> be in a particular relative path relationship.
>>> I wouldn't use relativePath for that, simply instruct people that they
>>> need to have a reasonably fresh checkouts and builds of any the
>>> upstream projects.
>>>
>>> Using Tika as an example (even though it's a lot smaller than UIMA),
>>> the tika-site component we use for managing the Tika web site has a
>>> SNAPSHOT dependency to Tika trunk. Anyone working on the web site is
>>> simply instructed to have a recent trunk build whenever changing the
>>> site.
>>>
>>> Note that the Apache parent POM also sets up a snapshot repository
>>> reference to repository.apache.org, so any SNAPSHOTs posted there will
>>> automatically be available to all projects that inherit from that POM
>>> without the need for modifying local Maven settings.
>> I noticed that, but it doesn't help for initial builds on new machines, unless
>> you build your build tooling, of course.
>> Which you do for small projects - just have everything together :-) ...
>>
>> I changed our website instructions for building from trunk to reflect the two
>> approaches:
>> either:
>>
>> 1) check-out and build the build tooling (no need to fiddle settings.xml for
>> Apache snapshots, because our poms use the Apache parent pom, as you note). or
>>
>> 2) fiddle your settings.xml to include the Apache snapshots, in which case, your
>> build will find our projects (shared) build artifacts there.
>>
>> Thanks for your help. I'm now off working on getting the README to go into the
>> right place in the source build for our top project parent pom. Looks like I
>> need a build plugin step to copy it into where the source assembly wants to find
>> the stuff that goes at the top of the source zip... so I'm adding something like
>> this:
>>
>> (I show this, in case I'm doing something silly, and there's an easier / more
>> obvious way to do this)
> This was silly. Just putting the README at the top level was enough to get it
> copied into the
> source-release.zip by the assembly process, when using -Papache-release profile.
>
So, well, this isn't quite correct. Things placed here get copied into the top
of the top project, not the top level of the archive.
That is, if the archive contains a top project TOP, the archive would have:
Archive-name
TOP
files-copied-to-top-of-the-top-project
...
LICENSE
NOTICE
etc - files at the top of the archive.
To get files to go here, using standard Apache source-release build conventions,
the files need to be placed in the
${project.build.directory}/maven-shared-archive-resources/META-INF, or a list of
files that should be copied from the project's top level to the archive top
level needs to be added to the assembly instructions.
I'll do this latter, I think.
-Marshall
> -Marshall
>> <!-- to copy README files -->
>> <plugin>
>> <artifactId>maven-resources-plugin</artifactId>
>> <executions>
>> <execution>
>> <id>copyReadme</id>
>> <inherited>false</inherited> <!-- only run at top level -->
>> <phase>process-resources</phase>
>> <goals><goal>copy-resources</goal></goals>
>> <configuration>
>> <!-- this is where the source assembly configuration gets its
>> files to put at the top level -->
>>
>> <outputDirectory>${project.build.directory}/maven-shared-archive-resources/META-INF</outputDirectory>
>> <resources>
>> <resource>
>> <directory>./</directory>
>> <includes><include>README.*</include></includes>
>> </resource>
>> </resources>
>> </configuration>
>> </execution>
>> </executions>
>>
>> </plugin>
>>
>> I tried this and it seems to work...
>>
>> -Marshall
>>>> Maybe there are other approaches too - please post suggestions :-)
>>> The easy option would be to simply have all of UIMA under the same
>>> trunk. That would mean a single release cycle for UIMA. But I guess
>>> there are good reasons not to do that.
>>>
>>> BR,
>>>
>>> Jukka Zitting
>>>
>>>
>
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
On 10/22/2010 3:21 PM, Marshall Schor wrote:
>> Hi,
>>
>> On Fri, Oct 22, 2010 at 7:44 PM, Marshall Schor <ms...@schor.com> wrote:
>>> We could copy the build tooling stuff into each project we release (say, the
>>> uimaj SDK, the uima-as add-on, and our Add-ons). Then each would be
>>> independent. But we would have 3 (maybe more) copies of common build tooling to
>>> keep in sync.
>> Yeah, that's obviously not a good idea. One way to make this work
>> would be to use svn:externals settings, but that's a bit hacky.
>>
>>> Or, we could keep the build tooling separate and shared, like it is now, but
>>> impose a requirement that it be checked out and put in a particular relative
>>> path with respect to the other projects' checkout, so we could use Maven's
>>> <relativePath> element to help find the parent poms, etc. We had this approach
>>> before, but it caused problems because of the necessity to have these checkouts
>>> be in a particular relative path relationship.
>> I wouldn't use relativePath for that, simply instruct people that they
>> need to have a reasonably fresh checkouts and builds of any the
>> upstream projects.
>>
>> Using Tika as an example (even though it's a lot smaller than UIMA),
>> the tika-site component we use for managing the Tika web site has a
>> SNAPSHOT dependency to Tika trunk. Anyone working on the web site is
>> simply instructed to have a recent trunk build whenever changing the
>> site.
>>
>> Note that the Apache parent POM also sets up a snapshot repository
>> reference to repository.apache.org, so any SNAPSHOTs posted there will
>> automatically be available to all projects that inherit from that POM
>> without the need for modifying local Maven settings.
> I noticed that, but it doesn't help for initial builds on new machines, unless
> you build your build tooling, of course.
> Which you do for small projects - just have everything together :-) ...
>
> I changed our website instructions for building from trunk to reflect the two
> approaches:
> either:
>
> 1) check-out and build the build tooling (no need to fiddle settings.xml for
> Apache snapshots, because our poms use the Apache parent pom, as you note). or
>
> 2) fiddle your settings.xml to include the Apache snapshots, in which case, your
> build will find our projects (shared) build artifacts there.
>
> Thanks for your help. I'm now off working on getting the README to go into the
> right place in the source build for our top project parent pom. Looks like I
> need a build plugin step to copy it into where the source assembly wants to find
> the stuff that goes at the top of the source zip... so I'm adding something like
> this:
>
> (I show this, in case I'm doing something silly, and there's an easier / more
> obvious way to do this)
This was silly. Just putting the README at the top level was enough to get it
copied into the
source-release.zip by the assembly process, when using -Papache-release profile.
-Marshall
> <!-- to copy README files -->
> <plugin>
> <artifactId>maven-resources-plugin</artifactId>
> <executions>
> <execution>
> <id>copyReadme</id>
> <inherited>false</inherited> <!-- only run at top level -->
> <phase>process-resources</phase>
> <goals><goal>copy-resources</goal></goals>
> <configuration>
> <!-- this is where the source assembly configuration gets its
> files to put at the top level -->
>
> <outputDirectory>${project.build.directory}/maven-shared-archive-resources/META-INF</outputDirectory>
> <resources>
> <resource>
> <directory>./</directory>
> <includes><include>README.*</include></includes>
> </resource>
> </resources>
> </configuration>
> </execution>
> </executions>
>
> </plugin>
>
> I tried this and it seems to work...
>
> -Marshall
>>> Maybe there are other approaches too - please post suggestions :-)
>> The easy option would be to simply have all of UIMA under the same
>> trunk. That would mean a single release cycle for UIMA. But I guess
>> there are good reasons not to do that.
>>
>> BR,
>>
>> Jukka Zitting
>>
>>
>
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
> Hi,
>
> On Fri, Oct 22, 2010 at 7:44 PM, Marshall Schor <ms...@schor.com> wrote:
>> We could copy the build tooling stuff into each project we release (say, the
>> uimaj SDK, the uima-as add-on, and our Add-ons). Then each would be
>> independent. But we would have 3 (maybe more) copies of common build tooling to
>> keep in sync.
> Yeah, that's obviously not a good idea. One way to make this work
> would be to use svn:externals settings, but that's a bit hacky.
>
>> Or, we could keep the build tooling separate and shared, like it is now, but
>> impose a requirement that it be checked out and put in a particular relative
>> path with respect to the other projects' checkout, so we could use Maven's
>> <relativePath> element to help find the parent poms, etc. We had this approach
>> before, but it caused problems because of the necessity to have these checkouts
>> be in a particular relative path relationship.
> I wouldn't use relativePath for that, simply instruct people that they
> need to have a reasonably fresh checkouts and builds of any the
> upstream projects.
>
> Using Tika as an example (even though it's a lot smaller than UIMA),
> the tika-site component we use for managing the Tika web site has a
> SNAPSHOT dependency to Tika trunk. Anyone working on the web site is
> simply instructed to have a recent trunk build whenever changing the
> site.
>
> Note that the Apache parent POM also sets up a snapshot repository
> reference to repository.apache.org, so any SNAPSHOTs posted there will
> automatically be available to all projects that inherit from that POM
> without the need for modifying local Maven settings.
I noticed that, but it doesn't help for initial builds on new machines, unless
you build your build tooling, of course.
Which you do for small projects - just have everything together :-) ...
I changed our website instructions for building from trunk to reflect the two
approaches:
either:
1) check-out and build the build tooling (no need to fiddle settings.xml for
Apache snapshots, because our poms use the Apache parent pom, as you note). or
2) fiddle your settings.xml to include the Apache snapshots, in which case, your
build will find our projects (shared) build artifacts there.
Thanks for your help. I'm now off working on getting the README to go into the
right place in the source build for our top project parent pom. Looks like I
need a build plugin step to copy it into where the source assembly wants to find
the stuff that goes at the top of the source zip... so I'm adding something like
this:
(I show this, in case I'm doing something silly, and there's an easier / more
obvious way to do this)
<!-- to copy README files -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>copyReadme</id>
<inherited>false</inherited> <!-- only run at top level -->
<phase>process-resources</phase>
<goals><goal>copy-resources</goal></goals>
<configuration>
<!-- this is where the source assembly configuration gets its
files to put at the top level -->
<outputDirectory>${project.build.directory}/maven-shared-archive-resources/META-INF</outputDirectory>
<resources>
<resource>
<directory>./</directory>
<includes><include>README.*</include></includes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
I tried this and it seems to work...
-Marshall
>> Maybe there are other approaches too - please post suggestions :-)
> The easy option would be to simply have all of UIMA under the same
> trunk. That would mean a single release cycle for UIMA. But I guess
> there are good reasons not to do that.
>
> BR,
>
> Jukka Zitting
>
>
Re: reducing the number of "releases" of build tooling
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Fri, Oct 22, 2010 at 7:44 PM, Marshall Schor <ms...@schor.com> wrote:
> We could copy the build tooling stuff into each project we release (say, the
> uimaj SDK, the uima-as add-on, and our Add-ons). Then each would be
> independent. But we would have 3 (maybe more) copies of common build tooling to
> keep in sync.
Yeah, that's obviously not a good idea. One way to make this work
would be to use svn:externals settings, but that's a bit hacky.
> Or, we could keep the build tooling separate and shared, like it is now, but
> impose a requirement that it be checked out and put in a particular relative
> path with respect to the other projects' checkout, so we could use Maven's
> <relativePath> element to help find the parent poms, etc. We had this approach
> before, but it caused problems because of the necessity to have these checkouts
> be in a particular relative path relationship.
I wouldn't use relativePath for that, simply instruct people that they
need to have a reasonably fresh checkouts and builds of any the
upstream projects.
Using Tika as an example (even though it's a lot smaller than UIMA),
the tika-site component we use for managing the Tika web site has a
SNAPSHOT dependency to Tika trunk. Anyone working on the web site is
simply instructed to have a recent trunk build whenever changing the
site.
Note that the Apache parent POM also sets up a snapshot repository
reference to repository.apache.org, so any SNAPSHOTs posted there will
automatically be available to all projects that inherit from that POM
without the need for modifying local Maven settings.
> Maybe there are other approaches too - please post suggestions :-)
The easy option would be to simply have all of UIMA under the same
trunk. That would mean a single release cycle for UIMA. But I guess
there are good reasons not to do that.
BR,
Jukka Zitting
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
> Hi,
>
> On Fri, Oct 22, 2010 at 5:16 PM, Marshall Schor <ms...@schor.com> wrote:
>> So, for instance, there is a docbook parent pom - it adds stuff needed for
>> processing docbooks.
> The functionality seems to be included in an automatically enabled
> profile. That could just as well go to the top-level POM, as it
> doesn't affect non-docbook builds.
True, I think.
> On the other hand, complex build extensions like that would be better
> handled in a proper Maven plugin (like
> uima-build-helper-maven-plugin), that docbook components could simply
> invoke with a normal <plugin/> entry instead of inheriting the rules
> from a parent POM.
OK, but would take some work :-) .
>> And there is an xxx -ibm-notice - this changes the normal "notice" for source
>> files to include additional text the incubator told us we needed to keep for
>> copyrights moved from original sources we got from IBM. Only those projects
>> that need this specify that parent, for instance.
> I'd simply add the extra postNoticeText property to all downstream
> POMs where its needed.
Good idea
>> The "parent-pom-distr" adds the assembly step we use to assemble our source and
>> binary distributions for UIMA and UIMA-AS.
> Why not inherit the source distribution settings directly from the
> Apache parent POM?
Yes, I've already proposed doing just that, in another thread, & we'll probably
move to that at some point (soon).
> The binary distribution settings could also be
> included in the top level UIMA parent POM by using the same
> runOnlyAtExecutionRoot setting used by the Apache parent POM.
?? I think we can embed this by using a profile that's triggered by something
in the project. The runOnlyAtExecutionRoot just keeps the assembly from being
run in sub-<modules> of a multi-module build, if I understand it correctly...
>> We could merge many of these (maybe all) into one parent, and then put in some
>> logic that activated the right parts of the pom based on what the project being
>> built, needed. Is that a better way?
> Yes, I'd say so, as at least it simplifies the project structure quite
> a bit. See above for concrete suggestions.
>
What do others think? I could go either way. The pros are it simplifies the
project structure, the cons are the parent pom becomes bigger and more complex.
Isn't this a similar trade off in software design - where you can have large,
complex classes or methods, versus doing some kind of decomposition, keeping
things which are strongly interconnected and have a particular "concern"
separated into smaller things?
>> Yes, once a project is released, then to build it just takes that one "mvn
>> install" command, done in the right directory, so it should be identical to yours.
> Then let's add those READMEs! :-)
Will do :-)
>> There is a difference, though, for building the non-released trunk version -
>> some bootstrapping is needed in your maven settings file, I think. (See other
>> parts of this thread). If you have a way around that, I'd love to hear about
>> that, too :-)
> I'm with Thilo here, it would be better if no extra settings were
> needed. Or if they really are needed, they need to be documented in
> the READMEs, for example by pointing out that you need to checkout and
> build project X before building project Y.
The way we could do this could be as follows:
We could copy the build tooling stuff into each project we release (say, the
uimaj SDK, the uima-as add-on, and our Add-ons). Then each would be
independent. But we would have 3 (maybe more) copies of common build tooling to
keep in sync.
Or, we could keep the build tooling separate and shared, like it is now, but
impose a requirement that it be checked out and put in a particular relative
path with respect to the other projects' checkout, so we could use Maven's
<relativePath> element to help find the parent poms, etc. We had this approach
before, but it caused problems because of the necessity to have these checkouts
be in a particular relative path relationship.
I'm guessing that the Tika project just keeps everything together, and that
everything is checked out at once, so the right relationships (relative
directories) are preserved. Is that right?
Maybe there are other approaches too - please post suggestions :-)
Thanks again for your feedback!
-Marshall
> BR,
>
> Jukka Zitting
>
>
Re: reducing the number of "releases" of build tooling
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Fri, Oct 22, 2010 at 5:16 PM, Marshall Schor <ms...@schor.com> wrote:
> So, for instance, there is a docbook parent pom - it adds stuff needed for
> processing docbooks.
The functionality seems to be included in an automatically enabled
profile. That could just as well go to the top-level POM, as it
doesn't affect non-docbook builds.
On the other hand, complex build extensions like that would be better
handled in a proper Maven plugin (like
uima-build-helper-maven-plugin), that docbook components could simply
invoke with a normal <plugin/> entry instead of inheriting the rules
from a parent POM.
> And there is an xxx -ibm-notice - this changes the normal "notice" for source
> files to include additional text the incubator told us we needed to keep for
> copyrights moved from original sources we got from IBM. Only those projects
> that need this specify that parent, for instance.
I'd simply add the extra postNoticeText property to all downstream
POMs where its needed.
> The "parent-pom-distr" adds the assembly step we use to assemble our source and
> binary distributions for UIMA and UIMA-AS.
Why not inherit the source distribution settings directly from the
Apache parent POM? The binary distribution settings could also be
included in the top level UIMA parent POM by using the same
runOnlyAtExecutionRoot setting used by the Apache parent POM.
> We could merge many of these (maybe all) into one parent, and then put in some
> logic that activated the right parts of the pom based on what the project being
> built, needed. Is that a better way?
Yes, I'd say so, as at least it simplifies the project structure quite
a bit. See above for concrete suggestions.
> Yes, once a project is released, then to build it just takes that one "mvn
> install" command, done in the right directory, so it should be identical to yours.
Then let's add those READMEs! :-)
> There is a difference, though, for building the non-released trunk version -
> some bootstrapping is needed in your maven settings file, I think. (See other
> parts of this thread). If you have a way around that, I'd love to hear about
> that, too :-)
I'm with Thilo here, it would be better if no extra settings were
needed. Or if they really are needed, they need to be documented in
the READMEs, for example by pointing out that you need to checkout and
build project X before building project Y.
BR,
Jukka Zitting
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
Hi, Jukka,
It's really great to get some experienced person review of what we're doing
here, and your comments are most appreciated!
> Hi,
>
> On Fri, Oct 22, 2010 at 2:02 PM, Marshall Schor <ms...@schor.com> wrote:
>> The advantage, I think, of this approach is that the UIMA distribution without
>> the build tooling is buildable from source using just the one mvn install; Maven
>> obtains the build tooling needed from maven central, via Maven's remote artifact
>> fetching mechanism, just like it would obtain the common Apache POM.
> OK, I see what you're after. The best way to handle this would
> probably to treat such cases as two parallel releases (one for the
> build tools, one for the dependant codebase) that are covered by a
> single vote ([VOTE] Release UIMA-SDK vX and build tools vY).
Right, that's what I'm thinking.
> More generally though, there are 14 POMs in uima/build/trunk,
> including *8* parent poms. Are these all really necessary? I don't
> know the details of the UIMA build that well, but this seems pretty
> excessive compared to other Maven projects I've worked with.
This may be due to (excessive perhaps) refactoring. The original intent was to
separate into individual parent poms particular aspects of the build which were
not necessarily needed by all projects.
So, for instance, there is a docbook parent pom - it adds stuff needed for
processing docbooks.
And there is an xxx -ibm-notice - this changes the normal "notice" for source
files to include additional text the incubator told us we needed to keep for
copyrights moved from original sources we got from IBM. Only those projects
that need this specify that parent, for instance.
The "parent-pom-distr" adds the assembly step we use to assemble our source and
binary distributions for UIMA and UIMA-AS.
We could merge many of these (maybe all) into one parent, and then put in some
logic that activated the right parts of the pom based on what the project being
built, needed. Is that a better way?
-Marshall
>> I do see one small issue: Our current source README refers to the UIMA website
>> for instructions on how to build, using maven. But over time, change will
>> happen :-) - so the source bundle ought to be more self-contained, and have
>> those instructions (on how to build) directly in the README, instead of via a
>> reference to the website. They are quite tiny, something like:
>>
>> a) download/install Maven 3
>> b) unzip the source distribution
>> c) cd to a directory (depending on what it is you want to build) and do mvn install
> Right, it would be great to have a README.txt with this information at
> the top of all trunks. For example, the build instructions for Apache
> Tika are [1]:
>
> Tika is based on Java 5 and uses the Maven 2 <http://maven.apache.org/>
> build system. To build Tika, use the following command in this directory:
>
> mvn clean install
>
> Ideally the UIMA build instructions should be just as simple.
Yes, once a project is released, then to build it just takes that one "mvn
install" command, done in the right directory, so it should be identical to yours.
There is a difference, though, for building the non-released trunk version -
some bootstrapping is needed in your maven settings file, I think. (See other
parts of this thread). If you have a way around that, I'd love to hear about
that, too :-)
-Marshall
> [1] http://svn.apache.org/repos/asf/tika/trunk/README.txt
>
> BR,
>
> Jukka Zitting
>
>
Re: reducing the number of "releases" of build tooling
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Fri, Oct 22, 2010 at 2:02 PM, Marshall Schor <ms...@schor.com> wrote:
> The advantage, I think, of this approach is that the UIMA distribution without
> the build tooling is buildable from source using just the one mvn install; Maven
> obtains the build tooling needed from maven central, via Maven's remote artifact
> fetching mechanism, just like it would obtain the common Apache POM.
OK, I see what you're after. The best way to handle this would
probably to treat such cases as two parallel releases (one for the
build tools, one for the dependant codebase) that are covered by a
single vote ([VOTE] Release UIMA-SDK vX and build tools vY).
More generally though, there are 14 POMs in uima/build/trunk,
including *8* parent poms. Are these all really necessary? I don't
know the details of the UIMA build that well, but this seems pretty
excessive compared to other Maven projects I've worked with.
> I do see one small issue: Our current source README refers to the UIMA website
> for instructions on how to build, using maven. But over time, change will
> happen :-) - so the source bundle ought to be more self-contained, and have
> those instructions (on how to build) directly in the README, instead of via a
> reference to the website. They are quite tiny, something like:
>
> a) download/install Maven 3
> b) unzip the source distribution
> c) cd to a directory (depending on what it is you want to build) and do mvn install
Right, it would be great to have a README.txt with this information at
the top of all trunks. For example, the build instructions for Apache
Tika are [1]:
Tika is based on Java 5 and uses the Maven 2 <http://maven.apache.org/>
build system. To build Tika, use the following command in this directory:
mvn clean install
Ideally the UIMA build instructions should be just as simple.
[1] http://svn.apache.org/repos/asf/tika/trunk/README.txt
BR,
Jukka Zitting
Re: reducing the number of "releases" of build tooling
Posted by Marshall Schor <ms...@schor.com>.
On 10/21/2010 12:26 PM, Jukka Zitting wrote:
> Hi,
>
> On Mon, Oct 18, 2010 at 4:22 PM, Marshall Schor <ms...@schor.com> wrote:
>> So, I'm thinking 3b) is the way to go. Other opinions?
> I don't quite understand where all this complexity comes from.
>
> The only thing you need for a release is a source archive with
> instructions on how to build it. If the release build depends on
> previously unreleased tools, simply include them in the release
> archive and make sure that the build instructions cover also how to
> build the tools.
Hi Jukka, nice to hear from you :-)
I'm thinking out-loud how this might work. Consider building using this source
archive, before the release has happened (while it's being voted on).
- You get the source archive
- You unzip it
- You look at the README and see instructions:
-- cd to this build tools dir, and do "mvn install"
-- cd to this uima dir, and do "mvn install"
After release, the first "mvn install" would no longer be needed, though
(because the build tooling would have been updated to mvn central).
Comparing that approach to 3b):
The build tooling is separately released to the "staging" repository. This
requires at release time, an additional "mvn release:prepare release:perform"
operation. And the source release for UIMA is smaller in that it doesn't
contain the maven release tooling.
The advantage, I think, of this approach is that the UIMA distribution without
the build tooling is buildable from source using just the one mvn install; Maven
obtains the build tooling needed from maven central, via Maven's remote artifact
fetching mechanism, just like it would obtain the common Apache POM. It more
clearly isolates the Maven build tooling from the rest of the project, and
promotes reuse of the build tooling among different releases (we're planning to
release several things - the base UIMA-SDK, the UIMA-AS add-on, and the various
"sandbox" projects). Also, by using this common build tooling approach, when we
find problems and fix them, then they are fixed for all subsequent project releases.
So, the additional complexity is really only in releasing the build tooling that
changes with a *separate* mvn release:prepare/perform.
I do see one small issue: Our current source README refers to the UIMA website
for instructions on how to build, using maven. But over time, change will
happen :-) - so the source bundle ought to be more self-contained, and have
those instructions (on how to build) directly in the README, instead of via a
reference to the website. They are quite tiny, something like:
a) download/install Maven 3
b) unzip the source distribution
c) cd to a directory (depending on what it is you want to build) and do mvn install
Please let us know if we've overlooked something. Does this explanation make sense?
-Marshall
> BR,
>
> Jukka Zitting
>
>
Re: reducing the number of "releases" of build tooling
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Mon, Oct 18, 2010 at 4:22 PM, Marshall Schor <ms...@schor.com> wrote:
> So, I'm thinking 3b) is the way to go. Other opinions?
I don't quite understand where all this complexity comes from.
The only thing you need for a release is a source archive with
instructions on how to build it. If the release build depends on
previously unreleased tools, simply include them in the release
archive and make sure that the build instructions cover also how to
build the tools.
BR,
Jukka Zitting