You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by Jean-Sebastien Delfino <js...@apache.org> on 2007/12/05 02:26:49 UTC
How to run incremental builds of the Tuscany SCA tree
I'm sharing this as I think it could be useful to others.
I like to build the java/sca tree before committing code changes to
check that my changes are not going to break any other module.
This gives me a peace of mind as I don't have to worry about build
breaks and is also nice to other contributors as I know that I'm not
breaking their modules.
Unfortunately Maven is not particularly smart about detecting changes
and rebuilding only impacted modules and the brute force "mvn clean
install" on the whole tree takes long and makes my Thinkpad's CPU overheat.
So I thought it would be nice to teach Maven how to do more incremental
builds:
- recompile and retest with mvn clean install the modules that have
compile dependencies on the modules I've changed
- only retest the modules that have runtime and test dependencies on the
modules I've changed.
I developed a little Maven plugin to do it. It's really simple and
leverages the Maven dependency model, see for yourself at [1].
Here's how I use the plugin to run incremental builds:
1. To mark that a module has changed, create a .modified marker file at
its root. This is what the plugin recognizes as an indication that the
module has changed. I was thinking about running "svn st" automatically
from the plugin but for now I've found simpler to just use a shell
script like follows to create/remove these .modified marker files:
st=`svn st | grep -v ".modified"`
if [ "x$st" != "x" ]
then
echo "Modified"
echo "$st"
touch .modified
else
echo "Not modified"
rm -f .modified
fi
This is for Linux, I'm sure that it can be translated to a Windows script.
2. To run the incremental build, from java/sca run:
mvn -o org.apache.tuscany.sca:tuscany-maven-incremental-build:build
This will recompile and/or retest only the modules affected by the
changed modules. Faster and more reliable, and greener as my Thinkpad
now uses less energy :)
Hope this helps. I'm interested in your feedback if you use it and like
it, or not :) or want to help improve it.
[1]
https://svn.apache.org/repos/asf/incubator/tuscany/java/sca/tools/maven/maven-incremental-build/src/main/java/org/apache/tuscany/sca/tools/incremental/build/plugin/IncrementalBuildMojo.java
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Daniel Kulp <da...@kulp.com>.
On Friday 14 December 2007, ant elder wrote:
> Thats interesting, an SVN location where any Apache committer can
> commit would be useful for Tuscany as well, for developing extensions
> or the Geronimo integration, Synapse or the DAS/LDAP work etc. How do
> you go about getting one of those set up?
That's a good question and I don't really know. My gut feeling says
that your (P)PMC would have to vote to allow that, then the IPMC would
have to approve it. Part of the issues with an "everyone can commit"
thing is the issues around IP vetting so having such a place does
increase that burden. In anycase, if the project agrees, it's
PROBABLY just create a sandbox directory and then have your mentors set
it up like the maven/sandbox area in the access control file. I don't
have access to that file (just the PMC chairs do) so I really cannot
tell them how to do it other than say "look at what maven did".
Dan
>
> ...ant
>
> On Dec 14, 2007 12:35 PM, Daniel Kulp <da...@kulp.com> wrote:
> > One additional point:
> >
> > maven has a sandbox for plugins:
> > http://svn.apache.org/repos/asf/maven/sandbox/trunk/plugins/
> > where ANYONE with apache commit access can commit and develop
> > plugins. It might be good to start it there so other maven folks can
> > really look at it, help out, etc.... The sandbox is specifically
> > setup so other Apache folks can help contribute cool ideas to maven
> > without the barriers of JIRA patches and such.
> >
> > When it looks good, they hold a vote and just svn mv it out of the
> > sandbox.
> >
> > Dan
> >
> > On Friday 14 December 2007, ant elder wrote:
> > > On Dec 14, 2007 1:05 AM, Jean-Sebastien Delfino
> > > <js...@apache.org>
> > >
> > > wrote:
> > > > Daniel Kulp wrote:
> > > > > That's pretty cool. Any thoughts about submitting that to
> > > > > the maven team?
> > > > >
> > > > > Dan
> > > >
> > > > I had started with posts [1],[2] to the Maven list. The only
> > > > response I got [3] gave me a good laugh, the next logical step
> > > > was to write the plugin :)
> > > >
> > > > I'd suggest to use and mature the plugin a bit then see if they
> > > > are open to revisit their "This is by design not what Maven
> > > > does" statement.
> > > >
> > > > [1]
> > > > http://www.mail-archive.com/users@maven.apache.org/msg76335.html
> > > > [2]
> > > > http://www.mail-archive.com/users@maven.apache.org/msg76622.html
> > > > [3]
> > > > http://www.mail-archive.com/users@maven.apache.org/msg77059.html
> > > > --
> > > > Jean-Sebastien
> > >
> > > +1 to trying to contribute this to Maven.
> > >
> > > This is exactly the type of thing we are looking for to help with
> > > graduation to show we're good Apache citizens and play well with
> > > other projects. I don't think one post should put you off, we
> > > don't all agree on everything here either, and now there's a
> > > direct invitation from Dan. I think it would actually be more
> > > beneficial to try to develop it over there instead of maturing it
> > > here. Those are the guys who know Maven so there will be more
> > > expert help and guidance there than here and it will provide a
> > > great opportunity to show people outside Tuscany how good the
> > > Tuscany people are at practising open development.
> > >
> > > ...ant
> >
> > --
> > Daniel Kulp
> > dan@kulp.com
> >
> > --------------------------------------------------------------------
> >- To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org For
> > additional commands, e-mail: tuscany-dev-help@ws.apache.org
--
Daniel Kulp
dan@kulp.com
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by ant elder <an...@gmail.com>.
Thats interesting, an SVN location where any Apache committer can commit
would be useful for Tuscany as well, for developing extensions or the
Geronimo integration, Synapse or the DAS/LDAP work etc. How do you go about
getting one of those set up?
...ant
On Dec 14, 2007 12:35 PM, Daniel Kulp <da...@kulp.com> wrote:
>
> One additional point:
>
> maven has a sandbox for plugins:
> http://svn.apache.org/repos/asf/maven/sandbox/trunk/plugins/
> where ANYONE with apache commit access can commit and develop plugins.
> It might be good to start it there so other maven folks can really look
> at it, help out, etc.... The sandbox is specifically setup so other
> Apache folks can help contribute cool ideas to maven without the
> barriers of JIRA patches and such.
>
> When it looks good, they hold a vote and just svn mv it out of the
> sandbox.
>
> Dan
>
>
>
> On Friday 14 December 2007, ant elder wrote:
> > On Dec 14, 2007 1:05 AM, Jean-Sebastien Delfino <js...@apache.org>
> >
> > wrote:
> > > Daniel Kulp wrote:
> > > > That's pretty cool. Any thoughts about submitting that to the
> > > > maven team?
> > > >
> > > > Dan
> > >
> > > I had started with posts [1],[2] to the Maven list. The only
> > > response I got [3] gave me a good laugh, the next logical step was
> > > to write the plugin :)
> > >
> > > I'd suggest to use and mature the plugin a bit then see if they are
> > > open to revisit their "This is by design not what Maven does"
> > > statement.
> > >
> > > [1] http://www.mail-archive.com/users@maven.apache.org/msg76335.html
> > > [2] http://www.mail-archive.com/users@maven.apache.org/msg76622.html
> > > [3] http://www.mail-archive.com/users@maven.apache.org/msg77059.html
> > > --
> > > Jean-Sebastien
> >
> > +1 to trying to contribute this to Maven.
> >
> > This is exactly the type of thing we are looking for to help with
> > graduation to show we're good Apache citizens and play well with other
> > projects. I don't think one post should put you off, we don't all
> > agree on everything here either, and now there's a direct invitation
> > from Dan. I think it would actually be more beneficial to try to
> > develop it over there instead of maturing it here. Those are the guys
> > who know Maven so there will be more expert help and guidance there
> > than here and it will provide a great opportunity to show people
> > outside Tuscany how good the Tuscany people are at practising open
> > development.
> >
> > ...ant
>
>
>
> --
> Daniel Kulp
> dan@kulp.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
>
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Jean-Sebastien Delfino <js...@apache.org>.
Daniel Kulp wrote:
> One additional point:
>
> maven has a sandbox for plugins:
> http://svn.apache.org/repos/asf/maven/sandbox/trunk/plugins/
> where ANYONE with apache commit access can commit and develop plugins.
> It might be good to start it there so other maven folks can really look
> at it, help out, etc.... The sandbox is specifically setup so other
> Apache folks can help contribute cool ideas to maven without the
> barriers of JIRA patches and such.
>
> When it looks good, they hold a vote and just svn mv it out of the
> sandbox.
>
> Dan
>
OK, I'll contribute it to the Maven sandbox.
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Daniel Kulp <da...@kulp.com>.
One additional point:
maven has a sandbox for plugins:
http://svn.apache.org/repos/asf/maven/sandbox/trunk/plugins/
where ANYONE with apache commit access can commit and develop plugins.
It might be good to start it there so other maven folks can really look
at it, help out, etc.... The sandbox is specifically setup so other
Apache folks can help contribute cool ideas to maven without the
barriers of JIRA patches and such.
When it looks good, they hold a vote and just svn mv it out of the
sandbox.
Dan
On Friday 14 December 2007, ant elder wrote:
> On Dec 14, 2007 1:05 AM, Jean-Sebastien Delfino <js...@apache.org>
>
> wrote:
> > Daniel Kulp wrote:
> > > That's pretty cool. Any thoughts about submitting that to the
> > > maven team?
> > >
> > > Dan
> >
> > I had started with posts [1],[2] to the Maven list. The only
> > response I got [3] gave me a good laugh, the next logical step was
> > to write the plugin :)
> >
> > I'd suggest to use and mature the plugin a bit then see if they are
> > open to revisit their "This is by design not what Maven does"
> > statement.
> >
> > [1] http://www.mail-archive.com/users@maven.apache.org/msg76335.html
> > [2] http://www.mail-archive.com/users@maven.apache.org/msg76622.html
> > [3] http://www.mail-archive.com/users@maven.apache.org/msg77059.html
> > --
> > Jean-Sebastien
>
> +1 to trying to contribute this to Maven.
>
> This is exactly the type of thing we are looking for to help with
> graduation to show we're good Apache citizens and play well with other
> projects. I don't think one post should put you off, we don't all
> agree on everything here either, and now there's a direct invitation
> from Dan. I think it would actually be more beneficial to try to
> develop it over there instead of maturing it here. Those are the guys
> who know Maven so there will be more expert help and guidance there
> than here and it will provide a great opportunity to show people
> outside Tuscany how good the Tuscany people are at practising open
> development.
>
> ...ant
--
Daniel Kulp
dan@kulp.com
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by ant elder <an...@gmail.com>.
On Dec 14, 2007 1:05 AM, Jean-Sebastien Delfino <js...@apache.org>
wrote:
> Daniel Kulp wrote:
> > That's pretty cool. Any thoughts about submitting that to the maven
> > team?
> >
> > Dan
> >
>
> I had started with posts [1],[2] to the Maven list. The only response I
> got [3] gave me a good laugh, the next logical step was to write the
> plugin :)
>
> I'd suggest to use and mature the plugin a bit then see if they are open
> to revisit their "This is by design not what Maven does" statement.
>
> [1] http://www.mail-archive.com/users@maven.apache.org/msg76335.html
> [2] http://www.mail-archive.com/users@maven.apache.org/msg76622.html
> [3] http://www.mail-archive.com/users@maven.apache.org/msg77059.html
> --
> Jean-Sebastien
>
>
+1 to trying to contribute this to Maven.
This is exactly the type of thing we are looking for to help with graduation
to show we're good Apache citizens and play well with other projects. I
don't think one post should put you off, we don't all agree on everything
here either, and now there's a direct invitation from Dan. I think it would
actually be more beneficial to try to develop it over there instead of
maturing it here. Those are the guys who know Maven so there will be more
expert help and guidance there than here and it will provide a great
opportunity to show people outside Tuscany how good the Tuscany people are
at practising open development.
...ant
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Raymond Feng <en...@gmail.com>.
Just try to document the status of our incremental build plugin, there is
one known issue to deal with pom projects (type="pom").
1) We need to find a way to detect if the pom.xml has been changed. There is
no archive produced by this type of project. We may have to check if the pom
in local maven repo is newer or not.
2) The maven invoker doesn't support non-recursive mode yet (mvn -N). As a
result, we cannot simply run the "mvn install" command on the pom project.
We can either request the maven invoker to support the non-recursive flag or
we can try to call the maven installer directly.
Thanks,
Raymond
----- Original Message -----
From: "Jean-Sebastien Delfino" <js...@apache.org>
To: <tu...@ws.apache.org>
Sent: Thursday, December 13, 2007 5:05 PM
Subject: Re: How to run incremental builds of the Tuscany SCA tree
> Daniel Kulp wrote:
>> That's pretty cool. Any thoughts about submitting that to the maven
>> team?
>>
>> Dan
>>
>
> I had started with posts [1],[2] to the Maven list. The only response I
> got [3] gave me a good laugh, the next logical step was to write the
> plugin :)
>
> I'd suggest to use and mature the plugin a bit then see if they are open
> to revisit their "This is by design not what Maven does" statement.
>
> [1] http://www.mail-archive.com/users@maven.apache.org/msg76335.html
> [2] http://www.mail-archive.com/users@maven.apache.org/msg76622.html
> [3] http://www.mail-archive.com/users@maven.apache.org/msg77059.html
> --
> Jean-Sebastien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Jean-Sebastien Delfino <js...@apache.org>.
Daniel Kulp wrote:
> That's pretty cool. Any thoughts about submitting that to the maven
> team?
>
> Dan
>
I had started with posts [1],[2] to the Maven list. The only response I
got [3] gave me a good laugh, the next logical step was to write the
plugin :)
I'd suggest to use and mature the plugin a bit then see if they are open
to revisit their "This is by design not what Maven does" statement.
[1] http://www.mail-archive.com/users@maven.apache.org/msg76335.html
[2] http://www.mail-archive.com/users@maven.apache.org/msg76622.html
[3] http://www.mail-archive.com/users@maven.apache.org/msg77059.html
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Raymond Feng <en...@gmail.com>.
Hi,
Thank you for the suggestion. We're thinking about that. To play safely, I
suggest that we wait a bit so that more people can try it out and further
improve the plugin to a reasonable good quality.
Thanks,
Raymond
----- Original Message -----
From: "Daniel Kulp" <da...@iona.com>
To: <tu...@ws.apache.org>
Sent: Thursday, December 13, 2007 8:55 AM
Subject: Re: How to run incremental builds of the Tuscany SCA tree
>
> That's pretty cool. Any thoughts about submitting that to the maven
> team?
>
> Dan
>
>
> On Thursday 13 December 2007, Raymond Feng wrote:
>> Yes, that's right.
>>
>> Thanks,
>> Raymond
>>
>> ----- Original Message -----
>> From: "Luciano Resende" <lu...@gmail.com>
>> To: <tu...@ws.apache.org>
>> Sent: Thursday, December 13, 2007 4:04 AM
>> Subject: Re: How to run incremental builds of the Tuscany SCA tree
>>
>> > On Dec 12, 2007 4:44 PM, Raymond Feng <en...@gmail.com> wrote:
>> >> Hi,
>> >>
>> >> Beyond performance, I don't think svn stat is good way to handle
>> >> the case that the project has been updated from svn but the build
>> >> is not run yet. I
>> >> now ported the timestamp check from the maven-compiler-plugin and
>> >> it should
>> >> give us the capability to be incremental.
>> >
>> > Does this mean i can have an incremental build just by executing the
>> > command below, and don't need the markers anymore ?
>> >
>> > mvn -o org.apache.tuscany.sca:tuscany-maven-incremental-build:build
>> >
>> >> http://svn.apache.org/viewvc?rev=603783&view=rev
>> >>
>> >> Thanks,
>> >> Raymond
>> >>
>> >> ----- Original Message -----
>> >> From: "Jean-Sebastien Delfino" <js...@apache.org>
>> >> To: <tu...@ws.apache.org>
>> >>
>> >> Sent: Wednesday, December 12, 2007 3:49 PM
>> >> Subject: Re: How to run incremental builds of the Tuscany SCA tree
>> >>
>> >> > Jean-Sebastien Delfino wrote:
>> >> >> Raymond Feng wrote:
>> >> >>> Hi,
>> >> >>>
>> >> >>> I tried the plugins and it works well.
>> >> >>>
>> >> >>> I also made the following improvements:
>> >> >>>
>> >> >>> 1) Integrate the plugin with maven scm capability, i.e. use
>> >> >>> APIs to call
>> >> >>> "svn status" to detect if there are any local changes against
>> >> >>> SVN repo.
>> >> >>> (Pretty fast for a single module).
>> >> >>
>> >> >> Any idea of how long it takes if you run it from the top of the
>> >> >> tree?
>> >> >
>> >> > I tried it myself. Svn status on each module takes too long,
>> >> > defeating the
>> >> > purpose of the plugin.
>> >> >
>> >> > If you don't want to create .modified marker files, you can
>> >> > compare the timestamps of the source files + pom.xml and the
>> >> > timestamp of a marker file that the plugin can create under the
>> >> > target directory to record when
>> >> > it was run.
>> >> >
>> >> > You could also check the timestamps of .class files and the .war
>> >> > or .jar
>> >> > files but that assumes more knowledge of the structure of the
>> >> > module. --
>> >> > Jean-Sebastien
>> >> >
>> >> > -----------------------------------------------------------------
>> >> >---- To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> >> > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>> >>
>> >> -------------------------------------------------------------------
>> >>-- To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org For
>> >> additional commands, e-mail: tuscany-dev-help@ws.apache.org
>> >
>> > --
>> > Luciano Resende
>> > Apache Tuscany Committer
>> > http://people.apache.org/~lresende
>> > http://lresende.blogspot.com/
>> >
>> > --------------------------------------------------------------------
>> >- To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org For
>> > additional commands, e-mail: tuscany-dev-help@ws.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
>
>
> --
> J. Daniel Kulp
> Principal Engineer
> IONA
> P: 781-902-8727 C: 508-380-7194
> daniel.kulp@iona.com
> http://www.dankulp.com/blog
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Daniel Kulp <da...@iona.com>.
That's pretty cool. Any thoughts about submitting that to the maven
team?
Dan
On Thursday 13 December 2007, Raymond Feng wrote:
> Yes, that's right.
>
> Thanks,
> Raymond
>
> ----- Original Message -----
> From: "Luciano Resende" <lu...@gmail.com>
> To: <tu...@ws.apache.org>
> Sent: Thursday, December 13, 2007 4:04 AM
> Subject: Re: How to run incremental builds of the Tuscany SCA tree
>
> > On Dec 12, 2007 4:44 PM, Raymond Feng <en...@gmail.com> wrote:
> >> Hi,
> >>
> >> Beyond performance, I don't think svn stat is good way to handle
> >> the case that the project has been updated from svn but the build
> >> is not run yet. I
> >> now ported the timestamp check from the maven-compiler-plugin and
> >> it should
> >> give us the capability to be incremental.
> >
> > Does this mean i can have an incremental build just by executing the
> > command below, and don't need the markers anymore ?
> >
> > mvn -o org.apache.tuscany.sca:tuscany-maven-incremental-build:build
> >
> >> http://svn.apache.org/viewvc?rev=603783&view=rev
> >>
> >> Thanks,
> >> Raymond
> >>
> >> ----- Original Message -----
> >> From: "Jean-Sebastien Delfino" <js...@apache.org>
> >> To: <tu...@ws.apache.org>
> >>
> >> Sent: Wednesday, December 12, 2007 3:49 PM
> >> Subject: Re: How to run incremental builds of the Tuscany SCA tree
> >>
> >> > Jean-Sebastien Delfino wrote:
> >> >> Raymond Feng wrote:
> >> >>> Hi,
> >> >>>
> >> >>> I tried the plugins and it works well.
> >> >>>
> >> >>> I also made the following improvements:
> >> >>>
> >> >>> 1) Integrate the plugin with maven scm capability, i.e. use
> >> >>> APIs to call
> >> >>> "svn status" to detect if there are any local changes against
> >> >>> SVN repo.
> >> >>> (Pretty fast for a single module).
> >> >>
> >> >> Any idea of how long it takes if you run it from the top of the
> >> >> tree?
> >> >
> >> > I tried it myself. Svn status on each module takes too long,
> >> > defeating the
> >> > purpose of the plugin.
> >> >
> >> > If you don't want to create .modified marker files, you can
> >> > compare the timestamps of the source files + pom.xml and the
> >> > timestamp of a marker file that the plugin can create under the
> >> > target directory to record when
> >> > it was run.
> >> >
> >> > You could also check the timestamps of .class files and the .war
> >> > or .jar
> >> > files but that assumes more knowledge of the structure of the
> >> > module. --
> >> > Jean-Sebastien
> >> >
> >> > -----------------------------------------------------------------
> >> >---- To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> >> > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> >>
> >> -------------------------------------------------------------------
> >>-- To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org For
> >> additional commands, e-mail: tuscany-dev-help@ws.apache.org
> >
> > --
> > Luciano Resende
> > Apache Tuscany Committer
> > http://people.apache.org/~lresende
> > http://lresende.blogspot.com/
> >
> > --------------------------------------------------------------------
> >- To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org For
> > additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
--
J. Daniel Kulp
Principal Engineer
IONA
P: 781-902-8727 C: 508-380-7194
daniel.kulp@iona.com
http://www.dankulp.com/blog
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Jean-Sebastien Delfino <js...@apache.org>.
Luciano Resende wrote:
> On Dec 12, 2007 4:44 PM, Raymond Feng <en...@gmail.com> wrote:
>> Hi,
>>
>> Beyond performance, I don't think svn stat is good way to handle the case
>> that the project has been updated from svn but the build is not run yet. I
>> now ported the timestamp check from the maven-compiler-plugin and it should
>> give us the capability to be incremental.
>>
>
> Does this mean i can have an incremental build just by executing the
> command below, and don't need the markers anymore ?
>
> mvn -o org.apache.tuscany.sca:tuscany-maven-incremental-build:build
>
Yes
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Raymond Feng <en...@gmail.com>.
Yes, that's right.
Thanks,
Raymond
----- Original Message -----
From: "Luciano Resende" <lu...@gmail.com>
To: <tu...@ws.apache.org>
Sent: Thursday, December 13, 2007 4:04 AM
Subject: Re: How to run incremental builds of the Tuscany SCA tree
> On Dec 12, 2007 4:44 PM, Raymond Feng <en...@gmail.com> wrote:
>> Hi,
>>
>> Beyond performance, I don't think svn stat is good way to handle the case
>> that the project has been updated from svn but the build is not run yet.
>> I
>> now ported the timestamp check from the maven-compiler-plugin and it
>> should
>> give us the capability to be incremental.
>>
>
> Does this mean i can have an incremental build just by executing the
> command below, and don't need the markers anymore ?
>
> mvn -o org.apache.tuscany.sca:tuscany-maven-incremental-build:build
>
>> http://svn.apache.org/viewvc?rev=603783&view=rev
>>
>> Thanks,
>> Raymond
>>
>> ----- Original Message -----
>> From: "Jean-Sebastien Delfino" <js...@apache.org>
>> To: <tu...@ws.apache.org>
>>
>> Sent: Wednesday, December 12, 2007 3:49 PM
>> Subject: Re: How to run incremental builds of the Tuscany SCA tree
>>
>>
>> > Jean-Sebastien Delfino wrote:
>> >> Raymond Feng wrote:
>> >>> Hi,
>> >>>
>> >>> I tried the plugins and it works well.
>> >>>
>> >>> I also made the following improvements:
>> >>>
>> >>> 1) Integrate the plugin with maven scm capability, i.e. use APIs to
>> >>> call
>> >>> "svn status" to detect if there are any local changes against SVN
>> >>> repo.
>> >>> (Pretty fast for a single module).
>> >>
>> >> Any idea of how long it takes if you run it from the top of the tree?
>> >>
>> >
>> > I tried it myself. Svn status on each module takes too long, defeating
>> > the
>> > purpose of the plugin.
>> >
>> > If you don't want to create .modified marker files, you can compare the
>> > timestamps of the source files + pom.xml and the timestamp of a marker
>> > file that the plugin can create under the target directory to record
>> > when
>> > it was run.
>> >
>> > You could also check the timestamps of .class files and the .war or
>> > .jar
>> > files but that assumes more knowledge of the structure of the module.
>> > --
>> > Jean-Sebastien
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>>
>>
>
>
>
> --
> Luciano Resende
> Apache Tuscany Committer
> http://people.apache.org/~lresende
> http://lresende.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Luciano Resende <lu...@gmail.com>.
On Dec 12, 2007 4:44 PM, Raymond Feng <en...@gmail.com> wrote:
> Hi,
>
> Beyond performance, I don't think svn stat is good way to handle the case
> that the project has been updated from svn but the build is not run yet. I
> now ported the timestamp check from the maven-compiler-plugin and it should
> give us the capability to be incremental.
>
Does this mean i can have an incremental build just by executing the
command below, and don't need the markers anymore ?
mvn -o org.apache.tuscany.sca:tuscany-maven-incremental-build:build
> http://svn.apache.org/viewvc?rev=603783&view=rev
>
> Thanks,
> Raymond
>
> ----- Original Message -----
> From: "Jean-Sebastien Delfino" <js...@apache.org>
> To: <tu...@ws.apache.org>
>
> Sent: Wednesday, December 12, 2007 3:49 PM
> Subject: Re: How to run incremental builds of the Tuscany SCA tree
>
>
> > Jean-Sebastien Delfino wrote:
> >> Raymond Feng wrote:
> >>> Hi,
> >>>
> >>> I tried the plugins and it works well.
> >>>
> >>> I also made the following improvements:
> >>>
> >>> 1) Integrate the plugin with maven scm capability, i.e. use APIs to call
> >>> "svn status" to detect if there are any local changes against SVN repo.
> >>> (Pretty fast for a single module).
> >>
> >> Any idea of how long it takes if you run it from the top of the tree?
> >>
> >
> > I tried it myself. Svn status on each module takes too long, defeating the
> > purpose of the plugin.
> >
> > If you don't want to create .modified marker files, you can compare the
> > timestamps of the source files + pom.xml and the timestamp of a marker
> > file that the plugin can create under the target directory to record when
> > it was run.
> >
> > You could also check the timestamps of .class files and the .war or .jar
> > files but that assumes more knowledge of the structure of the module.
> > --
> > Jean-Sebastien
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
>
--
Luciano Resende
Apache Tuscany Committer
http://people.apache.org/~lresende
http://lresende.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Raymond Feng <en...@gmail.com>.
Please see my comments below.
Thanks,
Raymond
----- Original Message -----
From: "Jean-Sebastien Delfino" <js...@apache.org>
To: <tu...@ws.apache.org>
Sent: Sunday, January 13, 2008 12:26 AM
Subject: Re: How to run incremental builds of the Tuscany SCA tree
> Raymond Feng wrote:
> [snip]
>> Beyond performance, I don't think svn stat is good way to handle the case
>> that the project has been updated from svn but the build is not run yet.
>> I now ported the timestamp check from the maven-compiler-plugin and it
>> should give us the capability to be incremental.
>>
>> http://svn.apache.org/viewvc?rev=603783&view=rev
> [snip]
>
> With these changes, I'm sorry to say that the incremental build plugin has
> become unusable for me, with a few basic flaws:
>
> - The logic that you ported from the maven-compiler-plugin fails to detect
> changes as usually my IDE has already compiled the .java files to .class
> files.
>
> - If module A has changed, module B depends on A, and if A builds OK but
> B's tests fail, a subsequent incremental build will not try to run B's
> tests as A will not appear as changed anymore.
These are two goog catches. I guess the only reliable way is a marker to
know if (and when) a module has been sucessfully buit by maven. Maybe we can
use the .modified file to keep the timestamp for the last successful build
of a module.
>
> - Did you remove the simplistic check for .modified files that I had
> initially, which worked for my naive but very common use case?
>
Yes. My previous assumption was that it not necessay any more with the
timestamp checks.
> I don't understand all the sophisticated code that was added to the plugin
> to support the logic that now breaks me. Could you please help and restore
> a simple version of the plugin that just works?
>
I added the check back. Can you try? But there is a flaw too. If a module is
just updated with changes from the svn and there is no diff between the
workspace and the repo. In this case, the module should still be marked as
changed.
> I'm happy if people want to play around with super sophisticated plugin
> solutions and spend time to debug them, but it would be great if we could
> keep this one simple, as I'm trying to get some work done with it.
>
> Thanks.
> --
> Jean-Sebastien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Jean-Sebastien Delfino <js...@apache.org>.
Raymond Feng wrote:
[snip]
> Beyond performance, I don't think svn stat is good way to handle the
> case that the project has been updated from svn but the build is not run
> yet. I now ported the timestamp check from the maven-compiler-plugin and
> it should give us the capability to be incremental.
>
> http://svn.apache.org/viewvc?rev=603783&view=rev
[snip]
With these changes, I'm sorry to say that the incremental build plugin
has become unusable for me, with a few basic flaws:
- The logic that you ported from the maven-compiler-plugin fails to
detect changes as usually my IDE has already compiled the .java files to
.class files.
- If module A has changed, module B depends on A, and if A builds OK but
B's tests fail, a subsequent incremental build will not try to run B's
tests as A will not appear as changed anymore.
- Did you remove the simplistic check for .modified files that I had
initially, which worked for my naive but very common use case?
I don't understand all the sophisticated code that was added to the
plugin to support the logic that now breaks me. Could you please help
and restore a simple version of the plugin that just works?
I'm happy if people want to play around with super sophisticated plugin
solutions and spend time to debug them, but it would be great if we
could keep this one simple, as I'm trying to get some work done with it.
Thanks.
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Raymond Feng <en...@gmail.com>.
Hi,
Beyond performance, I don't think svn stat is good way to handle the case
that the project has been updated from svn but the build is not run yet. I
now ported the timestamp check from the maven-compiler-plugin and it should
give us the capability to be incremental.
http://svn.apache.org/viewvc?rev=603783&view=rev
Thanks,
Raymond
----- Original Message -----
From: "Jean-Sebastien Delfino" <js...@apache.org>
To: <tu...@ws.apache.org>
Sent: Wednesday, December 12, 2007 3:49 PM
Subject: Re: How to run incremental builds of the Tuscany SCA tree
> Jean-Sebastien Delfino wrote:
>> Raymond Feng wrote:
>>> Hi,
>>>
>>> I tried the plugins and it works well.
>>>
>>> I also made the following improvements:
>>>
>>> 1) Integrate the plugin with maven scm capability, i.e. use APIs to call
>>> "svn status" to detect if there are any local changes against SVN repo.
>>> (Pretty fast for a single module).
>>
>> Any idea of how long it takes if you run it from the top of the tree?
>>
>
> I tried it myself. Svn status on each module takes too long, defeating the
> purpose of the plugin.
>
> If you don't want to create .modified marker files, you can compare the
> timestamps of the source files + pom.xml and the timestamp of a marker
> file that the plugin can create under the target directory to record when
> it was run.
>
> You could also check the timestamps of .class files and the .war or .jar
> files but that assumes more knowledge of the structure of the module.
> --
> Jean-Sebastien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Jean-Sebastien Delfino <js...@apache.org>.
Jean-Sebastien Delfino wrote:
> Raymond Feng wrote:
>> Hi,
>>
>> I tried the plugins and it works well.
>>
>> I also made the following improvements:
>>
>> 1) Integrate the plugin with maven scm capability, i.e. use APIs to
>> call "svn status" to detect if there are any local changes against SVN
>> repo. (Pretty fast for a single module).
>
> Any idea of how long it takes if you run it from the top of the tree?
>
I tried it myself. Svn status on each module takes too long, defeating
the purpose of the plugin.
If you don't want to create .modified marker files, you can compare the
timestamps of the source files + pom.xml and the timestamp of a marker
file that the plugin can create under the target directory to record
when it was run.
You could also check the timestamps of .class files and the .war or .jar
files but that assumes more knowledge of the structure of the module.
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Jean-Sebastien Delfino <js...@apache.org>.
Raymond Feng wrote:
> Hi,
>
> I tried the plugins and it works well.
>
> I also made the following improvements:
>
> 1) Integrate the plugin with maven scm capability, i.e. use APIs to call
> "svn status" to detect if there are any local changes against SVN repo.
> (Pretty fast for a single module).
Any idea of how long it takes if you run it from the top of the tree?
>
> 2) Add a "scm" parameter to configure if "svn status" will be used or
> not. For example,
> mvn org.apache.tuscany.sca:tuscany-maven-incremental-build:build
> -Dscm=false
>
> 3) Add an exception if the build is run against a pom module as we
> haven't implemented it yet.
>
> If there are no objections to my changes, I'll commit them.
>
+1 hoping that svn status on all modules doesn't take too long
> Thanks,
> Raymond
>
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: How to run incremental builds of the Tuscany SCA tree
Posted by Raymond Feng <en...@gmail.com>.
Hi,
I tried the plugins and it works well.
I also made the following improvements:
1) Integrate the plugin with maven scm capability, i.e. use APIs to call
"svn status" to detect if there are any local changes against SVN repo.
(Pretty fast for a single module).
2) Add a "scm" parameter to configure if "svn status" will be used or not.
For example,
mvn org.apache.tuscany.sca:tuscany-maven-incremental-build:build -Dscm=false
3) Add an exception if the build is run against a pom module as we haven't
implemented it yet.
If there are no objections to my changes, I'll commit them.
Thanks,
Raymond
----- Original Message -----
From: "Jean-Sebastien Delfino" <js...@apache.org>
To: <tu...@ws.apache.org>
Sent: Tuesday, December 04, 2007 5:26 PM
Subject: How to run incremental builds of the Tuscany SCA tree
> I'm sharing this as I think it could be useful to others.
>
> I like to build the java/sca tree before committing code changes to check
> that my changes are not going to break any other module.
>
> This gives me a peace of mind as I don't have to worry about build breaks
> and is also nice to other contributors as I know that I'm not breaking
> their modules.
>
> Unfortunately Maven is not particularly smart about detecting changes and
> rebuilding only impacted modules and the brute force "mvn clean install"
> on the whole tree takes long and makes my Thinkpad's CPU overheat.
>
> So I thought it would be nice to teach Maven how to do more incremental
> builds:
>
> - recompile and retest with mvn clean install the modules that have
> compile dependencies on the modules I've changed
>
> - only retest the modules that have runtime and test dependencies on the
> modules I've changed.
>
> I developed a little Maven plugin to do it. It's really simple and
> leverages the Maven dependency model, see for yourself at [1].
>
> Here's how I use the plugin to run incremental builds:
>
> 1. To mark that a module has changed, create a .modified marker file at
> its root. This is what the plugin recognizes as an indication that the
> module has changed. I was thinking about running "svn st" automatically
> from the plugin but for now I've found simpler to just use a shell script
> like follows to create/remove these .modified marker files:
>
> st=`svn st | grep -v ".modified"`
> if [ "x$st" != "x" ]
> then
> echo "Modified"
> echo "$st"
> touch .modified
> else
> echo "Not modified"
> rm -f .modified
> fi
>
> This is for Linux, I'm sure that it can be translated to a Windows script.
>
> 2. To run the incremental build, from java/sca run:
> mvn -o org.apache.tuscany.sca:tuscany-maven-incremental-build:build
>
> This will recompile and/or retest only the modules affected by the changed
> modules. Faster and more reliable, and greener as my Thinkpad now uses
> less energy :)
>
> Hope this helps. I'm interested in your feedback if you use it and like
> it, or not :) or want to help improve it.
>
> [1]
> https://svn.apache.org/repos/asf/incubator/tuscany/java/sca/tools/maven/maven-incremental-build/src/main/java/org/apache/tuscany/sca/tools/incremental/build/plugin/IncrementalBuildMojo.java
> --
> Jean-Sebastien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org