You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@buildr.apache.org by Peter Donald <pe...@realityforge.org> on 2010/03/20 00:58:12 UTC
Subprojects
Hi,
So I have a projects structure that goes something like
define 'A' do
...
define 'B' do
...
end
define 'C' do
...
end
end
When I run any task in the base project it runs "package" for all the
subprojects. This seems somewhat unintuitive when I am doing "buildr
clean" or some other task that is just meant to inspect the state of
project or even get help (i.e. "buildr help" does the same). It is
particularly annoying when you have 18 subprojects ;P I want to keep
the subproject structure as all the subprojects share settings with
the parent project.
So what am I doing wrong? How do I disable this recursing into child projects?
--
Cheers,
Peter Donald
Re: Subprojects
Posted by Alex Boisvert <al...@gmail.com>.
On Sat, Mar 20, 2010 at 8:54 AM, Alex Boisvert <al...@gmail.com>wrote:
> On Sat, Mar 20, 2010 at 2:12 AM, Peter Donald <pe...@realityforge.org>wrote:
>
>> On Sat, Mar 20, 2010 at 5:51 PM, Antoine Toulme <an...@lunar-ocean.com>
>> wrote:
>> > I would say you call compile.invoke somewhere... anyhow we can't help
>> you
>> > without your Buildfile.
>>
>> Thanks. Found it. Almost all of my projects have a "dist" target that
>> basically roles all the generated artifacts into one big zip and I
>> have been doing it something like the following. My question is how do
>> I change it so that the includes are deferred until the package is
>> called on this project rather than as the project file is parsed?
>>
>> package(:zip).tap do |zip|
>> prefix = "#{id}-#{version}"
>> zip.include( Buildr.artifacts([PAX_RUNNER]).each(&:invoke),
>> :path => "#{prefix}/bin")
>> zip.include( Buildr.artifacts(EQUINOX).each(&:invoke), :path =>
>> "#{prefix}/equinox")
>>
>> to_deploy = [OSGI_CORE, OSGI_COMPENDIUM, PAX_LOGGING,
>> PAX_LOGGING_SERVICE, CONFIG_ADMIN_SERVICE, PAX_CONFMAN] +
>> [BND_ANNOTATIONS, JMS] + projects('link', 'connection',
>> 'com.sun.messaging.mq.imq', 'routes')
>>
>> zip.include( Buildr.artifacts(to_deploy).each(&:invoke), :path
>> => "#{prefix}/lib")
>> zip.include( _('src/main/etc/*'), :path => "#{prefix}")
>> end
>>
>
> You need to wire dependencies between tasks. Instead of calling
> task.invoke(), use task.enhance() on the task which depends this task.
>
>
> package(:zip).tap do |zip|
> prefix = "#{id}-#{version}"
>
> artifact(PAX_RUNNER).tap do |a|
> zip.enhance a
> zip.include a, :path => "#{prefix}/bin")
> end
>
> artifact(EQUINOX).tap do |a|
> zip.enhance a
> zip.include a, :path => "#{prefix}/equinox")
> end
>
> libs = [OSGI_CORE, OSGI_COMPENDIUM, PAX_LOGGING,
>
> PAX_LOGGING_SERVICE, CONFIG_ADMIN_SERVICE,
> PAX_CONFMAN, BND_ANNOTATIONS, JMS].map { |a| artifact(a) }
> libs += projects('link', 'connection',
>
> 'com.sun.messaging.mq.imq',
> 'routes').map(&:packages)
> zip.enhance libs
> zip.include libs, :path => "#{prefix}/lib"
>
>
> zip.include _('src/main/etc/*'), :path => prefix
> end
>
> It wouldn't be a bad idea to have zip.include accept tasks and
> automatically wire things up. I'll look into this later.
>
Courtesy of Antoine Toulme and I, the above no longer requires calling
enhance(), artifact() or packages() on trunk and becomes the more succinct,
package(:zip).tap do |zip|
prefix = "#{id}-#{version}"
zip.include PAX_RUNNER, :path => "#{prefix}/bin")
zip.include EQUINOX, :path => "#{prefix}/equinox")
libs = [OSGI_CORE, OSGI_COMPENDIUM, PAX_LOGGING,
PAX_LOGGING_SERVICE, CONFIG_ADMIN_SERVICE,
PAX_CONFMAN, BND_ANNOTATIONS, JMS]
libs += projects('link', 'connection',
'com.sun.messaging.mq.imq',
'routes')
zip.include libs, :path => "#{prefix}/lib"
zip.include _('src/main/etc/*'), :path => prefix
end
alex
Re: Subprojects
Posted by Alex Boisvert <al...@gmail.com>.
On Sat, Mar 20, 2010 at 2:12 AM, Peter Donald <pe...@realityforge.org>wrote:
> On Sat, Mar 20, 2010 at 5:51 PM, Antoine Toulme <an...@lunar-ocean.com>
> wrote:
> > I would say you call compile.invoke somewhere... anyhow we can't help you
> > without your Buildfile.
>
> Thanks. Found it. Almost all of my projects have a "dist" target that
> basically roles all the generated artifacts into one big zip and I
> have been doing it something like the following. My question is how do
> I change it so that the includes are deferred until the package is
> called on this project rather than as the project file is parsed?
>
> package(:zip).tap do |zip|
> prefix = "#{id}-#{version}"
> zip.include( Buildr.artifacts([PAX_RUNNER]).each(&:invoke),
> :path => "#{prefix}/bin")
> zip.include( Buildr.artifacts(EQUINOX).each(&:invoke), :path =>
> "#{prefix}/equinox")
>
> to_deploy = [OSGI_CORE, OSGI_COMPENDIUM, PAX_LOGGING,
> PAX_LOGGING_SERVICE, CONFIG_ADMIN_SERVICE, PAX_CONFMAN] +
> [BND_ANNOTATIONS, JMS] + projects('link', 'connection',
> 'com.sun.messaging.mq.imq', 'routes')
>
> zip.include( Buildr.artifacts(to_deploy).each(&:invoke), :path
> => "#{prefix}/lib")
> zip.include( _('src/main/etc/*'), :path => "#{prefix}")
> end
>
You need to wire dependencies between tasks. Instead of calling
task.invoke(), use task.enhance() on the task which depends this task.
package(:zip).tap do |zip|
prefix = "#{id}-#{version}"
artifact(PAX_RUNNER).tap do |a|
zip.enhance a
zip.include a, :path => "#{prefix}/bin")
end
artifact(EQUINOX).tap do |a|
zip.enhance a
zip.include a, :path => "#{prefix}/equinox")
end
libs = [OSGI_CORE, OSGI_COMPENDIUM, PAX_LOGGING,
PAX_LOGGING_SERVICE, CONFIG_ADMIN_SERVICE,
PAX_CONFMAN, BND_ANNOTATIONS, JMS].map { |a| artifact(a) }
libs += projects('link', 'connection',
'com.sun.messaging.mq.imq',
'routes').map(&:packages)
zip.enhance libs
zip.include libs, :path => "#{prefix}/lib"
zip.include _('src/main/etc/*'), :path => prefix
end
It wouldn't be a bad idea to have zip.include accept tasks and automatically
wire things up. I'll look into this later.
Hope this helps... cheers,
alex
Re: Subprojects
Posted by Peter Donald <pe...@realityforge.org>.
On Sat, Mar 20, 2010 at 5:51 PM, Antoine Toulme <an...@lunar-ocean.com> wrote:
> I would say you call compile.invoke somewhere... anyhow we can't help you
> without your Buildfile.
Thanks. Found it. Almost all of my projects have a "dist" target that
basically roles all the generated artifacts into one big zip and I
have been doing it something like the following. My question is how do
I change it so that the includes are deferred until the package is
called on this project rather than as the project file is parsed?
package(:zip).tap do |zip|
prefix = "#{id}-#{version}"
zip.include( Buildr.artifacts([PAX_RUNNER]).each(&:invoke),
:path => "#{prefix}/bin")
zip.include( Buildr.artifacts(EQUINOX).each(&:invoke), :path =>
"#{prefix}/equinox")
to_deploy = [OSGI_CORE, OSGI_COMPENDIUM, PAX_LOGGING,
PAX_LOGGING_SERVICE, CONFIG_ADMIN_SERVICE, PAX_CONFMAN] +
[BND_ANNOTATIONS, JMS] + projects('link', 'connection',
'com.sun.messaging.mq.imq', 'routes')
zip.include( Buildr.artifacts(to_deploy).each(&:invoke), :path
=> "#{prefix}/lib")
zip.include( _('src/main/etc/*'), :path => "#{prefix}")
end
--
Cheers,
Peter Donald
Re: Subprojects
Posted by Antoine Toulme <an...@lunar-ocean.com>.
I would say you call compile.invoke somewhere... anyhow we can't help you
without your Buildfile.
Thanks,
Antoine
On Fri, Mar 19, 2010 at 22:49, Peter Donald <pe...@realityforge.org> wrote:
> Hi,
>
> > Then there's something very wrong happening because that's not how Buildr
> is
> > supposed to work. Either it's a bug, either it's an issue in your
> > buildfile. Obviously I can't reproduce this (on many, many builds I use
> > every day) so you'll have to show us how you get into that situation.
>
> Ok - It is almost certainly something I have got wrong with my build
> file but somehow I have managed to get it happening in multiple
> projects so it could be something about the way I write my build
> files. To reproduce it
>
> $ jgem install buildr
> $ buildr --version
> Buildr 1.3.5 (JRuby 1.4.0RC1)
> $ git clone git://github.com/rockninja/jamex.git
> $ cd jamex
> $ buildr help
> (in /home/peter/jamex, development)
> Compiling jamex:link into /home/peter/jamex/target/link/classes
> Running java aQute.bnd.main.bnd
> jamex-link-0.1.1-SNAPSHOT.jar 11 13012
> Running java aQute.bnd.main.bnd
> Packaging jamex-link-0.1.1-SNAPSHOT.jar
> Running java aQute.bnd.main.bnd
> jamex-com.sun.messaging.mq.imq-0.1.1-SNAPSHOT.jar 278 507327
> Running java aQute.bnd.main.bnd
> Packaging jamex-com.sun.messaging.mq.imq-0.1.1-SNAPSHOT.jar
> Compiling jamex:connection into /home/peter/jamex/target/connection/classes
> Running java aQute.bnd.main.bnd
> jamex-connection-0.1.1-SNAPSHOT.jar 5 8289
> Running java aQute.bnd.main.bnd
> Packaging jamex-connection-0.1.1-SNAPSHOT.jar
> Compiling jamex:routes into /home/peter/jamex/target/routes/classes
> Running java aQute.bnd.main.bnd
> jamex-routes-0.1.1-SNAPSHOT.jar 3 4419
> Running java aQute.bnd.main.bnd
> Packaging jamex-routes-0.1.1-SNAPSHOT.jar
> Usage:
> buildr [-f rakefile] {options} targets...
>
> Top-level projects (buildr help:projects for full list):
> jamex # An OSGi based JMS router in its infancy
>
> Common tasks:
> artifacts # Download all artifacts
> artifacts:sources # Download all artifacts' sources
> bnd:print # Does `bnd print` on the packaged bundle and
> stdouts the output for inspection
> build # Build the project
> clean # Clean files generated during a build
> compile # Compile all projects
> default # The default task is build
> eclipse # Generate Eclipse artifacts for all projects
> help:projects # List all projects defined by this buildfile
> help:tasks # List all tasks available from this buildfile
> idea # Generate Idea artifacts for all projects
> idea7x # Generate Idea 7.x artifacts for all projects
> install # Install packages created by the project
> javadoc # Create the Javadocs for this project
> junit:report # Generate JUnit tests report in reports/junit
> package # Create packages
> release # Make a release
> test # Run all tests
> uninstall # Remove previously installed packages
> upload # Upload packages created by the project
>
>
> For help on command line options:
> buildr --help
> To run a full build without running any tests:
> buildr test=no
> To run specific test:
> buildr test:MyTest
> To run integration tests:
> buildr integration
>
>
> Using Java 1.6.0_15, Ant 1.7.1.
>
> --
> Cheers,
>
> Peter Donald
>
Re: Subprojects
Posted by Peter Donald <pe...@realityforge.org>.
Hi,
> Then there's something very wrong happening because that's not how Buildr is
> supposed to work. Either it's a bug, either it's an issue in your
> buildfile. Obviously I can't reproduce this (on many, many builds I use
> every day) so you'll have to show us how you get into that situation.
Ok - It is almost certainly something I have got wrong with my build
file but somehow I have managed to get it happening in multiple
projects so it could be something about the way I write my build
files. To reproduce it
$ jgem install buildr
$ buildr --version
Buildr 1.3.5 (JRuby 1.4.0RC1)
$ git clone git://github.com/rockninja/jamex.git
$ cd jamex
$ buildr help
(in /home/peter/jamex, development)
Compiling jamex:link into /home/peter/jamex/target/link/classes
Running java aQute.bnd.main.bnd
jamex-link-0.1.1-SNAPSHOT.jar 11 13012
Running java aQute.bnd.main.bnd
Packaging jamex-link-0.1.1-SNAPSHOT.jar
Running java aQute.bnd.main.bnd
jamex-com.sun.messaging.mq.imq-0.1.1-SNAPSHOT.jar 278 507327
Running java aQute.bnd.main.bnd
Packaging jamex-com.sun.messaging.mq.imq-0.1.1-SNAPSHOT.jar
Compiling jamex:connection into /home/peter/jamex/target/connection/classes
Running java aQute.bnd.main.bnd
jamex-connection-0.1.1-SNAPSHOT.jar 5 8289
Running java aQute.bnd.main.bnd
Packaging jamex-connection-0.1.1-SNAPSHOT.jar
Compiling jamex:routes into /home/peter/jamex/target/routes/classes
Running java aQute.bnd.main.bnd
jamex-routes-0.1.1-SNAPSHOT.jar 3 4419
Running java aQute.bnd.main.bnd
Packaging jamex-routes-0.1.1-SNAPSHOT.jar
Usage:
buildr [-f rakefile] {options} targets...
Top-level projects (buildr help:projects for full list):
jamex # An OSGi based JMS router in its infancy
Common tasks:
artifacts # Download all artifacts
artifacts:sources # Download all artifacts' sources
bnd:print # Does `bnd print` on the packaged bundle and
stdouts the output for inspection
build # Build the project
clean # Clean files generated during a build
compile # Compile all projects
default # The default task is build
eclipse # Generate Eclipse artifacts for all projects
help:projects # List all projects defined by this buildfile
help:tasks # List all tasks available from this buildfile
idea # Generate Idea artifacts for all projects
idea7x # Generate Idea 7.x artifacts for all projects
install # Install packages created by the project
javadoc # Create the Javadocs for this project
junit:report # Generate JUnit tests report in reports/junit
package # Create packages
release # Make a release
test # Run all tests
uninstall # Remove previously installed packages
upload # Upload packages created by the project
For help on command line options:
buildr --help
To run a full build without running any tests:
buildr test=no
To run specific test:
buildr test:MyTest
To run integration tests:
buildr integration
Using Java 1.6.0_15, Ant 1.7.1.
--
Cheers,
Peter Donald
Re: Subprojects
Posted by Alex Boisvert <al...@gmail.com>.
On Fri, Mar 19, 2010 at 8:36 PM, Peter Donald <pe...@realityforge.org>wrote:
> When I go to the base directory of the project and do a "buildr
> help:tasks", "buildr help:projects", "buildr help" etc - I do not
> expect it to go and package the subprojects before giving me help.
> When I use "buildr clean" I expect it to delete a directory rather
> than package the subprojects then delete the directory.
>
And I have the same expectation.
> If "buildr clean" in the base directory went and invoked clean on all
> the subprojects that would make sense but currently it is packaging
> them all.
>
Then there's something very wrong happening because that's not how Buildr is
supposed to work. Either it's a bug, either it's an issue in your
buildfile. Obviously I can't reproduce this (on many, many builds I use
every day) so you'll have to show us how you get into that situation.
alex
Re: Subprojects
Posted by Alex Boisvert <al...@gmail.com>.
On Fri, Mar 19, 2010 at 8:36 PM, Peter Donald <pe...@realityforge.org>wrote:
> > Parent projects have an implicit dependency on child projects for all
> > recursive tasks (e.g. clean, build, package, ...).
> >
> > For other tasks, there should not be any such parent-child dependencies
> > unless you wire them yourself.
> >
> > If you provide a more detailed example of what you're doing / trying to
> do,
> > we can probably help you further.
>
> I guess the way that Buildr work seems unintuitive to me.
>
I must not have been clear enough.
Let's take the recursive :clean task. You have project Parent and project
Child. The clean task of Parent (Parent:clean) depends on the child's
(Child:clean). It does not depend on any other task. Not on
Parent:package, not on Child:package.
The dependencies between tasks within a project are illustrated in
http://cwiki.apache.org/confluence/display/BUILDR/12+Things+to+Know+About+Buildr
.
alex
Re: Subprojects
Posted by Peter Donald <pe...@realityforge.org>.
> Parent projects have an implicit dependency on child projects for all
> recursive tasks (e.g. clean, build, package, ...).
>
> For other tasks, there should not be any such parent-child dependencies
> unless you wire them yourself.
>
> If you provide a more detailed example of what you're doing / trying to do,
> we can probably help you further.
I guess the way that Buildr work seems unintuitive to me.
When I go to the base directory of the project and do a "buildr
help:tasks", "buildr help:projects", "buildr help" etc - I do not
expect it to go and package the subprojects before giving me help.
When I use "buildr clean" I expect it to delete a directory rather
than package the subprojects then delete the directory.
If "buildr clean" in the base directory went and invoked clean on all
the subprojects that would make sense but currently it is packaging
them all.
I guess my concern is the scalability of this approach. When a project
starts to have 40+ subprojects it could take a very long time to
perform even simple commands. It can be worked around by providing
another set of tasks to do things like clean and provide help but that
seems a bit hacky.
--
Cheers,
Peter Donald
Re: Subprojects
Posted by Alex Boisvert <al...@gmail.com>.
Parent projects have an implicit dependency on child projects for all
recursive tasks (e.g. clean, build, package, ...).
For other tasks, there should not be any such parent-child dependencies
unless you wire them yourself.
If you provide a more detailed example of what you're doing / trying to do,
we can probably help you further.
alex
On Fri, Mar 19, 2010 at 4:58 PM, Peter Donald <pe...@realityforge.org>wrote:
> Hi,
>
> So I have a projects structure that goes something like
>
> define 'A' do
> ...
> define 'B' do
> ...
> end
>
> define 'C' do
> ...
> end
> end
>
> When I run any task in the base project it runs "package" for all the
> subprojects. This seems somewhat unintuitive when I am doing "buildr
> clean" or some other task that is just meant to inspect the state of
> project or even get help (i.e. "buildr help" does the same). It is
> particularly annoying when you have 18 subprojects ;P I want to keep
> the subproject structure as all the subprojects share settings with
> the parent project.
>
> So what am I doing wrong? How do I disable this recursing into child
> projects?
>
> --
> Cheers,
>
> Peter Donald
>
Re: Subprojects
Posted by Antoine Toulme <an...@lunar-ocean.com>.
Your base project should just be a container for subprojects if you don't
want to tie their lifecycle.
You should move whatever behavior it has into a subproject, then call:
buildr container:A:compile
On Fri, Mar 19, 2010 at 16:58, Peter Donald <pe...@realityforge.org> wrote:
> Hi,
>
> So I have a projects structure that goes something like
>
> define 'A' do
> ...
> define 'B' do
> ...
> end
>
> define 'C' do
> ...
> end
> end
>
> When I run any task in the base project it runs "package" for all the
> subprojects. This seems somewhat unintuitive when I am doing "buildr
> clean" or some other task that is just meant to inspect the state of
> project or even get help (i.e. "buildr help" does the same). It is
> particularly annoying when you have 18 subprojects ;P I want to keep
> the subproject structure as all the subprojects share settings with
> the parent project.
>
> So what am I doing wrong? How do I disable this recursing into child
> projects?
>
> --
> Cheers,
>
> Peter Donald
>