Correct Way to Depend on Another Buildfile

[Daniel Spiewak <dj...@gmail.com>]

Consider: http://github.com/djspiewak/gll-combinators

Specifically, the examples/ directory defines a separate Buildr project
which depends upon the project in its parent directory (gll-combinators).  I
could have done this by using two sub-projects, but I wanted a very specific
directory organization.  In other words, I wanted the main sources to be
under gll-combinators/src/main/scala/ while the examples are under
examples/src/main/scala/.

Currently, I'm handling this by actually shelling out to Buildr from within
the examples/ buildfile, building the parent project and then explicitly
adding its target/classes/ directory to the examples/ dependencies.  Is
there a better way of doing this?  Better question: *shouldn't* there be a
better way of doing this?  :-)

Optimally, I would like to just `require` the parent's buildfile and then
using project('gll-combinators') to declare the dependency within the
examples/ buildfile.  However, Buildr doesn't seems to have some global
state relating to project definitions, meaning that this sort of thing
doesn't work at present.

Daniel

Re: Correct Way to Depend on Another Buildfile

[Alex Boisvert <bo...@intalio.com>]

Hi Assaf,

I tried to split a buildfile like Daniel's myself but tripped over several
technical issues and didn't manage to get something working.  Would you mind
posting a skeleton example, maybe something like before-and-after
buildfiles?   I'd like to see this added to the How To section on the wiki.

alex


On Sat, May 9, 2009 at 8:08 PM, Daniel Spiewak <dj...@gmail.com> wrote:

> Oh, gll-combinators doesn't depend on examples, the dependency is not
> inherently circular.  I think the problem is that both define(s) are both
> enhancing the compile task (in the default namespace) so that they somehow
> depend upon eachother.  At least, that's what I suspect the problem is, I
> haven't had time to seriously dig into it.
>
> Daniel
>
> On Sat, May 9, 2009 at 9:26 PM, Assaf Arkin <ar...@intalio.com> wrote:
>
> > On Fri, May 8, 2009 at 5:57 PM, Daniel Spiewak <dj...@gmail.com>
> > wrote:
> >
> > > Unfortunately, this technique leads to circular dependencies.  This
> might
> > > be
> > > something that we want to work on in future, since I would imagine that
> > > this
> > > sort of thing is fairly common.
> >
> >
> > If gll-combinators depends on examples and examples depends
> > on gll-combinators, then yes, you'll get circular dependencies. But if
> you
> > have a known-to-work buildfile then it should be possible (with some
> > effort)
> > to split it up, that in itself should not introduce circular
> dependencies.
> >
> > Assaf
> >
> >
> > >
> > >
> > > Daniel
> > >
> > > On Fri, May 8, 2009 at 11:22 AM, Assaf Arkin <ar...@intalio.com>
> wrote:
> > >
> > > > On Thu, May 7, 2009 at 8:18 AM, Daniel Spiewak <dj...@gmail.com>
> > > > wrote:
> > > >
> > > > > Consider: http://github.com/djspiewak/gll-combinators
> > > > >
> > > > > Specifically, the examples/ directory defines a separate Buildr
> > project
> > > > > which depends upon the project in its parent directory
> > > (gll-combinators).
> > > > >  I
> > > > > could have done this by using two sub-projects, but I wanted a very
> > > > > specific
> > > > > directory organization.  In other words, I wanted the main sources
> to
> > > be
> > > > > under gll-combinators/src/main/scala/ while the examples are under
> > > > > examples/src/main/scala/.
> > > > >
> > > > > Currently, I'm handling this by actually shelling out to Buildr
> from
> > > > within
> > > > > the examples/ buildfile, building the parent project and then
> > > explicitly
> > > > > adding its target/classes/ directory to the examples/ dependencies.
> >  Is
> > > > > there a better way of doing this?  Better question: *shouldn't*
> there
> > > be
> > > > a
> > > > > better way of doing this?  :-)
> > > >
> > > >
> > > > Yes.
> > > >
> > > > Project definitions should be portable, so if you did everything by
> the
> > > > book, you should be able to import gll-combinators from within
> examples
> > > (or
> > > > the other way around). Some gotaches:
> > > >
> > > > - You build now runs from a different directory, anything that
> assumes
> > > > relative paths is broken, but anything that uses _(..) will adjust.
> For
> > > > that, imported projects need to set their base_dir.
> > > > - Project references are relative, so if you have some 'master'
> project
> > > and
> > > > you import foo and bar, they can reference each other foo:module1,
> > > > bar:compA, etc.
> > > > - Globals may conflicts (and constants will warn you when they're
> > > > redefined)
> > > > - Build settings (build.yml, profile.yml): you can only use one of
> > these.
> > > >
> > > > Generally easier (and from what I've heard more common) to start with
> > the
> > > > outer buildfile and import nested buildfiles from its
> sub-directories.
> > > > Would
> > > > be easier if everything we do only supports that, so let's use that
> as
> > a
> > > > mental model for what you're doing.
> > > >
> > > > From examples/buildfile:
> > > >
> > > > - cd one level up, since you're effectively running the build from
> > here.
> > > > - require gll-combinators, everything you do afterwards should extend
> > it
> > > > - define the examples projects, base_dir should just point one level
> > down
> > > > (since you're current dir is now one level up)
> > > >
> > > > (You won't get tasks/build.yml/etc from the outer directory since the
> > > > directory change happens to late, but if the model works, we could
> make
> > > > Buildr do the right thing with some magic switch)
> > > >
> > > > Assaf
> > > >
> > > >
> > > > >
> > > > > Optimally, I would like to just `require` the parent's buildfile
> and
> > > then
> > > > > using project('gll-combinators') to declare the dependency within
> the
> > > > > examples/ buildfile.  However, Buildr doesn't seems to have some
> > global
> > > > > state relating to project definitions, meaning that this sort of
> > thing
> > > > > doesn't work at present.
> > > >
> > > >
> > > >
> > > > >
> > > > >
> > > > > Daniel
> > > > >
> > > >
> > >
> >
>

Re: Correct Way to Depend on Another Buildfile

[Daniel Spiewak <dj...@gmail.com>]

Oh, gll-combinators doesn't depend on examples, the dependency is not
inherently circular.  I think the problem is that both define(s) are both
enhancing the compile task (in the default namespace) so that they somehow
depend upon eachother.  At least, that's what I suspect the problem is, I
haven't had time to seriously dig into it.

Daniel

On Sat, May 9, 2009 at 9:26 PM, Assaf Arkin <ar...@intalio.com> wrote:

> On Fri, May 8, 2009 at 5:57 PM, Daniel Spiewak <dj...@gmail.com>
> wrote:
>
> > Unfortunately, this technique leads to circular dependencies.  This might
> > be
> > something that we want to work on in future, since I would imagine that
> > this
> > sort of thing is fairly common.
>
>
> If gll-combinators depends on examples and examples depends
> on gll-combinators, then yes, you'll get circular dependencies. But if you
> have a known-to-work buildfile then it should be possible (with some
> effort)
> to split it up, that in itself should not introduce circular dependencies.
>
> Assaf
>
>
> >
> >
> > Daniel
> >
> > On Fri, May 8, 2009 at 11:22 AM, Assaf Arkin <ar...@intalio.com> wrote:
> >
> > > On Thu, May 7, 2009 at 8:18 AM, Daniel Spiewak <dj...@gmail.com>
> > > wrote:
> > >
> > > > Consider: http://github.com/djspiewak/gll-combinators
> > > >
> > > > Specifically, the examples/ directory defines a separate Buildr
> project
> > > > which depends upon the project in its parent directory
> > (gll-combinators).
> > > >  I
> > > > could have done this by using two sub-projects, but I wanted a very
> > > > specific
> > > > directory organization.  In other words, I wanted the main sources to
> > be
> > > > under gll-combinators/src/main/scala/ while the examples are under
> > > > examples/src/main/scala/.
> > > >
> > > > Currently, I'm handling this by actually shelling out to Buildr from
> > > within
> > > > the examples/ buildfile, building the parent project and then
> > explicitly
> > > > adding its target/classes/ directory to the examples/ dependencies.
>  Is
> > > > there a better way of doing this?  Better question: *shouldn't* there
> > be
> > > a
> > > > better way of doing this?  :-)
> > >
> > >
> > > Yes.
> > >
> > > Project definitions should be portable, so if you did everything by the
> > > book, you should be able to import gll-combinators from within examples
> > (or
> > > the other way around). Some gotaches:
> > >
> > > - You build now runs from a different directory, anything that assumes
> > > relative paths is broken, but anything that uses _(..) will adjust. For
> > > that, imported projects need to set their base_dir.
> > > - Project references are relative, so if you have some 'master' project
> > and
> > > you import foo and bar, they can reference each other foo:module1,
> > > bar:compA, etc.
> > > - Globals may conflicts (and constants will warn you when they're
> > > redefined)
> > > - Build settings (build.yml, profile.yml): you can only use one of
> these.
> > >
> > > Generally easier (and from what I've heard more common) to start with
> the
> > > outer buildfile and import nested buildfiles from its sub-directories.
> > > Would
> > > be easier if everything we do only supports that, so let's use that as
> a
> > > mental model for what you're doing.
> > >
> > > From examples/buildfile:
> > >
> > > - cd one level up, since you're effectively running the build from
> here.
> > > - require gll-combinators, everything you do afterwards should extend
> it
> > > - define the examples projects, base_dir should just point one level
> down
> > > (since you're current dir is now one level up)
> > >
> > > (You won't get tasks/build.yml/etc from the outer directory since the
> > > directory change happens to late, but if the model works, we could make
> > > Buildr do the right thing with some magic switch)
> > >
> > > Assaf
> > >
> > >
> > > >
> > > > Optimally, I would like to just `require` the parent's buildfile and
> > then
> > > > using project('gll-combinators') to declare the dependency within the
> > > > examples/ buildfile.  However, Buildr doesn't seems to have some
> global
> > > > state relating to project definitions, meaning that this sort of
> thing
> > > > doesn't work at present.
> > >
> > >
> > >
> > > >
> > > >
> > > > Daniel
> > > >
> > >
> >
>

Re: Correct Way to Depend on Another Buildfile

[Assaf Arkin <ar...@intalio.com>]

On Fri, May 8, 2009 at 5:57 PM, Daniel Spiewak <dj...@gmail.com> wrote:

> Unfortunately, this technique leads to circular dependencies.  This might
> be
> something that we want to work on in future, since I would imagine that
> this
> sort of thing is fairly common.


If gll-combinators depends on examples and examples depends
on gll-combinators, then yes, you'll get circular dependencies. But if you
have a known-to-work buildfile then it should be possible (with some effort)
to split it up, that in itself should not introduce circular dependencies.

Assaf


>
>
> Daniel
>
> On Fri, May 8, 2009 at 11:22 AM, Assaf Arkin <ar...@intalio.com> wrote:
>
> > On Thu, May 7, 2009 at 8:18 AM, Daniel Spiewak <dj...@gmail.com>
> > wrote:
> >
> > > Consider: http://github.com/djspiewak/gll-combinators
> > >
> > > Specifically, the examples/ directory defines a separate Buildr project
> > > which depends upon the project in its parent directory
> (gll-combinators).
> > >  I
> > > could have done this by using two sub-projects, but I wanted a very
> > > specific
> > > directory organization.  In other words, I wanted the main sources to
> be
> > > under gll-combinators/src/main/scala/ while the examples are under
> > > examples/src/main/scala/.
> > >
> > > Currently, I'm handling this by actually shelling out to Buildr from
> > within
> > > the examples/ buildfile, building the parent project and then
> explicitly
> > > adding its target/classes/ directory to the examples/ dependencies.  Is
> > > there a better way of doing this?  Better question: *shouldn't* there
> be
> > a
> > > better way of doing this?  :-)
> >
> >
> > Yes.
> >
> > Project definitions should be portable, so if you did everything by the
> > book, you should be able to import gll-combinators from within examples
> (or
> > the other way around). Some gotaches:
> >
> > - You build now runs from a different directory, anything that assumes
> > relative paths is broken, but anything that uses _(..) will adjust. For
> > that, imported projects need to set their base_dir.
> > - Project references are relative, so if you have some 'master' project
> and
> > you import foo and bar, they can reference each other foo:module1,
> > bar:compA, etc.
> > - Globals may conflicts (and constants will warn you when they're
> > redefined)
> > - Build settings (build.yml, profile.yml): you can only use one of these.
> >
> > Generally easier (and from what I've heard more common) to start with the
> > outer buildfile and import nested buildfiles from its sub-directories.
> > Would
> > be easier if everything we do only supports that, so let's use that as a
> > mental model for what you're doing.
> >
> > From examples/buildfile:
> >
> > - cd one level up, since you're effectively running the build from here.
> > - require gll-combinators, everything you do afterwards should extend it
> > - define the examples projects, base_dir should just point one level down
> > (since you're current dir is now one level up)
> >
> > (You won't get tasks/build.yml/etc from the outer directory since the
> > directory change happens to late, but if the model works, we could make
> > Buildr do the right thing with some magic switch)
> >
> > Assaf
> >
> >
> > >
> > > Optimally, I would like to just `require` the parent's buildfile and
> then
> > > using project('gll-combinators') to declare the dependency within the
> > > examples/ buildfile.  However, Buildr doesn't seems to have some global
> > > state relating to project definitions, meaning that this sort of thing
> > > doesn't work at present.
> >
> >
> >
> > >
> > >
> > > Daniel
> > >
> >
>

Re: Correct Way to Depend on Another Buildfile

[Daniel Spiewak <dj...@gmail.com>]

Unfortunately, this technique leads to circular dependencies.  This might be
something that we want to work on in future, since I would imagine that this
sort of thing is fairly common.

Daniel

On Fri, May 8, 2009 at 11:22 AM, Assaf Arkin <ar...@intalio.com> wrote:

> On Thu, May 7, 2009 at 8:18 AM, Daniel Spiewak <dj...@gmail.com>
> wrote:
>
> > Consider: http://github.com/djspiewak/gll-combinators
> >
> > Specifically, the examples/ directory defines a separate Buildr project
> > which depends upon the project in its parent directory (gll-combinators).
> >  I
> > could have done this by using two sub-projects, but I wanted a very
> > specific
> > directory organization.  In other words, I wanted the main sources to be
> > under gll-combinators/src/main/scala/ while the examples are under
> > examples/src/main/scala/.
> >
> > Currently, I'm handling this by actually shelling out to Buildr from
> within
> > the examples/ buildfile, building the parent project and then explicitly
> > adding its target/classes/ directory to the examples/ dependencies.  Is
> > there a better way of doing this?  Better question: *shouldn't* there be
> a
> > better way of doing this?  :-)
>
>
> Yes.
>
> Project definitions should be portable, so if you did everything by the
> book, you should be able to import gll-combinators from within examples (or
> the other way around). Some gotaches:
>
> - You build now runs from a different directory, anything that assumes
> relative paths is broken, but anything that uses _(..) will adjust. For
> that, imported projects need to set their base_dir.
> - Project references are relative, so if you have some 'master' project and
> you import foo and bar, they can reference each other foo:module1,
> bar:compA, etc.
> - Globals may conflicts (and constants will warn you when they're
> redefined)
> - Build settings (build.yml, profile.yml): you can only use one of these.
>
> Generally easier (and from what I've heard more common) to start with the
> outer buildfile and import nested buildfiles from its sub-directories.
> Would
> be easier if everything we do only supports that, so let's use that as a
> mental model for what you're doing.
>
> From examples/buildfile:
>
> - cd one level up, since you're effectively running the build from here.
> - require gll-combinators, everything you do afterwards should extend it
> - define the examples projects, base_dir should just point one level down
> (since you're current dir is now one level up)
>
> (You won't get tasks/build.yml/etc from the outer directory since the
> directory change happens to late, but if the model works, we could make
> Buildr do the right thing with some magic switch)
>
> Assaf
>
>
> >
> > Optimally, I would like to just `require` the parent's buildfile and then
> > using project('gll-combinators') to declare the dependency within the
> > examples/ buildfile.  However, Buildr doesn't seems to have some global
> > state relating to project definitions, meaning that this sort of thing
> > doesn't work at present.
>
>
>
> >
> >
> > Daniel
> >
>

Re: Correct Way to Depend on Another Buildfile

[Assaf Arkin <ar...@intalio.com>]

On Thu, May 7, 2009 at 8:18 AM, Daniel Spiewak <dj...@gmail.com> wrote:

> Consider: http://github.com/djspiewak/gll-combinators
>
> Specifically, the examples/ directory defines a separate Buildr project
> which depends upon the project in its parent directory (gll-combinators).
>  I
> could have done this by using two sub-projects, but I wanted a very
> specific
> directory organization.  In other words, I wanted the main sources to be
> under gll-combinators/src/main/scala/ while the examples are under
> examples/src/main/scala/.
>
> Currently, I'm handling this by actually shelling out to Buildr from within
> the examples/ buildfile, building the parent project and then explicitly
> adding its target/classes/ directory to the examples/ dependencies.  Is
> there a better way of doing this?  Better question: *shouldn't* there be a
> better way of doing this?  :-)


Yes.

Project definitions should be portable, so if you did everything by the
book, you should be able to import gll-combinators from within examples (or
the other way around). Some gotaches:

- You build now runs from a different directory, anything that assumes
relative paths is broken, but anything that uses _(..) will adjust. For
that, imported projects need to set their base_dir.
- Project references are relative, so if you have some 'master' project and
you import foo and bar, they can reference each other foo:module1,
bar:compA, etc.
- Globals may conflicts (and constants will warn you when they're redefined)
- Build settings (build.yml, profile.yml): you can only use one of these.

Generally easier (and from what I've heard more common) to start with the
outer buildfile and import nested buildfiles from its sub-directories. Would
be easier if everything we do only supports that, so let's use that as a
mental model for what you're doing.

>From examples/buildfile:

- cd one level up, since you're effectively running the build from here.
- require gll-combinators, everything you do afterwards should extend it
- define the examples projects, base_dir should just point one level down
(since you're current dir is now one level up)

(You won't get tasks/build.yml/etc from the outer directory since the
directory change happens to late, but if the model works, we could make
Buildr do the right thing with some magic switch)

Assaf


>
> Optimally, I would like to just `require` the parent's buildfile and then
> using project('gll-combinators') to declare the dependency within the
> examples/ buildfile.  However, Buildr doesn't seems to have some global
> state relating to project definitions, meaning that this sort of thing
> doesn't work at present.



>
>
> Daniel
>