developer local override of dependencies

[Shawn Castrianni <Sh...@halliburton.com>]

I would like developers to be able to locally override the dependencies they get from the repository with modules they have built themselves.  Below are my use case descriptions:

Use Case 1:
Let's say I have modules A, B, and C such that A depends on B which depends on C.  A developer needs to do some work in modules B and C.  He has checked out sandboxes for both modules B and C.  He makes his changes to C in his local sandbox.  Before checking it into the source repository, he wants to make his changes to B and test against his changes to C.  So he publishes his locally built C module to a private repository locally on his machine.  He then switches to working on B.  He runs a resolve/retrieve.  His friendly CM guy has already configured his ivy settings file to look first in the private local repository using a chain resolver with returnFirst="true".  With these settings and a dependency revision of "latest", he will successfully pick up his locally published C for use with his changes in B.  He then can feel confident everything will work together and checks in changes to both B and C.  Everybody is happy.

Use Case 2:
Now let's say that a different developer wants to do the same workflow but he is working in modules A and C instead of B and C.  Just like the other developer he publishes his changes to C locally to a private repo.  However, when he does a resolve/retrieve from A, it does NOT pick up the private copy of C in the local repo.  This is because A uses the "latest" rule to get B which it find in the public repository.  Then IVY looks up C transitively by using the specific revision of C that was used to build the revision of B already found.  Since this specific revision of C is used in the lookup, it won't find the private local C since who know what revision number it has.

How can I fix Use Case 2 to work as elegantly as Use Case 1?  The new resolveMode feature is perfect for this problem.  If the developer in Use Case 2, changed the ivy settings file and added a module override to set resolveMode="dynamic", everything would work perfectly and the private local C would be found.  However, that is an extra step for the developer and he has to know all about ivy settings files.  I am trying to hide all of that from developers since most of them don't want to learn.  I added a special publish.local ANT target so that any developer could run it and publish locally to his own private repo.  I wouldn't want him to also go into the ivy settings file and manually add the resolveMode override too.  How could I make this more automatic with the ANT build file?

One idea I have is to use a variable in the resolveMode setting so that I could change its value and, therefore, the resolve behavior with an ANT property.  Then inside my ANT build file just before the resolve/retrieve task, I could scan the private repo looking for modules that have been published there and set the ANT properties to "dynamic" for each module I find.  I don't know if this would work, but it suffers from requiring me to add a module override with a resolveMode variable setting for EVERY module that exists.  That is a pain and requires updating everytime a new module gets created.

Any suggestions?  I use this workflow all the time when I need to make changes that span multiple modules.  I want to test all my changes to all modules together before checking in anything.

---
Shawn Castrianni

----------------------------------------------------------------------
This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient.  Any review, use, distribution, or disclosure by others is strictly prohibited.  If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

Re: developer local override of dependencies

[Niklas Matthies <ml...@nmhq.net>]

On Thu 2008-05-01 at 00:14h, Shawn Castrianni wrote on ivy-user:
> Anybody have any suggestions?

You could have the build script create a settings file, with settings
depending on the build target and/or on settings in a properties file,
and which includes the regular settings file.

-- Niklas Matthies

RE: developer local override of dependencies

[Shawn Castrianni <Sh...@halliburton.com>]

Anybody have any suggestions?

---
Shawn Castrianni


-----Original Message-----
From: Shawn Castrianni
Sent: Sunday, April 27, 2008 10:47 PM
To: 'ivy-user@ant.apache.org'
Subject: RE: developer local override of dependencies

True, but that is in a perfect world.  Imagine 100's of modules and assume the developer knows what he is doing and wants to change a module low in the  dependency hierarchy and a module high in the dependency hierarchy.  He doesn't want to have to build and publish locally all of the modules in between if he knows or feels comfortable that it won't affect them.  He would rely on the automated and continuous build system to catch any of those problems.  When a developer is in development mode, he wants to {code, build, test} over and over again until he is satisfied.  Therefore, this needs to be a fast workflow.  Having to build tons of modules in between just to get them all local would not be acceptable.  Plus, he may not have source code access to all of the ones in between so he couldn't build them locally even if he wanted.

---
Shawn Castrianni

-----Original Message-----
From: Jing Xue [mailto:jingxue@digizenstudio.com]
Sent: Sunday, April 27, 2008 10:41 PM
To: ivy-user@ant.apache.org
Subject: Re: developer local override of dependencies

On Sun, Apr 27, 2008 at 04:54:01AM -0500, Shawn Castrianni wrote:
> Use Case 2: Now let's say that a different developer wants to do the
> same workflow but he is working in modules A and C instead of B and C.
> Just like the other developer he publishes his changes to C locally to
> a private repo.  However, when he does a resolve/retrieve from A, it
> does NOT pick up the private copy of C in the local repo.  This is
> because A uses the "latest" rule to get B which it find in the public
> repository.  Then IVY looks up C transitively by using the specific
> revision of C that was used to build the revision of B already found.
> Since this specific revision of C is used in the lookup, it won't find
> the private local C since who know what revision number it has.

Sorry to answer with a question, but if C is being changed, shouldn't
the developer also be required to make sure the changes didn't break B?
IOW, B always needs to be rebuilt and tested against the local C before
checking C in. And in doing so, there would always be a local snapshot
of B depending on the local C, right?

--
Jing Xue

----------------------------------------------------------------------
This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient.  Any review, use, distribution, or disclosure by others is strictly prohibited.  If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

RE: developer local override of dependencies

[Shawn Castrianni <Sh...@halliburton.com>]

True, but that is in a perfect world.  Imagine 100's of modules and assume the developer knows what he is doing and wants to change a module low in the  dependency hierarchy and a module high in the dependency hierarchy.  He doesn't want to have to build and publish locally all of the modules in between if he knows or feels comfortable that it won't affect them.  He would rely on the automated and continuous build system to catch any of those problems.  When a developer is in development mode, he wants to {code, build, test} over and over again until he is satisfied.  Therefore, this needs to be a fast workflow.  Having to build tons of modules in between just to get them all local would not be acceptable.  Plus, he may not have source code access to all of the ones in between so he couldn't build them locally even if he wanted.

---
Shawn Castrianni

-----Original Message-----
From: Jing Xue [mailto:jingxue@digizenstudio.com]
Sent: Sunday, April 27, 2008 10:41 PM
To: ivy-user@ant.apache.org
Subject: Re: developer local override of dependencies

On Sun, Apr 27, 2008 at 04:54:01AM -0500, Shawn Castrianni wrote:
> Use Case 2: Now let's say that a different developer wants to do the
> same workflow but he is working in modules A and C instead of B and C.
> Just like the other developer he publishes his changes to C locally to
> a private repo.  However, when he does a resolve/retrieve from A, it
> does NOT pick up the private copy of C in the local repo.  This is
> because A uses the "latest" rule to get B which it find in the public
> repository.  Then IVY looks up C transitively by using the specific
> revision of C that was used to build the revision of B already found.
> Since this specific revision of C is used in the lookup, it won't find
> the private local C since who know what revision number it has.

Sorry to answer with a question, but if C is being changed, shouldn't
the developer also be required to make sure the changes didn't break B?
IOW, B always needs to be rebuilt and tested against the local C before
checking C in. And in doing so, there would always be a local snapshot
of B depending on the local C, right?

--
Jing Xue

----------------------------------------------------------------------
This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient.  Any review, use, distribution, or disclosure by others is strictly prohibited.  If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

Re: developer local override of dependencies

[Jing Xue <ji...@digizenstudio.com>]

On Sun, Apr 27, 2008 at 04:54:01AM -0500, Shawn Castrianni wrote:
> Use Case 2: Now let's say that a different developer wants to do the
> same workflow but he is working in modules A and C instead of B and C.
> Just like the other developer he publishes his changes to C locally to
> a private repo.  However, when he does a resolve/retrieve from A, it
> does NOT pick up the private copy of C in the local repo.  This is
> because A uses the "latest" rule to get B which it find in the public
> repository.  Then IVY looks up C transitively by using the specific
> revision of C that was used to build the revision of B already found.
> Since this specific revision of C is used in the lookup, it won't find
> the private local C since who know what revision number it has.

Sorry to answer with a question, but if C is being changed, shouldn't
the developer also be required to make sure the changes didn't break B?
IOW, B always needs to be rebuilt and tested against the local C before
checking C in. And in doing so, there would always be a local snapshot
of B depending on the local C, right?

-- 
Jing Xue