Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

Is there a way I can fail to resolve a dependency if it does not appear in
the remote repository?
For instance,

If I resolve against a <url/> resolver, and I find the dependency there, I
download it, my cache is
updated and I'll use the copy from my cache.

Now if that dependency is removed from the <url/> resolver and I resolve a
second time, I fail to
resolve the dependency, but I don't seem to invalidate the cache. So even
though the artifact no
longer exists in the repository I will resolve it out of my cache. I'll even
print a message like
"found [ whatever ] in
repository-that-acutally-no-longer-has-this-dependency"


-- 

- Eric

Re: Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

Caching is certainly useful for avoiding network I/O when possible, but
there is
more to a cache than copying a file somewhere on the local machine. Cache
consistency is equally important. A cache is not useful it contains the
wrong data.
This example is one use case where simply adding to a cache and never really
looking to see if you are caching the right things isn't enough for the
correct behavior.

Its interesting that you mention you'd like your cached copy of some
artifact to
still be available after its deleted. If I want to delete an artifact from
my repository
I wouldn't want people to be consuming it any longer. This could be part of
a path
to deprecation. I may notify my users that a package will soon be deprecated
and
simply remove it after fair warning. I may even want to add @Deprecated
annotations
to classes in a version of a package I plan to deprecate and do a
publication of the
same version of that package using overwrite="true". The path to deprecation
being
warning, package version that was warned about actually will begin to
generate
compiler warnings because I've republished with the annotations, and finally
removal.

I think for any local resolver checking the source and correctly
invalidating
the cache when you select useOrigin=true may be a reasonable behavior. Or
providing some other way for people who need this to be able to opt to have
a consistent cache when using remote resolvers.

Having a corrupt copy of an artifact in a cache is another use case where
the cache must be invalidated. This can happen in a number of ways.

---

That said, another way to look at this could be to ignore the caching aspect
for a moment. Step back and look at what I really want to do here.
Ultimately,
I'd like to publish a file to a local filesystem repo for overriding, and
then
be able to remove/unpublish that override w/o having to type a bunch of
commands or go looking for files in ivy's cache and the override - I really
don't
want people to go mucking with that.

One possible solution would be to make this erasing resolver that removes
files
for a package also in validate the cache. So to publish to this erasing
resolver
would mean please remove all artifacts and ivy files from the repository for

package "X" and also invalidate the cache for those versions.

The benefit here is that I can more effectively use this override
repository, and
I won't need to go tracking down files and deleting them manually (maybe I
get
ivyconf.xml from some else via a url and I don't know and shouldn't have to
know
where they configured the cache to be, maybe I also don't even know or care
the location of the local override. I could find out, but its not my job to
maintain
the ivyconf, thats someone elses role)

This would solve my particular use case for this local override repository.

However, you still have at least two other real use cases of cache
corruption
and deprecation that would be unsolved.


On 12/24/06, John Gill <ll...@gmail.com> wrote:
>
> I think the point is that the cache is there so that IVY doesn't have to
> check original repository were it came from (which could be remote, slow,
> offline, etc). In addition, how annoying would it be for you to have a
> copy
> of something in your cache, only to find it removed because it got deleted
> from some remote repository you were using.
>
> Personally, I think IVY is working they way it should, and wouldn't want
> it
> to check the remote repository to see if it needed to clean up the cache
> as
> that would make it slower.
>
> On 12/24/06, Eric Crahen <er...@gmail.com> wrote:
> >
> > Here is a simple reproduction:
> >
> > I have two packages, Example1 and Example2, both just create and publish
> a
> > dummy jar
> > to a local filesystem resolver in ~/.ivy/repository
> >
> > Both use a copy of this build.xml snippet to interact with ivy.
> >
> > build.xml:
> >
> >     <ivy:resolve   useOrigin="true"/>
> >     <ivy:cachepath pathid="compile.classpath"
> >       useOrigin="true" conf="java.compile"/>
> >     <ivy:cachepath pathid="test.classpath"
> >       useOrigin="true" conf="java.test"/>
> >     <ivy:cachepath pathid="runtime.classpath"
> >       useOrigin="true" conf="java.runtime"/>
> >
> >     <ivy:deliver status="release"/>
> >
> >     <mkdir dir="${build.dir}/lib"/>
> >     <jar destfile="${build.dir}/lib/${ant.project.name}.jar"/>
> >
> >     <ivy:publish resolver="override"/>
> >
> > This is the relevant ivyconfig.xml shared by both:
> >
> > ivyconfig.xml:
> >
> >     <filesystem name="override" local="true"
> > alwaysCheckExactRevision="false">
> >       <ivy
> >         pattern="${local.repository.dir}/${ivy.pattern}" />
> >       <artifact
> >         pattern="${local.repository.dir}/${ivy.pattern}" />
> >     </filesystem>
> >
> > Example ivy.xml:
> >
> > <ivy-module version="1.0">
> >
> >   <info organisation="example" module="${ant.project.name}" revision="
> > 0.0.1"
> > status="integration"/>
> >
> >   <configurations>
> >     <conf name="java.compile" visibility="public"/>
> >     <conf name="java.test"    extends="java.compile"
> > visibility="private"/>
> >     <conf name="java.runtime" extends="java.compile"
> > visibility="private"/>
> >   </configurations>
> >
> >   <publications>
> >     <artifact name="${ant.project.name}" type="lib" ext="jar" conf="
> > java.runtime"/>
> >   </publications>
> >
> >   <dependencies>
> >   </dependencies>
> >
> > </ivy-module>
> >
> > Example2 ivy.xml:
> >
> > <ivy-module version="1.0">
> >
> >   <info organisation="example" module="${ant.project.name}" revision="
> > 0.0.1"
> > status="integration"/>
> >
> >   <configurations>
> >     <conf name="java.compile" visibility="public"/>
> >     <conf name="java.test"    extends="java.compile"
> > visibility="private"/>
> >     <conf name="java.runtime" extends="java.compile"
> > visibility="private"/>
> >   </configurations>
> >
> >   <publications>
> >     <artifact name="${ant.project.name}" type="lib" ext="jar" conf="
> > java.runtime"/>
> >   </publications>
> >
> >   <dependencies>
> >     <dependency org="example" name="Example" rev="0.0.1" conf="*->*"/>
> >   </dependencies>
> >
> > </ivy-module>
> >
> > ---
> >
> > Building Example1 results in this:
> >
> > cd ~/.ivy && find
> > ./repository
> > ./repository/example
> > ./repository/example/Example
> > ./repository/example/Example/0.0.1
> > ./repository/example/Example/0.0.1/ivy
> > ./repository/example/Example/0.0.1/ivy/ivy.xml.sha1
> > ./repository/example/Example/0.0.1/ivy/ivy.xml.md5
> > ./repository/example/Example/0.0.1/ivy/ivy.xml
> > ./repository/example/Example/0.0.1/lib
> > ./repository/example/Example/0.0.1/lib/Example.jar
> > ./repository/example/Example/0.0.1/lib/Example.jar.sha1
> > ./repository/example/Example/0.0.1/lib/Example.jar.md5
> > ./cache
> > ./cache/ivy-report.css
> > ./cache/example-Example-java.compile.xml
> > ./cache/example-Example-java.runtime.xml
> > ./cache/resolved-example-Example-0.0.1.properties
> > ./cache/example-Example-java.test.xml
> > ./cache/resolved-example-Example-0.0.1.xml
> > ./cache/ivy-report.xsl
> >
> > I see the artifacts published to my filesystem resolver because they
> > appear
> > under repository.
> > Now I build Example2
> >
> > cd ~/.ivy && find
> > ./repository
> > ./repository/example
> > ./repository/example/Example
> > ./repository/example/Example/0.0.1
> > ./repository/example/Example/0.0.1/ivy
> > ./repository/example/Example/0.0.1/ivy/ivy.xml.sha1
> > ./repository/example/Example/0.0.1/ivy/ivy.xml.md5
> > ./repository/example/Example/0.0.1/ivy/ivy.xml
> > ./repository/example/Example/0.0.1/lib
> > ./repository/example/Example/0.0.1/lib/Example.jar
> > ./repository/example/Example/0.0.1/lib/Example.jar.sha1
> > ./repository/example/Example/0.0.1/lib/Example.jar.md5
> > ./repository/example/Example2
> > ./repository/example/Example2/0.0.1
> > ./repository/example/Example2/0.0.1/ivy
> > ./repository/example/Example2/0.0.1/ivy/ivy.xml.sha1
> > ./repository/example/Example2/0.0.1/ivy/ivy.xml.md5
> > ./repository/example/Example2/0.0.1/ivy/ivy.xml
> > ./repository/example/Example2/0.0.1/lib
> > ./repository/example/Example2/0.0.1/lib/Example2.jar.sha1
> > ./repository/example/Example2/0.0.1/lib/Example2.jar.md5
> > ./repository/example/Example2/0.0.1/lib/Example2.jar
> > ./cache
> > ./cache/ivy-report.css
> > ./cache/example-Example-java.compile.xml
> > ./cache/example-Example-java.runtime.xml
> > ./cache/resolved-example-Example-0.0.1.properties
> > ./cache/example-Example-java.test.xml
> > ./cache/resolved-example-Example2-0.0.1.properties
> > ./cache/example-Example2-java.compile.xml
> > ./cache/resolved-example-Example-0.0.1.xml
> > ./cache/example-Example2-java.runtime.xml
> > ./cache/example
> > ./cache/example/Example
> > ./cache/example/Example/ivydata-0.0.1.properties
> > ./cache/example/Example/ivy-0.0.1.xml
> > ./cache/example-Example2-java.test.xml
> > ./cache/ivy-report.xsl
> > ./cache/resolved-example-Example2-0.0.1.xml
> >
> >
> > I see the artifacts published to my filesystem resolver because they
> > appear
> > under repository.
> > HOWEVER, I also see that my cache was populated with the descriptor from
> > the
> > first project.
> > -> ./cache/example/Example
> > -> ./cache/example/Example/ivydata-0.0.1.properties
> > -> ./cache/example/Example/ivy-0.0.1.xml
> >
> > Now when I remove Example from my repository and build Example2 again,
> Ivy
> > resolves
> > Example correctly, even though it has actually been removed.
> >
> > rm -r ~/.ivy/repository/example/Example
> >
> > It does this because the cache doesn't get invalidated. When the resolve
> > task runs during the
> > second build of Example2 in which I've removed the artifacts and ivy
> files
> > from the filesystem
> > repository by force, it seems to only consult the cache and actually
> says
> > "found [ example | Example
> > | 0.0.1 ] in cache" which doesn't make sense since I've specified
> > useOrigin=true.
> >
> > ----
> >
> > The use case for this is that I want to do something, like override
> > commons-http client locally to work
> > with a bugfix that hasn't been released yet. I use a chain with a local
> > filesystem resolver in the front
> > so that returnFirst="true". This way whatever I put in my local resolve
> > will
> > effectively override the
> > offical release. Now at some point I don't need the override - maybe the
> > bugfix is integrated into the
> > offical release - maybe I just decide that bug is not relevant to me
> > anymore
> > and I want to go back to
> > building against the offical copy. I want to just remove the local
> > override
> > by removing that artifact and
> > ivy file from my local filesystem. This fails because the cache is never
> > invalidated correctly.
> >
> > The work around is nuke the cache, which isn't ideal. Ultimately I'd
> like
> > to
> > never be typing rm to use
> > ivy effectively which is why this isn't ideal. I don't want to manually
> be
> > blasting things in ivy's cache.
> > Not having to manually blast things in my local repository is what the
> > https://issues.apache.org/jira/browse/IVY-358 request is for.
> >
> >
> >
> >
> > On 12/23/06, Eric Crahen <er...@gmail.com> wrote:
> > >
> > > Is there a way I can fail to resolve a dependency if it does not
> appear
> > in
> > > the remote repository?
> > > For instance,
> > >
> > > If I resolve against a <url/> resolver, and I find the dependency
> there,
> > I
> > > download it, my cache is
> > > updated and I'll use the copy from my cache.
> > >
> > > Now if that dependency is removed from the <url/> resolver and I
> resolve
> > a
> > > second time, I fail to
> > > resolve the dependency, but I don't seem to invalidate the cache. So
> > even
> > > though the artifact no
> > > longer exists in the repository I will resolve it out of my cache.
> I'll
> > > even print a message like
> > > "found [ whatever ] in
> > > repository-that-acutally-no-longer-has-this-dependency"
> > >
> > >
> > > --
> > >
> > > - Eric
> >
> >
> >
> >
> > --
> >
> > - Eric
> >
> >
>
>
> --
> Regards,
> John Gill
>
>


-- 

- Eric

Re: Opt out of the cache for a resolver

[John Gill <ll...@gmail.com>]

I think the point is that the cache is there so that IVY doesn't have to
check original repository were it came from (which could be remote, slow,
offline, etc). In addition, how annoying would it be for you to have a copy
of something in your cache, only to find it removed because it got deleted
from some remote repository you were using.

Personally, I think IVY is working they way it should, and wouldn't want it
to check the remote repository to see if it needed to clean up the cache as
that would make it slower.

On 12/24/06, Eric Crahen <er...@gmail.com> wrote:
>
> Here is a simple reproduction:
>
> I have two packages, Example1 and Example2, both just create and publish a
> dummy jar
> to a local filesystem resolver in ~/.ivy/repository
>
> Both use a copy of this build.xml snippet to interact with ivy.
>
> build.xml:
>
>     <ivy:resolve   useOrigin="true"/>
>     <ivy:cachepath pathid="compile.classpath"
>       useOrigin="true" conf="java.compile"/>
>     <ivy:cachepath pathid="test.classpath"
>       useOrigin="true" conf="java.test"/>
>     <ivy:cachepath pathid="runtime.classpath"
>       useOrigin="true" conf="java.runtime"/>
>
>     <ivy:deliver status="release"/>
>
>     <mkdir dir="${build.dir}/lib"/>
>     <jar destfile="${build.dir}/lib/${ant.project.name}.jar"/>
>
>     <ivy:publish resolver="override"/>
>
> This is the relevant ivyconfig.xml shared by both:
>
> ivyconfig.xml:
>
>     <filesystem name="override" local="true"
> alwaysCheckExactRevision="false">
>       <ivy
>         pattern="${local.repository.dir}/${ivy.pattern}" />
>       <artifact
>         pattern="${local.repository.dir}/${ivy.pattern}" />
>     </filesystem>
>
> Example ivy.xml:
>
> <ivy-module version="1.0">
>
>   <info organisation="example" module="${ant.project.name}" revision="
> 0.0.1"
> status="integration"/>
>
>   <configurations>
>     <conf name="java.compile" visibility="public"/>
>     <conf name="java.test"    extends="java.compile"
> visibility="private"/>
>     <conf name="java.runtime" extends="java.compile"
> visibility="private"/>
>   </configurations>
>
>   <publications>
>     <artifact name="${ant.project.name}" type="lib" ext="jar" conf="
> java.runtime"/>
>   </publications>
>
>   <dependencies>
>   </dependencies>
>
> </ivy-module>
>
> Example2 ivy.xml:
>
> <ivy-module version="1.0">
>
>   <info organisation="example" module="${ant.project.name}" revision="
> 0.0.1"
> status="integration"/>
>
>   <configurations>
>     <conf name="java.compile" visibility="public"/>
>     <conf name="java.test"    extends="java.compile"
> visibility="private"/>
>     <conf name="java.runtime" extends="java.compile"
> visibility="private"/>
>   </configurations>
>
>   <publications>
>     <artifact name="${ant.project.name}" type="lib" ext="jar" conf="
> java.runtime"/>
>   </publications>
>
>   <dependencies>
>     <dependency org="example" name="Example" rev="0.0.1" conf="*->*"/>
>   </dependencies>
>
> </ivy-module>
>
> ---
>
> Building Example1 results in this:
>
> cd ~/.ivy && find
> ./repository
> ./repository/example
> ./repository/example/Example
> ./repository/example/Example/0.0.1
> ./repository/example/Example/0.0.1/ivy
> ./repository/example/Example/0.0.1/ivy/ivy.xml.sha1
> ./repository/example/Example/0.0.1/ivy/ivy.xml.md5
> ./repository/example/Example/0.0.1/ivy/ivy.xml
> ./repository/example/Example/0.0.1/lib
> ./repository/example/Example/0.0.1/lib/Example.jar
> ./repository/example/Example/0.0.1/lib/Example.jar.sha1
> ./repository/example/Example/0.0.1/lib/Example.jar.md5
> ./cache
> ./cache/ivy-report.css
> ./cache/example-Example-java.compile.xml
> ./cache/example-Example-java.runtime.xml
> ./cache/resolved-example-Example-0.0.1.properties
> ./cache/example-Example-java.test.xml
> ./cache/resolved-example-Example-0.0.1.xml
> ./cache/ivy-report.xsl
>
> I see the artifacts published to my filesystem resolver because they
> appear
> under repository.
> Now I build Example2
>
> cd ~/.ivy && find
> ./repository
> ./repository/example
> ./repository/example/Example
> ./repository/example/Example/0.0.1
> ./repository/example/Example/0.0.1/ivy
> ./repository/example/Example/0.0.1/ivy/ivy.xml.sha1
> ./repository/example/Example/0.0.1/ivy/ivy.xml.md5
> ./repository/example/Example/0.0.1/ivy/ivy.xml
> ./repository/example/Example/0.0.1/lib
> ./repository/example/Example/0.0.1/lib/Example.jar
> ./repository/example/Example/0.0.1/lib/Example.jar.sha1
> ./repository/example/Example/0.0.1/lib/Example.jar.md5
> ./repository/example/Example2
> ./repository/example/Example2/0.0.1
> ./repository/example/Example2/0.0.1/ivy
> ./repository/example/Example2/0.0.1/ivy/ivy.xml.sha1
> ./repository/example/Example2/0.0.1/ivy/ivy.xml.md5
> ./repository/example/Example2/0.0.1/ivy/ivy.xml
> ./repository/example/Example2/0.0.1/lib
> ./repository/example/Example2/0.0.1/lib/Example2.jar.sha1
> ./repository/example/Example2/0.0.1/lib/Example2.jar.md5
> ./repository/example/Example2/0.0.1/lib/Example2.jar
> ./cache
> ./cache/ivy-report.css
> ./cache/example-Example-java.compile.xml
> ./cache/example-Example-java.runtime.xml
> ./cache/resolved-example-Example-0.0.1.properties
> ./cache/example-Example-java.test.xml
> ./cache/resolved-example-Example2-0.0.1.properties
> ./cache/example-Example2-java.compile.xml
> ./cache/resolved-example-Example-0.0.1.xml
> ./cache/example-Example2-java.runtime.xml
> ./cache/example
> ./cache/example/Example
> ./cache/example/Example/ivydata-0.0.1.properties
> ./cache/example/Example/ivy-0.0.1.xml
> ./cache/example-Example2-java.test.xml
> ./cache/ivy-report.xsl
> ./cache/resolved-example-Example2-0.0.1.xml
>
>
> I see the artifacts published to my filesystem resolver because they
> appear
> under repository.
> HOWEVER, I also see that my cache was populated with the descriptor from
> the
> first project.
> -> ./cache/example/Example
> -> ./cache/example/Example/ivydata-0.0.1.properties
> -> ./cache/example/Example/ivy-0.0.1.xml
>
> Now when I remove Example from my repository and build Example2 again, Ivy
> resolves
> Example correctly, even though it has actually been removed.
>
> rm -r ~/.ivy/repository/example/Example
>
> It does this because the cache doesn't get invalidated. When the resolve
> task runs during the
> second build of Example2 in which I've removed the artifacts and ivy files
> from the filesystem
> repository by force, it seems to only consult the cache and actually says
> "found [ example | Example
> | 0.0.1 ] in cache" which doesn't make sense since I've specified
> useOrigin=true.
>
> ----
>
> The use case for this is that I want to do something, like override
> commons-http client locally to work
> with a bugfix that hasn't been released yet. I use a chain with a local
> filesystem resolver in the front
> so that returnFirst="true". This way whatever I put in my local resolve
> will
> effectively override the
> offical release. Now at some point I don't need the override - maybe the
> bugfix is integrated into the
> offical release - maybe I just decide that bug is not relevant to me
> anymore
> and I want to go back to
> building against the offical copy. I want to just remove the local
> override
> by removing that artifact and
> ivy file from my local filesystem. This fails because the cache is never
> invalidated correctly.
>
> The work around is nuke the cache, which isn't ideal. Ultimately I'd like
> to
> never be typing rm to use
> ivy effectively which is why this isn't ideal. I don't want to manually be
> blasting things in ivy's cache.
> Not having to manually blast things in my local repository is what the
> https://issues.apache.org/jira/browse/IVY-358 request is for.
>
>
>
>
> On 12/23/06, Eric Crahen <er...@gmail.com> wrote:
> >
> > Is there a way I can fail to resolve a dependency if it does not appear
> in
> > the remote repository?
> > For instance,
> >
> > If I resolve against a <url/> resolver, and I find the dependency there,
> I
> > download it, my cache is
> > updated and I'll use the copy from my cache.
> >
> > Now if that dependency is removed from the <url/> resolver and I resolve
> a
> > second time, I fail to
> > resolve the dependency, but I don't seem to invalidate the cache. So
> even
> > though the artifact no
> > longer exists in the repository I will resolve it out of my cache. I'll
> > even print a message like
> > "found [ whatever ] in
> > repository-that-acutally-no-longer-has-this-dependency"
> >
> >
> > --
> >
> > - Eric
>
>
>
>
> --
>
> - Eric
>
>


-- 
Regards,
John Gill

Re: Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

Yes, what I am trying to do here is override! I am talking about
making changes local to my workspace. No one can see the override
resolver except me. Each developers override resolver would use a
private piece of their file system.

What you just described is a way to do this using versions.
I make changes to package-1.1 and Bob makes changes to package-1.1
and 10 other people do the same. We can all make up a new version.

There is the official shared/approved version 1.1, and the private version
1.1.X that we all created independently.

You can restate this as

Independent private shadows/overrides of version 1.1
and the official shared/approved version 1.1

Do you agree that this is the same thing, and the only difference really
is that your approach is to use versioning names and I am exploring a
different
approach?

--

Please, can you walk me through the scenario I described to John a few posts
back (where I describe A-> B -> C -> D-> X-> Y)?

I would like you demonstrate how that scenario would work by renaming and
reversioning things.



On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
>
> Eric Crahen wrote:
> > When I use the copy in my override repository its the patched version.
> > When I remove the copy in my override repository it is the released
> > version.
> >
> > This is the point of a local override repository.
> > There are plenty of valid reasons for this:
> [...]
>
> Your vision seems to be restricted to 'overriding' without thinking to
> 'versioning'. Which ultimately leads to version 'shadowing'.
> A version is a version and must be uniquely identifiable. period. no jar
> flying around which is 1.1 which is not a 1.1.
>
> As John mentioned. Change the name. Change the version number. (no
> excuses that you cannot predict and that you are thinking that the next
> version of 1.1 may be postfixed by the name of your company or your dog
> name - this is _unlikely_). Change the resolver statuses. Whatever makes
> sense in term of version.
>
> Patch your version 1.1 and call it 1.1-patched-by-my-company and make
> sure that your resolver is actually able to make the distinction between
> 'patched-by-my-company' > 'final' or if you don't want to go that road,
> make sure to force the version to '1.1-patched-by-my-company]'.
> Whatever. Makes this humanly identifiable.
>
> -- stephane
>
>


-- 

- Eric

Re: Opt out of the cache for a resolver

[Stephane Bailliez <sb...@gmail.com>]

Eric Crahen wrote:
> When I use the copy in my override repository its the patched version.
> When I remove the copy in my override repository it is the released 
> version.
>
> This is the point of a local override repository.
> There are plenty of valid reasons for this:
[...]

Your vision seems to be restricted to 'overriding' without thinking to 
'versioning'. Which ultimately leads to version 'shadowing'.
A version is a version and must be uniquely identifiable. period. no jar 
flying around which is 1.1 which is not a 1.1.

As John mentioned. Change the name. Change the version number. (no 
excuses that you cannot predict and that you are thinking that the next 
version of 1.1 may be postfixed by the name of your company or your dog 
name - this is _unlikely_). Change the resolver statuses. Whatever makes 
sense in term of version.

Patch your version 1.1 and call it 1.1-patched-by-my-company and make 
sure that your resolver is actually able to make the distinction between 
'patched-by-my-company' > 'final' or if you don't want to go that road, 
make sure to force the version to '1.1-patched-by-my-company]'. 
Whatever. Makes this humanly identifiable.

-- stephane

Re: Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

It should be in there under one of the recent items I opened

On 12/26/06, Xavier Hanin <xa...@gmail.com> wrote:
>
> On 12/26/06, Eric Crahen <er...@gmail.com> wrote:
> >
> > One of the interesting parts of the discussion to read through when you
> > have
> > the time is the difference between development and release use cases.
> >
> > ---
> >
> > I think I would have to disagree about adding deprecation annotations
> > being
> > a
> > dangerous change. The library behaves identically; line numbers in a
> stack
> > trace
> > may change, but the content and meaning and the trace is identical. The
> > only
> > danger really would be relying on line numbers in the stack trace, which
> > is
> > a
> > questionable practice to begin with.
> >
> > Certainly, there are a lot of "gotchas" library owners will have to
> > consider, but
> > none of these are things that Ivy can just solve for everyone, even if
> you
> > try to
> > enforce a certain pattern of usage.
> >
> > Each library owner will have their own set of concerns, their own
> > development
> > process, etc. I don't believe there is a one size fits all solution. It
> > makes sense to
> > provide a best practices recommendation, but I think it would be a
> mistake
> > to limit
> > thought to only those solutions. Ultimately, the library owner will need
> > to
> > weigh these
> > things, understand the impact what they are doing and use Ivy as a
> *tool*
> > to
> >
> > accomplish their goal.
>
>
> I strongly agree with that, and this has always be Ivy philosophy: try to
> adapt to several usages, and do not force the user into a single way of
> working. That's why I underlined that even if I strongly suggest to avoid
> changing the content of a version, Ivy supports this kind of process, so I
> consider your problem a bug which should be fixed, as long as we are
> talking
> about a changing revision. So I suggest you add a new bug report in JIRA.
>
> Regards,
>
> Xavier
>
> If you don't trust someone to be able to make a decision on how to release
> > their code,
> > you don't have faith they can do something as simple as associate a name
> > and
> >
> > number with it, then I'm really suprised you'd trust they could do the
> > more
> > complicated
> > task of writing the code itself.
> >
> >
> > On 12/26/06, Xavier Hanin <xa...@gmail.com> wrote:
> > >
> > > Hello All,
> > >
> > > Interesting discussion here, I don't have time to read everything and
> > > catch
> > > u^p on the discussion, but I'd like to give my opinion:
> > > - I agree with Stephane and John that a version is a version and you
> > > should
> > > never change what has been released. Even adding deprecated
> annotations
> > > can
> > > change line numbers if you are not careful, and you will have troubles
> > > inspecting a stack trace if your line numbers are not correct. And
> this
> > is
> > > just an example of all the problems you may run into when you don't
> > > actually
> > > know the exact content of a jar when you know its version
> > > - Even if I believe a version is a version and should never change,
> Ivy
> > > support two kind of changes in a version:
> > > ** the module descriptor change (i.e. a change in the ivy.xml) when
> you
> > > enable checkModified
> > > ** a change of the artifacts (i.e. a jar for instance), when you
> declare
> > > it
> > > as a changing revision (either in your ivy.xml or by using the
> > > changingPattern)
> > >
> > > In this last case I think that supporting cache invalidation as
> > requested
> > > by
> > > Eric woud be consistent with the behaviour expected. And when you say
> > that
> > > your revision is changing Ivy has to check if the cache is valid each
> > > time,
> > > so it wouldn't introduce a new performance overhead.
> > >
> > > Regards,
> > > Xavier
> > >
> > >
> > > On 12/24/06, Eric Crahen <er...@gmail.com> wrote:
> > > >
> > > > Thanks, I think we have just both described the need to invalidate a
> > > > version in the ivy cache. I'll compare the two approaches
> objectively:
> > > >
> > > > In both cases I'm using a chain resolver in which the first element
> is
> > > > just my local filesystem and the second is the shared repository
> which
> > > > contains offical/approved copies of all internal and external
> > libraries.
> > > > In both cases all changes are visible only to me because we publish
> > > > only to our filesystem.
> > > >
> > > > The key parts of your description are:
> > > > "change the version for Y, then update your references."
> > > >
> > > > This effectively means,
> > > >
> > > > #1 checkout Y, change the version, fix it, publish it
> > > > #2 checkout X, change the version because I'm changing its
> > dependencies
> > > >     to point to the fixed Y, publish it.
> > > > #3 check out D, change the version because I'm changing its
> > dependencies
> > > >     to point to the fixed Y, publish it.
> > > > #4 check out C, change the version because I'm changing its
> > dependencies
> > > >     to point to the fixed D, publish it.
> > > > #5 check out B, change the version because I'm changing its
> > dependencies
> > > >     to point to the fixed C, publish it.
> > > > #6 check out A, change the version because I'm changing its
> > dependencies
> > > >     to point to the fixed B, publish it.
> > > > #7 Hope that I chose version numbers that will become automatically
> > > > obsolete
> > > >     when the official fix is released.
> > > >
> > > > So to fix a simple bug, locally using private versions only I can
> see,
> > > in
> > > > 1
> > > > package,
> > > > I actually have to alter 6 ivy.xmls and perform 6 builds.
> > > >
> > > > With what I suggested I would change 2 packages.
> > > >
> > > > #1 checkout Y, fix it, publish it to the local override resolver
> > > > #2 checkout A, build it.
> > > >
> > > > This required changing no ivy.xmls and building only 2 packages.
> > > >
> > > > Ultimately we have accomplished the same thing.
> > > >
> > > >
> > > > The difference is that you preferred to use a very strict version
> > > scheme,
> > > > where as I opted to use the fact that something exists in my local
> > > > resolver
> > > > as an implicit private local version.
> > > >
> > > > The strong version comes at the cost of more work, and the amount of
> > > > that work (changing ivy.xml and building files) is proportional to
> the
> > > > number of packages. I would actually need to transitively enumerate
> > > > my dependencies and be sure that I update the versions correctly
> > > > in all dependencies - which may be tedious and error prone.
> > > >
> > > > The implied version uses the chain to resolver preference to let
> > > > ivy to the transitive traversal for me; and since this is an unnamed
> > > > implied version, there was no need to guess a version number.
> > > >
> > > > Both have visual cues, it depends on where you look. The cue in the
> > > > strong version is in the ivy.xml and the cue in the implied version
> is
> > > > existence of a file in the local repository.
> > > >
> > > > Which one is most suitable is a choice a developer is free to make.
> I
> > > > don't Ivy needs to dictate one or the other, this really is a matter
> > of
> > > > preference
> > > > and a matter of what a developer is comfortable with. Both very
> > possible
> > > > and valid solutions. There are situations when both are useful.
> > > > I do not want to engage in a subject discussion about which is the
> > right
> > > > thing to do.
> > > >
> > > > Now here is what we have not addressed, which is the reason for the
> > > entire
> > > > thread.
> > > > ** How do we evict these private copies from the cache when they are
> > no
> > > > longer
> > > > needed? **
> > > >
> > > > In the case of the strong versioning approach, part of what we have
> > done
> > > > is
> > > > hope to have chosen versions that would be considered older than the
> > > > official
> > > > version once released. We may not have guessed right and need to
> > remove
> > > > that
> > > > private local copy from the ivy cache. After all its lifetime is
> over,
> > > > this
> > > > was a private
> > > > local copy that should no longer exist in the cache; we may to
> choose
> > to
> > > > keep it
> > > > in the local resolver where it can be selected at any time, but it
> > > > certainly
> > > > has no
> > > > need to be in the cache once its outlived its usefulness. Unless
> > someone
> > > > never
> > > > makes mistakes, then maybe they will never need to invalidate the
> > cache.
> > > > Maybe
> > > > some of us are just that good but I make mistakes, so I would really
> > > like
> > > > the ability
> > > > to do so when needed when I use this approach.
> > > >
> > > > In the case of the implied version approach, we used the simple fact
> > > that
> > > > artifacts
> > > > existed in the local filesystem repository to indicate that those
> are
> > > the
> > > > preferred
> > > > artifacts to use. Now that we have a better release available in the
> > > > shared
> > > > repository
> > > > we'd like to invalidate the artifacts in the cache and remove these
> > > files
> > > > from the resolver.
> > > >
> > > > BOTH describe a use-case for invalidating the ivy cache for a
> > particular
> > > > verison.
> > > >
> > > > ---
> > > >
> > > > One possible way to do invalidate the cache would be that erasing
> > > resolver
> > > > I
> > > > described which
> > > > would only work in association with a local filesystem resolver.
> > > > When I published any version, then it would invalidate that version
> in
> > > the
> > > > cache.
> > > >
> > > > This would allow me to use that same resolver chain
> > > >
> > > > // psuedo-chain
> > > > <chain>
> > > > <filesystem name="local" local="true"/>
> > > > <url name="shared/>
> > > > </chain>
> > > >
> > > > <invalidate name="invalidate" ref="local" erase=<<optional>> />
> > > >
> > > > With either approach my workflow for making local changes,
> regardless
> > of
> > > > how
> > > > I choose to
> > > > version, is
> > > >
> > > > Check out.
> > > > Resolve with chain.
> > > > Build.
> > > > Publish to local repository.
> > > > Publish to invalidate when its time to invalidate the cache.
> > > >
> > > > Erasing the copy of that version from the local resolver could be
> > > optional
> > > > to support both
> > > > use cases where people do want to remove packages, and use cases
> where
> > > > people do
> > > > not.
> > > >
> > > > ---
> > > >
> > > > You said that you feared that people using this patching/overriding
> > > > logic means people could make a mistake in external repositories.
> > > > This can happen at any time regardless of the technique used to
> > > > make local changes. This is orthogonal to using one technique or
> > > > the other for local changes. Releases require care, which is another
> > > > issue entirely.
> > > >
> > > > As to your other points 1-5 at the end or you post, this is good
> > advice
> > > > but orthogonal to solving the problem of making a local change and
> > > > effectively invalidating the cache when its no longer needed.
> > > >
> > > >
> > > > On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
> > > > >
> > > > > Eric Crahen wrote:
> > > > >
> > > > > > I've found that there is a bug in Y.
> > > > > > Several things need to happen for me at this point:
> > > > > > #1.
> > > > > > I need to keep working on A. So I need to be able to patch Y.
> > > > > > This means I need a local copy. Now you can label this
> > > > > > Eric's-hacked-library
> > > > >
> > > > > This means fix the bug, change the version number or append an
> > > > > additional string or whatever that match your status. Then store
> it
> > in
> > > > > your repository. Then update your references so that you force the
> > use
> > > > > of this specific version.
> > > > >
> > > > > You must strive to have y-1.1-patched-by-eric.jar > y-1.1.jar or
> > > > > y-1.1-patched-by-eric.jar evicting y-1.1.jar depending your
> internal
> > > > > policy., etc...
> > > > > When a version 1.2 will be released (which hopefully will
> > incorporate
> > > > > your patch), then 1.2 automatically obsolete 1.1-patched-by-eric
> > (unless
> > > > > the version is really forced with force=true)
> > > > >
> > > > > When you do this dependencies are really managed rather than
> hacked
> > > and
> > > > > there is always the visual element to make sure that the version
> > that
> > > is
> > > > > included is really the latest version and it conveys immediately
> the
> > > > > information that this dependency is patched.
> > > > >
> > > > >
> > > > > If we follow this patching/overriding logic, that means that
> people
> > > > > could well do the same on external repositories and silently swap
> > one
> > > > > jar with another patched version because they are too lazy to do a
> > > > > release, which basically means that 2 people could be using the
> same
> > > > > version with a different code. That's brilliant to convince
> > developers
> > > > > to commit suicide. :)
> > > > >
> > > > > And this is also another reason why I don't use external
> > repositories.
> > > > > 1) I trust no one.
> > > > > 2) dependencies are generally not well expressed
> > > > > 3) versioning is sometimes not even well done correctly
> > > > > 4) I keep full control of all my dependencies
> > > > > 5) I have flexibility.
> > > > >
> > > > > -- stephane
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > - Eric
> > > >
> > > >
> > >
> > >
> >
> >
> > --
> >
> > - Eric
> >
> >
>
>


-- 

- Eric

Re: Opt out of the cache for a resolver

[Xavier Hanin <xa...@gmail.com>]

On 12/26/06, Eric Crahen <er...@gmail.com> wrote:
>
> One of the interesting parts of the discussion to read through when you
> have
> the time is the difference between development and release use cases.
>
> ---
>
> I think I would have to disagree about adding deprecation annotations
> being
> a
> dangerous change. The library behaves identically; line numbers in a stack
> trace
> may change, but the content and meaning and the trace is identical. The
> only
> danger really would be relying on line numbers in the stack trace, which
> is
> a
> questionable practice to begin with.
>
> Certainly, there are a lot of "gotchas" library owners will have to
> consider, but
> none of these are things that Ivy can just solve for everyone, even if you
> try to
> enforce a certain pattern of usage.
>
> Each library owner will have their own set of concerns, their own
> development
> process, etc. I don't believe there is a one size fits all solution. It
> makes sense to
> provide a best practices recommendation, but I think it would be a mistake
> to limit
> thought to only those solutions. Ultimately, the library owner will need
> to
> weigh these
> things, understand the impact what they are doing and use Ivy as a *tool*
> to
>
> accomplish their goal.


I strongly agree with that, and this has always be Ivy philosophy: try to
adapt to several usages, and do not force the user into a single way of
working. That's why I underlined that even if I strongly suggest to avoid
changing the content of a version, Ivy supports this kind of process, so I
consider your problem a bug which should be fixed, as long as we are talking
about a changing revision. So I suggest you add a new bug report in JIRA.

Regards,

Xavier

If you don't trust someone to be able to make a decision on how to release
> their code,
> you don't have faith they can do something as simple as associate a name
> and
>
> number with it, then I'm really suprised you'd trust they could do the
> more
> complicated
> task of writing the code itself.
>
>
> On 12/26/06, Xavier Hanin <xa...@gmail.com> wrote:
> >
> > Hello All,
> >
> > Interesting discussion here, I don't have time to read everything and
> > catch
> > u^p on the discussion, but I'd like to give my opinion:
> > - I agree with Stephane and John that a version is a version and you
> > should
> > never change what has been released. Even adding deprecated annotations
> > can
> > change line numbers if you are not careful, and you will have troubles
> > inspecting a stack trace if your line numbers are not correct. And this
> is
> > just an example of all the problems you may run into when you don't
> > actually
> > know the exact content of a jar when you know its version
> > - Even if I believe a version is a version and should never change, Ivy
> > support two kind of changes in a version:
> > ** the module descriptor change (i.e. a change in the ivy.xml) when you
> > enable checkModified
> > ** a change of the artifacts (i.e. a jar for instance), when you declare
> > it
> > as a changing revision (either in your ivy.xml or by using the
> > changingPattern)
> >
> > In this last case I think that supporting cache invalidation as
> requested
> > by
> > Eric woud be consistent with the behaviour expected. And when you say
> that
> > your revision is changing Ivy has to check if the cache is valid each
> > time,
> > so it wouldn't introduce a new performance overhead.
> >
> > Regards,
> > Xavier
> >
> >
> > On 12/24/06, Eric Crahen <er...@gmail.com> wrote:
> > >
> > > Thanks, I think we have just both described the need to invalidate a
> > > version in the ivy cache. I'll compare the two approaches objectively:
> > >
> > > In both cases I'm using a chain resolver in which the first element is
> > > just my local filesystem and the second is the shared repository which
> > > contains offical/approved copies of all internal and external
> libraries.
> > > In both cases all changes are visible only to me because we publish
> > > only to our filesystem.
> > >
> > > The key parts of your description are:
> > > "change the version for Y, then update your references."
> > >
> > > This effectively means,
> > >
> > > #1 checkout Y, change the version, fix it, publish it
> > > #2 checkout X, change the version because I'm changing its
> dependencies
> > >     to point to the fixed Y, publish it.
> > > #3 check out D, change the version because I'm changing its
> dependencies
> > >     to point to the fixed Y, publish it.
> > > #4 check out C, change the version because I'm changing its
> dependencies
> > >     to point to the fixed D, publish it.
> > > #5 check out B, change the version because I'm changing its
> dependencies
> > >     to point to the fixed C, publish it.
> > > #6 check out A, change the version because I'm changing its
> dependencies
> > >     to point to the fixed B, publish it.
> > > #7 Hope that I chose version numbers that will become automatically
> > > obsolete
> > >     when the official fix is released.
> > >
> > > So to fix a simple bug, locally using private versions only I can see,
> > in
> > > 1
> > > package,
> > > I actually have to alter 6 ivy.xmls and perform 6 builds.
> > >
> > > With what I suggested I would change 2 packages.
> > >
> > > #1 checkout Y, fix it, publish it to the local override resolver
> > > #2 checkout A, build it.
> > >
> > > This required changing no ivy.xmls and building only 2 packages.
> > >
> > > Ultimately we have accomplished the same thing.
> > >
> > >
> > > The difference is that you preferred to use a very strict version
> > scheme,
> > > where as I opted to use the fact that something exists in my local
> > > resolver
> > > as an implicit private local version.
> > >
> > > The strong version comes at the cost of more work, and the amount of
> > > that work (changing ivy.xml and building files) is proportional to the
> > > number of packages. I would actually need to transitively enumerate
> > > my dependencies and be sure that I update the versions correctly
> > > in all dependencies - which may be tedious and error prone.
> > >
> > > The implied version uses the chain to resolver preference to let
> > > ivy to the transitive traversal for me; and since this is an unnamed
> > > implied version, there was no need to guess a version number.
> > >
> > > Both have visual cues, it depends on where you look. The cue in the
> > > strong version is in the ivy.xml and the cue in the implied version is
> > > existence of a file in the local repository.
> > >
> > > Which one is most suitable is a choice a developer is free to make. I
> > > don't Ivy needs to dictate one or the other, this really is a matter
> of
> > > preference
> > > and a matter of what a developer is comfortable with. Both very
> possible
> > > and valid solutions. There are situations when both are useful.
> > > I do not want to engage in a subject discussion about which is the
> right
> > > thing to do.
> > >
> > > Now here is what we have not addressed, which is the reason for the
> > entire
> > > thread.
> > > ** How do we evict these private copies from the cache when they are
> no
> > > longer
> > > needed? **
> > >
> > > In the case of the strong versioning approach, part of what we have
> done
> > > is
> > > hope to have chosen versions that would be considered older than the
> > > official
> > > version once released. We may not have guessed right and need to
> remove
> > > that
> > > private local copy from the ivy cache. After all its lifetime is over,
> > > this
> > > was a private
> > > local copy that should no longer exist in the cache; we may to choose
> to
> > > keep it
> > > in the local resolver where it can be selected at any time, but it
> > > certainly
> > > has no
> > > need to be in the cache once its outlived its usefulness. Unless
> someone
> > > never
> > > makes mistakes, then maybe they will never need to invalidate the
> cache.
> > > Maybe
> > > some of us are just that good but I make mistakes, so I would really
> > like
> > > the ability
> > > to do so when needed when I use this approach.
> > >
> > > In the case of the implied version approach, we used the simple fact
> > that
> > > artifacts
> > > existed in the local filesystem repository to indicate that those are
> > the
> > > preferred
> > > artifacts to use. Now that we have a better release available in the
> > > shared
> > > repository
> > > we'd like to invalidate the artifacts in the cache and remove these
> > files
> > > from the resolver.
> > >
> > > BOTH describe a use-case for invalidating the ivy cache for a
> particular
> > > verison.
> > >
> > > ---
> > >
> > > One possible way to do invalidate the cache would be that erasing
> > resolver
> > > I
> > > described which
> > > would only work in association with a local filesystem resolver.
> > > When I published any version, then it would invalidate that version in
> > the
> > > cache.
> > >
> > > This would allow me to use that same resolver chain
> > >
> > > // psuedo-chain
> > > <chain>
> > > <filesystem name="local" local="true"/>
> > > <url name="shared/>
> > > </chain>
> > >
> > > <invalidate name="invalidate" ref="local" erase=<<optional>> />
> > >
> > > With either approach my workflow for making local changes, regardless
> of
> > > how
> > > I choose to
> > > version, is
> > >
> > > Check out.
> > > Resolve with chain.
> > > Build.
> > > Publish to local repository.
> > > Publish to invalidate when its time to invalidate the cache.
> > >
> > > Erasing the copy of that version from the local resolver could be
> > optional
> > > to support both
> > > use cases where people do want to remove packages, and use cases where
> > > people do
> > > not.
> > >
> > > ---
> > >
> > > You said that you feared that people using this patching/overriding
> > > logic means people could make a mistake in external repositories.
> > > This can happen at any time regardless of the technique used to
> > > make local changes. This is orthogonal to using one technique or
> > > the other for local changes. Releases require care, which is another
> > > issue entirely.
> > >
> > > As to your other points 1-5 at the end or you post, this is good
> advice
> > > but orthogonal to solving the problem of making a local change and
> > > effectively invalidating the cache when its no longer needed.
> > >
> > >
> > > On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
> > > >
> > > > Eric Crahen wrote:
> > > >
> > > > > I've found that there is a bug in Y.
> > > > > Several things need to happen for me at this point:
> > > > > #1.
> > > > > I need to keep working on A. So I need to be able to patch Y.
> > > > > This means I need a local copy. Now you can label this
> > > > > Eric's-hacked-library
> > > >
> > > > This means fix the bug, change the version number or append an
> > > > additional string or whatever that match your status. Then store it
> in
> > > > your repository. Then update your references so that you force the
> use
> > > > of this specific version.
> > > >
> > > > You must strive to have y-1.1-patched-by-eric.jar > y-1.1.jar or
> > > > y-1.1-patched-by-eric.jar evicting y-1.1.jar depending your internal
> > > > policy., etc...
> > > > When a version 1.2 will be released (which hopefully will
> incorporate
> > > > your patch), then 1.2 automatically obsolete 1.1-patched-by-eric
> (unless
> > > > the version is really forced with force=true)
> > > >
> > > > When you do this dependencies are really managed rather than hacked
> > and
> > > > there is always the visual element to make sure that the version
> that
> > is
> > > > included is really the latest version and it conveys immediately the
> > > > information that this dependency is patched.
> > > >
> > > >
> > > > If we follow this patching/overriding logic, that means that people
> > > > could well do the same on external repositories and silently swap
> one
> > > > jar with another patched version because they are too lazy to do a
> > > > release, which basically means that 2 people could be using the same
> > > > version with a different code. That's brilliant to convince
> developers
> > > > to commit suicide. :)
> > > >
> > > > And this is also another reason why I don't use external
> repositories.
> > > > 1) I trust no one.
> > > > 2) dependencies are generally not well expressed
> > > > 3) versioning is sometimes not even well done correctly
> > > > 4) I keep full control of all my dependencies
> > > > 5) I have flexibility.
> > > >
> > > > -- stephane
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > - Eric
> > >
> > >
> >
> >
>
>
> --
>
> - Eric
>
>

Re: Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

One of the interesting parts of the discussion to read through when you have
the time is the difference between development and release use cases.

---

I think I would have to disagree about adding deprecation annotations being
a
dangerous change. The library behaves identically; line numbers in a stack
trace
may change, but the content and meaning and the trace is identical. The only
danger really would be relying on line numbers in the stack trace, which is
a
questionable practice to begin with.

Certainly, there are a lot of "gotchas" library owners will have to
consider, but
none of these are things that Ivy can just solve for everyone, even if you
try to
enforce a certain pattern of usage.

Each library owner will have their own set of concerns, their own
development
process, etc. I don't believe there is a one size fits all solution. It
makes sense to
provide a best practices recommendation, but I think it would be a mistake
to limit
thought to only those solutions. Ultimately, the library owner will need to
weigh these
things, understand the impact what they are doing and use Ivy as a *tool* to

accomplish their goal.

If you don't trust someone to be able to make a decision on how to release
their code,
you don't have faith they can do something as simple as associate a name and

number with it, then I'm really suprised you'd trust they could do the more
complicated
task of writing the code itself.


On 12/26/06, Xavier Hanin <xa...@gmail.com> wrote:
>
> Hello All,
>
> Interesting discussion here, I don't have time to read everything and
> catch
> u^p on the discussion, but I'd like to give my opinion:
> - I agree with Stephane and John that a version is a version and you
> should
> never change what has been released. Even adding deprecated annotations
> can
> change line numbers if you are not careful, and you will have troubles
> inspecting a stack trace if your line numbers are not correct. And this is
> just an example of all the problems you may run into when you don't
> actually
> know the exact content of a jar when you know its version
> - Even if I believe a version is a version and should never change, Ivy
> support two kind of changes in a version:
> ** the module descriptor change (i.e. a change in the ivy.xml) when you
> enable checkModified
> ** a change of the artifacts (i.e. a jar for instance), when you declare
> it
> as a changing revision (either in your ivy.xml or by using the
> changingPattern)
>
> In this last case I think that supporting cache invalidation as requested
> by
> Eric woud be consistent with the behaviour expected. And when you say that
> your revision is changing Ivy has to check if the cache is valid each
> time,
> so it wouldn't introduce a new performance overhead.
>
> Regards,
> Xavier
>
>
> On 12/24/06, Eric Crahen <er...@gmail.com> wrote:
> >
> > Thanks, I think we have just both described the need to invalidate a
> > version in the ivy cache. I'll compare the two approaches objectively:
> >
> > In both cases I'm using a chain resolver in which the first element is
> > just my local filesystem and the second is the shared repository which
> > contains offical/approved copies of all internal and external libraries.
> > In both cases all changes are visible only to me because we publish
> > only to our filesystem.
> >
> > The key parts of your description are:
> > "change the version for Y, then update your references."
> >
> > This effectively means,
> >
> > #1 checkout Y, change the version, fix it, publish it
> > #2 checkout X, change the version because I'm changing its dependencies
> >     to point to the fixed Y, publish it.
> > #3 check out D, change the version because I'm changing its dependencies
> >     to point to the fixed Y, publish it.
> > #4 check out C, change the version because I'm changing its dependencies
> >     to point to the fixed D, publish it.
> > #5 check out B, change the version because I'm changing its dependencies
> >     to point to the fixed C, publish it.
> > #6 check out A, change the version because I'm changing its dependencies
> >     to point to the fixed B, publish it.
> > #7 Hope that I chose version numbers that will become automatically
> > obsolete
> >     when the official fix is released.
> >
> > So to fix a simple bug, locally using private versions only I can see,
> in
> > 1
> > package,
> > I actually have to alter 6 ivy.xmls and perform 6 builds.
> >
> > With what I suggested I would change 2 packages.
> >
> > #1 checkout Y, fix it, publish it to the local override resolver
> > #2 checkout A, build it.
> >
> > This required changing no ivy.xmls and building only 2 packages.
> >
> > Ultimately we have accomplished the same thing.
> >
> >
> > The difference is that you preferred to use a very strict version
> scheme,
> > where as I opted to use the fact that something exists in my local
> > resolver
> > as an implicit private local version.
> >
> > The strong version comes at the cost of more work, and the amount of
> > that work (changing ivy.xml and building files) is proportional to the
> > number of packages. I would actually need to transitively enumerate
> > my dependencies and be sure that I update the versions correctly
> > in all dependencies - which may be tedious and error prone.
> >
> > The implied version uses the chain to resolver preference to let
> > ivy to the transitive traversal for me; and since this is an unnamed
> > implied version, there was no need to guess a version number.
> >
> > Both have visual cues, it depends on where you look. The cue in the
> > strong version is in the ivy.xml and the cue in the implied version is
> > existence of a file in the local repository.
> >
> > Which one is most suitable is a choice a developer is free to make. I
> > don't Ivy needs to dictate one or the other, this really is a matter of
> > preference
> > and a matter of what a developer is comfortable with. Both very possible
> > and valid solutions. There are situations when both are useful.
> > I do not want to engage in a subject discussion about which is the right
> > thing to do.
> >
> > Now here is what we have not addressed, which is the reason for the
> entire
> > thread.
> > ** How do we evict these private copies from the cache when they are no
> > longer
> > needed? **
> >
> > In the case of the strong versioning approach, part of what we have done
> > is
> > hope to have chosen versions that would be considered older than the
> > official
> > version once released. We may not have guessed right and need to remove
> > that
> > private local copy from the ivy cache. After all its lifetime is over,
> > this
> > was a private
> > local copy that should no longer exist in the cache; we may to choose to
> > keep it
> > in the local resolver where it can be selected at any time, but it
> > certainly
> > has no
> > need to be in the cache once its outlived its usefulness. Unless someone
> > never
> > makes mistakes, then maybe they will never need to invalidate the cache.
> > Maybe
> > some of us are just that good but I make mistakes, so I would really
> like
> > the ability
> > to do so when needed when I use this approach.
> >
> > In the case of the implied version approach, we used the simple fact
> that
> > artifacts
> > existed in the local filesystem repository to indicate that those are
> the
> > preferred
> > artifacts to use. Now that we have a better release available in the
> > shared
> > repository
> > we'd like to invalidate the artifacts in the cache and remove these
> files
> > from the resolver.
> >
> > BOTH describe a use-case for invalidating the ivy cache for a particular
> > verison.
> >
> > ---
> >
> > One possible way to do invalidate the cache would be that erasing
> resolver
> > I
> > described which
> > would only work in association with a local filesystem resolver.
> > When I published any version, then it would invalidate that version in
> the
> > cache.
> >
> > This would allow me to use that same resolver chain
> >
> > // psuedo-chain
> > <chain>
> > <filesystem name="local" local="true"/>
> > <url name="shared/>
> > </chain>
> >
> > <invalidate name="invalidate" ref="local" erase=<<optional>> />
> >
> > With either approach my workflow for making local changes, regardless of
> > how
> > I choose to
> > version, is
> >
> > Check out.
> > Resolve with chain.
> > Build.
> > Publish to local repository.
> > Publish to invalidate when its time to invalidate the cache.
> >
> > Erasing the copy of that version from the local resolver could be
> optional
> > to support both
> > use cases where people do want to remove packages, and use cases where
> > people do
> > not.
> >
> > ---
> >
> > You said that you feared that people using this patching/overriding
> > logic means people could make a mistake in external repositories.
> > This can happen at any time regardless of the technique used to
> > make local changes. This is orthogonal to using one technique or
> > the other for local changes. Releases require care, which is another
> > issue entirely.
> >
> > As to your other points 1-5 at the end or you post, this is good advice
> > but orthogonal to solving the problem of making a local change and
> > effectively invalidating the cache when its no longer needed.
> >
> >
> > On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
> > >
> > > Eric Crahen wrote:
> > >
> > > > I've found that there is a bug in Y.
> > > > Several things need to happen for me at this point:
> > > > #1.
> > > > I need to keep working on A. So I need to be able to patch Y.
> > > > This means I need a local copy. Now you can label this
> > > > Eric's-hacked-library
> > >
> > > This means fix the bug, change the version number or append an
> > > additional string or whatever that match your status. Then store it in
> > > your repository. Then update your references so that you force the use
> > > of this specific version.
> > >
> > > You must strive to have y-1.1-patched-by-eric.jar > y-1.1.jar or
> > > y-1.1-patched-by-eric.jar evicting y-1.1.jar depending your internal
> > > policy., etc...
> > > When a version 1.2 will be released (which hopefully will incorporate
> > > your patch), then 1.2 automatically obsolete 1.1-patched-by-eric(unless
> > > the version is really forced with force=true)
> > >
> > > When you do this dependencies are really managed rather than hacked
> and
> > > there is always the visual element to make sure that the version that
> is
> > > included is really the latest version and it conveys immediately the
> > > information that this dependency is patched.
> > >
> > >
> > > If we follow this patching/overriding logic, that means that people
> > > could well do the same on external repositories and silently swap one
> > > jar with another patched version because they are too lazy to do a
> > > release, which basically means that 2 people could be using the same
> > > version with a different code. That's brilliant to convince developers
> > > to commit suicide. :)
> > >
> > > And this is also another reason why I don't use external repositories.
> > > 1) I trust no one.
> > > 2) dependencies are generally not well expressed
> > > 3) versioning is sometimes not even well done correctly
> > > 4) I keep full control of all my dependencies
> > > 5) I have flexibility.
> > >
> > > -- stephane
> > >
> >
> >
> >
> > --
> >
> > - Eric
> >
> >
>
>


-- 

- Eric

Re: Opt out of the cache for a resolver

[Xavier Hanin <xa...@gmail.com>]

Hello All,

Interesting discussion here, I don't have time to read everything and catch
u^p on the discussion, but I'd like to give my opinion:
- I agree with Stephane and John that a version is a version and you should
never change what has been released. Even adding deprecated annotations can
change line numbers if you are not careful, and you will have troubles
inspecting a stack trace if your line numbers are not correct. And this is
just an example of all the problems you may run into when you don't actually
know the exact content of a jar when you know its version
- Even if I believe a version is a version and should never change, Ivy
support two kind of changes in a version:
** the module descriptor change (i.e. a change in the ivy.xml) when you
enable checkModified
** a change of the artifacts (i.e. a jar for instance), when you declare it
as a changing revision (either in your ivy.xml or by using the
changingPattern)

In this last case I think that supporting cache invalidation as requested by
Eric woud be consistent with the behaviour expected. And when you say that
your revision is changing Ivy has to check if the cache is valid each time,
so it wouldn't introduce a new performance overhead.

Regards,
Xavier


On 12/24/06, Eric Crahen <er...@gmail.com> wrote:
>
> Thanks, I think we have just both described the need to invalidate a
> version in the ivy cache. I'll compare the two approaches objectively:
>
> In both cases I'm using a chain resolver in which the first element is
> just my local filesystem and the second is the shared repository which
> contains offical/approved copies of all internal and external libraries.
> In both cases all changes are visible only to me because we publish
> only to our filesystem.
>
> The key parts of your description are:
> "change the version for Y, then update your references."
>
> This effectively means,
>
> #1 checkout Y, change the version, fix it, publish it
> #2 checkout X, change the version because I'm changing its dependencies
>     to point to the fixed Y, publish it.
> #3 check out D, change the version because I'm changing its dependencies
>     to point to the fixed Y, publish it.
> #4 check out C, change the version because I'm changing its dependencies
>     to point to the fixed D, publish it.
> #5 check out B, change the version because I'm changing its dependencies
>     to point to the fixed C, publish it.
> #6 check out A, change the version because I'm changing its dependencies
>     to point to the fixed B, publish it.
> #7 Hope that I chose version numbers that will become automatically
> obsolete
>     when the official fix is released.
>
> So to fix a simple bug, locally using private versions only I can see, in
> 1
> package,
> I actually have to alter 6 ivy.xmls and perform 6 builds.
>
> With what I suggested I would change 2 packages.
>
> #1 checkout Y, fix it, publish it to the local override resolver
> #2 checkout A, build it.
>
> This required changing no ivy.xmls and building only 2 packages.
>
> Ultimately we have accomplished the same thing.
>
>
> The difference is that you preferred to use a very strict version scheme,
> where as I opted to use the fact that something exists in my local
> resolver
> as an implicit private local version.
>
> The strong version comes at the cost of more work, and the amount of
> that work (changing ivy.xml and building files) is proportional to the
> number of packages. I would actually need to transitively enumerate
> my dependencies and be sure that I update the versions correctly
> in all dependencies - which may be tedious and error prone.
>
> The implied version uses the chain to resolver preference to let
> ivy to the transitive traversal for me; and since this is an unnamed
> implied version, there was no need to guess a version number.
>
> Both have visual cues, it depends on where you look. The cue in the
> strong version is in the ivy.xml and the cue in the implied version is
> existence of a file in the local repository.
>
> Which one is most suitable is a choice a developer is free to make. I
> don't Ivy needs to dictate one or the other, this really is a matter of
> preference
> and a matter of what a developer is comfortable with. Both very possible
> and valid solutions. There are situations when both are useful.
> I do not want to engage in a subject discussion about which is the right
> thing to do.
>
> Now here is what we have not addressed, which is the reason for the entire
> thread.
> ** How do we evict these private copies from the cache when they are no
> longer
> needed? **
>
> In the case of the strong versioning approach, part of what we have done
> is
> hope to have chosen versions that would be considered older than the
> official
> version once released. We may not have guessed right and need to remove
> that
> private local copy from the ivy cache. After all its lifetime is over,
> this
> was a private
> local copy that should no longer exist in the cache; we may to choose to
> keep it
> in the local resolver where it can be selected at any time, but it
> certainly
> has no
> need to be in the cache once its outlived its usefulness. Unless someone
> never
> makes mistakes, then maybe they will never need to invalidate the cache.
> Maybe
> some of us are just that good but I make mistakes, so I would really like
> the ability
> to do so when needed when I use this approach.
>
> In the case of the implied version approach, we used the simple fact that
> artifacts
> existed in the local filesystem repository to indicate that those are the
> preferred
> artifacts to use. Now that we have a better release available in the
> shared
> repository
> we'd like to invalidate the artifacts in the cache and remove these files
> from the resolver.
>
> BOTH describe a use-case for invalidating the ivy cache for a particular
> verison.
>
> ---
>
> One possible way to do invalidate the cache would be that erasing resolver
> I
> described which
> would only work in association with a local filesystem resolver.
> When I published any version, then it would invalidate that version in the
> cache.
>
> This would allow me to use that same resolver chain
>
> // psuedo-chain
> <chain>
> <filesystem name="local" local="true"/>
> <url name="shared/>
> </chain>
>
> <invalidate name="invalidate" ref="local" erase=<<optional>> />
>
> With either approach my workflow for making local changes, regardless of
> how
> I choose to
> version, is
>
> Check out.
> Resolve with chain.
> Build.
> Publish to local repository.
> Publish to invalidate when its time to invalidate the cache.
>
> Erasing the copy of that version from the local resolver could be optional
> to support both
> use cases where people do want to remove packages, and use cases where
> people do
> not.
>
> ---
>
> You said that you feared that people using this patching/overriding
> logic means people could make a mistake in external repositories.
> This can happen at any time regardless of the technique used to
> make local changes. This is orthogonal to using one technique or
> the other for local changes. Releases require care, which is another
> issue entirely.
>
> As to your other points 1-5 at the end or you post, this is good advice
> but orthogonal to solving the problem of making a local change and
> effectively invalidating the cache when its no longer needed.
>
>
> On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
> >
> > Eric Crahen wrote:
> >
> > > I've found that there is a bug in Y.
> > > Several things need to happen for me at this point:
> > > #1.
> > > I need to keep working on A. So I need to be able to patch Y.
> > > This means I need a local copy. Now you can label this
> > > Eric's-hacked-library
> >
> > This means fix the bug, change the version number or append an
> > additional string or whatever that match your status. Then store it in
> > your repository. Then update your references so that you force the use
> > of this specific version.
> >
> > You must strive to have y-1.1-patched-by-eric.jar > y-1.1.jar or
> > y-1.1-patched-by-eric.jar evicting y-1.1.jar depending your internal
> > policy., etc...
> > When a version 1.2 will be released (which hopefully will incorporate
> > your patch), then 1.2 automatically obsolete 1.1-patched-by-eric (unless
> > the version is really forced with force=true)
> >
> > When you do this dependencies are really managed rather than hacked and
> > there is always the visual element to make sure that the version that is
> > included is really the latest version and it conveys immediately the
> > information that this dependency is patched.
> >
> >
> > If we follow this patching/overriding logic, that means that people
> > could well do the same on external repositories and silently swap one
> > jar with another patched version because they are too lazy to do a
> > release, which basically means that 2 people could be using the same
> > version with a different code. That's brilliant to convince developers
> > to commit suicide. :)
> >
> > And this is also another reason why I don't use external repositories.
> > 1) I trust no one.
> > 2) dependencies are generally not well expressed
> > 3) versioning is sometimes not even well done correctly
> > 4) I keep full control of all my dependencies
> > 5) I have flexibility.
> >
> > -- stephane
> >
>
>
>
> --
>
> - Eric
>
>

Re: Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

Thanks, I think we have just both described the need to invalidate a
version in the ivy cache. I'll compare the two approaches objectively:

In both cases I'm using a chain resolver in which the first element is
just my local filesystem and the second is the shared repository which
contains offical/approved copies of all internal and external libraries.
In both cases all changes are visible only to me because we publish
only to our filesystem.

The key parts of your description are:
"change the version for Y, then update your references."

This effectively means,

#1 checkout Y, change the version, fix it, publish it
#2 checkout X, change the version because I'm changing its dependencies
     to point to the fixed Y, publish it.
#3 check out D, change the version because I'm changing its dependencies
     to point to the fixed Y, publish it.
#4 check out C, change the version because I'm changing its dependencies
     to point to the fixed D, publish it.
#5 check out B, change the version because I'm changing its dependencies
     to point to the fixed C, publish it.
#6 check out A, change the version because I'm changing its dependencies
     to point to the fixed B, publish it.
#7 Hope that I chose version numbers that will become automatically obsolete
     when the official fix is released.

So to fix a simple bug, locally using private versions only I can see, in 1
package,
I actually have to alter 6 ivy.xmls and perform 6 builds.

With what I suggested I would change 2 packages.

#1 checkout Y, fix it, publish it to the local override resolver
#2 checkout A, build it.

This required changing no ivy.xmls and building only 2 packages.

Ultimately we have accomplished the same thing.


The difference is that you preferred to use a very strict version scheme,
where as I opted to use the fact that something exists in my local resolver
as an implicit private local version.

The strong version comes at the cost of more work, and the amount of
that work (changing ivy.xml and building files) is proportional to the
number of packages. I would actually need to transitively enumerate
my dependencies and be sure that I update the versions correctly
in all dependencies - which may be tedious and error prone.

The implied version uses the chain to resolver preference to let
ivy to the transitive traversal for me; and since this is an unnamed
implied version, there was no need to guess a version number.

Both have visual cues, it depends on where you look. The cue in the
strong version is in the ivy.xml and the cue in the implied version is
existence of a file in the local repository.

Which one is most suitable is a choice a developer is free to make. I
don't Ivy needs to dictate one or the other, this really is a matter of
preference
and a matter of what a developer is comfortable with. Both very possible
and valid solutions. There are situations when both are useful.
I do not want to engage in a subject discussion about which is the right
thing to do.

Now here is what we have not addressed, which is the reason for the entire
thread.
** How do we evict these private copies from the cache when they are no
longer
needed? **

In the case of the strong versioning approach, part of what we have done is
hope to have chosen versions that would be considered older than the
official
version once released. We may not have guessed right and need to remove that
private local copy from the ivy cache. After all its lifetime is over, this
was a private
local copy that should no longer exist in the cache; we may to choose to
keep it
in the local resolver where it can be selected at any time, but it certainly
has no
need to be in the cache once its outlived its usefulness. Unless someone
never
makes mistakes, then maybe they will never need to invalidate the cache.
Maybe
some of us are just that good but I make mistakes, so I would really like
the ability
to do so when needed when I use this approach.

In the case of the implied version approach, we used the simple fact that
artifacts
existed in the local filesystem repository to indicate that those are the
preferred
artifacts to use. Now that we have a better release available in the shared
repository
we'd like to invalidate the artifacts in the cache and remove these files
from the resolver.

BOTH describe a use-case for invalidating the ivy cache for a particular
verison.

---

One possible way to do invalidate the cache would be that erasing resolver I
described which
would only work in association with a local filesystem resolver.
When I published any version, then it would invalidate that version in the
cache.

This would allow me to use that same resolver chain

// psuedo-chain
<chain>
  <filesystem name="local" local="true"/>
  <url name="shared/>
</chain>

<invalidate name="invalidate" ref="local" erase=<<optional>> />

With either approach my workflow for making local changes, regardless of how
I choose to
version, is

Check out.
Resolve with chain.
Build.
Publish to local repository.
Publish to invalidate when its time to invalidate the cache.

Erasing the copy of that version from the local resolver could be optional
to support both
use cases where people do want to remove packages, and use cases where
people do
not.

---

You said that you feared that people using this patching/overriding
logic means people could make a mistake in external repositories.
This can happen at any time regardless of the technique used to
make local changes. This is orthogonal to using one technique or
the other for local changes. Releases require care, which is another
issue entirely.

As to your other points 1-5 at the end or you post, this is good advice
but orthogonal to solving the problem of making a local change and
effectively invalidating the cache when its no longer needed.


On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
>
> Eric Crahen wrote:
>
> > I've found that there is a bug in Y.
> > Several things need to happen for me at this point:
> > #1.
> > I need to keep working on A. So I need to be able to patch Y.
> > This means I need a local copy. Now you can label this
> > Eric's-hacked-library
>
> This means fix the bug, change the version number or append an
> additional string or whatever that match your status. Then store it in
> your repository. Then update your references so that you force the use
> of this specific version.
>
> You must strive to have y-1.1-patched-by-eric.jar > y-1.1.jar or
> y-1.1-patched-by-eric.jar evicting y-1.1.jar depending your internal
> policy., etc...
> When a version 1.2 will be released (which hopefully will incorporate
> your patch), then 1.2 automatically obsolete 1.1-patched-by-eric (unless
> the version is really forced with force=true)
>
> When you do this dependencies are really managed rather than hacked and
> there is always the visual element to make sure that the version that is
> included is really the latest version and it conveys immediately the
> information that this dependency is patched.
>
>
> If we follow this patching/overriding logic, that means that people
> could well do the same on external repositories and silently swap one
> jar with another patched version because they are too lazy to do a
> release, which basically means that 2 people could be using the same
> version with a different code. That's brilliant to convince developers
> to commit suicide. :)
>
> And this is also another reason why I don't use external repositories.
> 1) I trust no one.
> 2) dependencies are generally not well expressed
> 3) versioning is sometimes not even well done correctly
> 4) I keep full control of all my dependencies
> 5) I have flexibility.
>
> -- stephane
>



-- 

- Eric

Re: Opt out of the cache for a resolver

[Stephane Bailliez <sb...@gmail.com>]

Eric Crahen wrote:

> I've found that there is a bug in Y.
> Several things need to happen for me at this point:
> #1.
> I need to keep working on A. So I need to be able to patch Y.
> This means I need a local copy. Now you can label this 
> Eric's-hacked-library

This means fix the bug, change the version number or append an 
additional string or whatever that match your status. Then store it in 
your repository. Then update your references so that you force the use 
of this specific version.

You must strive to have y-1.1-patched-by-eric.jar > y-1.1.jar or 
y-1.1-patched-by-eric.jar evicting y-1.1.jar depending your internal 
policy., etc...
When a version 1.2 will be released (which hopefully will incorporate 
your patch), then 1.2 automatically obsolete 1.1-patched-by-eric (unless 
the version is really forced with force=true)

When you do this dependencies are really managed rather than hacked and 
there is always the visual element to make sure that the version that is 
included is really the latest version and it conveys immediately the 
information that this dependency is patched.


If we follow this patching/overriding logic, that means that people 
could well do the same on external repositories and silently swap one 
jar with another patched version because they are too lazy to do a 
release, which basically means that 2 people could be using the same 
version with a different code. That's brilliant to convince developers 
to commit suicide. :)

And this is also another reason why I don't use external repositories.
1) I trust no one.
2) dependencies are generally not well expressed
3) versioning is sometimes not even well done correctly
4) I keep full control of all my dependencies
5) I have flexibility.

-- stephane

Re: Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

I don't see how the use-case of needing to use a work around while
an official patch is integrated to a package can be so easily discounted
as not valid.

Can you walk me through how I would do what you are proposing. Maybe you
have a better way to do this.

I'll change the names to protect the innocent.

I'm working on my library A, library A depends on B, library B depends C,
library C depends on D. library D depends on external A. external X depends
on
external Y.

Here is the picture form of the dependency.

A -> B -> C -> D -> X -> Y

I've inherited Y as a transitive dependency by depending on X. Both are
librariess
I do not own, and I do not have influence over the version numbers they use,
or will
use when releases are made to these libraries.

I've found that there is a bug in Y.

Several things need to happen for me at this point:

#1.

I need to keep working on A. So I need to be able to patch Y.

This means I need a local copy. Now you can label this Eric's-hacked-library

if you want. I really don't think that's a fair characterization. My intent
is to repair
the bug, and submit a patch and bug report with repro steps to the owner of
library Y.

- I need to be able to build Y locally so I can unit test Y and put the fix
together.

#2

- I need to be able to build any of my libraries against the bugfix Y in
order
to run integration tests.

The bugfix does not involve a signature change, it is binary compatible.
There is no need to recompile all the libraries, but I would like to run
integration
tests on any of A,B,C or D that I choose to. Integration tests are part of
my
build - they run after unit tests.

#3

When the bugfix I submitted is accepted, or the bug is corrected by someone
else's patch,

- I need to stop using the local bugfix version and switch back to the
current
official version.


Now in this real scenario, I don't need to make any releases of my own
libraries
for others to consume. All I need to do is unblock my progress by patching a
bug
in someone elses code. I don't want to wait and not accomplish anything for
an
unknown amount of time while an external party repairs thier code. Because
I'm
a good citizen I will report the bug and give them a fix. Ultimately, the
goal here
is that I need to keep working while the bug gets sorted out.

Can you explain how I would do this. Please walk me through very
specifically
what you are suggesting I change in the ivy.xml's. Where I publish these
changes
to. And how I remove the changes when the bug is fixed.

The three things I listed must happen. This is a requirement.

I want to do it without manually removing a thing from the cache, but since
that
cache issue really is secondary in what we are talking about now, you can
ignore
that request for the time being. (But I'll probably come back to it later)


On 12/24/06, John Gill <ll...@gmail.com> wrote:
>
> My view of the world is that a version is a version, and if you release
> something, then it is released forever, and every package that is release
> should be uniquely identifyable. So you need to given your hacked/debug
> releases a unique version number (or maybe a different name like
> Erics-hacked-commons-http version 1.1 :-) That way, when you don't want
> your
> team to use Erics-hacked-commons-http version 1.1 anymore, they simply go
> back to using commons-http version 1.1.1 or 1.2 or whatever it finally
> gets
> released as. This way, you won't have to clean out your cache.
>
>
>
> On 12/24/06, Eric Crahen <er...@gmail.com> wrote:
> >
> > When I use the copy in my override repository its the patched version.
> > When I remove the copy in my override repository it is the released
> > version.
> >
> > This is the point of a local override repository.
> > There are plenty of valid reasons for this:
> >
> > For one, schedules do not always line up. I may need to continue with
> > development efforts in parallel with some other group. Having a
> mechanism
> > to
> > allow me to use a work around while someone else integrates an offical
> > patch
> > for a bug. My development isn't on hold while an external group who is
> not
> > on the same schedule works things out.
> >
> > Another is simply to test/explore. I may have several librariess I own.
> A
> > depends on B
> > depends on C. I may need to modify a private copy of B temporarily to
> add
> > additional logging, or enable an MBean, switch a connection pool
> > implementation
> > from one thing to another. Maybe I'm simply doing this as an exercise to
> > understand
> > how some code works.
> >
> > In neither of cases do I plan on doing a release, and in neither does it
> > make sense
> > to play games with version numbers. In the first, I can't predict what
> > version number
> > the official fix for a bug will be in - will it be commons-http-1.1.1 or
> > commons-http-1.2?
> > I don't know, and its not up to me. Probably depends what other bug
> fixes
> > or
> > enhancements
> > they decide to release.
> >
> > In the second I have no reason to bump the version. I'm not making a
> > release, I'm debugging.
> > I might have inherited this dependency transitively and I don't want to
> go
> > tracking down the
> > libraries I depend on that depend on libraries, that depend on libraries
> > that depend on
> > this one library I want to add some trace to. Its a waste of time. All I
> > want to do is better
> > understand how something works - and I know the version of the thing I'm
> > trying to understand,
> > I just want to modify it locally. Having to change version/revisions for
> > this can, in the case of
> > dependencies inherited transitively, mean chasing down dependencies
> > editing
> > the ivy.xml's to
> > depend on ${bumped_version}, and publishing those into my local override
> > as
> > well. That's
> > a lot of extra steps when I know I just want to add some debugging code
> to
> > a
> > very specific
> > package version.
> >
> > There are other use cases that revolve around working on multiple
> > libraries
> > at once. LibraryA
> > my depend on LibraryB. They are both mine, and I am working bugs that
> > involve changes to both.
> > I may have multiple bugs to work on in different copies one in ~/bugfix1
> > and
> > one in ~/bugfix2. Again,
> > I don't care to bump version numbers. Perhaps a code review won't be
> > conducted until the end of
> > the week, and its part of a process that happens after that review where
> > the
> > appropriate versions
> > are decided. I really don't care about playing guessing games with
> version
> > numbers. I know the
> > library versions that cause a bad behavior and my job right now is to
> find
> > out what is wrong, and
> > propose a solution.
> >
> >
> > Having a way to override a library for these reasons is not shooting
> > myself
> > in the foot. What I've
> > described is very useful. I'm shooting myself in the foot by not
> enabling
> > this sort of behavior.
> >
> >
> > On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
> > >
> > > Eric Crahen wrote:
> > > >
> > > > The use case for this is that I want to do something, like override
> > > > commons-http client locally to work
> > > > with a bugfix that hasn't been released yet. I use a chain with a
> > local
> > > > filesystem resolver in the front
> > > > so that returnFirst="true". This way whatever I put in my local
> > > > resolve will
> > > > effectively override the
> > > > offical release. Now at some point I don't need the override - maybe
> > the
> > > > bugfix is integrated into the
> > > > offical release - maybe I just decide that bug is not relevant to me
> > > > anymore
> > > > and I want to go back to
> > > > building against the offical copy. I want to just remove the local
> > > > override
> > > > by removing that artifact and
> > > > ivy file from my local filesystem. This fails because the cache is
> > never
> > > > invalidated correctly.
> > > So you will end up with commons-http-1.1.jar and commons-http-1.1.jar?
> > > which one is the genuine 1.1 ? the first one or the second one ?
> > >
> > > Don't do that. It is a crazy idea to hack the jar and silently swap it
> > > while keeping the same version.
> > > You are just shooting yourself in the foot by doing that.
> > >
> > > -- stephane
> > >
> > >
> >
> >
> > --
> >
> > - Eric
> >
> >
>
>
> --
> Regards,
> John Gill
>
>


-- 

- Eric

Re: Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

Also to go back to a package being forever released as a side thread.

I may have many small releases of libraries within my organization. The
reality is these packages have a lifetime and will be deprecated. They may
exist forever, but I will not support them forever. I mark classes in
deprecated
versions @Deprecated to warn those who still depend on them they are on
their own.

I want to just re-release this package version with those annotations to do
this
I can do that via the overwrite property on release quite easily.

What I can't do is invalidate peoples caches. I already have a very fast
network, or a replicated shared filesystem, so I am willing to accept the
cost
of doing a checksum to invalidate my cache or not. Consistency is important
for my use case, and I really don't see a strong reason not to enable such
behavior. This need not be the default behavior, but would be available to
those who need it. Those who don't care to deprecate package are unaffected.


On 12/24/06, John Gill <ll...@gmail.com> wrote:
>
> My view of the world is that a version is a version, and if you release
> something, then it is released forever, and every package that is release
> should be uniquely identifyable. So you need to given your hacked/debug
> releases a unique version number (or maybe a different name like
> Erics-hacked-commons-http version 1.1 :-) That way, when you don't want
> your
> team to use Erics-hacked-commons-http version 1.1 anymore, they simply go
> back to using commons-http version 1.1.1 or 1.2 or whatever it finally
> gets
> released as. This way, you won't have to clean out your cache.
>
>
>
> On 12/24/06, Eric Crahen <er...@gmail.com> wrote:
> >
> > When I use the copy in my override repository its the patched version.
> > When I remove the copy in my override repository it is the released
> > version.
> >
> > This is the point of a local override repository.
> > There are plenty of valid reasons for this:
> >
> > For one, schedules do not always line up. I may need to continue with
> > development efforts in parallel with some other group. Having a
> mechanism
> > to
> > allow me to use a work around while someone else integrates an offical
> > patch
> > for a bug. My development isn't on hold while an external group who is
> not
> > on the same schedule works things out.
> >
> > Another is simply to test/explore. I may have several librariess I own.
> A
> > depends on B
> > depends on C. I may need to modify a private copy of B temporarily to
> add
> > additional logging, or enable an MBean, switch a connection pool
> > implementation
> > from one thing to another. Maybe I'm simply doing this as an exercise to
> > understand
> > how some code works.
> >
> > In neither of cases do I plan on doing a release, and in neither does it
> > make sense
> > to play games with version numbers. In the first, I can't predict what
> > version number
> > the official fix for a bug will be in - will it be commons-http-1.1.1 or
> > commons-http-1.2?
> > I don't know, and its not up to me. Probably depends what other bug
> fixes
> > or
> > enhancements
> > they decide to release.
> >
> > In the second I have no reason to bump the version. I'm not making a
> > release, I'm debugging.
> > I might have inherited this dependency transitively and I don't want to
> go
> > tracking down the
> > libraries I depend on that depend on libraries, that depend on libraries
> > that depend on
> > this one library I want to add some trace to. Its a waste of time. All I
> > want to do is better
> > understand how something works - and I know the version of the thing I'm
> > trying to understand,
> > I just want to modify it locally. Having to change version/revisions for
> > this can, in the case of
> > dependencies inherited transitively, mean chasing down dependencies
> > editing
> > the ivy.xml's to
> > depend on ${bumped_version}, and publishing those into my local override
> > as
> > well. That's
> > a lot of extra steps when I know I just want to add some debugging code
> to
> > a
> > very specific
> > package version.
> >
> > There are other use cases that revolve around working on multiple
> > libraries
> > at once. LibraryA
> > my depend on LibraryB. They are both mine, and I am working bugs that
> > involve changes to both.
> > I may have multiple bugs to work on in different copies one in ~/bugfix1
> > and
> > one in ~/bugfix2. Again,
> > I don't care to bump version numbers. Perhaps a code review won't be
> > conducted until the end of
> > the week, and its part of a process that happens after that review where
> > the
> > appropriate versions
> > are decided. I really don't care about playing guessing games with
> version
> > numbers. I know the
> > library versions that cause a bad behavior and my job right now is to
> find
> > out what is wrong, and
> > propose a solution.
> >
> >
> > Having a way to override a library for these reasons is not shooting
> > myself
> > in the foot. What I've
> > described is very useful. I'm shooting myself in the foot by not
> enabling
> > this sort of behavior.
> >
> >
> > On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
> > >
> > > Eric Crahen wrote:
> > > >
> > > > The use case for this is that I want to do something, like override
> > > > commons-http client locally to work
> > > > with a bugfix that hasn't been released yet. I use a chain with a
> > local
> > > > filesystem resolver in the front
> > > > so that returnFirst="true". This way whatever I put in my local
> > > > resolve will
> > > > effectively override the
> > > > offical release. Now at some point I don't need the override - maybe
> > the
> > > > bugfix is integrated into the
> > > > offical release - maybe I just decide that bug is not relevant to me
> > > > anymore
> > > > and I want to go back to
> > > > building against the offical copy. I want to just remove the local
> > > > override
> > > > by removing that artifact and
> > > > ivy file from my local filesystem. This fails because the cache is
> > never
> > > > invalidated correctly.
> > > So you will end up with commons-http-1.1.jar and commons-http-1.1.jar?
> > > which one is the genuine 1.1 ? the first one or the second one ?
> > >
> > > Don't do that. It is a crazy idea to hack the jar and silently swap it
> > > while keeping the same version.
> > > You are just shooting yourself in the foot by doing that.
> > >
> > > -- stephane
> > >
> > >
> >
> >
> > --
> >
> > - Eric
> >
> >
>
>
> --
> Regards,
> John Gill
>
>


-- 

- Eric

Re: Opt out of the cache for a resolver

[John Gill <ll...@gmail.com>]

My view of the world is that a version is a version, and if you release
something, then it is released forever, and every package that is release
should be uniquely identifyable. So you need to given your hacked/debug
releases a unique version number (or maybe a different name like
Erics-hacked-commons-http version 1.1 :-) That way, when you don't want your
team to use Erics-hacked-commons-http version 1.1 anymore, they simply go
back to using commons-http version 1.1.1 or 1.2 or whatever it finally gets
released as. This way, you won't have to clean out your cache.



On 12/24/06, Eric Crahen <er...@gmail.com> wrote:
>
> When I use the copy in my override repository its the patched version.
> When I remove the copy in my override repository it is the released
> version.
>
> This is the point of a local override repository.
> There are plenty of valid reasons for this:
>
> For one, schedules do not always line up. I may need to continue with
> development efforts in parallel with some other group. Having a mechanism
> to
> allow me to use a work around while someone else integrates an offical
> patch
> for a bug. My development isn't on hold while an external group who is not
> on the same schedule works things out.
>
> Another is simply to test/explore. I may have several librariess I own. A
> depends on B
> depends on C. I may need to modify a private copy of B temporarily to add
> additional logging, or enable an MBean, switch a connection pool
> implementation
> from one thing to another. Maybe I'm simply doing this as an exercise to
> understand
> how some code works.
>
> In neither of cases do I plan on doing a release, and in neither does it
> make sense
> to play games with version numbers. In the first, I can't predict what
> version number
> the official fix for a bug will be in - will it be commons-http-1.1.1 or
> commons-http-1.2?
> I don't know, and its not up to me. Probably depends what other bug fixes
> or
> enhancements
> they decide to release.
>
> In the second I have no reason to bump the version. I'm not making a
> release, I'm debugging.
> I might have inherited this dependency transitively and I don't want to go
> tracking down the
> libraries I depend on that depend on libraries, that depend on libraries
> that depend on
> this one library I want to add some trace to. Its a waste of time. All I
> want to do is better
> understand how something works - and I know the version of the thing I'm
> trying to understand,
> I just want to modify it locally. Having to change version/revisions for
> this can, in the case of
> dependencies inherited transitively, mean chasing down dependencies
> editing
> the ivy.xml's to
> depend on ${bumped_version}, and publishing those into my local override
> as
> well. That's
> a lot of extra steps when I know I just want to add some debugging code to
> a
> very specific
> package version.
>
> There are other use cases that revolve around working on multiple
> libraries
> at once. LibraryA
> my depend on LibraryB. They are both mine, and I am working bugs that
> involve changes to both.
> I may have multiple bugs to work on in different copies one in ~/bugfix1
> and
> one in ~/bugfix2. Again,
> I don't care to bump version numbers. Perhaps a code review won't be
> conducted until the end of
> the week, and its part of a process that happens after that review where
> the
> appropriate versions
> are decided. I really don't care about playing guessing games with version
> numbers. I know the
> library versions that cause a bad behavior and my job right now is to find
> out what is wrong, and
> propose a solution.
>
>
> Having a way to override a library for these reasons is not shooting
> myself
> in the foot. What I've
> described is very useful. I'm shooting myself in the foot by not enabling
> this sort of behavior.
>
>
> On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
> >
> > Eric Crahen wrote:
> > >
> > > The use case for this is that I want to do something, like override
> > > commons-http client locally to work
> > > with a bugfix that hasn't been released yet. I use a chain with a
> local
> > > filesystem resolver in the front
> > > so that returnFirst="true". This way whatever I put in my local
> > > resolve will
> > > effectively override the
> > > offical release. Now at some point I don't need the override - maybe
> the
> > > bugfix is integrated into the
> > > offical release - maybe I just decide that bug is not relevant to me
> > > anymore
> > > and I want to go back to
> > > building against the offical copy. I want to just remove the local
> > > override
> > > by removing that artifact and
> > > ivy file from my local filesystem. This fails because the cache is
> never
> > > invalidated correctly.
> > So you will end up with commons-http-1.1.jar and commons-http-1.1.jar ?
> > which one is the genuine 1.1 ? the first one or the second one ?
> >
> > Don't do that. It is a crazy idea to hack the jar and silently swap it
> > while keeping the same version.
> > You are just shooting yourself in the foot by doing that.
> >
> > -- stephane
> >
> >
>
>
> --
>
> - Eric
>
>


-- 
Regards,
John Gill

Re: Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

When I use the copy in my override repository its the patched version.
When I remove the copy in my override repository it is the released version.

This is the point of a local override repository.
There are plenty of valid reasons for this:

For one, schedules do not always line up. I may need to continue with
development efforts in parallel with some other group. Having a mechanism to
allow me to use a work around while someone else integrates an offical patch
for a bug. My development isn't on hold while an external group who is not
on the same schedule works things out.

Another is simply to test/explore. I may have several librariess I own. A
depends on B
depends on C. I may need to modify a private copy of B temporarily to add
additional logging, or enable an MBean, switch a connection pool
implementation
from one thing to another. Maybe I'm simply doing this as an exercise to
understand
how some code works.

In neither of cases do I plan on doing a release, and in neither does it
make sense
to play games with version numbers. In the first, I can't predict what
version number
the official fix for a bug will be in - will it be commons-http-1.1.1 or
commons-http-1.2?
I don't know, and its not up to me. Probably depends what other bug fixes or
enhancements
they decide to release.

In the second I have no reason to bump the version. I'm not making a
release, I'm debugging.
I might have inherited this dependency transitively and I don't want to go
tracking down the
libraries I depend on that depend on libraries, that depend on libraries
that depend on
this one library I want to add some trace to. Its a waste of time. All I
want to do is better
understand how something works - and I know the version of the thing I'm
trying to understand,
I just want to modify it locally. Having to change version/revisions for
this can, in the case of
dependencies inherited transitively, mean chasing down dependencies editing
the ivy.xml's to
depend on ${bumped_version}, and publishing those into my local override as
well. That's
a lot of extra steps when I know I just want to add some debugging code to a
very specific
package version.

There are other use cases that revolve around working on multiple libraries
at once. LibraryA
my depend on LibraryB. They are both mine, and I am working bugs that
involve changes to both.
I may have multiple bugs to work on in different copies one in ~/bugfix1 and
one in ~/bugfix2. Again,
I don't care to bump version numbers. Perhaps a code review won't be
conducted until the end of
the week, and its part of a process that happens after that review where the
appropriate versions
are decided. I really don't care about playing guessing games with version
numbers. I know the
library versions that cause a bad behavior and my job right now is to find
out what is wrong, and
propose a solution.


Having a way to override a library for these reasons is not shooting myself
in the foot. What I've
described is very useful. I'm shooting myself in the foot by not enabling
this sort of behavior.


On 12/24/06, Stephane Bailliez <sb...@gmail.com> wrote:
>
> Eric Crahen wrote:
> >
> > The use case for this is that I want to do something, like override
> > commons-http client locally to work
> > with a bugfix that hasn't been released yet. I use a chain with a local
> > filesystem resolver in the front
> > so that returnFirst="true". This way whatever I put in my local
> > resolve will
> > effectively override the
> > offical release. Now at some point I don't need the override - maybe the
> > bugfix is integrated into the
> > offical release - maybe I just decide that bug is not relevant to me
> > anymore
> > and I want to go back to
> > building against the offical copy. I want to just remove the local
> > override
> > by removing that artifact and
> > ivy file from my local filesystem. This fails because the cache is never
> > invalidated correctly.
> So you will end up with commons-http-1.1.jar and commons-http-1.1.jar ?
> which one is the genuine 1.1 ? the first one or the second one ?
>
> Don't do that. It is a crazy idea to hack the jar and silently swap it
> while keeping the same version.
> You are just shooting yourself in the foot by doing that.
>
> -- stephane
>
>


-- 

- Eric

Re: Opt out of the cache for a resolver

[Stephane Bailliez <sb...@gmail.com>]

Eric Crahen wrote:
>
> The use case for this is that I want to do something, like override
> commons-http client locally to work
> with a bugfix that hasn't been released yet. I use a chain with a local
> filesystem resolver in the front
> so that returnFirst="true". This way whatever I put in my local 
> resolve will
> effectively override the
> offical release. Now at some point I don't need the override - maybe the
> bugfix is integrated into the
> offical release - maybe I just decide that bug is not relevant to me 
> anymore
> and I want to go back to
> building against the offical copy. I want to just remove the local 
> override
> by removing that artifact and
> ivy file from my local filesystem. This fails because the cache is never
> invalidated correctly.
So you will end up with commons-http-1.1.jar and commons-http-1.1.jar ?
which one is the genuine 1.1 ? the first one or the second one ?

Don't do that. It is a crazy idea to hack the jar and silently swap it 
while keeping the same version.
You are just shooting yourself in the foot by doing that.

-- stephane

Re: Opt out of the cache for a resolver

[Eric Crahen <er...@gmail.com>]

Here is a simple reproduction:

I have two packages, Example1 and Example2, both just create and publish a
dummy jar
to a local filesystem resolver in ~/.ivy/repository

Both use a copy of this build.xml snippet to interact with ivy.

build.xml:

    <ivy:resolve   useOrigin="true"/>
    <ivy:cachepath pathid="compile.classpath"
      useOrigin="true" conf="java.compile"/>
    <ivy:cachepath pathid="test.classpath"
      useOrigin="true" conf="java.test"/>
    <ivy:cachepath pathid="runtime.classpath"
      useOrigin="true" conf="java.runtime"/>

    <ivy:deliver status="release"/>

    <mkdir dir="${build.dir}/lib"/>
    <jar destfile="${build.dir}/lib/${ant.project.name}.jar"/>

    <ivy:publish resolver="override"/>

This is the relevant ivyconfig.xml shared by both:

ivyconfig.xml:

    <filesystem name="override" local="true"
alwaysCheckExactRevision="false">
      <ivy
        pattern="${local.repository.dir}/${ivy.pattern}" />
      <artifact
        pattern="${local.repository.dir}/${ivy.pattern}" />
    </filesystem>

Example ivy.xml:

<ivy-module version="1.0">

  <info organisation="example" module="${ant.project.name}" revision="0.0.1"
status="integration"/>

  <configurations>
    <conf name="java.compile" visibility="public"/>
    <conf name="java.test"    extends="java.compile" visibility="private"/>
    <conf name="java.runtime" extends="java.compile" visibility="private"/>
  </configurations>

  <publications>
    <artifact name="${ant.project.name}" type="lib" ext="jar" conf="
java.runtime"/>
  </publications>

  <dependencies>
  </dependencies>

</ivy-module>

Example2 ivy.xml:

<ivy-module version="1.0">

  <info organisation="example" module="${ant.project.name}" revision="0.0.1"
status="integration"/>

  <configurations>
    <conf name="java.compile" visibility="public"/>
    <conf name="java.test"    extends="java.compile" visibility="private"/>
    <conf name="java.runtime" extends="java.compile" visibility="private"/>
  </configurations>

  <publications>
    <artifact name="${ant.project.name}" type="lib" ext="jar" conf="
java.runtime"/>
  </publications>

  <dependencies>
    <dependency org="example" name="Example" rev="0.0.1" conf="*->*"/>
  </dependencies>

</ivy-module>

---

Building Example1 results in this:

cd ~/.ivy && find
./repository
./repository/example
./repository/example/Example
./repository/example/Example/0.0.1
./repository/example/Example/0.0.1/ivy
./repository/example/Example/0.0.1/ivy/ivy.xml.sha1
./repository/example/Example/0.0.1/ivy/ivy.xml.md5
./repository/example/Example/0.0.1/ivy/ivy.xml
./repository/example/Example/0.0.1/lib
./repository/example/Example/0.0.1/lib/Example.jar
./repository/example/Example/0.0.1/lib/Example.jar.sha1
./repository/example/Example/0.0.1/lib/Example.jar.md5
./cache
./cache/ivy-report.css
./cache/example-Example-java.compile.xml
./cache/example-Example-java.runtime.xml
./cache/resolved-example-Example-0.0.1.properties
./cache/example-Example-java.test.xml
./cache/resolved-example-Example-0.0.1.xml
./cache/ivy-report.xsl

I see the artifacts published to my filesystem resolver because they appear
under repository.
Now I build Example2

cd ~/.ivy && find
./repository
./repository/example
./repository/example/Example
./repository/example/Example/0.0.1
./repository/example/Example/0.0.1/ivy
./repository/example/Example/0.0.1/ivy/ivy.xml.sha1
./repository/example/Example/0.0.1/ivy/ivy.xml.md5
./repository/example/Example/0.0.1/ivy/ivy.xml
./repository/example/Example/0.0.1/lib
./repository/example/Example/0.0.1/lib/Example.jar
./repository/example/Example/0.0.1/lib/Example.jar.sha1
./repository/example/Example/0.0.1/lib/Example.jar.md5
./repository/example/Example2
./repository/example/Example2/0.0.1
./repository/example/Example2/0.0.1/ivy
./repository/example/Example2/0.0.1/ivy/ivy.xml.sha1
./repository/example/Example2/0.0.1/ivy/ivy.xml.md5
./repository/example/Example2/0.0.1/ivy/ivy.xml
./repository/example/Example2/0.0.1/lib
./repository/example/Example2/0.0.1/lib/Example2.jar.sha1
./repository/example/Example2/0.0.1/lib/Example2.jar.md5
./repository/example/Example2/0.0.1/lib/Example2.jar
./cache
./cache/ivy-report.css
./cache/example-Example-java.compile.xml
./cache/example-Example-java.runtime.xml
./cache/resolved-example-Example-0.0.1.properties
./cache/example-Example-java.test.xml
./cache/resolved-example-Example2-0.0.1.properties
./cache/example-Example2-java.compile.xml
./cache/resolved-example-Example-0.0.1.xml
./cache/example-Example2-java.runtime.xml
./cache/example
./cache/example/Example
./cache/example/Example/ivydata-0.0.1.properties
./cache/example/Example/ivy-0.0.1.xml
./cache/example-Example2-java.test.xml
./cache/ivy-report.xsl
./cache/resolved-example-Example2-0.0.1.xml


I see the artifacts published to my filesystem resolver because they appear
under repository.
HOWEVER, I also see that my cache was populated with the descriptor from the
first project.
-> ./cache/example/Example
-> ./cache/example/Example/ivydata-0.0.1.properties
-> ./cache/example/Example/ivy-0.0.1.xml

Now when I remove Example from my repository and build Example2 again, Ivy
resolves
Example correctly, even though it has actually been removed.

rm -r ~/.ivy/repository/example/Example

It does this because the cache doesn't get invalidated. When the resolve
task runs during the
second build of Example2 in which I've removed the artifacts and ivy files
from the filesystem
repository by force, it seems to only consult the cache and actually says
"found [ example | Example
| 0.0.1 ] in cache" which doesn't make sense since I've specified
useOrigin=true.

----

The use case for this is that I want to do something, like override
commons-http client locally to work
with a bugfix that hasn't been released yet. I use a chain with a local
filesystem resolver in the front
so that returnFirst="true". This way whatever I put in my local resolve will
effectively override the
offical release. Now at some point I don't need the override - maybe the
bugfix is integrated into the
offical release - maybe I just decide that bug is not relevant to me anymore
and I want to go back to
building against the offical copy. I want to just remove the local override
by removing that artifact and
ivy file from my local filesystem. This fails because the cache is never
invalidated correctly.

The work around is nuke the cache, which isn't ideal. Ultimately I'd like to
never be typing rm to use
ivy effectively which is why this isn't ideal. I don't want to manually be
blasting things in ivy's cache.
Not having to manually blast things in my local repository is what the
https://issues.apache.org/jira/browse/IVY-358 request is for.




On 12/23/06, Eric Crahen <er...@gmail.com> wrote:
>
> Is there a way I can fail to resolve a dependency if it does not appear in
> the remote repository?
> For instance,
>
> If I resolve against a <url/> resolver, and I find the dependency there, I
> download it, my cache is
> updated and I'll use the copy from my cache.
>
> Now if that dependency is removed from the <url/> resolver and I resolve a
> second time, I fail to
> resolve the dependency, but I don't seem to invalidate the cache. So even
> though the artifact no
> longer exists in the repository I will resolve it out of my cache. I'll
> even print a message like
> "found [ whatever ] in
> repository-that-acutally-no-longer-has-this-dependency"
>
>
> --
>
> - Eric




-- 

- Eric