Dealing with deleted revisions

[John Williams <jr...@pobox.com>]

I've found a somewhat common situation that Ivy deals with very
poorly.  I have a fix in mind, but it involves extending the ivy.xml
syntax.

The problem occurs when a required revision of a module is deleted,
such as when a developer decides to get rid of his/her local version
of a module and go back to using a shared version.  The problem arises
because I've decided (through painful experience) that local
publications should use different revision numbers from shared
publications.

Suppose there are 3 modules, A, B, and C, and each has a 1.0-local
version in the local repository and a 1.0 version in the shared
repository.  Also, C depends on B with rev=1+, and B depends on A with
rev=1+.  Now, suppose I decide I'm done working on module A, so I
delete the local revision.  Now when I try to resolve module C, it
fails because the published ivy.xml of B depends on rev 1.0-local of
A, which no longer exists.

Ideally, I'd like Ivy to use rev 1.0 of A, which would be allowed by
the ivy.xml file for B, but the *published* ivy.xml for B has the
revision of A changed from 1+ to 1.0-local, so Ivy has no way of
knowing that 1.0 is an acceptable revision.  The change I want to make
in the ivy.xml files is that each dependency in a published ivy.xml
should have *two* revisions--one containing the actual revision in use
at the time of publication, and one copied verbatim from the
hand-written ivy.xml.  During resolution, the actual revision would
always be preferred, and the original pattern would be used as a
backup when the original revision is not available.

--jw

Re: Dealing with deleted revisions

[Xavier Hanin <xa...@gmail.com>]

On 1/31/07, John Williams <jr...@pobox.com> wrote:
>
> On 1/31/07, Xavier Hanin <xa...@gmail.com> wrote:
> > The problem is quite common, and the idea is interesting... However
> > I'm not sure to share your vision about the solution. What I don't
> > like is to split the version constraint in two separate attributes.
> > What you want is a mechanism to express a version constraint like
> > "take 1.0-local, and if it doesn't exist, take 1+". For me this is
> > only one version constraint, and I'd prefer expressing it as one
> > expression in the rev attribute. If you think about it, this is very
> > similar to the fallback mechanism used for configurations. If we
> > follow the same notation it would be rev="1.0-local(1+)".
> >
> > One advantage I see to this solution is that it may be done without
> > even modifying Ivy: create your own VersionMatcher, recognising this
> > syntax, and you're done for the resolve part. And to put use this
> > notation during deliver, using a custom
> > PublishingDependencyRevisionResolver should be ok.
> >
> > Now I don't think this should be the default behaviour, even if it
> > could be useful for others. Maybe something using the status to do
> > that only if the dependency may be deleted (if we say for example that
> > an integration version may be deleted, but not a milestone or a
> > release).
> >
> > What do you think?
>
> Keeping a single 'rev' attribute does seem like a better choice,
> though for the syntax I'd rather see a comma-separated list to match
> how configs are done.


The fallback mechanism in confs use parenthesis. But I'm not opposed to
another syntax.

When you say "without even modifying Ivy", are you suggesting that
> users of Ivy should implement the functionality themselves, or just
> saying how you'd want the feature to be implemented in the Ivy
> distribution?


I say that this can be developed and tested without modifying Ivy, then if
the community is ok to adopt it in the Ivy distribution, it would better be
part of the official distrib.

I really think this behavior should be available out of
> the box, even if it's not the default.


I agree. Another advantage I see is that people could use the fallback
version constraint even on purpose, I mean outside what would be done by the
deliver

I agree with your suggestion of assuming integration versions can be
> deleted but not other versions, but I think including both the exact
> revision number and the revision pattern is always the right thing to
> do when publishing, since the resolver can always ignore anything but
> the exact revision.


I'm not sure to agree, this require more thoughts. I don't like having
dynamic version constraints on ivy files in the repository. In this case
it's special, since the version constraint is dynamic only in a special
case. So I don't know. If we can configure the status for which the fallback
version should be used, I think it's more flexible and not too difficult to
achieve.

BTW, note that as Eric said, the whole problem could be addressed
differently with a more flexible cache and local repository management. But
I don't know if this means we shouldn't attempt to go in this direction
(which has the merit to be pretty simple to implement).

- Xavier

jw
>

Re: Dealing with deleted revisions

[John Williams <jr...@pobox.com>]

On 1/31/07, Xavier Hanin <xa...@gmail.com> wrote:
> The problem is quite common, and the idea is interesting... However
> I'm not sure to share your vision about the solution. What I don't
> like is to split the version constraint in two separate attributes.
> What you want is a mechanism to express a version constraint like
> "take 1.0-local, and if it doesn't exist, take 1+". For me this is
> only one version constraint, and I'd prefer expressing it as one
> expression in the rev attribute. If you think about it, this is very
> similar to the fallback mechanism used for configurations. If we
> follow the same notation it would be rev="1.0-local(1+)".
>
> One advantage I see to this solution is that it may be done without
> even modifying Ivy: create your own VersionMatcher, recognising this
> syntax, and you're done for the resolve part. And to put use this
> notation during deliver, using a custom
> PublishingDependencyRevisionResolver should be ok.
>
> Now I don't think this should be the default behaviour, even if it
> could be useful for others. Maybe something using the status to do
> that only if the dependency may be deleted (if we say for example that
> an integration version may be deleted, but not a milestone or a
> release).
>
> What do you think?

Keeping a single 'rev' attribute does seem like a better choice,
though for the syntax I'd rather see a comma-separated list to match
how configs are done.

When you say "without even modifying Ivy", are you suggesting that
users of Ivy should implement the functionality themselves, or just
saying how you'd want the feature to be implemented in the Ivy
distribution?  I really think this behavior should be available out of
the box, even if it's not the default.

I agree with your suggestion of assuming integration versions can be
deleted but not other versions, but I think including both the exact
revision number and the revision pattern is always the right thing to
do when publishing, since the resolver can always ignore anything but
the exact revision.

jw

Re: Dealing with deleted revisions

[Eric Crahen <er...@gmail.com>]

On 1/31/07, Xavier Hanin <xa...@gmail.com> wrote:
>
> Yes, the use of decorators for caching would address this issue in a
> much cleaner way, even if I'm still not sure to see the complete
> picture. But the more I think about it, the more I think it would
> really be nice to have this new cache management in the next version.
> What do you think?
>

I would love nothing more than to see this in the new version. I'm currently
awaiting approval from employer to contribute to this project. Once it comes
through I hope to be able to assist somewhat with this instead of just
begging =)


-- 

- Eric

Re: Dealing with deleted revisions

[Xavier Hanin <xa...@gmail.com>]

On 1/31/07, Eric Crahen <er...@gmail.com> wrote:
> I think this would another issue that could be resolved by the decorators
> for caching.
> Some of the posts I've made before about doing local overrides go into more
> depth on approaches to that w/o changing local revision numbers and the
> cache control would address the gotchas.
Yes, the use of decorators for caching would address this issue in a
much cleaner way, even if I'm still not sure to see the complete
picture. But the more I think about it, the more I think it would
really be nice to have this new cache management in the next version.
What do you think?

- Xavier

>
> On 1/31/07, Xavier Hanin <xa...@gmail.com> wrote:
> >
> > On 1/31/07, John Williams <jr...@pobox.com> wrote:
> > > I've found a somewhat common situation that Ivy deals with very
> > > poorly.  I have a fix in mind, but it involves extending the ivy.xml
> > > syntax.
> > >
> > > The problem occurs when a required revision of a module is deleted,
> > > such as when a developer decides to get rid of his/her local version
> > > of a module and go back to using a shared version.  The problem arises
> > > because I've decided (through painful experience) that local
> > > publications should use different revision numbers from shared
> > > publications.
> > >
> > > Suppose there are 3 modules, A, B, and C, and each has a 1.0-local
> > > version in the local repository and a 1.0 version in the shared
> > > repository.  Also, C depends on B with rev=1+, and B depends on A with
> > > rev=1+.  Now, suppose I decide I'm done working on module A, so I
> > > delete the local revision.  Now when I try to resolve module C, it
> > > fails because the published ivy.xml of B depends on rev 1.0-local of
> > > A, which no longer exists.
> > >
> > > Ideally, I'd like Ivy to use rev 1.0 of A, which would be allowed by
> > > the ivy.xml file for B, but the *published* ivy.xml for B has the
> > > revision of A changed from 1+ to 1.0-local, so Ivy has no way of
> > > knowing that 1.0 is an acceptable revision.  The change I want to make
> > > in the ivy.xml files is that each dependency in a published ivy.xml
> > > should have *two* revisions--one containing the actual revision in use
> > > at the time of publication, and one copied verbatim from the
> > > hand-written ivy.xml.  During resolution, the actual revision would
> > > always be preferred, and the original pattern would be used as a
> > > backup when the original revision is not available.
> >
> > The problem is quite common, and the idea is interesting... However
> > I'm not sure to share your vision about the solution. What I don't
> > like is to split the version constraint in two separate attributes.
> > What you want is a mechanism to express a version constraint like
> > "take 1.0-local, and if it doesn't exist, take 1+". For me this is
> > only one version constraint, and I'd prefer expressing it as one
> > expression in the rev attribute. If you think about it, this is very
> > similar to the fallback mechanism used for configurations. If we
> > follow the same notation it would be rev="1.0-local(1+)".
> >
> > One advantage I see to this solution is that it may be done without
> > even modifying Ivy: create your own VersionMatcher, recognising this
> > syntax, and you're done for the resolve part. And to put use this
> > notation during deliver, using a custom
> > PublishingDependencyRevisionResolver should be ok.
> >
> > Now I don't think this should be the default behaviour, even if it
> > could be useful for others. Maybe something using the status to do
> > that only if the dependency may be deleted (if we say for example that
> > an integration version may be deleted, but not a milestone or a
> > release).
> >
> > What do you think?
> >
> > Xavier
> >
> > >
> > > --jw
> > >
> >
>
>
>
> --
>
> - Eric
>
>

Re: Dealing with deleted revisions

[Eric Crahen <er...@gmail.com>]

I think this would another issue that could be resolved by the decorators
for caching.
Some of the posts I've made before about doing local overrides go into more
depth on approaches to that w/o changing local revision numbers and the
cache control would address the gotchas.

On 1/31/07, Xavier Hanin <xa...@gmail.com> wrote:
>
> On 1/31/07, John Williams <jr...@pobox.com> wrote:
> > I've found a somewhat common situation that Ivy deals with very
> > poorly.  I have a fix in mind, but it involves extending the ivy.xml
> > syntax.
> >
> > The problem occurs when a required revision of a module is deleted,
> > such as when a developer decides to get rid of his/her local version
> > of a module and go back to using a shared version.  The problem arises
> > because I've decided (through painful experience) that local
> > publications should use different revision numbers from shared
> > publications.
> >
> > Suppose there are 3 modules, A, B, and C, and each has a 1.0-local
> > version in the local repository and a 1.0 version in the shared
> > repository.  Also, C depends on B with rev=1+, and B depends on A with
> > rev=1+.  Now, suppose I decide I'm done working on module A, so I
> > delete the local revision.  Now when I try to resolve module C, it
> > fails because the published ivy.xml of B depends on rev 1.0-local of
> > A, which no longer exists.
> >
> > Ideally, I'd like Ivy to use rev 1.0 of A, which would be allowed by
> > the ivy.xml file for B, but the *published* ivy.xml for B has the
> > revision of A changed from 1+ to 1.0-local, so Ivy has no way of
> > knowing that 1.0 is an acceptable revision.  The change I want to make
> > in the ivy.xml files is that each dependency in a published ivy.xml
> > should have *two* revisions--one containing the actual revision in use
> > at the time of publication, and one copied verbatim from the
> > hand-written ivy.xml.  During resolution, the actual revision would
> > always be preferred, and the original pattern would be used as a
> > backup when the original revision is not available.
>
> The problem is quite common, and the idea is interesting... However
> I'm not sure to share your vision about the solution. What I don't
> like is to split the version constraint in two separate attributes.
> What you want is a mechanism to express a version constraint like
> "take 1.0-local, and if it doesn't exist, take 1+". For me this is
> only one version constraint, and I'd prefer expressing it as one
> expression in the rev attribute. If you think about it, this is very
> similar to the fallback mechanism used for configurations. If we
> follow the same notation it would be rev="1.0-local(1+)".
>
> One advantage I see to this solution is that it may be done without
> even modifying Ivy: create your own VersionMatcher, recognising this
> syntax, and you're done for the resolve part. And to put use this
> notation during deliver, using a custom
> PublishingDependencyRevisionResolver should be ok.
>
> Now I don't think this should be the default behaviour, even if it
> could be useful for others. Maybe something using the status to do
> that only if the dependency may be deleted (if we say for example that
> an integration version may be deleted, but not a milestone or a
> release).
>
> What do you think?
>
> Xavier
>
> >
> > --jw
> >
>



-- 

- Eric

Re: Dealing with deleted revisions

[Xavier Hanin <xa...@gmail.com>]

On 1/31/07, John Williams <jr...@pobox.com> wrote:
> I've found a somewhat common situation that Ivy deals with very
> poorly.  I have a fix in mind, but it involves extending the ivy.xml
> syntax.
>
> The problem occurs when a required revision of a module is deleted,
> such as when a developer decides to get rid of his/her local version
> of a module and go back to using a shared version.  The problem arises
> because I've decided (through painful experience) that local
> publications should use different revision numbers from shared
> publications.
>
> Suppose there are 3 modules, A, B, and C, and each has a 1.0-local
> version in the local repository and a 1.0 version in the shared
> repository.  Also, C depends on B with rev=1+, and B depends on A with
> rev=1+.  Now, suppose I decide I'm done working on module A, so I
> delete the local revision.  Now when I try to resolve module C, it
> fails because the published ivy.xml of B depends on rev 1.0-local of
> A, which no longer exists.
>
> Ideally, I'd like Ivy to use rev 1.0 of A, which would be allowed by
> the ivy.xml file for B, but the *published* ivy.xml for B has the
> revision of A changed from 1+ to 1.0-local, so Ivy has no way of
> knowing that 1.0 is an acceptable revision.  The change I want to make
> in the ivy.xml files is that each dependency in a published ivy.xml
> should have *two* revisions--one containing the actual revision in use
> at the time of publication, and one copied verbatim from the
> hand-written ivy.xml.  During resolution, the actual revision would
> always be preferred, and the original pattern would be used as a
> backup when the original revision is not available.

The problem is quite common, and the idea is interesting... However
I'm not sure to share your vision about the solution. What I don't
like is to split the version constraint in two separate attributes.
What you want is a mechanism to express a version constraint like
"take 1.0-local, and if it doesn't exist, take 1+". For me this is
only one version constraint, and I'd prefer expressing it as one
expression in the rev attribute. If you think about it, this is very
similar to the fallback mechanism used for configurations. If we
follow the same notation it would be rev="1.0-local(1+)".

One advantage I see to this solution is that it may be done without
even modifying Ivy: create your own VersionMatcher, recognising this
syntax, and you're done for the resolve part. And to put use this
notation during deliver, using a custom
PublishingDependencyRevisionResolver should be ok.

Now I don't think this should be the default behaviour, even if it
could be useful for others. Maybe something using the status to do
that only if the dependency may be deleted (if we say for example that
an integration version may be deleted, but not a milestone or a
release).

What do you think?

Xavier

>
> --jw
>