You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Garima Bathla <ga...@gmail.com> on 2009/07/29 20:10:42 UTC
feature request: configuration intersections only implemented halfway
I've come to the conclusion that the new configuration intersections feature
is implemented only halfway, and as a result it presents an inconsistent,
misleading, confusing contract. Either we yank it out from Ivy entirely or
we implement it fully.
Suppose I have an Ivy module called system with configurations like so:
<configurations>
<conf name="windows" grouping="platform" />
<conf name="linux" grouping="platform" />
<conf name="release" grouping="quality" />
<conf name="debug" grouping="quality" />
</configurations>
Plus publication artifacts like so:
<publications>
<artifact name="art1" type="dll" conf="windows,release" />
<artifact name="art1d" type="dll" conf="windows,debug" />
<artifact name="art1" type="so" conf="linux,release" />
<artifact name="art1d" type="so" conf="linux,debug" />
</publications>
Suppose I have an Ivy module called consumer with a dependency on system
like so:
<dependency … conf="default->windows+debug" />
Now, this + notation is allowed by Ivy, thanks to the configuration
intersections feature. However, according to system's ivy.xml, this
windows+debug syntax is useless and meaningless. The implicit request for
art1d.dll could just as well be expressed as:
<dependency … conf="default->windows,debug" />
Why? Because art1d.dll is being published in the union, so if you want
art1d.dll, then you should just ask for the union. (Never mind that, if you
only consider only the configurations and not the publications to be the Ivy
module's interface, then the fact that art1d.dll is being delivered via the
union should be encapsulated from the consumer.) Anyway, if you ask simply
for the union, you'd also get art1.dll and art1d.so, which you don't want.
So now you're having to delve into what the particulars of the publication
artifacts (again, violating idea of configurations being the interface and
encapsulating publications) to try to divine something that is not
explicitly there.
Worse, what if the creator of the system Ivy module really did want the
unions to work as unions? Asking for intersections is an attempt to undercut
the interface offered by the dependency module. There's nothing about the
following interface (to the extent that publication artifacts can be
considered an interface) to suggest that intersections are involved—unless
you're really, really clever at reading between the lines:
<artifact name="art1" type="dll" conf="windows,release" />
<artifact name="art1d" type="dll" conf="windows,debug" />
<artifact name="art1" type="so" conf="linux,release" />
<artifact name="art1d" type="so" conf="linux,debug" />
What if you were really, really clever though? You could divine that, "Hey,
look at all those combination patterns. You could make intersections out of
them." Wow, though, that's complicated.
Here's what bothers me about this contract as a senior enterprise developer.
We put a premium on programming languages' ability to express constructs in
as simple, expressive, and intuitive a manner as possible. But when you get
right down to it, Ivy too is a language. Or to be precise, Ivy provides a
general-purpose language for transitive dependency management/artifact
retrieval. In that context, Ivy should strive to express dependencies in as
simple, intuitive, and expressive a manner as possible. It's our instinct as
developers to make sure we're no less smart than the next guy, to try to
understand anything, no matter how complex, that gets thrown at us. But in
the course of trying to understand complexity, we tend to lose sight of when
complexity becomes needless. In today's world, we don't use Roman numerals
so that we can bask in our cleverness, we use decimals so we can get the job
done. The implicit intersection contract above is too complicated.
So again, let's either implement configuration intersections fully or remove
configuration intersections entirely from Ivy. What we have now is a
confusing, complicated compromise that suggests that we couldn't make up our
minds about this feature.
You can tell where I stand on that either/or. I want configuration
intersections implemented fully, and that means simply that the + notation
should be allowed on artifact confs, like so:
<artifact name="art1" type="dll" conf="windows+release" />
<artifact name="art1d" type="dll" conf="windows+debug" />
<artifact name="art1" type="so" conf="linux+release" />
<artifact name="art1d" type="so" conf="linux+debug" />
I can absolutely attest that I have a real-world business case that doesn't
involve cute pets or cute coffee drinks where, if I'm not able to express
something like "windows+release" on a publication, I'm stuck between a rock
and a hard place:
· Rock: If I'm forced to create combination Ivy confs like
"windows-and-release", my Ivy module descriptors become totally
unmaintainable really fast.
· Hard place: If I have to rely on consumer Ivy modules knowing to
express their dependencies using "windows+release" (with all the
communication that involves), then whenever a consumer Ivy module mistakenly
does "windows,release", my deployment is hosed.
Now, someone might argue, "It's ambiguous how configuration intersections
affect transitive dependencies." And I say to that, not so. It would be easy
enough to do a logical proof that configuration intersections like
"windows+release" would behave no differently than manufactured composite
configurations like "windows-and-release".
Someone might also argue that intersections are too restrictive and will get
too few artifacts. And to that I say, restrictiveness is the whole point.
Who is anyone but the module publisher to know what is too restrictive?
There is one counterargument I am sympathetic to. If the configurations
section of a module descriptor is considered the Ivy module's sole
interface, then configuration intersections, as something that would be
expressed only in the publications section, are not part of the interface.
But that train has already left the station. If we allow configuration
intersections on dependencies, then we're already expressing a request that
goes beyond the configurations section; we're already saying we've taken a
peek at the publications section.
Still, I'm sympathetic to that argument. And I think the ideal solution
would be to acknowledge that Ivy confs, as originally conceived, are not
composite constructs; so we need to introduce composite elements that can be
used to construct confs. Frankly, though, I'd rather not wait for the ideal
solution.
I'm hoping it's not too late for the simple fix of allowing + notation on
artifact confs to get incorporated into Ivy 2.1 before its final release.
And since the JIRA issue IVY1093 is not yet resolved, I'm thinking this
change would be a natural fit to just incorporate into that fix.
I will be more than happy to open a JIRA issue for this problem, but I
definitely will like to know your opinion on this before I do open one.
Regards,
Garima.
Re: feature request: configuration intersections only implemented halfway
Posted by Niklas Matthies <ml...@nmhq.net>.
On Wed 2009-07-29 at 11:10h, Garima Bathla wrote on ivy-user:
:
> Now, someone might argue, "It's ambiguous how configuration
> intersections affect transitive dependencies." And I say to that,
> not so. It would be easy enough to do a logical proof that
> configuration intersections like "windows+release" would behave no
> differently than manufactured composite configurations like
> "windows-and-release".
Consider a diamond-shaped dependency where the module with these
"windows" and "release" confs is the transitive dependency (= the
"tip" of the diamond). Now imagine the left side of the diamond
depends only on "windows", while the right side depends only on
"release". It's clear that the "bottom" of the diamond will receive
both the artifacts of "windows" and of "release", but will they get
your "windows+release" artifacts? Should they? How does it interact
with configuration intersections of dependencies? Is this compatible
with Ivy's resolution algorithm? These issues don't arise with a
"windows-and-release" dependency.
As for configuration intersections used as a conf for retrieving,
the problem there is that with "windows+release" you need to somehow
combine the conf mappings for "windows" and "release". Suppose that
for some dependency M you have "windows->foo" and "release->bar".
Which of the artifacts of the configurations "foo" and "bar" of
dependency M should be retrieved for "windows+release"?
Either you take the union of "foo" and "bar", which would contradict
the intent of getting an intersection, in particular if the bulk of
the artifact payload resides in the dependencies. Or you actually take
the intersection, which is likely to cause trouble, because only the
maintainer of M knows whether such an intersection of "foo" and "bar"
makes any sense (it probably doesn't), and even this he only knows for
the publications of M proper, not for the artifacts of the
dependencies of M -- same problem as with the original module.
And then there's the aspect of multiple dependencies. Should the
intersection be taken here too? Should it matter whether artifacts
come from two different configurations of the same dependency, as
opposed to coming from two different dependencies? (I don't think it
should.) It's clear that with multiple dependencies no one really
knows whether intersections of the artifacts from those dependencies
make any sense.
> Someone might also argue that intersections are too restrictive and
> will get too few artifacts.
In your use case of an "intersection artifact", i.e. an artifact that
is only applicable when some combination of configurations is
requested, the requester actually gets more artifacts than when
requesting each configuration separately. But I guess you were
referring to configuration intersections as dependencies here.
> And to that I say, restrictiveness is the whole point. Who is anyone
> but the module publisher to know what is too restrictive?
Well, he doesn't know what is too restrictive for the modules he
depends on, exactly because only the publishers of those modules can
know that -- though actually they can't if they have dependencies
themselves.
Intersections simply don't work for dependencies. The root reason for
this is that dependencies are defined on the level of modules and
configurations, not on the level of artifacts, and hence its not
possible to determine the actual dependencies of arbitrary subsets of
artifacts like the ones created by configuration intersecions.
> There is one counterargument I am sympathetic to. If the configurations
> section of a module descriptor is considered the Ivy module's sole
> interface, then configuration intersections, as something that would be
> expressed only in the publications section, are not part of the interface.
> But that train has already left the station. If we allow configuration
> intersections on dependencies, then we're already expressing a request that
> goes beyond the configurations section; we're already saying we've taken a
> peek at the publications section.
Yes, that's why I think it's unsound. A client module shouldn't depend
on whether an artifact comes from a direct dependency or from a
transitive dependency. For example one should be able to create a
"virtual" module that has no publications, but just depends on other
modules, such that the effect of depending on the virtual module is
the same as directly depending on those other modules. Or, a module
should be able to decide to "outsource" some of its artifacts into a
dependency, without this breaking any client modules. The only way to
achieve this equivalence in the presence of intersections is for the
intersections to be applied transitively, which, as pointed out above,
cuts across the declared dependencies of the modules.
-- Niklas Matthies
Re: feature request: configuration intersections only implemented
halfway
Posted by Garima Bathla <ga...@gmail.com>.
Shawn, I apologize for putting the idea out there that configuration
intersections should be removed entirely from Ivy. Truth be told, I'd be
having just as big a conniption as you would if that really were to happen. The
reason I broached that thought was merely to acknowledge that there may be
some philosophical opposition to the entire concept.
I think Mitch has pretty accurately described the real-life business problem
I'm facing now. I don't want to specify an artifact that by itself is
inaccurate:
<artifact ... conf="windows,debug" />
And then see a consumer Ivy module ask for "windows,debug". If that happens,
my system blows up. The way things are now, there's no way to prevent that
from happening and there's no EASY way to communicate when a union is really
a union and when it's not. I tried to convey the problem I'm facing with the
"rock" and "hard place" passage of my original post.
It seems like the way configuration intersections are implemented now is a
bit like the sort of sausage-making compromise that happens with
legislation. Some people don't want something at all, other people want
something in its full form, and in the process of meeting halfway, neither
party is left entirely happy.
So what I would propose is a two-step process: a quick fix followed by a
long-term solution.
The quick fix I'd like to see for Ivy 2.1 is to allow intersections to be
specified on publication artifacts. The long-term solution, which I think
would appease the people (including me) who have certain philosophical
concerns about intersections, is to transition to a module descriptor API
whereby, in addition to the existing simple Ivy confs, you also can have
conf-like elements out of which composite intersection Ivy confs can be
constructed.
I really appreciate your feedback on my posts.
Regards,
Garima.
On Wed, Jul 29, 2009 at 11:39 AM, Shawn Castrianni <
Shawn.Castrianni@halliburton.com> wrote:
> First, I will admit I did not ready every word of your post as it was
> lengthy. Therefore, I will only comment on what caught my eye.
>
>
> 1. " However, according to system's ivy.xml, this windows+debug syntax is
> useless and meaningless. The implicit request for
> art1d.dll could just as well be expressed as: <dependency ...
> conf="default->windows,debug" />"
>
> This is not true. "default->windows,debug" means give me all of the
> artifacts that exist in either (union) configurations windows and debug.
> Whereas, "default->windows+debug" means give me all of the artifacts that
> exist only in both (intersection) configurations windows and debug. If I am
> trying to build a windows debug version of my module, than I don't want all
> the windows artifacts and all the debug artifacts. All the windows
> artifacts will give me release artifacts too. All the debug artifacts will
> give me other platforms artifacts too. Since I am building my module in
> debug mode, release artifacts do me no good. Since I am building on
> windows, linux artifacts do me no good. Therefore, the only way for me to
> get exactly what I want is the intersection notataion of windows+debug.
>
>
> 2. "So again, let's either implement configuration intersections fully or
> remove configuration intersections entirely from Ivy."
>
> I requested this feature. I have a real world use case for it as described
> above. It solves my problem beautifully. Its existence in ivy does not
> hurt anybody if they don't use it. Removing it only hurts me and doesn't
> help anybody else.
>
> ---
> Shawn Castrianni
>
> -----Original Message-----
> From: Garima Bathla [mailto:garima.bathla@gmail.com]
> Sent: Wednesday, July 29, 2009 1:11 PM
> To: ivy-user@ant.apache.org
> Subject: feature request: configuration intersections only implemented
> halfway
>
> I've come to the conclusion that the new configuration intersections
> feature
> is implemented only halfway, and as a result it presents an inconsistent,
> misleading, confusing contract. Either we yank it out from Ivy entirely or
> we implement it fully.
>
> Suppose I have an Ivy module called system with configurations like so:
> <configurations>
> <conf name="windows" grouping="platform" />
> <conf name="linux" grouping="platform" />
> <conf name="release" grouping="quality" />
> <conf name="debug" grouping="quality" />
> </configurations>
>
> Plus publication artifacts like so:
> <publications>
> <artifact name="art1" type="dll" conf="windows,release" />
> <artifact name="art1d" type="dll" conf="windows,debug" />
> <artifact name="art1" type="so" conf="linux,release" />
> <artifact name="art1d" type="so" conf="linux,debug" />
> </publications>
>
> Suppose I have an Ivy module called consumer with a dependency on system
> like so:
> <dependency ... conf="default->windows+debug" />
>
> Now, this + notation is allowed by Ivy, thanks to the configuration
> intersections feature. However, according to system's ivy.xml, this
> windows+debug syntax is useless and meaningless. The implicit request for
> art1d.dll could just as well be expressed as:
> <dependency ... conf="default->windows,debug" />
>
> Why? Because art1d.dll is being published in the union, so if you want
> art1d.dll, then you should just ask for the union. (Never mind that, if you
> only consider only the configurations and not the publications to be the
> Ivy
> module's interface, then the fact that art1d.dll is being delivered via the
> union should be encapsulated from the consumer.) Anyway, if you ask simply
> for the union, you'd also get art1.dll and art1d.so, which you don't want.
> So now you're having to delve into what the particulars of the publication
> artifacts (again, violating idea of configurations being the interface and
> encapsulating publications) to try to divine something that is not
> explicitly there.
>
> Worse, what if the creator of the system Ivy module really did want the
> unions to work as unions? Asking for intersections is an attempt to
> undercut
> the interface offered by the dependency module. There's nothing about the
> following interface (to the extent that publication artifacts can be
> considered an interface) to suggest that intersections are involved-unless
> you're really, really clever at reading between the lines:
> <artifact name="art1" type="dll" conf="windows,release" />
> <artifact name="art1d" type="dll" conf="windows,debug" />
> <artifact name="art1" type="so" conf="linux,release" />
> <artifact name="art1d" type="so" conf="linux,debug" />
>
> What if you were really, really clever though? You could divine that, "Hey,
> look at all those combination patterns. You could make intersections out of
> them." Wow, though, that's complicated.
> Here's what bothers me about this contract as a senior enterprise
> developer.
>
>
> We put a premium on programming languages' ability to express constructs in
> as simple, expressive, and intuitive a manner as possible. But when you get
> right down to it, Ivy too is a language. Or to be precise, Ivy provides a
> general-purpose language for transitive dependency management/artifact
> retrieval. In that context, Ivy should strive to express dependencies in as
> simple, intuitive, and expressive a manner as possible. It's our instinct
> as
> developers to make sure we're no less smart than the next guy, to try to
> understand anything, no matter how complex, that gets thrown at us. But in
> the course of trying to understand complexity, we tend to lose sight of
> when
> complexity becomes needless. In today's world, we don't use Roman numerals
> so that we can bask in our cleverness, we use decimals so we can get the
> job
> done. The implicit intersection contract above is too complicated.
>
> So again, let's either implement configuration intersections fully or
> remove
> configuration intersections entirely from Ivy. What we have now is a
> confusing, complicated compromise that suggests that we couldn't make up
> our
> minds about this feature.
>
> You can tell where I stand on that either/or. I want configuration
> intersections implemented fully, and that means simply that the + notation
> should be allowed on artifact confs, like so:
> <artifact name="art1" type="dll" conf="windows+release" />
> <artifact name="art1d" type="dll" conf="windows+debug" />
> <artifact name="art1" type="so" conf="linux+release" />
> <artifact name="art1d" type="so" conf="linux+debug" />
>
> I can absolutely attest that I have a real-world business case that doesn't
> involve cute pets or cute coffee drinks where, if I'm not able to express
> something like "windows+release" on a publication, I'm stuck between a rock
> and a hard place:
> * Rock: If I'm forced to create combination Ivy confs like
> "windows-and-release", my Ivy module descriptors become totally
> unmaintainable really fast.
> * Hard place: If I have to rely on consumer Ivy modules knowing to
> express their dependencies using "windows+release" (with all the
> communication that involves), then whenever a consumer Ivy module
> mistakenly
> does "windows,release", my deployment is hosed.
>
> Now, someone might argue, "It's ambiguous how configuration intersections
> affect transitive dependencies." And I say to that, not so. It would be
> easy
> enough to do a logical proof that configuration intersections like
> "windows+release" would behave no differently than manufactured composite
> configurations like "windows-and-release".
>
> Someone might also argue that intersections are too restrictive and will
> get
> too few artifacts. And to that I say, restrictiveness is the whole point.
> Who is anyone but the module publisher to know what is too restrictive?
>
> There is one counterargument I am sympathetic to. If the configurations
> section of a module descriptor is considered the Ivy module's sole
> interface, then configuration intersections, as something that would be
> expressed only in the publications section, are not part of the interface.
> But that train has already left the station. If we allow configuration
> intersections on dependencies, then we're already expressing a request that
> goes beyond the configurations section; we're already saying we've taken a
> peek at the publications section.
>
> Still, I'm sympathetic to that argument. And I think the ideal solution
> would be to acknowledge that Ivy confs, as originally conceived, are not
> composite constructs; so we need to introduce composite elements that can
> be
> used to construct confs. Frankly, though, I'd rather not wait for the ideal
> solution.
>
> I'm hoping it's not too late for the simple fix of allowing + notation on
> artifact confs to get incorporated into Ivy 2.1 before its final release.
> And since the JIRA issue IVY1093 is not yet resolved, I'm thinking this
> change would be a natural fit to just incorporate into that fix.
>
> I will be more than happy to open a JIRA issue for this problem, but I
> definitely will like to know your opinion on this before I do open one.
>
> Regards,
> Garima.
>
> ----------------------------------------------------------------------
> This e-mail, including any attached files, may contain confidential and
> privileged information for the sole use of the intended recipient. Any
> review, use, distribution, or disclosure by others is strictly prohibited.
> If you are not the intended recipient (or authorized to receive information
> for the intended recipient), please contact the sender by reply e-mail and
> delete all copies of this message.
>
Re: feature request: configuration intersections only implemented halfway
Posted by Niklas Matthies <ml...@nmhq.net>.
On Wed 2009-07-29 at 23:22h, Garima Bathla wrote on ivy-user:
:
> So I need to come up with a palette of Ivy confs and a combination of them
> to specify to deliver me the following artifacts:
> mocha.jar
> mocha-tall.jar
> mocha-tall-iced.jar
>
> Well, I know how to do this the hard way:
> <conf name="mocha" />
> <conf name="mocha-tall" extends="mocha" />
> <conf name="mocha-tall-iced" extends="mocha-tall" />
>
> Then in my dependency module, I would ask for "mocha-tall-iced". Only
> problem: now I'm stuck with 4+16+32 Ivy confs. There's got to be a better
> way. (Again, I realize that I'm producing a proliferation of artifacts
> regardless.)
Isn't the number of additional confs just proportional to the number
of those special artifacts though? If yes, then this means you have
only O(1) of extra work to do for each such artifact, so to speak.
No extra combinatorial explosion. Or am I missing something?
I'm still curious what the real (non-coffee) domain context is. ;)
-- Niklas Matthies
Re: feature request: configuration intersections only implemented
halfway
Posted by Garima Bathla <ga...@gmail.com>.
Thank you all for actively pursuing this issue. I need to take a step back
from my feature request for a moment, because I just realized my situation
is even more complicated than I had thought.
Forgive me for going back to my coffee analogy. In addition to something
like mocha-tall-iced.jar that I only want delivered in that combination
situation, I also have something like mocha-tall.jar that I simultaneously
want delivered whether it's iced or hot. And to top all that off, I also
have something called mocha.jar that I likewise simultaneously want
delivered regardless of the size or temperature.
So I need to come up with a palette of Ivy confs and a combination of them
to specify to deliver me the following artifacts:
mocha.jar
mocha-tall.jar
mocha-tall-iced.jar
Well, I know how to do this the hard way:
<conf name="mocha" />
<conf name="mocha-tall" extends="mocha" />
<conf name="mocha-tall-iced" extends="mocha-tall" />
Then in my dependency module, I would ask for "mocha-tall-iced". Only
problem: now I'm stuck with 4+16+32 Ivy confs. There's got to be a better
way. (Again, I realize that I'm producing a proliferation of artifacts
regardless.)
It may very well be that I wind up doing some serious re-factoring and get
rid of one of my dimensions. I'll also take a second look at configuration
groupings.
Regards,
Garima.
On Wed, Jul 29, 2009 at 4:09 PM, Mitch Gitman <mg...@gmail.com> wrote:
> Shawn, I think your last paragraph really captures the fundamental
> difference between your use case and the one I'm facing (which parallels
> the
> use case Garima broached):
> I don't understand Garima's request as putting the intersection logic into
> the <publications> section of the ivy.xml file like: <artifact
> name="art1d"
> type="dll" conf="windows+debug"/> means that the ONLY way a consumer can
> get
> this artifact is by specifying "default->windows+debug" in the
> <dependencies> section. This prevents me from getting all windows
> artifacts
> or all debug artifacts. It just doesn't make sense to me.
>
> For me, being able to get "ALL Windows artifacts" or "ALL Windows OR debug
> artifacts" would be at best an academic exercise. As a practical matter,
> though, the simple ability to do this is trouble. The simple inability to
> express intersection-only means that I can't communicate in Ivy the true
> semantics of my Ivy module.
>
> In other words, I really do have artifacts that should only be available in
> the intersection of certain confs of their Ivy module. Being able to get
> unions containing these particular artifacts does me no good, but it does
> potentially do me harm.
>
> To put things in simple terms, what Shawn asked for was to make the
> following possible on the *consumer *Ivy module:
> if (A && B)
> In addition to:
> if (A || B)
>
> What Garima is asking for—and what I'm seconding—is to make the same
> possible on the *producer *Ivy module.
>
> On Wed, Jul 29, 2009 at 3:15 PM, Shawn Castrianni <
> Shawn.Castrianni@halliburton.com> wrote:
>
> > 1. Yes, I agree that adding what Garima wants will not compromise my use
> > case. I just haven't been convinced why Garima is requesting this.
> Either
> > Garima is not understanding configurations in general (which might be the
> > case as Garima admitted not knowing about the '@' character) or you are
> not
> > understanding configurations in general (which might be the case based on
> > your comment that "windows,debug" is inaccurate) or I am not
> understanding
> > your use case.
> >
> > 2. "The following, by itself, is inaccurate: <artifact name="art1d"
> > type="dll" conf="windows,debug" />"
> >
> > How can you say this is inaccurate? I am thinking you are getting
> confused
> > about configurations (or maybe I am). The conf attribute of an artifact
> is
> > simply a comma delimited list of all configuration names that this
> artifact
> > is applicable for. The art1d.dll is a windows related file so it is
> > definitely applicable for the windows configuration. The art1d.dll is a
> > debug version of the dll so it is definitely applicable for the debug
> > configuration. The "windows,debug" does NOT mean union in this section
> of
> > the ivy.xml file where the artifacts are defined.
> >
> > It is in the <dependencies? section of the ivy.xml file where
> > "windows,debug" would mean union. So the comma notation of
> configurations
> > means two different things depending on what section of the ivy.xml you
> are
> > talking about. In the <publications> section, the comma doesn't mean
> > anything. It is just a delimiter separating configuration names for all
> of
> > the configurations that this artifact is applicable for. In the
> > <dependencies> section, the conf mapping attribute like
> > "default->windows,debug" means the default configuration of my current
> > module is dependent on BOTH (UNION) of the windows and debug
> configurations
> > of the dependent module.
> >
> > IVY configurations are just a way to subset all the artifacts of a module
> > since you may not want all artifacts of a module all the time. Sometimes
> > you just want all windows artifacts so you would use the conf mapping in
> the
> > <dependencies> section as "default->windows". Sometimes you just want
> all
> > debug artifacts so you would use the conf mapping in the <dependencies>
> > section as "default->debug". Sometimes you just want the windows debug
> > artifacts so you would use the new intersection capability in the
> > <dependencies> section as "default->windows+debug". It is purely a
> download
> > efficiency enhancement for me. Before the intersection capability, I
> would
> > have to use "default->windows" and then manually delete the non-debug
> > artifacts. But even with that, I had to wait for IVY to download them
> into
> > the cache. Over a slow network line, this can waste a lot of time for
> > artifacts that are just going to be deleted manually.
> >
> > I don't understand Garima's request as putting the intersection logic
> into
> > the <publications> section of the ivy.xml file like: <artifact
> name="art1d"
> > type="dll" conf="windows+debug"/> means that the ONLY way a consumer can
> get
> > this artifact is by specifying "default->windows+debug" in the
> > <dependencies> section. This prevents me from getting all windows
> artifacts
> > or all debug artifacts. It just doesn't make sense to me.
> >
> > ---
> > Shawn Castrianni
> >
> >
>
Re: feature request: configuration intersections only implemented
halfway
Posted by Mitch Gitman <mg...@gmail.com>.
Shawn, I think your last paragraph really captures the fundamental
difference between your use case and the one I'm facing (which parallels the
use case Garima broached):
I don't understand Garima's request as putting the intersection logic into
the <publications> section of the ivy.xml file like: <artifact name="art1d"
type="dll" conf="windows+debug"/> means that the ONLY way a consumer can get
this artifact is by specifying "default->windows+debug" in the
<dependencies> section. This prevents me from getting all windows artifacts
or all debug artifacts. It just doesn't make sense to me.
For me, being able to get "ALL Windows artifacts" or "ALL Windows OR debug
artifacts" would be at best an academic exercise. As a practical matter,
though, the simple ability to do this is trouble. The simple inability to
express intersection-only means that I can't communicate in Ivy the true
semantics of my Ivy module.
In other words, I really do have artifacts that should only be available in
the intersection of certain confs of their Ivy module. Being able to get
unions containing these particular artifacts does me no good, but it does
potentially do me harm.
To put things in simple terms, what Shawn asked for was to make the
following possible on the *consumer *Ivy module:
if (A && B)
In addition to:
if (A || B)
What Garima is asking for—and what I'm seconding—is to make the same
possible on the *producer *Ivy module.
On Wed, Jul 29, 2009 at 3:15 PM, Shawn Castrianni <
Shawn.Castrianni@halliburton.com> wrote:
> 1. Yes, I agree that adding what Garima wants will not compromise my use
> case. I just haven't been convinced why Garima is requesting this. Either
> Garima is not understanding configurations in general (which might be the
> case as Garima admitted not knowing about the '@' character) or you are not
> understanding configurations in general (which might be the case based on
> your comment that "windows,debug" is inaccurate) or I am not understanding
> your use case.
>
> 2. "The following, by itself, is inaccurate: <artifact name="art1d"
> type="dll" conf="windows,debug" />"
>
> How can you say this is inaccurate? I am thinking you are getting confused
> about configurations (or maybe I am). The conf attribute of an artifact is
> simply a comma delimited list of all configuration names that this artifact
> is applicable for. The art1d.dll is a windows related file so it is
> definitely applicable for the windows configuration. The art1d.dll is a
> debug version of the dll so it is definitely applicable for the debug
> configuration. The "windows,debug" does NOT mean union in this section of
> the ivy.xml file where the artifacts are defined.
>
> It is in the <dependencies? section of the ivy.xml file where
> "windows,debug" would mean union. So the comma notation of configurations
> means two different things depending on what section of the ivy.xml you are
> talking about. In the <publications> section, the comma doesn't mean
> anything. It is just a delimiter separating configuration names for all of
> the configurations that this artifact is applicable for. In the
> <dependencies> section, the conf mapping attribute like
> "default->windows,debug" means the default configuration of my current
> module is dependent on BOTH (UNION) of the windows and debug configurations
> of the dependent module.
>
> IVY configurations are just a way to subset all the artifacts of a module
> since you may not want all artifacts of a module all the time. Sometimes
> you just want all windows artifacts so you would use the conf mapping in the
> <dependencies> section as "default->windows". Sometimes you just want all
> debug artifacts so you would use the conf mapping in the <dependencies>
> section as "default->debug". Sometimes you just want the windows debug
> artifacts so you would use the new intersection capability in the
> <dependencies> section as "default->windows+debug". It is purely a download
> efficiency enhancement for me. Before the intersection capability, I would
> have to use "default->windows" and then manually delete the non-debug
> artifacts. But even with that, I had to wait for IVY to download them into
> the cache. Over a slow network line, this can waste a lot of time for
> artifacts that are just going to be deleted manually.
>
> I don't understand Garima's request as putting the intersection logic into
> the <publications> section of the ivy.xml file like: <artifact name="art1d"
> type="dll" conf="windows+debug"/> means that the ONLY way a consumer can get
> this artifact is by specifying "default->windows+debug" in the
> <dependencies> section. This prevents me from getting all windows artifacts
> or all debug artifacts. It just doesn't make sense to me.
>
> ---
> Shawn Castrianni
>
>
RE: feature request: configuration intersections only implemented
halfway
Posted by Shawn Castrianni <Sh...@halliburton.com>.
1. Yes, I agree that adding what Garima wants will not compromise my use case. I just haven't been convinced why Garima is requesting this. Either Garima is not understanding configurations in general (which might be the case as Garima admitted not knowing about the '@' character) or you are not understanding configurations in general (which might be the case based on your comment that "windows,debug" is inaccurate) or I am not understanding your use case.
2. "The following, by itself, is inaccurate: <artifact name="art1d" type="dll" conf="windows,debug" />"
How can you say this is inaccurate? I am thinking you are getting confused about configurations (or maybe I am). The conf attribute of an artifact is simply a comma delimited list of all configuration names that this artifact is applicable for. The art1d.dll is a windows related file so it is definitely applicable for the windows configuration. The art1d.dll is a debug version of the dll so it is definitely applicable for the debug configuration. The "windows,debug" does NOT mean union in this section of the ivy.xml file where the artifacts are defined.
It is in the <dependencies? section of the ivy.xml file where "windows,debug" would mean union. So the comma notation of configurations means two different things depending on what section of the ivy.xml you are talking about. In the <publications> section, the comma doesn't mean anything. It is just a delimiter separating configuration names for all of the configurations that this artifact is applicable for. In the <dependencies> section, the conf mapping attribute like "default->windows,debug" means the default configuration of my current module is dependent on BOTH (UNION) of the windows and debug configurations of the dependent module.
IVY configurations are just a way to subset all the artifacts of a module since you may not want all artifacts of a module all the time. Sometimes you just want all windows artifacts so you would use the conf mapping in the <dependencies> section as "default->windows". Sometimes you just want all debug artifacts so you would use the conf mapping in the <dependencies> section as "default->debug". Sometimes you just want the windows debug artifacts so you would use the new intersection capability in the <dependencies> section as "default->windows+debug". It is purely a download efficiency enhancement for me. Before the intersection capability, I would have to use "default->windows" and then manually delete the non-debug artifacts. But even with that, I had to wait for IVY to download them into the cache. Over a slow network line, this can waste a lot of time for artifacts that are just going to be deleted manually.
I don't understand Garima's request as putting the intersection logic into the <publications> section of the ivy.xml file like: <artifact name="art1d" type="dll" conf="windows+debug"/> means that the ONLY way a consumer can get this artifact is by specifying "default->windows+debug" in the <dependencies> section. This prevents me from getting all windows artifacts or all debug artifacts. It just doesn't make sense to me.
---
Shawn Castrianni
-----Original Message-----
From: Mitch Gitman [mailto:mgitman@gmail.com]
Sent: Wednesday, July 29, 2009 4:07 PM
To: ivy-user@ant.apache.org
Subject: Re: feature request: configuration intersections only implemented halfway
I would heartily second Garima's feature request. I'll describe below a
potential error condition with the way configuration intersections are
implemented now.
It looks to me like adding what she's asking for in no way compromises the
functionality Shawn already enjoys. It just more explicitly, clearly, and
robustly enforces the contract involved in intersections--and prevents the
error I'll describe shortly.
Shawn, a question for you. Would you agree that merely *allowing *intersections
on the *artifact *element--while still supporting the existing behavior--in
no way compromises what you're already able to accomplish?
And a question for Maarten or Xavier. What is the proper channel to go
through for requesting a new feature in Ivy?
Few responses to Shawn inline...
On Wed, Jul 29, 2009 at 11:39 AM, Shawn Castrianni <
Shawn.Castrianni@halliburton.com> wrote:
> First, I will admit I did not ready every word of your post as it was
> lengthy. Therefore, I will only comment on what caught my eye.
>
>
> 1. " However, according to system's ivy.xml, this windows+debug syntax is
> useless and meaningless. The implicit request for
> art1d.dll could just as well be expressed as: <dependency ...
> conf="default->windows,debug" />"
>
> This is not true. "default->windows,debug" means give me all of the
> artifacts that exist in either (union) configurations windows and debug.
> Whereas, "default->windows+debug" means give me all of the artifacts that
> exist only in both (intersection) configurations windows and debug. If I am
> trying to build a windows debug version of my module, than I don't want all
> the windows artifacts and all the debug artifacts. All the windows
> artifacts will give me release artifacts too. All the debug artifacts will
> give me other platforms artifacts too. Since I am building my module in
> debug mode, release artifacts do me no good. Since I am building on
> windows, linux artifacts do me no good. Therefore, the only way for me to
> get exactly what I want is the intersection notataion of windows+debug.
>
Shawn, this may be a precious distinction. But what I think Garima is trying
to say is that, *strictly on the face of it*, "windows+debug" is useless and
meaningless when the artifact is specified as "windows,debug". It's only by
doing the exercise of going through the artifacts and producing the
intersection that you're able to obtain just that one file, art1d.dll.
So this brings me to the error condition. If the client were to specify
"windows,debug", they would get something that, in your own example, is
incorrect. Therefore, you're relying on the consumer to specify things
correctly.
What troubles me is that you can have an *artifact *specification that,
standing by itself, is inaccurate. The following, by itself, is inaccurate:
<artifact name="art1d" type="dll" conf="windows,debug" />
This artifact should not be available in the union of windows and debug but
it is. Shawn, would you agree with that narrow statement?
>
>
> 2. "So again, let's either implement configuration intersections fully or
> remove configuration intersections entirely from Ivy."
>
> I requested this feature. I have a real world use case for it as described
> above. It solves my problem beautifully. Its existence in ivy does not
> hurt anybody if they don't use it. Removing it only hurts me and doesn't
> help anybody else.
Shawn, I appreciate that Garima's message was a lot to digest. But it looks
like you're grabbing a bit of devil's advocacy and taking it out of context.
I think Garima has made it clear as day that she too believes configuration
intersections are a great and necessary feature and that the idea of
removing it is not the path she wants to go down.
It so happens that I myself am facing a real-world use case for
configuration intersections and that I myself find the current support for
the feature overly complex and an invitation for trouble. Maybe this is an
admission that I'm not too bright, but I would hate to have to look at a
single *artifact *element, see the union of confs, and not be aware that the
artifact actually should *not *be delivered in that situation--and that I'm
going to have to go through a little mathematical exercise to figure out
what should be delivered when.
>
>
> ---
> Shawn Castrianni
>
>
----------------------------------------------------------------------
This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.
Re: feature request: configuration intersections only implemented
halfway
Posted by Mitch Gitman <mg...@gmail.com>.
I would heartily second Garima's feature request. I'll describe below a
potential error condition with the way configuration intersections are
implemented now.
It looks to me like adding what she's asking for in no way compromises the
functionality Shawn already enjoys. It just more explicitly, clearly, and
robustly enforces the contract involved in intersections--and prevents the
error I'll describe shortly.
Shawn, a question for you. Would you agree that merely *allowing *intersections
on the *artifact *element--while still supporting the existing behavior--in
no way compromises what you're already able to accomplish?
And a question for Maarten or Xavier. What is the proper channel to go
through for requesting a new feature in Ivy?
Few responses to Shawn inline...
On Wed, Jul 29, 2009 at 11:39 AM, Shawn Castrianni <
Shawn.Castrianni@halliburton.com> wrote:
> First, I will admit I did not ready every word of your post as it was
> lengthy. Therefore, I will only comment on what caught my eye.
>
>
> 1. " However, according to system's ivy.xml, this windows+debug syntax is
> useless and meaningless. The implicit request for
> art1d.dll could just as well be expressed as: <dependency ...
> conf="default->windows,debug" />"
>
> This is not true. "default->windows,debug" means give me all of the
> artifacts that exist in either (union) configurations windows and debug.
> Whereas, "default->windows+debug" means give me all of the artifacts that
> exist only in both (intersection) configurations windows and debug. If I am
> trying to build a windows debug version of my module, than I don't want all
> the windows artifacts and all the debug artifacts. All the windows
> artifacts will give me release artifacts too. All the debug artifacts will
> give me other platforms artifacts too. Since I am building my module in
> debug mode, release artifacts do me no good. Since I am building on
> windows, linux artifacts do me no good. Therefore, the only way for me to
> get exactly what I want is the intersection notataion of windows+debug.
>
Shawn, this may be a precious distinction. But what I think Garima is trying
to say is that, *strictly on the face of it*, "windows+debug" is useless and
meaningless when the artifact is specified as "windows,debug". It's only by
doing the exercise of going through the artifacts and producing the
intersection that you're able to obtain just that one file, art1d.dll.
So this brings me to the error condition. If the client were to specify
"windows,debug", they would get something that, in your own example, is
incorrect. Therefore, you're relying on the consumer to specify things
correctly.
What troubles me is that you can have an *artifact *specification that,
standing by itself, is inaccurate. The following, by itself, is inaccurate:
<artifact name="art1d" type="dll" conf="windows,debug" />
This artifact should not be available in the union of windows and debug but
it is. Shawn, would you agree with that narrow statement?
>
>
> 2. "So again, let's either implement configuration intersections fully or
> remove configuration intersections entirely from Ivy."
>
> I requested this feature. I have a real world use case for it as described
> above. It solves my problem beautifully. Its existence in ivy does not
> hurt anybody if they don't use it. Removing it only hurts me and doesn't
> help anybody else.
Shawn, I appreciate that Garima's message was a lot to digest. But it looks
like you're grabbing a bit of devil's advocacy and taking it out of context.
I think Garima has made it clear as day that she too believes configuration
intersections are a great and necessary feature and that the idea of
removing it is not the path she wants to go down.
It so happens that I myself am facing a real-world use case for
configuration intersections and that I myself find the current support for
the feature overly complex and an invitation for trouble. Maybe this is an
admission that I'm not too bright, but I would hate to have to look at a
single *artifact *element, see the union of confs, and not be aware that the
artifact actually should *not *be delivered in that situation--and that I'm
going to have to go through a little mathematical exercise to figure out
what should be delivered when.
>
>
> ---
> Shawn Castrianni
>
>
RE: feature request: configuration intersections only implemented
halfway
Posted by Shawn Castrianni <Sh...@halliburton.com>.
First, I will admit I did not ready every word of your post as it was lengthy. Therefore, I will only comment on what caught my eye.
1. " However, according to system's ivy.xml, this windows+debug syntax is useless and meaningless. The implicit request for
art1d.dll could just as well be expressed as: <dependency ... conf="default->windows,debug" />"
This is not true. "default->windows,debug" means give me all of the artifacts that exist in either (union) configurations windows and debug. Whereas, "default->windows+debug" means give me all of the artifacts that exist only in both (intersection) configurations windows and debug. If I am trying to build a windows debug version of my module, than I don't want all the windows artifacts and all the debug artifacts. All the windows artifacts will give me release artifacts too. All the debug artifacts will give me other platforms artifacts too. Since I am building my module in debug mode, release artifacts do me no good. Since I am building on windows, linux artifacts do me no good. Therefore, the only way for me to get exactly what I want is the intersection notataion of windows+debug.
2. "So again, let's either implement configuration intersections fully or remove configuration intersections entirely from Ivy."
I requested this feature. I have a real world use case for it as described above. It solves my problem beautifully. Its existence in ivy does not hurt anybody if they don't use it. Removing it only hurts me and doesn't help anybody else.
---
Shawn Castrianni
-----Original Message-----
From: Garima Bathla [mailto:garima.bathla@gmail.com]
Sent: Wednesday, July 29, 2009 1:11 PM
To: ivy-user@ant.apache.org
Subject: feature request: configuration intersections only implemented halfway
I've come to the conclusion that the new configuration intersections feature
is implemented only halfway, and as a result it presents an inconsistent,
misleading, confusing contract. Either we yank it out from Ivy entirely or
we implement it fully.
Suppose I have an Ivy module called system with configurations like so:
<configurations>
<conf name="windows" grouping="platform" />
<conf name="linux" grouping="platform" />
<conf name="release" grouping="quality" />
<conf name="debug" grouping="quality" />
</configurations>
Plus publication artifacts like so:
<publications>
<artifact name="art1" type="dll" conf="windows,release" />
<artifact name="art1d" type="dll" conf="windows,debug" />
<artifact name="art1" type="so" conf="linux,release" />
<artifact name="art1d" type="so" conf="linux,debug" />
</publications>
Suppose I have an Ivy module called consumer with a dependency on system
like so:
<dependency ... conf="default->windows+debug" />
Now, this + notation is allowed by Ivy, thanks to the configuration
intersections feature. However, according to system's ivy.xml, this
windows+debug syntax is useless and meaningless. The implicit request for
art1d.dll could just as well be expressed as:
<dependency ... conf="default->windows,debug" />
Why? Because art1d.dll is being published in the union, so if you want
art1d.dll, then you should just ask for the union. (Never mind that, if you
only consider only the configurations and not the publications to be the Ivy
module's interface, then the fact that art1d.dll is being delivered via the
union should be encapsulated from the consumer.) Anyway, if you ask simply
for the union, you'd also get art1.dll and art1d.so, which you don't want.
So now you're having to delve into what the particulars of the publication
artifacts (again, violating idea of configurations being the interface and
encapsulating publications) to try to divine something that is not
explicitly there.
Worse, what if the creator of the system Ivy module really did want the
unions to work as unions? Asking for intersections is an attempt to undercut
the interface offered by the dependency module. There's nothing about the
following interface (to the extent that publication artifacts can be
considered an interface) to suggest that intersections are involved-unless
you're really, really clever at reading between the lines:
<artifact name="art1" type="dll" conf="windows,release" />
<artifact name="art1d" type="dll" conf="windows,debug" />
<artifact name="art1" type="so" conf="linux,release" />
<artifact name="art1d" type="so" conf="linux,debug" />
What if you were really, really clever though? You could divine that, "Hey,
look at all those combination patterns. You could make intersections out of
them." Wow, though, that's complicated.
Here's what bothers me about this contract as a senior enterprise developer.
We put a premium on programming languages' ability to express constructs in
as simple, expressive, and intuitive a manner as possible. But when you get
right down to it, Ivy too is a language. Or to be precise, Ivy provides a
general-purpose language for transitive dependency management/artifact
retrieval. In that context, Ivy should strive to express dependencies in as
simple, intuitive, and expressive a manner as possible. It's our instinct as
developers to make sure we're no less smart than the next guy, to try to
understand anything, no matter how complex, that gets thrown at us. But in
the course of trying to understand complexity, we tend to lose sight of when
complexity becomes needless. In today's world, we don't use Roman numerals
so that we can bask in our cleverness, we use decimals so we can get the job
done. The implicit intersection contract above is too complicated.
So again, let's either implement configuration intersections fully or remove
configuration intersections entirely from Ivy. What we have now is a
confusing, complicated compromise that suggests that we couldn't make up our
minds about this feature.
You can tell where I stand on that either/or. I want configuration
intersections implemented fully, and that means simply that the + notation
should be allowed on artifact confs, like so:
<artifact name="art1" type="dll" conf="windows+release" />
<artifact name="art1d" type="dll" conf="windows+debug" />
<artifact name="art1" type="so" conf="linux+release" />
<artifact name="art1d" type="so" conf="linux+debug" />
I can absolutely attest that I have a real-world business case that doesn't
involve cute pets or cute coffee drinks where, if I'm not able to express
something like "windows+release" on a publication, I'm stuck between a rock
and a hard place:
* Rock: If I'm forced to create combination Ivy confs like
"windows-and-release", my Ivy module descriptors become totally
unmaintainable really fast.
* Hard place: If I have to rely on consumer Ivy modules knowing to
express their dependencies using "windows+release" (with all the
communication that involves), then whenever a consumer Ivy module mistakenly
does "windows,release", my deployment is hosed.
Now, someone might argue, "It's ambiguous how configuration intersections
affect transitive dependencies." And I say to that, not so. It would be easy
enough to do a logical proof that configuration intersections like
"windows+release" would behave no differently than manufactured composite
configurations like "windows-and-release".
Someone might also argue that intersections are too restrictive and will get
too few artifacts. And to that I say, restrictiveness is the whole point.
Who is anyone but the module publisher to know what is too restrictive?
There is one counterargument I am sympathetic to. If the configurations
section of a module descriptor is considered the Ivy module's sole
interface, then configuration intersections, as something that would be
expressed only in the publications section, are not part of the interface.
But that train has already left the station. If we allow configuration
intersections on dependencies, then we're already expressing a request that
goes beyond the configurations section; we're already saying we've taken a
peek at the publications section.
Still, I'm sympathetic to that argument. And I think the ideal solution
would be to acknowledge that Ivy confs, as originally conceived, are not
composite constructs; so we need to introduce composite elements that can be
used to construct confs. Frankly, though, I'd rather not wait for the ideal
solution.
I'm hoping it's not too late for the simple fix of allowing + notation on
artifact confs to get incorporated into Ivy 2.1 before its final release.
And since the JIRA issue IVY1093 is not yet resolved, I'm thinking this
change would be a natural fit to just incorporate into that fix.
I will be more than happy to open a JIRA issue for this problem, but I
definitely will like to know your opinion on this before I do open one.
Regards,
Garima.
----------------------------------------------------------------------
This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.