You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Brett Porter <br...@apache.org> on 2008/06/22 06:37:21 UTC
initial comments on SAT solver document
Hi Oleg, all,
I haven't dug into any of the code yet, but I've read through the
document you were working on. It seems positive and makes sense. I did
get a bit lost on the meaning of the "Tree Builder" section as it
doesn't seem to describe the current tree or graph technique, but
maybe illustrate the possible combinations?
Anyway, a few things caught my attention...
You mentioned about the inconsistency of whether 3.8.1 means "must be
3.8.1" vs "I'm using 3.8.1 right now but I'm not really sure which
versions it works with". Pretty much everyone putting POMs together
does the latter, as they are not trained in specifying ranges, and
because projects do not always follow the guidelines on versioning
such that compatibility can be measured. You also mentioned that some
change to the way ranges are specified might be needed. I'm wondering
what thoughts you have on how we specify dependencies need to be
different from today?
I do think the compatibility question is an interesting one. We could
theoretically build up a compatibility database in the central
repository, even using clirr to make initial assessment on java
artifacts. However this is unfortunately never a binary decision
either, as you may not be using the incompatible parts, and
compatibility can break in non-API ways.
I'm curious about how these work in terms of remote retrieval (ie,
where consulting metadatasource is a costly operation). When I looked
at p2 in Sept last year, they stored all the repository artifact
metadata in a single, large file. Are we expecting to need some
partial or full resolution of the repository content?
Do you think depth resolution is still relevant if we have a better
method? I believe the only depth we really should care about is "1" -
those immediately specified (and particularly those managed by
depMgmt). With that in mind, how can the use of depMgmt help in
reducing the problem space during resolution?
You also mention OSGi resolution at the end. Is it your intent that
one of the goals of the mechanism is to be able to be a complete OSGi
resolver?
I'm also interested in seeing explicit listing of the problems we aim
to solve to ensure we have objectives/scope. For example:
- lack of alternate conflict resolution in current artifact mechanism
- reduction in amount of metadata and artifacts dragged in (ie, stuff
that could use maven 2.0.9 still pulls down whatever version of maven
it was built against, even when it's not used)
- deterministic building over time
Just my initial thoughts - I'll follow your progress as you continue
investigating :)
Cheers,
Brett
--
Brett Porter
brett@apache.org
http://blogs.exist.com/bporter/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Jason van Zyl <ja...@maven.org>.
On 22-Jun-08, at 12:41 AM, Ralph Goers wrote:
> Jason van Zyl wrote:
>> In the world where there is some rigour in that the said project
>> doesn't change the API within a major release they know very well.
>> In Eclipse a great deal of care is taken to make sure that APIs
>> don't change within a major release of the platform and ranges in
>> dependencies are usually specified as ranges that allow the
>> consumption of new versions within a major release version. We're
>> not accustom to this here at Apache because there just isn't that
>> level of rigour yet.
> There are at least hundreds of various software projects that could
> possibly become a dependency to a given software development
> project. While it is great that Eclipse takes great care to maintain
> some measure of compatibility between releases but the same can't be
> said of the vast majority of things a project could be dependent on.
> While it would be great to live in a Utopian world where API
> compatibility is rarely, if ever, broken, that isn't the world we
> will be living in for the forseeable future.
You are used to Apache, Eclipse is not a Utopia and has managed to do
it pretty well. It's a practical upshot of dire need, it's not a "nice
to have" when you have such a large ecosystem, it's a requirement.
> Expecting a project to understand all the compatibility issues of
> its dependencies is not only unreasonable and burdensome, it is
> impossible when those dependencies can dynamically change due to
> version ranges.
>
I disagree, there are lots of projects that have that notion
engrained. I'm not saying that you are not free to lock down your
version but lots of people use ranges in OSGi with commercial and
Eclipse-based dependencies and they have success.
> Ironically, although Eclipse might take great care at maintaining
> compatibility, none of its components have ever found their way into
> one of my projects.
>
> While commercial vendors might be better I've encountered many of
> the same issues with them. They will do a "major upgrade" once every
> 18 months or so and break compatibility if it suits their needs.
> Only point upgrades (purely bug fixes - no new features) are
> guaranteed to be compatible. Remember, breaking compatibility
> doesn't necessarily require an API change.
I fully understand the configuration and behavioral changes also need
to be taken into account.
> It might involve a configuration change, a change in a database
> schema, changes in its dependencies that impact your project in
> other ways, etc.
>>
>>>
>>>
>>> Frankly, even if version ranges worked as above I'd still probably
>>> not use them during a build simply because I want to specify
>>> exactly what is used in the build.
>>
>> A lot of people probably do too, but that's not what you've been
>> doing with the POMs you use now I'm willing to bet. Almost everyone
>> with the mechanism we have now can be subject to exactly what you
>> describe above without knowing it. Most people don't know the "1.0"
>> in their POMs doesn't actually mean use "1.0" all the time.
> If that's what you think then you forget why I've been so slow in
> actually converting my projects from Maven 1 to Maven 2 (which I
> finally found the time to do by the way). We used the jar override
> feature in Maven 1 to specify the version of EVERY dependency. I
> wasn't able to convert to Maven 2 until we could accomplish the same
> thing - which we are now able to do in a much better way.
>
> However, if simply specifying a specific version in a managed
> dependency doesn't guarantee that the specified version of that
> dependency is used then I see three problems:
> 1. The documentation isn't clear enough.
> 2. I need to modify my managed dependencies to specify the start and
> end of the version range to the same version.
> 3. Since most people don't expect this behavior one would normally
> consider it to be a bug. Certainly forcing people to have to do the
> previous step is downright awful. It would be much better to provide
> an option that forces only the specified version to be used.
>>
>>> However, it would be great to know that multiple projects have
>>> dependencies on incompatible versions of the same thing. It would
>>> also be great to have reports that indicate which dependencies
>>> could be safely upgraded.
>>
>> It should be the exception that it changes in a major version.
>>
> That's all well and good but it completely misses the point. If you
> really think that Maven can cause every possible project that a
> project might list as a dependency to do things "correctly" you are
> going to end up very frustrated. Maven needs to deal with the world
> as it is.
Where did I say anything resembling the assumption that this happens
everywhere? I'm simply stating it's not as rare as you might think.
I'm also saying use ranges where it makes sense and there is a
difference in how the ranges are used in development versus runtime
and here I'm specifically talking about OSGi. After testing something
and passing through QA it would obviously be not very useful to let
the range calculate something differently after it had been tested.
>
>
> Again, I will reiterate. IMO, without projects declaring themselves
> which prior versions they are compatible with no version ranging
> scheme will ever be able to produce a build you can be even close to
> 100% confident in. As always, you are free to disagree.
I'm not disagreeing with you, and if that is what you need then you
just specify that known set in your dependency range. You were saying
it's not supported in the POM, but all you need to do is state the
values in the ranges that you know yourself to be valid for your
project. As you update, leave the previous version in the range if you
know it to be compatible and add the next one.
I wasn't in anyway implying the information is good enough anywhere
but a few places to rely on the "within a major version this remains
API (and behaviorally) compatible." rule. You learn pretty fast where
this is possible or not.
>
>
> Ralph
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
Selfish deeds are the shortest path to self destruction.
-- The Seven Samuari, Akira Kirosawa
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Ralph Goers <Ra...@dslextreme.com>.
Jason van Zyl wrote:
> In the world where there is some rigour in that the said project
> doesn't change the API within a major release they know very well. In
> Eclipse a great deal of care is taken to make sure that APIs don't
> change within a major release of the platform and ranges in
> dependencies are usually specified as ranges that allow the
> consumption of new versions within a major release version. We're not
> accustom to this here at Apache because there just isn't that level of
> rigour yet.
There are at least hundreds of various software projects that could
possibly become a dependency to a given software development project.
While it is great that Eclipse takes great care to maintain some measure
of compatibility between releases but the same can't be said of the vast
majority of things a project could be dependent on. While it would be
great to live in a Utopian world where API compatibility is rarely, if
ever, broken, that isn't the world we will be living in for the
forseeable future. Expecting a project to understand all the
compatibility issues of its dependencies is not only unreasonable and
burdensome, it is impossible when those dependencies can dynamically
change due to version ranges.
Ironically, although Eclipse might take great care at maintaining
compatibility, none of its components have ever found their way into one
of my projects.
While commercial vendors might be better I've encountered many of the
same issues with them. They will do a "major upgrade" once every 18
months or so and break compatibility if it suits their needs. Only point
upgrades (purely bug fixes - no new features) are guaranteed to be
compatible. Remember, breaking compatibility doesn't necessarily require
an API change. It might involve a configuration change, a change in a
database schema, changes in its dependencies that impact your project in
other ways, etc.
>
>>
>>
>> Frankly, even if version ranges worked as above I'd still probably
>> not use them during a build simply because I want to specify exactly
>> what is used in the build.
>
> A lot of people probably do too, but that's not what you've been doing
> with the POMs you use now I'm willing to bet. Almost everyone with the
> mechanism we have now can be subject to exactly what you describe
> above without knowing it. Most people don't know the "1.0" in their
> POMs doesn't actually mean use "1.0" all the time.
If that's what you think then you forget why I've been so slow in
actually converting my projects from Maven 1 to Maven 2 (which I finally
found the time to do by the way). We used the jar override feature in
Maven 1 to specify the version of EVERY dependency. I wasn't able to
convert to Maven 2 until we could accomplish the same thing - which we
are now able to do in a much better way.
However, if simply specifying a specific version in a managed dependency
doesn't guarantee that the specified version of that dependency is used
then I see three problems:
1. The documentation isn't clear enough.
2. I need to modify my managed dependencies to specify the start and end
of the version range to the same version.
3. Since most people don't expect this behavior one would normally
consider it to be a bug. Certainly forcing people to have to do the
previous step is downright awful. It would be much better to provide an
option that forces only the specified version to be used.
>
>> However, it would be great to know that multiple projects have
>> dependencies on incompatible versions of the same thing. It would
>> also be great to have reports that indicate which dependencies could
>> be safely upgraded.
>
> It should be the exception that it changes in a major version.
>
That's all well and good but it completely misses the point. If you
really think that Maven can cause every possible project that a project
might list as a dependency to do things "correctly" you are going to end
up very frustrated. Maven needs to deal with the world as it is.
Again, I will reiterate. IMO, without projects declaring themselves
which prior versions they are compatible with no version ranging scheme
will ever be able to produce a build you can be even close to 100%
confident in. As always, you are free to disagree.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Jason van Zyl <ja...@maven.org>.
On 21-Jun-08, at 11:16 PM, Ralph Goers wrote:
> See below
>
> Brett Porter wrote:
>> Hi Oleg, all,
>>
>> I haven't dug into any of the code yet, but I've read through the
>> document you were working on. It seems positive and makes sense. I
>> did get a bit lost on the meaning of the "Tree Builder" section as
>> it doesn't seem to describe the current tree or graph technique,
>> but maybe illustrate the possible combinations?
>>
>> Anyway, a few things caught my attention...
>>
>> You mentioned about the inconsistency of whether 3.8.1 means "must
>> be 3.8.1" vs "I'm using 3.8.1 right now but I'm not really sure
>> which versions it works with". Pretty much everyone putting POMs
>> together does the latter, as they are not trained in specifying
>> ranges, and because projects do not always follow the guidelines on
>> versioning such that compatibility can be measured. You also
>> mentioned that some change to the way ranges are specified might be
>> needed. I'm wondering what thoughts you have on how we specify
>> dependencies need to be different from today?
> This is one of the problems that has bothered me for a while. It
> makes no sense for a project to say I'm compatible with 3.8.x. How
> in the world could they know that?
In the world where there is some rigour in that the said project
doesn't change the API within a major release they know very well. In
Eclipse a great deal of care is taken to make sure that APIs don't
change within a major release of the platform and ranges in
dependencies are usually specified as ranges that allow the
consumption of new versions within a major release version. We're not
accustom to this here at Apache because there just isn't that level of
rigour yet.
> The dependent project might release a new 3.8.x release that isn't
> compatible.
Commercial vendors take a lot more care because people complain
wildly, and Eclipse takes a lot more care. It's not rocket science to
use CLIRR, we along with a lot of other open source projects have just
been lazy.
> Thus, for any kind of version range to have any meaning, projects
> must declare what prior versions they are compatible with using a
> declaration that isn't part of the current pom syntax.
I'm not sure it has to be, there generally isn't a very good reason to
break API compatibility in a major release.
> Then when a project specifies a dependency on 3.8.1 if there is a
> 3.8.2 that specifies it is compatible with any prior 3.8.x release
> it would be fine to use 3.8.2 in the project with the dependency.
> Without such a specification, in my opinion version ranges aren't
> particularly useful.
I disagree. There are generally accepted rules about changes in APIs
in major and minor releases. Takes a conscious effort but tooling can
help as well. There's a big set of tooling available in the eclipse
platform to help with this coming in 3.4.
>
>
> Frankly, even if version ranges worked as above I'd still probably
> not use them during a build simply because I want to specify exactly
> what is used in the build.
A lot of people probably do too, but that's not what you've been doing
with the POMs you use now I'm willing to bet. Almost everyone with the
mechanism we have now can be subject to exactly what you describe
above without knowing it. Most people don't know the "1.0" in their
POMs doesn't actually mean use "1.0" all the time.
> However, it would be great to know that multiple projects have
> dependencies on incompatible versions of the same thing. It would
> also be great to have reports that indicate which dependencies could
> be safely upgraded.
It should be the exception that it changes in a major version.
>
>
> Ralph
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
People develop abstractions by generalizing from concrete examples.
Every attempt to determine the correct abstraction on paper without
actually developing a running system is doomed to failure. No one
is that smart. A framework is a resuable design, so you develop it by
looking at the things it is supposed to be a design of. The more
examples
you look at, the more general your framework will be.
-- Ralph Johnson & Don Roberts, Patterns for Evolving Frameworks
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Michael McCallum <gh...@apache.org>.
On Mon, 23 Jun 2008 14:26:38 Michael McCallum wrote:
> have a meeting will explain in more detail later...
have not forgotten just been really busy and want to give this a proper
treatment...
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Michael McCallum <gh...@apache.org>.
On Mon, 23 Jun 2008 14:18:24 Ralph Goers wrote:
> > 2) because and i might be wrong but dependencyManagement messes with the
> > version defined up the chain when what I want is a localised enforcement,
> > its important to isolate the dependency and apply exclusions and version
> > restrictions to guarantee that I get consistent resolution.
>
> What do you mean "up the chain"? dependencyManagement should behave as
> if you specified the version wherever it is encountered in an actual
> dependency. So managed dependencies specified in one part of a
> dependency tree should only affect other parts of the tree in the same
> way they would with the fully specified dependency.
thats my concern...
> As far as I know version ranges should work just fine. I haven't tried
> it because I don't use them. I don't know what you mean by defining
> dependencies "in terms of each other". Do you mean versions 1.1-1.1.5 of
> Project A can use versions 1.0-1.x of Project B? This works until you
> start throwing in Project C, D, E, F, etc which also have dependencies.
> At some point understanding what is really compatible with what is
> extremely difficult.
have a meeting will explain in more detail later...
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Ralph Goers <Ra...@dslextreme.com>.
Michael McCallum wrote:
> On Mon, 23 Jun 2008 11:20:40 Ralph Goers wrote:
>
>> Why aren't you using dependencyManagement instead of the technique shown
>> below?
>>
> Good question, please refute/correct/flame if/where possible
>
> 1) there was no way to use dependencyManagement with deps when I adopted the
> technique. This works now with pom depedencies i think but I'm not certain it
> provides all the benefits.
>
I believe it should work now.
> 2) because and i might be wrong but dependencyManagement messes with the
> version defined up the chain when what I want is a localised enforcement, its
> important to isolate the dependency and apply exclusions and version
> restrictions to guarantee that I get consistent resolution.
>
What do you mean "up the chain"? dependencyManagement should behave as
if you specified the version wherever it is encountered in an actual
dependency. So managed dependencies specified in one part of a
dependency tree should only affect other parts of the tree in the same
way they would with the fully specified dependency.
> 3) also because i actually use jars for the composites/constraints so that the
> poms and pom properties end up in my aggregations - they are single merged
> together... reasons are OT - so i can unravel what actually ended up in the
> jar if needed.
>
I have no idea how this is a benefit. Probably just me not understanding
something.
> 4) is it possible to use version ranges in dependencyManagement? I was
> thinking not but have not tried recently. Once you get a reasonable set of
> constrained external depdencies you naturally want to define them in terms of
> each other to reduce overhead and to ensure better consistency
>
As far as I know version ranges should work just fine. I haven't tried
it because I don't use them. I don't know what you mean by defining
dependencies "in terms of each other". Do you mean versions 1.1-1.1.5 of
Project A can use versions 1.0-1.x of Project B? This works until you
start throwing in Project C, D, E, F, etc which also have dependencies.
At some point understanding what is really compatible with what is
extremely difficult.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Michael McCallum <gh...@apache.org>.
On Mon, 23 Jun 2008 11:20:40 Ralph Goers wrote:
> Why aren't you using dependencyManagement instead of the technique shown
> below?
Good question, please refute/correct/flame if/where possible
1) there was no way to use dependencyManagement with deps when I adopted the
technique. This works now with pom depedencies i think but I'm not certain it
provides all the benefits.
2) because and i might be wrong but dependencyManagement messes with the
version defined up the chain when what I want is a localised enforcement, its
important to isolate the dependency and apply exclusions and version
restrictions to guarantee that I get consistent resolution.
3) also because i actually use jars for the composites/constraints so that the
poms and pom properties end up in my aggregations - they are single merged
together... reasons are OT - so i can unravel what actually ended up in the
jar if needed.
4) is it possible to use version ranges in dependencyManagement? I was
thinking not but have not tried recently. Once you get a reasonable set of
constrained external depdencies you naturally want to define them in terms of
each other to reduce overhead and to ensure better consistency
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Ralph Goers <Ra...@dslextreme.com>.
Why aren't you using dependencyManagement instead of the technique shown
below?
Michael McCallum wrote:
> On Sun, 22 Jun 2008 18:16:37 Ralph Goers wrote:
>
>> This is one of the problems that has bothered me for a while. It makes
>> no sense for a project to say I'm compatible with 3.8.x. How in the
>> world could they know that? The dependent project might release a new
>> 3.8.x release that isn't compatible. Thus, for any kind of version range
>> to have any meaning, projects must declare what prior versions they are
>> compatible with using a declaration that isn't part of the current pom
>> syntax. Then when a project specifies a dependency on 3.8.1 if there is
>> a 3.8.2 that specifies it is compatible with any prior 3.8.x release it
>> would be fine to use 3.8.2 in the project with the dependency. Without
>> such a specification, in my opinion version ranges aren't particularly
>> useful.
>>
> I get around this by wrapping a well defined version around a strict range
> e.g.
>
> i have a special pom project that holds the version of spring that all my
> projects should use currently at 7.1 which says use 2.0.7
>
> so all my projects use a range of
> <dependency>
> <version>[7,8-!)</version>
> <groupId>my.domain.constraint</groupId>
> <artifactId>my.domain.constraint.spring</artifactId>
> </dependency>
>
> and my.domain.constraint.spring is defined in brief as
>
> <project ...>
> ...
> <dependency>
> <version>[2.0.7]</version>
> <groupId>org.springframework</groupId>
> <artifactId>context</artifactId>
> </dependency>
> ...
> </project>
>
> the only overhead to this is a few extra artifacts with no code... hardly a
> major... there is also the added bonus that you have a convenient place to
> control exclusions like... commons-logging
>
> having read more of the thread... most of the apache projects are small and
> have fairly simple trees you can get away with locking vrsion down simply. In
> my case the trees are fairly horendous and using various pattern and
> refactoring techiniques was necessary to get clean consistent trees.
>
> It does mean that I encapsulate the the third party version as shown above and
> it also lets me guarantee that dependency graph produced by spring,
> hibernate, jmx, etc etc is the same when used in all my other projects. Each
> time there is a new release I can just do depedency:resolve and/or
> depedency:tree and make sure the trees look the same and then run tests on
> all my projects against the new version to ensure nothing breaks. If
> something does I can decide to bump the major version of the 'constraint'
> project or fix the problem in the broken project...
> i don't think that you can ever make a generic solution to this problem you
> always need a flexible solution close to home, because ultimately you need to
> manage when and where things change if you ever hope to produce stable
> software in a timely fashion.
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Michael McCallum <gh...@apache.org>.
On Sun, 22 Jun 2008 18:16:37 Ralph Goers wrote:
> This is one of the problems that has bothered me for a while. It makes
> no sense for a project to say I'm compatible with 3.8.x. How in the
> world could they know that? The dependent project might release a new
> 3.8.x release that isn't compatible. Thus, for any kind of version range
> to have any meaning, projects must declare what prior versions they are
> compatible with using a declaration that isn't part of the current pom
> syntax. Then when a project specifies a dependency on 3.8.1 if there is
> a 3.8.2 that specifies it is compatible with any prior 3.8.x release it
> would be fine to use 3.8.2 in the project with the dependency. Without
> such a specification, in my opinion version ranges aren't particularly
> useful.
I get around this by wrapping a well defined version around a strict range
e.g.
i have a special pom project that holds the version of spring that all my
projects should use currently at 7.1 which says use 2.0.7
so all my projects use a range of
<dependency>
<version>[7,8-!)</version>
<groupId>my.domain.constraint</groupId>
<artifactId>my.domain.constraint.spring</artifactId>
</dependency>
and my.domain.constraint.spring is defined in brief as
<project ...>
...
<dependency>
<version>[2.0.7]</version>
<groupId>org.springframework</groupId>
<artifactId>context</artifactId>
</dependency>
...
</project>
the only overhead to this is a few extra artifacts with no code... hardly a
major... there is also the added bonus that you have a convenient place to
control exclusions like... commons-logging
having read more of the thread... most of the apache projects are small and
have fairly simple trees you can get away with locking vrsion down simply. In
my case the trees are fairly horendous and using various pattern and
refactoring techiniques was necessary to get clean consistent trees.
It does mean that I encapsulate the the third party version as shown above and
it also lets me guarantee that dependency graph produced by spring,
hibernate, jmx, etc etc is the same when used in all my other projects. Each
time there is a new release I can just do depedency:resolve and/or
depedency:tree and make sure the trees look the same and then run tests on
all my projects against the new version to ensure nothing breaks. If
something does I can decide to bump the major version of the 'constraint'
project or fix the problem in the broken project...
i don't think that you can ever make a generic solution to this problem you
always need a flexible solution close to home, because ultimately you need to
manage when and where things change if you ever hope to produce stable
software in a timely fashion.
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Ralph Goers <Ra...@dslextreme.com>.
Jason van Zyl wrote:
>>
>>>
>>> Frankly, even if version ranges worked as above I'd still probably
>>> not use them during a build simply because I want to specify exactly
>>> what is used in the build. However, it would be great to know that
>>> multiple projects have dependencies on incompatible versions of the
>>> same thing. It would also be great to have reports that indicate
>>> which dependencies could be safely upgraded.
>> The approach above goes in this direction - it might give some
>> guarantees. But not too strong - we cannot test behavioral
>> compatibility.
>
> Not without some decent integration tests no. But you have a pretty
> good chance with good ITs. We catch lots of shit that compiles, but
> bombs in our ITs.
>
I believe Oleg meant behavior incompatibilities of third party
artifacts, not Maven. Maven's ITs can't be expected to detect problems
with other projects.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Jason van Zyl <ja...@maven.org>.
On 22-Jun-08, at 12:03 AM, Oleg Gusakov wrote:
>
>
> Ralph Goers wrote:
>> See below
>>
>> Brett Porter wrote:
>>> Hi Oleg, all,
>>>
>>> I haven't dug into any of the code yet, but I've read through the
>>> document you were working on. It seems positive and makes sense. I
>>> did get a bit lost on the meaning of the "Tree Builder" section as
>>> it doesn't seem to describe the current tree or graph technique,
>>> but maybe illustrate the possible combinations?
>>>
>>> Anyway, a few things caught my attention...
>>>
>>> You mentioned about the inconsistency of whether 3.8.1 means "must
>>> be 3.8.1" vs "I'm using 3.8.1 right now but I'm not really sure
>>> which versions it works with". Pretty much everyone putting POMs
>>> together does the latter, as they are not trained in specifying
>>> ranges, and because projects do not always follow the guidelines
>>> on versioning such that compatibility can be measured. You also
>>> mentioned that some change to the way ranges are specified might
>>> be needed. I'm wondering what thoughts you have on how we specify
>>> dependencies need to be different from today?
>> This is one of the problems that has bothered me for a while. It
>> makes no sense for a project to say I'm compatible with 3.8.x. How
>> in the world could they know that? The dependent project might
>> release a new 3.8.x release that isn't compatible. Thus, for any
>> kind of version range to have any meaning, projects must declare
>> what prior versions they are compatible with using a declaration
>> that isn't part of the current pom syntax. Then when a project
>> specifies a dependency on 3.8.1 if there is a 3.8.2 that specifies
>> it is compatible with any prior 3.8.x release it would be fine to
>> use 3.8.2 in the project with the dependency. Without such a
>> specification, in my opinion version ranges aren't particularly
>> useful.
> In my personal opinion - wild versioning should go away. We need to
> impose stardards on versions: at least major.minor.build-[etc]. So -
> any additive changes increment minir version, any incompatible
> changes increment major. Build - for developer/QA test/release
> iterations. We can enforce those rules for binary compatibility, but
> behavioral compatibility will still be volatile :(
>>
>> Frankly, even if version ranges worked as above I'd still probably
>> not use them during a build simply because I want to specify
>> exactly what is used in the build. However, it would be great to
>> know that multiple projects have dependencies on incompatible
>> versions of the same thing. It would also be great to have reports
>> that indicate which dependencies could be safely upgraded.
> The approach above goes in this direction - it might give some
> guarantees. But not too strong - we cannot test behavioral
> compatibility.
Not without some decent integration tests no. But you have a pretty
good chance with good ITs. We catch lots of shit that compiles, but
bombs in our ITs.
>
>
> Thanks,
> Oleg
>>
>> Ralph
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
You are never dedicated to something you have complete confidence in.
No one is fanatically shouting that the sun is going to rise tomorrow.
They know it is going to rise tomorrow. When people are fanatically
dedicated to political or religious faiths or any other kind of
dogmas or goals, it's always because these dogmas or
goals are in doubt.
-- Robert Pirzig, Zen and the Art of Motorcycle Maintenance
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Oleg Gusakov <ol...@gmail.com>.
Ralph Goers wrote:
> See below
>
> Brett Porter wrote:
>> Hi Oleg, all,
>>
>> I haven't dug into any of the code yet, but I've read through the
>> document you were working on. It seems positive and makes sense. I
>> did get a bit lost on the meaning of the "Tree Builder" section as it
>> doesn't seem to describe the current tree or graph technique, but
>> maybe illustrate the possible combinations?
>>
>> Anyway, a few things caught my attention...
>>
>> You mentioned about the inconsistency of whether 3.8.1 means "must be
>> 3.8.1" vs "I'm using 3.8.1 right now but I'm not really sure which
>> versions it works with". Pretty much everyone putting POMs together
>> does the latter, as they are not trained in specifying ranges, and
>> because projects do not always follow the guidelines on versioning
>> such that compatibility can be measured. You also mentioned that some
>> change to the way ranges are specified might be needed. I'm wondering
>> what thoughts you have on how we specify dependencies need to be
>> different from today?
> This is one of the problems that has bothered me for a while. It makes
> no sense for a project to say I'm compatible with 3.8.x. How in the
> world could they know that? The dependent project might release a new
> 3.8.x release that isn't compatible. Thus, for any kind of version
> range to have any meaning, projects must declare what prior versions
> they are compatible with using a declaration that isn't part of the
> current pom syntax. Then when a project specifies a dependency on
> 3.8.1 if there is a 3.8.2 that specifies it is compatible with any
> prior 3.8.x release it would be fine to use 3.8.2 in the project with
> the dependency. Without such a specification, in my opinion version
> ranges aren't particularly useful.
In my personal opinion - wild versioning should go away. We need to
impose stardards on versions: at least major.minor.build-[etc]. So - any
additive changes increment minir version, any incompatible changes
increment major. Build - for developer/QA test/release iterations. We
can enforce those rules for binary compatibility, but behavioral
compatibility will still be volatile :(
>
> Frankly, even if version ranges worked as above I'd still probably not
> use them during a build simply because I want to specify exactly what
> is used in the build. However, it would be great to know that multiple
> projects have dependencies on incompatible versions of the same thing.
> It would also be great to have reports that indicate which
> dependencies could be safely upgraded.
The approach above goes in this direction - it might give some
guarantees. But not too strong - we cannot test behavioral compatibility.
Thanks,
Oleg
>
> Ralph
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Ralph Goers <Ra...@dslextreme.com>.
See below
Brett Porter wrote:
> Hi Oleg, all,
>
> I haven't dug into any of the code yet, but I've read through the
> document you were working on. It seems positive and makes sense. I did
> get a bit lost on the meaning of the "Tree Builder" section as it
> doesn't seem to describe the current tree or graph technique, but
> maybe illustrate the possible combinations?
>
> Anyway, a few things caught my attention...
>
> You mentioned about the inconsistency of whether 3.8.1 means "must be
> 3.8.1" vs "I'm using 3.8.1 right now but I'm not really sure which
> versions it works with". Pretty much everyone putting POMs together
> does the latter, as they are not trained in specifying ranges, and
> because projects do not always follow the guidelines on versioning
> such that compatibility can be measured. You also mentioned that some
> change to the way ranges are specified might be needed. I'm wondering
> what thoughts you have on how we specify dependencies need to be
> different from today?
This is one of the problems that has bothered me for a while. It makes
no sense for a project to say I'm compatible with 3.8.x. How in the
world could they know that? The dependent project might release a new
3.8.x release that isn't compatible. Thus, for any kind of version range
to have any meaning, projects must declare what prior versions they are
compatible with using a declaration that isn't part of the current pom
syntax. Then when a project specifies a dependency on 3.8.1 if there is
a 3.8.2 that specifies it is compatible with any prior 3.8.x release it
would be fine to use 3.8.2 in the project with the dependency. Without
such a specification, in my opinion version ranges aren't particularly
useful.
Frankly, even if version ranges worked as above I'd still probably not
use them during a build simply because I want to specify exactly what is
used in the build. However, it would be great to know that multiple
projects have dependencies on incompatible versions of the same thing.
It would also be great to have reports that indicate which dependencies
could be safely upgraded.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: initial comments on SAT solver document
Posted by Jason van Zyl <ja...@maven.org>.
On 21-Jun-08, at 9:37 PM, Brett Porter wrote:
> Hi Oleg, all,
>
> I haven't dug into any of the code yet, but I've read through the
> document you were working on. It seems positive and makes sense. I
> did get a bit lost on the meaning of the "Tree Builder" section as
> it doesn't seem to describe the current tree or graph technique, but
> maybe illustrate the possible combinations?
>
> Anyway, a few things caught my attention...
>
> You mentioned about the inconsistency of whether 3.8.1 means "must
> be 3.8.1" vs "I'm using 3.8.1 right now but I'm not really sure
> which versions it works with". Pretty much everyone putting POMs
> together does the latter, as they are not trained in specifying
> ranges, and because projects do not always follow the guidelines on
> versioning such that compatibility can be measured. You also
> mentioned that some change to the way ranges are specified might be
> needed. I'm wondering what thoughts you have on how we specify
> dependencies need to be different from today?
>
> I do think the compatibility question is an interesting one. We
> could theoretically build up a compatibility database in the central
> repository, even using clirr to make initial assessment on java
> artifacts. However this is unfortunately never a binary decision
> either, as you may not be using the incompatible parts, and
> compatibility can break in non-API ways.
>
There will never be a heuristic that can replace people just taking
some care with the APIs and their testing. Without that combination on
the part of the project an external party trying to collect these
metrics to use is probably not going to be of great value. Retaining
API compatibility along with a high degree of coverage and you
probably have a useful metric and really if it's not a conscious
effort the API will be broken even if inadvertently. The mechanical
turk is not going to help us much there.
> I'm curious about how these work in terms of remote retrieval (ie,
> where consulting metadatasource is a costly operation). When I
> looked at p2 in Sept last year, they stored all the repository
> artifact metadata in a single, large file.
This is just one implementation of the MetadataRepository API, it's
not very costly because they only deal with dependencies. Our
currently because of the POM swizzling we do.
> Are we expecting to need some partial or full resolution of the
> repository content?
>
When you deal with ranges the number of possible paths grows very
quickly, and the SAT solver is very fast. You could predigest
dependency information but pre-calculated subgraphs are probably not
useful as one change in a range in a path and you're recalculating
anyway.
> Do you think depth resolution is still relevant if we have a better
> method? I believe the only depth we really should care about is "1"
> - those immediately specified (and particularly those managed by
> depMgmt). With that in mind, how can the use of depMgmt help in
> reducing the problem space during resolution?
It will boil down to something far simpler. The SAT solver is
guaranteed to find you a working solution. All ranges are a set where
that set can be one, or fixed. If you want it weighted to what you say
in your POM then fix the version. The depMan should not change the
calculation like it does now, again if you want it fixed then fix it.
We will let the SAT solver find the solution and we take the magic out.
>
>
> You also mention OSGi resolution at the end. Is it your intent that
> one of the goals of the mechanism is to be able to be a complete
> OSGi resolver?
We already have an OSGi resolver in Tycho, this is not the same thing.
What is in p2 uses SAT, but this is not what's OSGi. No one has made
it clear if that's what OSGi proper will use but right now the OSGi
resolver is a simple state machine. What's in p2 is not an OSGi
resolver even though what it retrieves is ultimately fed into an OSGi
state machine.
>
>
> I'm also interested in seeing explicit listing of the problems we
> aim to solve to ensure we have objectives/scope. For example:
> - lack of alternate conflict resolution in current artifact mechanism
> - reduction in amount of metadata and artifacts dragged in (ie,
> stuff that could use maven 2.0.9 still pulls down whatever version
> of maven it was built against, even when it's not used)
> - deterministic building over time
>
> Just my initial thoughts - I'll follow your progress as you continue
> investigating :)
>
> Cheers,
> Brett
>
> --
> Brett Porter
> brett@apache.org
> http://blogs.exist.com/bporter/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
Three people can keep a secret provided two of them are dead.
-- Unknown
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org