You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Stephen Connolly <st...@gmail.com> on 2014/06/20 14:51:14 UTC
"supplies" concept proposal (was "provides" could be "proffers")
"supplies" concept proposal
===========================
Introduction
------------
The following is a proposal for Maven in a post-modelVersion-4.0.0 era. The
aim of this proposal is to simplify the management of dependency trees in
the decentralised era of artifact production that we find ourselves in.
The core issue is that different organisations can produce artifacts that
may overlap. The easiest example is the servlet-api. If we restrict
ourselves to version 2.5 of the servlet specification there are quite a few
artifacts that all deliver the exact same content:
* `jetty:servlet-api:2.5-6.0.2`
* `org.daisy.libs:servlet-api:2.5.0`
* `org.mortbay.jetty:servlet-api-2.5:6.1.14`
* `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final`
* etc
**Note:** this is a generic problem that is not restricted to the
servlet-api, the servlet-api just provides the example that will be most
familiar to everyone.
So where these multiple artifacts supplying the equivalent content becomes
a problem is when the dependency tree is being calculated. If you have two
dependencies each declaring transitive dependencies on different artifacts
that supply equivalent content, then you end up with two copies of the same
JAR file in your classpath.
In the case of the servlet-api, the hack most people use is to declare the
servlet-api with scope `provided` thus preventing it from being transitive.
This is, however, a hack. In a more ideal world it would be better to let
the servlet-api be transitive and only when we get to the WAR module would
we declare that a specific servlet-api is to be provided in the containers
that the WAR is targets for deployment into.
We can take a second example that does not have the luxury of a *de facto*
hack.
* `javax.faces:jsf-api:2.1`
* `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final`
* `org.apache.myfaces.core:myfaces-api:2.1.13`
Now in the case of the JSF API, you are supposed to bundle the JSF API in
your WAR file. So if I use three JSF component libraries, I could very well
end up with three different but equivalent JSF API jars in my WAR file.
Ideally what we want is some way of telling Maven that these artifacts are
equivalent.
Proposal
--------
Introduce the concept of "supplies" to the project model. The concept needs
three changes to the project model:
1. An explicit top level construct for a project to explicitly declare
up-front artifacts that it knows - at the time the project is being
authored - to contain equivalent content to at least a subset of the
project's content. Declarations could include a claim from: `subset`,
`superset`, `disjoint` and `equivalent` with the default being `disjoint`.
2. An explicit sub-element of the `dependency` construct to allow consumers
to *post-facto* declare a specific dependency as supplying equivalent
content for other dependencies
3. An extension to the `dependency/excludes/exclude` construct to allow
consumers to remove claims a dependency makes with respect to supplying
equivalent content
By way of illustration, here are some examples of these constructs mapped
to a Model Version 4.0.0 like XML schema. As the post-modelVersion-4.0.0
schema is not yet known, this represents the best way to illustrate how the
concept will work, but note that this proposal does not suggest a schema
for this concept.
### Example 1
This illustrates how we would want, say, the `myfaces-api` project model to
look.
```
<project>
<groupId>org.apache.myfaces.core</groupId>
<artifactId>myfaces-api</artifactId>
<version>2.1.3</version>
...
<supplyManagement>
<supplies>
<groupId>javax.faces</groupId>
<artifactId>jsf-api</artifactId>
<version>[2.1,2.2)</version>
<claim>superset</claim>
<supplies>
<supplies>
<groupId>org.jboss.spec.javax.faces</groupId>
<artifactId>jboss-jsf-api_2.1_spec</artifactId>
<claim>equivalent</claim>
<supplies>
</supplyManagement>
...
</project>
```
This indicates that the `myfaces-api` artifact is intended to be useable as
a drop-in replacement for either the `javax.faces:jsf-api` artifact within
a bounded range or for any version of the
`org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you get a
supplier conflict in your classpath, then Maven should fail the build.
For example if somebody forked `myfaces-api` but did not list `myfaces-api`
in the fork's supplyManagement and you end up with both `myfaces-api` and
`myfaces-fork-api` in your classpath. Maven can detect that there are two
dependencies that both claim to supply `javax.faces:jsf-api` and fail the
build, thereby letting the user add either exclusions or additional
supplies information to one of the artifacts and preventing duplicate
artifacts on the classpath. The build need not be failed if the supplies
claims provide a resolution. e.g. if the claim is `equivalent` then that
implies that there is a 1:1 mapping and hence the artifacts are drop-in
replacements. However where the claim is `superset` we cannot know that the
extra content in our artifact is the same as the extra content in another
artifact which has a superset of `javax.faces:jsf-api`.
### Example 2
This illustrates a JSF component library that is working with the existing
JSF APIs
```
<project>
...
<dependencies>
<dependency>
<groupId>javax.faces</groupId>
<artifactId>jsf-api</artifactId>
<version>2.1</version>
<supplyManagement>
<supplies>
<groupId>org.jboss.spec.javax.faces</groupId>
<artifactId>jboss-jsf-api_2.1_spec</artifactId>
<claim>equivalent</claim>
<supplies>
<supplies>
<groupId>org.apache.myfaces.core</groupId>
<artifactId>myfaces-api</artifactId>
<version>[2.1,2.2)</version>
<claim>equivalent</claim>
</supplies>
</supplyManagement>
<dependency>
</dependencies>
...
</project>
```
In this case we are publishing a transitive dependency with additional
supplyManagement injected. Consumers of this project would thus gain the
benefit of collapsing their transitive dependencies for any of these three
artifacts. As all artifacts are declared with `equivalent` claim, thus the
nearest of those three artifacts to the project will win as per the
standard dependency resolution rules when dealing with conflicting version
requirements in the transitive dependency tree.
### Example 3
Finally, there is the case where you need to correct an incorrect claim of
supply
```
<project>
...
<dependencies>
<dependency>
<groupId>javax.faces</groupId>
<artifactId>jsf-api</artifactId>
<version>2.1</version>
<exclusions>
<exclusion>
<groupId>org.jboss.spec.javax.faces</groupId>
<artifactId>jboss-jsf-api_2.2_spec</artifactId>
<scope>supplies</scope>
<exclusion>
</exclusions>
<dependency>
</dependencies>
...
</project>
```
This would typically be coupled with adding back in a correct supplies
definition, but we need to allow for people to correct the graph after the
fact of their dependencies being deployed to the remote repository.
### Claim conflict resolution
The four classes of claim can be resolved using the following matrix
```
+---------------------------------------------------+
| A |
+------------+------------+------------+------------+
| subset | equivalent | superset | disjoint |
+---+------------+------------+------------+------------+------------+
| | subset | conflict | A wins | A wins | conflict |
| +------------+------------+------------+------------+------------+
| | equivalent | B wins | A or B | A wins | conflict |
| B +------------+------------+------------+------------+------------+
| | superset | B wins | B wins | conflict | conflict |
| +------------+------------+------------+------------+------------+
| | disjoint | conflict | conflict | conflict | conflict |
+---+------------+------------+------------+------------+------------+
```
The default unspecified claim is `disjoint` which indicates that some of
the content is reproduced, but not all and there is additional content
added. With such a claim there will always be conflict and the build should
fail until the Project Model is updated to either remove some of the claims
or resolve the dependency clash.
The ideal claim is `equivalent` which indicates that the two artifacts are
bi-directionally substitutable. This does not mean that the contents are
identical. It does mean that they both deliver on the same contract in an
equivalent fashion.
The `subset` and `superset` claims are for aggregation APIs. So for example
the Java EE Web Profile API is a superset of the various spec APIs that
make up the Java EE Web Profile and a subset of the full Java EE
specification. The use of the `subset` claim should be reserved to those
cases that are strict subsets. If anything is added that is not in the
supplied artifact then the correct claim is `disjoint`.
### Validation of supplies claims
We do not want to introduce Java bias with this feature. As a result the
validation of claims and supplies directives will be left to plugins. For
the Java case we should probably provide either/both an enforcer rule or a
maven hosted plugin to assist in checking JAR projects against the declared
supplies declarations, but Maven core should not be concerned with solving
the validation problem.
Similarly, while there may be advantages in a more fine grained API
contract negotiation between dependencies, to bind such a concept into the
project model would significantly taint the Maven project model with more
Java-centric concepts. Given that current software development typically
uses at least two core languages: Java and JavaScript, we should be aiming
to reduce Java-centric constructs from Maven rather than increase them.
### Backporting
It will not be possible to fully express this concept in a modelVersion
4.0.0 project model. Thus if generating 4.0.0 compatible project models,
the aim should be to fully resolve the dependencies of the project using
all available information and express that as the transitive dependencies.
Thus we will not expose the "supplies" information to modelVersion 4.0.0
parsers but we will expose the end results of that and present the final
effective flattened dependency tree.
modelVersion 4.0.0 consumers will thus be no worse off than they already
are and those consumers understanding newer modelVersions can get the
enhanced tree resolution that they would not get otherwise.
Re: "supplies" concept proposal (was "provides" could be "proffers")
Posted by Mark Derricutt <ma...@talios.com>.
On 21 Jun 2014, at 0:51, Stephen Connolly wrote:
> "supplies" concept proposal
I've not yet had a good read thru this thread yet, but I thought I'd
point out the OSGi "requirements and capabilities" work that's going on
with those specs, which is similar/related to a degree.
http://blog.osgi.org/2012/03/requirements-and-capabilities.html
http://wiki.osgi.org/wiki/Provide-Capability
http://wiki.osgi.org/wiki/Require-Capability
Whilst the "supplies" is more about aliasing, and/or working with
automating a <dependency><globalExclude> I think there's potentially
cross over in intent...
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: "supplies" concept proposal (was "provides" could be "proffers")
Posted by Stephen Connolly <st...@gmail.com>.
Write up your proposal and start a separate thread for it... no hijacking
;-)
On 21 June 2014 15:48, Benson Margulies <bi...@gmail.com> wrote:
> At the risk of redundancy, my favorite was the scm element. We
> implement or consider all kinds of nasty kludges to sneak scm-specific
> information through the tiny funnel of the current schema. It should
> be possible to extend the pom for cases like this without all the
> angst about 'breaking tools that read poms.' Whatever else is true of
> pom 5, it should, I claim, have a documented extensibility strategy
> that forces such tools to deal gracefully with incremental additions.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: "supplies" concept proposal (was "provides" could be "proffers")
Posted by Benson Margulies <bi...@gmail.com>.
At the risk of redundancy, my favorite was the scm element. We
implement or consider all kinds of nasty kludges to sneak scm-specific
information through the tiny funnel of the current schema. It should
be possible to extend the pom for cases like this without all the
angst about 'breaking tools that read poms.' Whatever else is true of
pom 5, it should, I claim, have a documented extensibility strategy
that forces such tools to deal gracefully with incremental additions.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: "supplies" concept proposal (was "provides" could be "proffers")
Posted by Stephen Connolly <st...@gmail.com>.
*http://jira.codehaus.org/browse/MNG-5652
<http://jira.codehaus.org/browse/MNG-5652>*
On 21 June 2014 15:28, Jason van Zyl <ja...@takari.io> wrote:
> Throw it into a JIRA and tag the component FDPFC. I can only find 8 issues
> related to features requiring POM format changes. I seem to recall more.
>
> On Jun 20, 2014, at 11:55 AM, Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
> > Well I've had this proposal in one form or another on and off for the
> past
> > 2-3 years. Jason recently stated that he only wants to look at concrete
> > proposals as the first stab for what/how to extend the pom format... so
> > here is a very concrete example proposal ;-)
> >
> >
> > On 20 June 2014 16:50, Paul Benedict <pb...@apache.org> wrote:
> >
> >> Ah, the woes of fragmented jars. That's the bane of EE dependency
> >> management. Figuring out the superset/subset stuff is indeed terrible.
> Good
> >> job, Stephen, to propose a solution for it.
> >>
> >>
> >> Cheers,
> >> Paul
> >>
> >>
> >> On Fri, Jun 20, 2014 at 10:03 AM, Stephen Connolly <
> >> stephen.alan.connolly@gmail.com> wrote:
> >>
> >>> Well this started out from a need for a more effective <excludes>.
> >>>
> >>> The issue with <excludes> is that you have to apply it to each and
> every
> >>> dependency... when what you really want to say is "see this dependency
> >>> here? well that is the same as X, Y and Z so don't pull them in from
> the
> >>> transitive dependencies"
> >>>
> >>> If you like that is the second use case in my proposal.
> >>>
> >>> The first use case is less relevant from the "better excludes" point of
> >>> view, but I believe it to be more useful.
> >>>
> >>> The third use case is for when you don't trust upstream to get things
> >>> right. Keep in mind that if, as a general principle, you don't trust
> >>> upstream to express things mostly correct then what the hell are you
> >> doing
> >>> trusting them to express their transitive dependencies correctly?
> >>>
> >>> Tooling would allow us to validate a pom's claims of equivalence. But
> >>> ultimately I think we need this concept to deal with te fragmentation
> of
> >>> API jars and implementations of APIs that we see taking place in
> central.
> >>>
> >>>
> >>> On 20 June 2014 15:27, Paul Benedict <pb...@apache.org> wrote:
> >>>
> >>>> I am having trouble understanding how grouping together artifacts that
> >>> are
> >>>> "equivalent" gain me anything in my project building. I am already
> >> doing
> >>>> this in my POM today by excluding what's equivalent. Is this proposal
> >>>> simply about adding semantic information to the POM so it's more
> >> apparent
> >>>> what the intent is?
> >>>>
> >>>> Furthermore, whoever defines the equivalence needs to be extremely
> >>> careful.
> >>>> Not all "equivalent" jars are actually equivalent. Most people have
> >>>> learned, for example, that the javaee-api:6.0 jar in Maven is all
> stubs
> >>> and
> >>>> can't be used for unit testing; so people go to find EE jars from
> JBoss
> >>> or
> >>>> GlassFish, to get the full functionality. So I am not sure I'd ever
> >> want
> >>>> someone telling me what's equivalent during consuming.
> >>>>
> >>>>
> >>>> Cheers,
> >>>> Paul
> >>>>
> >>>>
> >>>> On Fri, Jun 20, 2014 at 7:51 AM, Stephen Connolly <
> >>>> stephen.alan.connolly@gmail.com> wrote:
> >>>>
> >>>>> "supplies" concept proposal
> >>>>> ===========================
> >>>>>
> >>>>> Introduction
> >>>>> ------------
> >>>>>
> >>>>> The following is a proposal for Maven in a post-modelVersion-4.0.0
> >> era.
> >>>> The
> >>>>> aim of this proposal is to simplify the management of dependency
> >> trees
> >>> in
> >>>>> the decentralised era of artifact production that we find ourselves
> >> in.
> >>>>>
> >>>>> The core issue is that different organisations can produce artifacts
> >>> that
> >>>>> may overlap. The easiest example is the servlet-api. If we restrict
> >>>>> ourselves to version 2.5 of the servlet specification there are
> >> quite a
> >>>> few
> >>>>> artifacts that all deliver the exact same content:
> >>>>>
> >>>>> * `jetty:servlet-api:2.5-6.0.2`
> >>>>> * `org.daisy.libs:servlet-api:2.5.0`
> >>>>> * `org.mortbay.jetty:servlet-api-2.5:6.1.14`
> >>>>> *
> >> `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final`
> >>>>> * etc
> >>>>>
> >>>>> **Note:** this is a generic problem that is not restricted to the
> >>>>> servlet-api, the servlet-api just provides the example that will be
> >>> most
> >>>>> familiar to everyone.
> >>>>>
> >>>>> So where these multiple artifacts supplying the equivalent content
> >>>> becomes
> >>>>> a problem is when the dependency tree is being calculated. If you
> >> have
> >>>> two
> >>>>> dependencies each declaring transitive dependencies on different
> >>>> artifacts
> >>>>> that supply equivalent content, then you end up with two copies of
> >> the
> >>>> same
> >>>>> JAR file in your classpath.
> >>>>>
> >>>>> In the case of the servlet-api, the hack most people use is to
> >> declare
> >>>> the
> >>>>> servlet-api with scope `provided` thus preventing it from being
> >>>> transitive.
> >>>>> This is, however, a hack. In a more ideal world it would be better to
> >>> let
> >>>>> the servlet-api be transitive and only when we get to the WAR module
> >>>> would
> >>>>> we declare that a specific servlet-api is to be provided in the
> >>>> containers
> >>>>> that the WAR is targets for deployment into.
> >>>>>
> >>>>> We can take a second example that does not have the luxury of a *de
> >>>> facto*
> >>>>> hack.
> >>>>>
> >>>>> * `javax.faces:jsf-api:2.1`
> >>>>> * `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final`
> >>>>> * `org.apache.myfaces.core:myfaces-api:2.1.13`
> >>>>>
> >>>>> Now in the case of the JSF API, you are supposed to bundle the JSF
> >> API
> >>> in
> >>>>> your WAR file. So if I use three JSF component libraries, I could
> >> very
> >>>> well
> >>>>> end up with three different but equivalent JSF API jars in my WAR
> >> file.
> >>>>>
> >>>>> Ideally what we want is some way of telling Maven that these
> >> artifacts
> >>>> are
> >>>>> equivalent.
> >>>>>
> >>>>> Proposal
> >>>>> --------
> >>>>>
> >>>>> Introduce the concept of "supplies" to the project model. The concept
> >>>> needs
> >>>>> three changes to the project model:
> >>>>>
> >>>>> 1. An explicit top level construct for a project to explicitly
> >> declare
> >>>>> up-front artifacts that it knows - at the time the project is being
> >>>>> authored - to contain equivalent content to at least a subset of the
> >>>>> project's content. Declarations could include a claim from: `subset`,
> >>>>> `superset`, `disjoint` and `equivalent` with the default being
> >>>> `disjoint`.
> >>>>> 2. An explicit sub-element of the `dependency` construct to allow
> >>>> consumers
> >>>>> to *post-facto* declare a specific dependency as supplying equivalent
> >>>>> content for other dependencies
> >>>>> 3. An extension to the `dependency/excludes/exclude` construct to
> >> allow
> >>>>> consumers to remove claims a dependency makes with respect to
> >> supplying
> >>>>> equivalent content
> >>>>>
> >>>>> By way of illustration, here are some examples of these constructs
> >>> mapped
> >>>>> to a Model Version 4.0.0 like XML schema. As the
> >>> post-modelVersion-4.0.0
> >>>>> schema is not yet known, this represents the best way to illustrate
> >> how
> >>>> the
> >>>>> concept will work, but note that this proposal does not suggest a
> >>> schema
> >>>>> for this concept.
> >>>>>
> >>>>> ### Example 1
> >>>>>
> >>>>> This illustrates how we would want, say, the `myfaces-api` project
> >>> model
> >>>> to
> >>>>> look.
> >>>>>
> >>>>> ```
> >>>>> <project>
> >>>>> <groupId>org.apache.myfaces.core</groupId>
> >>>>> <artifactId>myfaces-api</artifactId>
> >>>>> <version>2.1.3</version>
> >>>>> ...
> >>>>> <supplyManagement>
> >>>>> <supplies>
> >>>>> <groupId>javax.faces</groupId>
> >>>>> <artifactId>jsf-api</artifactId>
> >>>>> <version>[2.1,2.2)</version>
> >>>>> <claim>superset</claim>
> >>>>> <supplies>
> >>>>> <supplies>
> >>>>> <groupId>org.jboss.spec.javax.faces</groupId>
> >>>>> <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> >>>>> <claim>equivalent</claim>
> >>>>> <supplies>
> >>>>> </supplyManagement>
> >>>>> ...
> >>>>> </project>
> >>>>> ```
> >>>>>
> >>>>> This indicates that the `myfaces-api` artifact is intended to be
> >>> useable
> >>>> as
> >>>>> a drop-in replacement for either the `javax.faces:jsf-api` artifact
> >>>> within
> >>>>> a bounded range or for any version of the
> >>>>> `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you
> >>> get
> >>>> a
> >>>>> supplier conflict in your classpath, then Maven should fail the
> >> build.
> >>>>>
> >>>>> For example if somebody forked `myfaces-api` but did not list
> >>>> `myfaces-api`
> >>>>> in the fork's supplyManagement and you end up with both `myfaces-api`
> >>> and
> >>>>> `myfaces-fork-api` in your classpath. Maven can detect that there are
> >>> two
> >>>>> dependencies that both claim to supply `javax.faces:jsf-api` and fail
> >>> the
> >>>>> build, thereby letting the user add either exclusions or additional
> >>>>> supplies information to one of the artifacts and preventing duplicate
> >>>>> artifacts on the classpath. The build need not be failed if the
> >>> supplies
> >>>>> claims provide a resolution. e.g. if the claim is `equivalent` then
> >>> that
> >>>>> implies that there is a 1:1 mapping and hence the artifacts are
> >> drop-in
> >>>>> replacements. However where the claim is `superset` we cannot know
> >> that
> >>>> the
> >>>>> extra content in our artifact is the same as the extra content in
> >>> another
> >>>>> artifact which has a superset of `javax.faces:jsf-api`.
> >>>>>
> >>>>> ### Example 2
> >>>>>
> >>>>> This illustrates a JSF component library that is working with the
> >>>> existing
> >>>>> JSF APIs
> >>>>>
> >>>>> ```
> >>>>> <project>
> >>>>> ...
> >>>>> <dependencies>
> >>>>> <dependency>
> >>>>> <groupId>javax.faces</groupId>
> >>>>> <artifactId>jsf-api</artifactId>
> >>>>> <version>2.1</version>
> >>>>> <supplyManagement>
> >>>>> <supplies>
> >>>>> <groupId>org.jboss.spec.javax.faces</groupId>
> >>>>> <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> >>>>> <claim>equivalent</claim>
> >>>>> <supplies>
> >>>>> <supplies>
> >>>>> <groupId>org.apache.myfaces.core</groupId>
> >>>>> <artifactId>myfaces-api</artifactId>
> >>>>> <version>[2.1,2.2)</version>
> >>>>> <claim>equivalent</claim>
> >>>>> </supplies>
> >>>>> </supplyManagement>
> >>>>> <dependency>
> >>>>> </dependencies>
> >>>>> ...
> >>>>> </project>
> >>>>> ```
> >>>>>
> >>>>> In this case we are publishing a transitive dependency with
> >> additional
> >>>>> supplyManagement injected. Consumers of this project would thus gain
> >>> the
> >>>>> benefit of collapsing their transitive dependencies for any of these
> >>>> three
> >>>>> artifacts. As all artifacts are declared with `equivalent` claim,
> >> thus
> >>>> the
> >>>>> nearest of those three artifacts to the project will win as per the
> >>>>> standard dependency resolution rules when dealing with conflicting
> >>>> version
> >>>>> requirements in the transitive dependency tree.
> >>>>>
> >>>>> ### Example 3
> >>>>>
> >>>>> Finally, there is the case where you need to correct an incorrect
> >> claim
> >>>> of
> >>>>> supply
> >>>>>
> >>>>>
> >>>>> ```
> >>>>> <project>
> >>>>> ...
> >>>>> <dependencies>
> >>>>> <dependency>
> >>>>> <groupId>javax.faces</groupId>
> >>>>> <artifactId>jsf-api</artifactId>
> >>>>> <version>2.1</version>
> >>>>> <exclusions>
> >>>>> <exclusion>
> >>>>> <groupId>org.jboss.spec.javax.faces</groupId>
> >>>>> <artifactId>jboss-jsf-api_2.2_spec</artifactId>
> >>>>> <scope>supplies</scope>
> >>>>> <exclusion>
> >>>>> </exclusions>
> >>>>> <dependency>
> >>>>> </dependencies>
> >>>>> ...
> >>>>> </project>
> >>>>> ```
> >>>>>
> >>>>> This would typically be coupled with adding back in a correct
> >> supplies
> >>>>> definition, but we need to allow for people to correct the graph
> >> after
> >>>> the
> >>>>> fact of their dependencies being deployed to the remote repository.
> >>>>>
> >>>>> ### Claim conflict resolution
> >>>>>
> >>>>> The four classes of claim can be resolved using the following matrix
> >>>>>
> >>>>> ```
> >>>>>
> >> +---------------------------------------------------+
> >>>>> | A
> >> |
> >>>>>
> >> +------------+------------+------------+------------+
> >>>>> | subset | equivalent | superset | disjoint
> >> |
> >>>>>
> >> +---+------------+------------+------------+------------+------------+
> >>>>> | | subset | conflict | A wins | A wins | conflict
> >> |
> >>>>> |
> >> +------------+------------+------------+------------+------------+
> >>>>> | | equivalent | B wins | A or B | A wins | conflict
> >> |
> >>>>> | B
> >> +------------+------------+------------+------------+------------+
> >>>>> | | superset | B wins | B wins | conflict | conflict
> >> |
> >>>>> |
> >> +------------+------------+------------+------------+------------+
> >>>>> | | disjoint | conflict | conflict | conflict | conflict
> >> |
> >>>>>
> >> +---+------------+------------+------------+------------+------------+
> >>>>> ```
> >>>>>
> >>>>> The default unspecified claim is `disjoint` which indicates that some
> >>> of
> >>>>> the content is reproduced, but not all and there is additional
> >> content
> >>>>> added. With such a claim there will always be conflict and the build
> >>>> should
> >>>>> fail until the Project Model is updated to either remove some of the
> >>>> claims
> >>>>> or resolve the dependency clash.
> >>>>>
> >>>>> The ideal claim is `equivalent` which indicates that the two
> >> artifacts
> >>>> are
> >>>>> bi-directionally substitutable. This does not mean that the contents
> >>> are
> >>>>> identical. It does mean that they both deliver on the same contract
> >> in
> >>> an
> >>>>> equivalent fashion.
> >>>>>
> >>>>> The `subset` and `superset` claims are for aggregation APIs. So for
> >>>> example
> >>>>> the Java EE Web Profile API is a superset of the various spec APIs
> >> that
> >>>>> make up the Java EE Web Profile and a subset of the full Java EE
> >>>>> specification. The use of the `subset` claim should be reserved to
> >>> those
> >>>>> cases that are strict subsets. If anything is added that is not in
> >> the
> >>>>> supplied artifact then the correct claim is `disjoint`.
> >>>>>
> >>>>> ### Validation of supplies claims
> >>>>>
> >>>>> We do not want to introduce Java bias with this feature. As a result
> >>> the
> >>>>> validation of claims and supplies directives will be left to plugins.
> >>> For
> >>>>> the Java case we should probably provide either/both an enforcer rule
> >>> or
> >>>> a
> >>>>> maven hosted plugin to assist in checking JAR projects against the
> >>>> declared
> >>>>> supplies declarations, but Maven core should not be concerned with
> >>>> solving
> >>>>> the validation problem.
> >>>>>
> >>>>> Similarly, while there may be advantages in a more fine grained API
> >>>>> contract negotiation between dependencies, to bind such a concept
> >> into
> >>>> the
> >>>>> project model would significantly taint the Maven project model with
> >>> more
> >>>>> Java-centric concepts. Given that current software development
> >>> typically
> >>>>> uses at least two core languages: Java and JavaScript, we should be
> >>>> aiming
> >>>>> to reduce Java-centric constructs from Maven rather than increase
> >> them.
> >>>>>
> >>>>> ### Backporting
> >>>>>
> >>>>> It will not be possible to fully express this concept in a
> >> modelVersion
> >>>>> 4.0.0 project model. Thus if generating 4.0.0 compatible project
> >>> models,
> >>>>> the aim should be to fully resolve the dependencies of the project
> >>> using
> >>>>> all available information and express that as the transitive
> >>>> dependencies.
> >>>>>
> >>>>> Thus we will not expose the "supplies" information to modelVersion
> >>> 4.0.0
> >>>>> parsers but we will expose the end results of that and present the
> >>> final
> >>>>> effective flattened dependency tree.
> >>>>>
> >>>>> modelVersion 4.0.0 consumers will thus be no worse off than they
> >>> already
> >>>>> are and those consumers understanding newer modelVersions can get the
> >>>>> enhanced tree resolution that they would not get otherwise.
> >>>>>
> >>>>
> >>>
> >>
>
> Thanks,
>
> Jason
>
> ----------------------------------------------------------
> Jason van Zyl
> Founder, Apache Maven
> http://twitter.com/jvanzyl
> http://twitter.com/takari_io
> ---------------------------------------------------------
>
> We all have problems. How we deal with them is a measure of our worth.
>
> -- Unknown
>
>
>
>
>
>
>
>
>
>
Re: "supplies" concept proposal (was "provides" could be "proffers")
Posted by Jason van Zyl <ja...@takari.io>.
Throw it into a JIRA and tag the component FDPFC. I can only find 8 issues related to features requiring POM format changes. I seem to recall more.
On Jun 20, 2014, at 11:55 AM, Stephen Connolly <st...@gmail.com> wrote:
> Well I've had this proposal in one form or another on and off for the past
> 2-3 years. Jason recently stated that he only wants to look at concrete
> proposals as the first stab for what/how to extend the pom format... so
> here is a very concrete example proposal ;-)
>
>
> On 20 June 2014 16:50, Paul Benedict <pb...@apache.org> wrote:
>
>> Ah, the woes of fragmented jars. That's the bane of EE dependency
>> management. Figuring out the superset/subset stuff is indeed terrible. Good
>> job, Stephen, to propose a solution for it.
>>
>>
>> Cheers,
>> Paul
>>
>>
>> On Fri, Jun 20, 2014 at 10:03 AM, Stephen Connolly <
>> stephen.alan.connolly@gmail.com> wrote:
>>
>>> Well this started out from a need for a more effective <excludes>.
>>>
>>> The issue with <excludes> is that you have to apply it to each and every
>>> dependency... when what you really want to say is "see this dependency
>>> here? well that is the same as X, Y and Z so don't pull them in from the
>>> transitive dependencies"
>>>
>>> If you like that is the second use case in my proposal.
>>>
>>> The first use case is less relevant from the "better excludes" point of
>>> view, but I believe it to be more useful.
>>>
>>> The third use case is for when you don't trust upstream to get things
>>> right. Keep in mind that if, as a general principle, you don't trust
>>> upstream to express things mostly correct then what the hell are you
>> doing
>>> trusting them to express their transitive dependencies correctly?
>>>
>>> Tooling would allow us to validate a pom's claims of equivalence. But
>>> ultimately I think we need this concept to deal with te fragmentation of
>>> API jars and implementations of APIs that we see taking place in central.
>>>
>>>
>>> On 20 June 2014 15:27, Paul Benedict <pb...@apache.org> wrote:
>>>
>>>> I am having trouble understanding how grouping together artifacts that
>>> are
>>>> "equivalent" gain me anything in my project building. I am already
>> doing
>>>> this in my POM today by excluding what's equivalent. Is this proposal
>>>> simply about adding semantic information to the POM so it's more
>> apparent
>>>> what the intent is?
>>>>
>>>> Furthermore, whoever defines the equivalence needs to be extremely
>>> careful.
>>>> Not all "equivalent" jars are actually equivalent. Most people have
>>>> learned, for example, that the javaee-api:6.0 jar in Maven is all stubs
>>> and
>>>> can't be used for unit testing; so people go to find EE jars from JBoss
>>> or
>>>> GlassFish, to get the full functionality. So I am not sure I'd ever
>> want
>>>> someone telling me what's equivalent during consuming.
>>>>
>>>>
>>>> Cheers,
>>>> Paul
>>>>
>>>>
>>>> On Fri, Jun 20, 2014 at 7:51 AM, Stephen Connolly <
>>>> stephen.alan.connolly@gmail.com> wrote:
>>>>
>>>>> "supplies" concept proposal
>>>>> ===========================
>>>>>
>>>>> Introduction
>>>>> ------------
>>>>>
>>>>> The following is a proposal for Maven in a post-modelVersion-4.0.0
>> era.
>>>> The
>>>>> aim of this proposal is to simplify the management of dependency
>> trees
>>> in
>>>>> the decentralised era of artifact production that we find ourselves
>> in.
>>>>>
>>>>> The core issue is that different organisations can produce artifacts
>>> that
>>>>> may overlap. The easiest example is the servlet-api. If we restrict
>>>>> ourselves to version 2.5 of the servlet specification there are
>> quite a
>>>> few
>>>>> artifacts that all deliver the exact same content:
>>>>>
>>>>> * `jetty:servlet-api:2.5-6.0.2`
>>>>> * `org.daisy.libs:servlet-api:2.5.0`
>>>>> * `org.mortbay.jetty:servlet-api-2.5:6.1.14`
>>>>> *
>> `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final`
>>>>> * etc
>>>>>
>>>>> **Note:** this is a generic problem that is not restricted to the
>>>>> servlet-api, the servlet-api just provides the example that will be
>>> most
>>>>> familiar to everyone.
>>>>>
>>>>> So where these multiple artifacts supplying the equivalent content
>>>> becomes
>>>>> a problem is when the dependency tree is being calculated. If you
>> have
>>>> two
>>>>> dependencies each declaring transitive dependencies on different
>>>> artifacts
>>>>> that supply equivalent content, then you end up with two copies of
>> the
>>>> same
>>>>> JAR file in your classpath.
>>>>>
>>>>> In the case of the servlet-api, the hack most people use is to
>> declare
>>>> the
>>>>> servlet-api with scope `provided` thus preventing it from being
>>>> transitive.
>>>>> This is, however, a hack. In a more ideal world it would be better to
>>> let
>>>>> the servlet-api be transitive and only when we get to the WAR module
>>>> would
>>>>> we declare that a specific servlet-api is to be provided in the
>>>> containers
>>>>> that the WAR is targets for deployment into.
>>>>>
>>>>> We can take a second example that does not have the luxury of a *de
>>>> facto*
>>>>> hack.
>>>>>
>>>>> * `javax.faces:jsf-api:2.1`
>>>>> * `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final`
>>>>> * `org.apache.myfaces.core:myfaces-api:2.1.13`
>>>>>
>>>>> Now in the case of the JSF API, you are supposed to bundle the JSF
>> API
>>> in
>>>>> your WAR file. So if I use three JSF component libraries, I could
>> very
>>>> well
>>>>> end up with three different but equivalent JSF API jars in my WAR
>> file.
>>>>>
>>>>> Ideally what we want is some way of telling Maven that these
>> artifacts
>>>> are
>>>>> equivalent.
>>>>>
>>>>> Proposal
>>>>> --------
>>>>>
>>>>> Introduce the concept of "supplies" to the project model. The concept
>>>> needs
>>>>> three changes to the project model:
>>>>>
>>>>> 1. An explicit top level construct for a project to explicitly
>> declare
>>>>> up-front artifacts that it knows - at the time the project is being
>>>>> authored - to contain equivalent content to at least a subset of the
>>>>> project's content. Declarations could include a claim from: `subset`,
>>>>> `superset`, `disjoint` and `equivalent` with the default being
>>>> `disjoint`.
>>>>> 2. An explicit sub-element of the `dependency` construct to allow
>>>> consumers
>>>>> to *post-facto* declare a specific dependency as supplying equivalent
>>>>> content for other dependencies
>>>>> 3. An extension to the `dependency/excludes/exclude` construct to
>> allow
>>>>> consumers to remove claims a dependency makes with respect to
>> supplying
>>>>> equivalent content
>>>>>
>>>>> By way of illustration, here are some examples of these constructs
>>> mapped
>>>>> to a Model Version 4.0.0 like XML schema. As the
>>> post-modelVersion-4.0.0
>>>>> schema is not yet known, this represents the best way to illustrate
>> how
>>>> the
>>>>> concept will work, but note that this proposal does not suggest a
>>> schema
>>>>> for this concept.
>>>>>
>>>>> ### Example 1
>>>>>
>>>>> This illustrates how we would want, say, the `myfaces-api` project
>>> model
>>>> to
>>>>> look.
>>>>>
>>>>> ```
>>>>> <project>
>>>>> <groupId>org.apache.myfaces.core</groupId>
>>>>> <artifactId>myfaces-api</artifactId>
>>>>> <version>2.1.3</version>
>>>>> ...
>>>>> <supplyManagement>
>>>>> <supplies>
>>>>> <groupId>javax.faces</groupId>
>>>>> <artifactId>jsf-api</artifactId>
>>>>> <version>[2.1,2.2)</version>
>>>>> <claim>superset</claim>
>>>>> <supplies>
>>>>> <supplies>
>>>>> <groupId>org.jboss.spec.javax.faces</groupId>
>>>>> <artifactId>jboss-jsf-api_2.1_spec</artifactId>
>>>>> <claim>equivalent</claim>
>>>>> <supplies>
>>>>> </supplyManagement>
>>>>> ...
>>>>> </project>
>>>>> ```
>>>>>
>>>>> This indicates that the `myfaces-api` artifact is intended to be
>>> useable
>>>> as
>>>>> a drop-in replacement for either the `javax.faces:jsf-api` artifact
>>>> within
>>>>> a bounded range or for any version of the
>>>>> `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you
>>> get
>>>> a
>>>>> supplier conflict in your classpath, then Maven should fail the
>> build.
>>>>>
>>>>> For example if somebody forked `myfaces-api` but did not list
>>>> `myfaces-api`
>>>>> in the fork's supplyManagement and you end up with both `myfaces-api`
>>> and
>>>>> `myfaces-fork-api` in your classpath. Maven can detect that there are
>>> two
>>>>> dependencies that both claim to supply `javax.faces:jsf-api` and fail
>>> the
>>>>> build, thereby letting the user add either exclusions or additional
>>>>> supplies information to one of the artifacts and preventing duplicate
>>>>> artifacts on the classpath. The build need not be failed if the
>>> supplies
>>>>> claims provide a resolution. e.g. if the claim is `equivalent` then
>>> that
>>>>> implies that there is a 1:1 mapping and hence the artifacts are
>> drop-in
>>>>> replacements. However where the claim is `superset` we cannot know
>> that
>>>> the
>>>>> extra content in our artifact is the same as the extra content in
>>> another
>>>>> artifact which has a superset of `javax.faces:jsf-api`.
>>>>>
>>>>> ### Example 2
>>>>>
>>>>> This illustrates a JSF component library that is working with the
>>>> existing
>>>>> JSF APIs
>>>>>
>>>>> ```
>>>>> <project>
>>>>> ...
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>javax.faces</groupId>
>>>>> <artifactId>jsf-api</artifactId>
>>>>> <version>2.1</version>
>>>>> <supplyManagement>
>>>>> <supplies>
>>>>> <groupId>org.jboss.spec.javax.faces</groupId>
>>>>> <artifactId>jboss-jsf-api_2.1_spec</artifactId>
>>>>> <claim>equivalent</claim>
>>>>> <supplies>
>>>>> <supplies>
>>>>> <groupId>org.apache.myfaces.core</groupId>
>>>>> <artifactId>myfaces-api</artifactId>
>>>>> <version>[2.1,2.2)</version>
>>>>> <claim>equivalent</claim>
>>>>> </supplies>
>>>>> </supplyManagement>
>>>>> <dependency>
>>>>> </dependencies>
>>>>> ...
>>>>> </project>
>>>>> ```
>>>>>
>>>>> In this case we are publishing a transitive dependency with
>> additional
>>>>> supplyManagement injected. Consumers of this project would thus gain
>>> the
>>>>> benefit of collapsing their transitive dependencies for any of these
>>>> three
>>>>> artifacts. As all artifacts are declared with `equivalent` claim,
>> thus
>>>> the
>>>>> nearest of those three artifacts to the project will win as per the
>>>>> standard dependency resolution rules when dealing with conflicting
>>>> version
>>>>> requirements in the transitive dependency tree.
>>>>>
>>>>> ### Example 3
>>>>>
>>>>> Finally, there is the case where you need to correct an incorrect
>> claim
>>>> of
>>>>> supply
>>>>>
>>>>>
>>>>> ```
>>>>> <project>
>>>>> ...
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>javax.faces</groupId>
>>>>> <artifactId>jsf-api</artifactId>
>>>>> <version>2.1</version>
>>>>> <exclusions>
>>>>> <exclusion>
>>>>> <groupId>org.jboss.spec.javax.faces</groupId>
>>>>> <artifactId>jboss-jsf-api_2.2_spec</artifactId>
>>>>> <scope>supplies</scope>
>>>>> <exclusion>
>>>>> </exclusions>
>>>>> <dependency>
>>>>> </dependencies>
>>>>> ...
>>>>> </project>
>>>>> ```
>>>>>
>>>>> This would typically be coupled with adding back in a correct
>> supplies
>>>>> definition, but we need to allow for people to correct the graph
>> after
>>>> the
>>>>> fact of their dependencies being deployed to the remote repository.
>>>>>
>>>>> ### Claim conflict resolution
>>>>>
>>>>> The four classes of claim can be resolved using the following matrix
>>>>>
>>>>> ```
>>>>>
>> +---------------------------------------------------+
>>>>> | A
>> |
>>>>>
>> +------------+------------+------------+------------+
>>>>> | subset | equivalent | superset | disjoint
>> |
>>>>>
>> +---+------------+------------+------------+------------+------------+
>>>>> | | subset | conflict | A wins | A wins | conflict
>> |
>>>>> |
>> +------------+------------+------------+------------+------------+
>>>>> | | equivalent | B wins | A or B | A wins | conflict
>> |
>>>>> | B
>> +------------+------------+------------+------------+------------+
>>>>> | | superset | B wins | B wins | conflict | conflict
>> |
>>>>> |
>> +------------+------------+------------+------------+------------+
>>>>> | | disjoint | conflict | conflict | conflict | conflict
>> |
>>>>>
>> +---+------------+------------+------------+------------+------------+
>>>>> ```
>>>>>
>>>>> The default unspecified claim is `disjoint` which indicates that some
>>> of
>>>>> the content is reproduced, but not all and there is additional
>> content
>>>>> added. With such a claim there will always be conflict and the build
>>>> should
>>>>> fail until the Project Model is updated to either remove some of the
>>>> claims
>>>>> or resolve the dependency clash.
>>>>>
>>>>> The ideal claim is `equivalent` which indicates that the two
>> artifacts
>>>> are
>>>>> bi-directionally substitutable. This does not mean that the contents
>>> are
>>>>> identical. It does mean that they both deliver on the same contract
>> in
>>> an
>>>>> equivalent fashion.
>>>>>
>>>>> The `subset` and `superset` claims are for aggregation APIs. So for
>>>> example
>>>>> the Java EE Web Profile API is a superset of the various spec APIs
>> that
>>>>> make up the Java EE Web Profile and a subset of the full Java EE
>>>>> specification. The use of the `subset` claim should be reserved to
>>> those
>>>>> cases that are strict subsets. If anything is added that is not in
>> the
>>>>> supplied artifact then the correct claim is `disjoint`.
>>>>>
>>>>> ### Validation of supplies claims
>>>>>
>>>>> We do not want to introduce Java bias with this feature. As a result
>>> the
>>>>> validation of claims and supplies directives will be left to plugins.
>>> For
>>>>> the Java case we should probably provide either/both an enforcer rule
>>> or
>>>> a
>>>>> maven hosted plugin to assist in checking JAR projects against the
>>>> declared
>>>>> supplies declarations, but Maven core should not be concerned with
>>>> solving
>>>>> the validation problem.
>>>>>
>>>>> Similarly, while there may be advantages in a more fine grained API
>>>>> contract negotiation between dependencies, to bind such a concept
>> into
>>>> the
>>>>> project model would significantly taint the Maven project model with
>>> more
>>>>> Java-centric concepts. Given that current software development
>>> typically
>>>>> uses at least two core languages: Java and JavaScript, we should be
>>>> aiming
>>>>> to reduce Java-centric constructs from Maven rather than increase
>> them.
>>>>>
>>>>> ### Backporting
>>>>>
>>>>> It will not be possible to fully express this concept in a
>> modelVersion
>>>>> 4.0.0 project model. Thus if generating 4.0.0 compatible project
>>> models,
>>>>> the aim should be to fully resolve the dependencies of the project
>>> using
>>>>> all available information and express that as the transitive
>>>> dependencies.
>>>>>
>>>>> Thus we will not expose the "supplies" information to modelVersion
>>> 4.0.0
>>>>> parsers but we will expose the end results of that and present the
>>> final
>>>>> effective flattened dependency tree.
>>>>>
>>>>> modelVersion 4.0.0 consumers will thus be no worse off than they
>>> already
>>>>> are and those consumers understanding newer modelVersions can get the
>>>>> enhanced tree resolution that they would not get otherwise.
>>>>>
>>>>
>>>
>>
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
http://twitter.com/jvanzyl
http://twitter.com/takari_io
---------------------------------------------------------
We all have problems. How we deal with them is a measure of our worth.
-- Unknown
Re: "supplies" concept proposal (was "provides" could be "proffers")
Posted by Stephen Connolly <st...@gmail.com>.
Well I've had this proposal in one form or another on and off for the past
2-3 years. Jason recently stated that he only wants to look at concrete
proposals as the first stab for what/how to extend the pom format... so
here is a very concrete example proposal ;-)
On 20 June 2014 16:50, Paul Benedict <pb...@apache.org> wrote:
> Ah, the woes of fragmented jars. That's the bane of EE dependency
> management. Figuring out the superset/subset stuff is indeed terrible. Good
> job, Stephen, to propose a solution for it.
>
>
> Cheers,
> Paul
>
>
> On Fri, Jun 20, 2014 at 10:03 AM, Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
> > Well this started out from a need for a more effective <excludes>.
> >
> > The issue with <excludes> is that you have to apply it to each and every
> > dependency... when what you really want to say is "see this dependency
> > here? well that is the same as X, Y and Z so don't pull them in from the
> > transitive dependencies"
> >
> > If you like that is the second use case in my proposal.
> >
> > The first use case is less relevant from the "better excludes" point of
> > view, but I believe it to be more useful.
> >
> > The third use case is for when you don't trust upstream to get things
> > right. Keep in mind that if, as a general principle, you don't trust
> > upstream to express things mostly correct then what the hell are you
> doing
> > trusting them to express their transitive dependencies correctly?
> >
> > Tooling would allow us to validate a pom's claims of equivalence. But
> > ultimately I think we need this concept to deal with te fragmentation of
> > API jars and implementations of APIs that we see taking place in central.
> >
> >
> > On 20 June 2014 15:27, Paul Benedict <pb...@apache.org> wrote:
> >
> > > I am having trouble understanding how grouping together artifacts that
> > are
> > > "equivalent" gain me anything in my project building. I am already
> doing
> > > this in my POM today by excluding what's equivalent. Is this proposal
> > > simply about adding semantic information to the POM so it's more
> apparent
> > > what the intent is?
> > >
> > > Furthermore, whoever defines the equivalence needs to be extremely
> > careful.
> > > Not all "equivalent" jars are actually equivalent. Most people have
> > > learned, for example, that the javaee-api:6.0 jar in Maven is all stubs
> > and
> > > can't be used for unit testing; so people go to find EE jars from JBoss
> > or
> > > GlassFish, to get the full functionality. So I am not sure I'd ever
> want
> > > someone telling me what's equivalent during consuming.
> > >
> > >
> > > Cheers,
> > > Paul
> > >
> > >
> > > On Fri, Jun 20, 2014 at 7:51 AM, Stephen Connolly <
> > > stephen.alan.connolly@gmail.com> wrote:
> > >
> > > > "supplies" concept proposal
> > > > ===========================
> > > >
> > > > Introduction
> > > > ------------
> > > >
> > > > The following is a proposal for Maven in a post-modelVersion-4.0.0
> era.
> > > The
> > > > aim of this proposal is to simplify the management of dependency
> trees
> > in
> > > > the decentralised era of artifact production that we find ourselves
> in.
> > > >
> > > > The core issue is that different organisations can produce artifacts
> > that
> > > > may overlap. The easiest example is the servlet-api. If we restrict
> > > > ourselves to version 2.5 of the servlet specification there are
> quite a
> > > few
> > > > artifacts that all deliver the exact same content:
> > > >
> > > > * `jetty:servlet-api:2.5-6.0.2`
> > > > * `org.daisy.libs:servlet-api:2.5.0`
> > > > * `org.mortbay.jetty:servlet-api-2.5:6.1.14`
> > > > *
> `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final`
> > > > * etc
> > > >
> > > > **Note:** this is a generic problem that is not restricted to the
> > > > servlet-api, the servlet-api just provides the example that will be
> > most
> > > > familiar to everyone.
> > > >
> > > > So where these multiple artifacts supplying the equivalent content
> > > becomes
> > > > a problem is when the dependency tree is being calculated. If you
> have
> > > two
> > > > dependencies each declaring transitive dependencies on different
> > > artifacts
> > > > that supply equivalent content, then you end up with two copies of
> the
> > > same
> > > > JAR file in your classpath.
> > > >
> > > > In the case of the servlet-api, the hack most people use is to
> declare
> > > the
> > > > servlet-api with scope `provided` thus preventing it from being
> > > transitive.
> > > > This is, however, a hack. In a more ideal world it would be better to
> > let
> > > > the servlet-api be transitive and only when we get to the WAR module
> > > would
> > > > we declare that a specific servlet-api is to be provided in the
> > > containers
> > > > that the WAR is targets for deployment into.
> > > >
> > > > We can take a second example that does not have the luxury of a *de
> > > facto*
> > > > hack.
> > > >
> > > > * `javax.faces:jsf-api:2.1`
> > > > * `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final`
> > > > * `org.apache.myfaces.core:myfaces-api:2.1.13`
> > > >
> > > > Now in the case of the JSF API, you are supposed to bundle the JSF
> API
> > in
> > > > your WAR file. So if I use three JSF component libraries, I could
> very
> > > well
> > > > end up with three different but equivalent JSF API jars in my WAR
> file.
> > > >
> > > > Ideally what we want is some way of telling Maven that these
> artifacts
> > > are
> > > > equivalent.
> > > >
> > > > Proposal
> > > > --------
> > > >
> > > > Introduce the concept of "supplies" to the project model. The concept
> > > needs
> > > > three changes to the project model:
> > > >
> > > > 1. An explicit top level construct for a project to explicitly
> declare
> > > > up-front artifacts that it knows - at the time the project is being
> > > > authored - to contain equivalent content to at least a subset of the
> > > > project's content. Declarations could include a claim from: `subset`,
> > > > `superset`, `disjoint` and `equivalent` with the default being
> > > `disjoint`.
> > > > 2. An explicit sub-element of the `dependency` construct to allow
> > > consumers
> > > > to *post-facto* declare a specific dependency as supplying equivalent
> > > > content for other dependencies
> > > > 3. An extension to the `dependency/excludes/exclude` construct to
> allow
> > > > consumers to remove claims a dependency makes with respect to
> supplying
> > > > equivalent content
> > > >
> > > > By way of illustration, here are some examples of these constructs
> > mapped
> > > > to a Model Version 4.0.0 like XML schema. As the
> > post-modelVersion-4.0.0
> > > > schema is not yet known, this represents the best way to illustrate
> how
> > > the
> > > > concept will work, but note that this proposal does not suggest a
> > schema
> > > > for this concept.
> > > >
> > > > ### Example 1
> > > >
> > > > This illustrates how we would want, say, the `myfaces-api` project
> > model
> > > to
> > > > look.
> > > >
> > > > ```
> > > > <project>
> > > > <groupId>org.apache.myfaces.core</groupId>
> > > > <artifactId>myfaces-api</artifactId>
> > > > <version>2.1.3</version>
> > > > ...
> > > > <supplyManagement>
> > > > <supplies>
> > > > <groupId>javax.faces</groupId>
> > > > <artifactId>jsf-api</artifactId>
> > > > <version>[2.1,2.2)</version>
> > > > <claim>superset</claim>
> > > > <supplies>
> > > > <supplies>
> > > > <groupId>org.jboss.spec.javax.faces</groupId>
> > > > <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> > > > <claim>equivalent</claim>
> > > > <supplies>
> > > > </supplyManagement>
> > > > ...
> > > > </project>
> > > > ```
> > > >
> > > > This indicates that the `myfaces-api` artifact is intended to be
> > useable
> > > as
> > > > a drop-in replacement for either the `javax.faces:jsf-api` artifact
> > > within
> > > > a bounded range or for any version of the
> > > > `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you
> > get
> > > a
> > > > supplier conflict in your classpath, then Maven should fail the
> build.
> > > >
> > > > For example if somebody forked `myfaces-api` but did not list
> > > `myfaces-api`
> > > > in the fork's supplyManagement and you end up with both `myfaces-api`
> > and
> > > > `myfaces-fork-api` in your classpath. Maven can detect that there are
> > two
> > > > dependencies that both claim to supply `javax.faces:jsf-api` and fail
> > the
> > > > build, thereby letting the user add either exclusions or additional
> > > > supplies information to one of the artifacts and preventing duplicate
> > > > artifacts on the classpath. The build need not be failed if the
> > supplies
> > > > claims provide a resolution. e.g. if the claim is `equivalent` then
> > that
> > > > implies that there is a 1:1 mapping and hence the artifacts are
> drop-in
> > > > replacements. However where the claim is `superset` we cannot know
> that
> > > the
> > > > extra content in our artifact is the same as the extra content in
> > another
> > > > artifact which has a superset of `javax.faces:jsf-api`.
> > > >
> > > > ### Example 2
> > > >
> > > > This illustrates a JSF component library that is working with the
> > > existing
> > > > JSF APIs
> > > >
> > > > ```
> > > > <project>
> > > > ...
> > > > <dependencies>
> > > > <dependency>
> > > > <groupId>javax.faces</groupId>
> > > > <artifactId>jsf-api</artifactId>
> > > > <version>2.1</version>
> > > > <supplyManagement>
> > > > <supplies>
> > > > <groupId>org.jboss.spec.javax.faces</groupId>
> > > > <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> > > > <claim>equivalent</claim>
> > > > <supplies>
> > > > <supplies>
> > > > <groupId>org.apache.myfaces.core</groupId>
> > > > <artifactId>myfaces-api</artifactId>
> > > > <version>[2.1,2.2)</version>
> > > > <claim>equivalent</claim>
> > > > </supplies>
> > > > </supplyManagement>
> > > > <dependency>
> > > > </dependencies>
> > > > ...
> > > > </project>
> > > > ```
> > > >
> > > > In this case we are publishing a transitive dependency with
> additional
> > > > supplyManagement injected. Consumers of this project would thus gain
> > the
> > > > benefit of collapsing their transitive dependencies for any of these
> > > three
> > > > artifacts. As all artifacts are declared with `equivalent` claim,
> thus
> > > the
> > > > nearest of those three artifacts to the project will win as per the
> > > > standard dependency resolution rules when dealing with conflicting
> > > version
> > > > requirements in the transitive dependency tree.
> > > >
> > > > ### Example 3
> > > >
> > > > Finally, there is the case where you need to correct an incorrect
> claim
> > > of
> > > > supply
> > > >
> > > >
> > > > ```
> > > > <project>
> > > > ...
> > > > <dependencies>
> > > > <dependency>
> > > > <groupId>javax.faces</groupId>
> > > > <artifactId>jsf-api</artifactId>
> > > > <version>2.1</version>
> > > > <exclusions>
> > > > <exclusion>
> > > > <groupId>org.jboss.spec.javax.faces</groupId>
> > > > <artifactId>jboss-jsf-api_2.2_spec</artifactId>
> > > > <scope>supplies</scope>
> > > > <exclusion>
> > > > </exclusions>
> > > > <dependency>
> > > > </dependencies>
> > > > ...
> > > > </project>
> > > > ```
> > > >
> > > > This would typically be coupled with adding back in a correct
> supplies
> > > > definition, but we need to allow for people to correct the graph
> after
> > > the
> > > > fact of their dependencies being deployed to the remote repository.
> > > >
> > > > ### Claim conflict resolution
> > > >
> > > > The four classes of claim can be resolved using the following matrix
> > > >
> > > > ```
> > > >
> +---------------------------------------------------+
> > > > | A
> |
> > > >
> +------------+------------+------------+------------+
> > > > | subset | equivalent | superset | disjoint
> |
> > > >
> +---+------------+------------+------------+------------+------------+
> > > > | | subset | conflict | A wins | A wins | conflict
> |
> > > > |
> +------------+------------+------------+------------+------------+
> > > > | | equivalent | B wins | A or B | A wins | conflict
> |
> > > > | B
> +------------+------------+------------+------------+------------+
> > > > | | superset | B wins | B wins | conflict | conflict
> |
> > > > |
> +------------+------------+------------+------------+------------+
> > > > | | disjoint | conflict | conflict | conflict | conflict
> |
> > > >
> +---+------------+------------+------------+------------+------------+
> > > > ```
> > > >
> > > > The default unspecified claim is `disjoint` which indicates that some
> > of
> > > > the content is reproduced, but not all and there is additional
> content
> > > > added. With such a claim there will always be conflict and the build
> > > should
> > > > fail until the Project Model is updated to either remove some of the
> > > claims
> > > > or resolve the dependency clash.
> > > >
> > > > The ideal claim is `equivalent` which indicates that the two
> artifacts
> > > are
> > > > bi-directionally substitutable. This does not mean that the contents
> > are
> > > > identical. It does mean that they both deliver on the same contract
> in
> > an
> > > > equivalent fashion.
> > > >
> > > > The `subset` and `superset` claims are for aggregation APIs. So for
> > > example
> > > > the Java EE Web Profile API is a superset of the various spec APIs
> that
> > > > make up the Java EE Web Profile and a subset of the full Java EE
> > > > specification. The use of the `subset` claim should be reserved to
> > those
> > > > cases that are strict subsets. If anything is added that is not in
> the
> > > > supplied artifact then the correct claim is `disjoint`.
> > > >
> > > > ### Validation of supplies claims
> > > >
> > > > We do not want to introduce Java bias with this feature. As a result
> > the
> > > > validation of claims and supplies directives will be left to plugins.
> > For
> > > > the Java case we should probably provide either/both an enforcer rule
> > or
> > > a
> > > > maven hosted plugin to assist in checking JAR projects against the
> > > declared
> > > > supplies declarations, but Maven core should not be concerned with
> > > solving
> > > > the validation problem.
> > > >
> > > > Similarly, while there may be advantages in a more fine grained API
> > > > contract negotiation between dependencies, to bind such a concept
> into
> > > the
> > > > project model would significantly taint the Maven project model with
> > more
> > > > Java-centric concepts. Given that current software development
> > typically
> > > > uses at least two core languages: Java and JavaScript, we should be
> > > aiming
> > > > to reduce Java-centric constructs from Maven rather than increase
> them.
> > > >
> > > > ### Backporting
> > > >
> > > > It will not be possible to fully express this concept in a
> modelVersion
> > > > 4.0.0 project model. Thus if generating 4.0.0 compatible project
> > models,
> > > > the aim should be to fully resolve the dependencies of the project
> > using
> > > > all available information and express that as the transitive
> > > dependencies.
> > > >
> > > > Thus we will not expose the "supplies" information to modelVersion
> > 4.0.0
> > > > parsers but we will expose the end results of that and present the
> > final
> > > > effective flattened dependency tree.
> > > >
> > > > modelVersion 4.0.0 consumers will thus be no worse off than they
> > already
> > > > are and those consumers understanding newer modelVersions can get the
> > > > enhanced tree resolution that they would not get otherwise.
> > > >
> > >
> >
>
Re: "supplies" concept proposal (was "provides" could be "proffers")
Posted by Paul Benedict <pb...@apache.org>.
Ah, the woes of fragmented jars. That's the bane of EE dependency
management. Figuring out the superset/subset stuff is indeed terrible. Good
job, Stephen, to propose a solution for it.
Cheers,
Paul
On Fri, Jun 20, 2014 at 10:03 AM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> Well this started out from a need for a more effective <excludes>.
>
> The issue with <excludes> is that you have to apply it to each and every
> dependency... when what you really want to say is "see this dependency
> here? well that is the same as X, Y and Z so don't pull them in from the
> transitive dependencies"
>
> If you like that is the second use case in my proposal.
>
> The first use case is less relevant from the "better excludes" point of
> view, but I believe it to be more useful.
>
> The third use case is for when you don't trust upstream to get things
> right. Keep in mind that if, as a general principle, you don't trust
> upstream to express things mostly correct then what the hell are you doing
> trusting them to express their transitive dependencies correctly?
>
> Tooling would allow us to validate a pom's claims of equivalence. But
> ultimately I think we need this concept to deal with te fragmentation of
> API jars and implementations of APIs that we see taking place in central.
>
>
> On 20 June 2014 15:27, Paul Benedict <pb...@apache.org> wrote:
>
> > I am having trouble understanding how grouping together artifacts that
> are
> > "equivalent" gain me anything in my project building. I am already doing
> > this in my POM today by excluding what's equivalent. Is this proposal
> > simply about adding semantic information to the POM so it's more apparent
> > what the intent is?
> >
> > Furthermore, whoever defines the equivalence needs to be extremely
> careful.
> > Not all "equivalent" jars are actually equivalent. Most people have
> > learned, for example, that the javaee-api:6.0 jar in Maven is all stubs
> and
> > can't be used for unit testing; so people go to find EE jars from JBoss
> or
> > GlassFish, to get the full functionality. So I am not sure I'd ever want
> > someone telling me what's equivalent during consuming.
> >
> >
> > Cheers,
> > Paul
> >
> >
> > On Fri, Jun 20, 2014 at 7:51 AM, Stephen Connolly <
> > stephen.alan.connolly@gmail.com> wrote:
> >
> > > "supplies" concept proposal
> > > ===========================
> > >
> > > Introduction
> > > ------------
> > >
> > > The following is a proposal for Maven in a post-modelVersion-4.0.0 era.
> > The
> > > aim of this proposal is to simplify the management of dependency trees
> in
> > > the decentralised era of artifact production that we find ourselves in.
> > >
> > > The core issue is that different organisations can produce artifacts
> that
> > > may overlap. The easiest example is the servlet-api. If we restrict
> > > ourselves to version 2.5 of the servlet specification there are quite a
> > few
> > > artifacts that all deliver the exact same content:
> > >
> > > * `jetty:servlet-api:2.5-6.0.2`
> > > * `org.daisy.libs:servlet-api:2.5.0`
> > > * `org.mortbay.jetty:servlet-api-2.5:6.1.14`
> > > * `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final`
> > > * etc
> > >
> > > **Note:** this is a generic problem that is not restricted to the
> > > servlet-api, the servlet-api just provides the example that will be
> most
> > > familiar to everyone.
> > >
> > > So where these multiple artifacts supplying the equivalent content
> > becomes
> > > a problem is when the dependency tree is being calculated. If you have
> > two
> > > dependencies each declaring transitive dependencies on different
> > artifacts
> > > that supply equivalent content, then you end up with two copies of the
> > same
> > > JAR file in your classpath.
> > >
> > > In the case of the servlet-api, the hack most people use is to declare
> > the
> > > servlet-api with scope `provided` thus preventing it from being
> > transitive.
> > > This is, however, a hack. In a more ideal world it would be better to
> let
> > > the servlet-api be transitive and only when we get to the WAR module
> > would
> > > we declare that a specific servlet-api is to be provided in the
> > containers
> > > that the WAR is targets for deployment into.
> > >
> > > We can take a second example that does not have the luxury of a *de
> > facto*
> > > hack.
> > >
> > > * `javax.faces:jsf-api:2.1`
> > > * `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final`
> > > * `org.apache.myfaces.core:myfaces-api:2.1.13`
> > >
> > > Now in the case of the JSF API, you are supposed to bundle the JSF API
> in
> > > your WAR file. So if I use three JSF component libraries, I could very
> > well
> > > end up with three different but equivalent JSF API jars in my WAR file.
> > >
> > > Ideally what we want is some way of telling Maven that these artifacts
> > are
> > > equivalent.
> > >
> > > Proposal
> > > --------
> > >
> > > Introduce the concept of "supplies" to the project model. The concept
> > needs
> > > three changes to the project model:
> > >
> > > 1. An explicit top level construct for a project to explicitly declare
> > > up-front artifacts that it knows - at the time the project is being
> > > authored - to contain equivalent content to at least a subset of the
> > > project's content. Declarations could include a claim from: `subset`,
> > > `superset`, `disjoint` and `equivalent` with the default being
> > `disjoint`.
> > > 2. An explicit sub-element of the `dependency` construct to allow
> > consumers
> > > to *post-facto* declare a specific dependency as supplying equivalent
> > > content for other dependencies
> > > 3. An extension to the `dependency/excludes/exclude` construct to allow
> > > consumers to remove claims a dependency makes with respect to supplying
> > > equivalent content
> > >
> > > By way of illustration, here are some examples of these constructs
> mapped
> > > to a Model Version 4.0.0 like XML schema. As the
> post-modelVersion-4.0.0
> > > schema is not yet known, this represents the best way to illustrate how
> > the
> > > concept will work, but note that this proposal does not suggest a
> schema
> > > for this concept.
> > >
> > > ### Example 1
> > >
> > > This illustrates how we would want, say, the `myfaces-api` project
> model
> > to
> > > look.
> > >
> > > ```
> > > <project>
> > > <groupId>org.apache.myfaces.core</groupId>
> > > <artifactId>myfaces-api</artifactId>
> > > <version>2.1.3</version>
> > > ...
> > > <supplyManagement>
> > > <supplies>
> > > <groupId>javax.faces</groupId>
> > > <artifactId>jsf-api</artifactId>
> > > <version>[2.1,2.2)</version>
> > > <claim>superset</claim>
> > > <supplies>
> > > <supplies>
> > > <groupId>org.jboss.spec.javax.faces</groupId>
> > > <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> > > <claim>equivalent</claim>
> > > <supplies>
> > > </supplyManagement>
> > > ...
> > > </project>
> > > ```
> > >
> > > This indicates that the `myfaces-api` artifact is intended to be
> useable
> > as
> > > a drop-in replacement for either the `javax.faces:jsf-api` artifact
> > within
> > > a bounded range or for any version of the
> > > `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you
> get
> > a
> > > supplier conflict in your classpath, then Maven should fail the build.
> > >
> > > For example if somebody forked `myfaces-api` but did not list
> > `myfaces-api`
> > > in the fork's supplyManagement and you end up with both `myfaces-api`
> and
> > > `myfaces-fork-api` in your classpath. Maven can detect that there are
> two
> > > dependencies that both claim to supply `javax.faces:jsf-api` and fail
> the
> > > build, thereby letting the user add either exclusions or additional
> > > supplies information to one of the artifacts and preventing duplicate
> > > artifacts on the classpath. The build need not be failed if the
> supplies
> > > claims provide a resolution. e.g. if the claim is `equivalent` then
> that
> > > implies that there is a 1:1 mapping and hence the artifacts are drop-in
> > > replacements. However where the claim is `superset` we cannot know that
> > the
> > > extra content in our artifact is the same as the extra content in
> another
> > > artifact which has a superset of `javax.faces:jsf-api`.
> > >
> > > ### Example 2
> > >
> > > This illustrates a JSF component library that is working with the
> > existing
> > > JSF APIs
> > >
> > > ```
> > > <project>
> > > ...
> > > <dependencies>
> > > <dependency>
> > > <groupId>javax.faces</groupId>
> > > <artifactId>jsf-api</artifactId>
> > > <version>2.1</version>
> > > <supplyManagement>
> > > <supplies>
> > > <groupId>org.jboss.spec.javax.faces</groupId>
> > > <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> > > <claim>equivalent</claim>
> > > <supplies>
> > > <supplies>
> > > <groupId>org.apache.myfaces.core</groupId>
> > > <artifactId>myfaces-api</artifactId>
> > > <version>[2.1,2.2)</version>
> > > <claim>equivalent</claim>
> > > </supplies>
> > > </supplyManagement>
> > > <dependency>
> > > </dependencies>
> > > ...
> > > </project>
> > > ```
> > >
> > > In this case we are publishing a transitive dependency with additional
> > > supplyManagement injected. Consumers of this project would thus gain
> the
> > > benefit of collapsing their transitive dependencies for any of these
> > three
> > > artifacts. As all artifacts are declared with `equivalent` claim, thus
> > the
> > > nearest of those three artifacts to the project will win as per the
> > > standard dependency resolution rules when dealing with conflicting
> > version
> > > requirements in the transitive dependency tree.
> > >
> > > ### Example 3
> > >
> > > Finally, there is the case where you need to correct an incorrect claim
> > of
> > > supply
> > >
> > >
> > > ```
> > > <project>
> > > ...
> > > <dependencies>
> > > <dependency>
> > > <groupId>javax.faces</groupId>
> > > <artifactId>jsf-api</artifactId>
> > > <version>2.1</version>
> > > <exclusions>
> > > <exclusion>
> > > <groupId>org.jboss.spec.javax.faces</groupId>
> > > <artifactId>jboss-jsf-api_2.2_spec</artifactId>
> > > <scope>supplies</scope>
> > > <exclusion>
> > > </exclusions>
> > > <dependency>
> > > </dependencies>
> > > ...
> > > </project>
> > > ```
> > >
> > > This would typically be coupled with adding back in a correct supplies
> > > definition, but we need to allow for people to correct the graph after
> > the
> > > fact of their dependencies being deployed to the remote repository.
> > >
> > > ### Claim conflict resolution
> > >
> > > The four classes of claim can be resolved using the following matrix
> > >
> > > ```
> > > +---------------------------------------------------+
> > > | A |
> > > +------------+------------+------------+------------+
> > > | subset | equivalent | superset | disjoint |
> > > +---+------------+------------+------------+------------+------------+
> > > | | subset | conflict | A wins | A wins | conflict |
> > > | +------------+------------+------------+------------+------------+
> > > | | equivalent | B wins | A or B | A wins | conflict |
> > > | B +------------+------------+------------+------------+------------+
> > > | | superset | B wins | B wins | conflict | conflict |
> > > | +------------+------------+------------+------------+------------+
> > > | | disjoint | conflict | conflict | conflict | conflict |
> > > +---+------------+------------+------------+------------+------------+
> > > ```
> > >
> > > The default unspecified claim is `disjoint` which indicates that some
> of
> > > the content is reproduced, but not all and there is additional content
> > > added. With such a claim there will always be conflict and the build
> > should
> > > fail until the Project Model is updated to either remove some of the
> > claims
> > > or resolve the dependency clash.
> > >
> > > The ideal claim is `equivalent` which indicates that the two artifacts
> > are
> > > bi-directionally substitutable. This does not mean that the contents
> are
> > > identical. It does mean that they both deliver on the same contract in
> an
> > > equivalent fashion.
> > >
> > > The `subset` and `superset` claims are for aggregation APIs. So for
> > example
> > > the Java EE Web Profile API is a superset of the various spec APIs that
> > > make up the Java EE Web Profile and a subset of the full Java EE
> > > specification. The use of the `subset` claim should be reserved to
> those
> > > cases that are strict subsets. If anything is added that is not in the
> > > supplied artifact then the correct claim is `disjoint`.
> > >
> > > ### Validation of supplies claims
> > >
> > > We do not want to introduce Java bias with this feature. As a result
> the
> > > validation of claims and supplies directives will be left to plugins.
> For
> > > the Java case we should probably provide either/both an enforcer rule
> or
> > a
> > > maven hosted plugin to assist in checking JAR projects against the
> > declared
> > > supplies declarations, but Maven core should not be concerned with
> > solving
> > > the validation problem.
> > >
> > > Similarly, while there may be advantages in a more fine grained API
> > > contract negotiation between dependencies, to bind such a concept into
> > the
> > > project model would significantly taint the Maven project model with
> more
> > > Java-centric concepts. Given that current software development
> typically
> > > uses at least two core languages: Java and JavaScript, we should be
> > aiming
> > > to reduce Java-centric constructs from Maven rather than increase them.
> > >
> > > ### Backporting
> > >
> > > It will not be possible to fully express this concept in a modelVersion
> > > 4.0.0 project model. Thus if generating 4.0.0 compatible project
> models,
> > > the aim should be to fully resolve the dependencies of the project
> using
> > > all available information and express that as the transitive
> > dependencies.
> > >
> > > Thus we will not expose the "supplies" information to modelVersion
> 4.0.0
> > > parsers but we will expose the end results of that and present the
> final
> > > effective flattened dependency tree.
> > >
> > > modelVersion 4.0.0 consumers will thus be no worse off than they
> already
> > > are and those consumers understanding newer modelVersions can get the
> > > enhanced tree resolution that they would not get otherwise.
> > >
> >
>
Re: "supplies" concept proposal (was "provides" could be "proffers")
Posted by Stephen Connolly <st...@gmail.com>.
Well this started out from a need for a more effective <excludes>.
The issue with <excludes> is that you have to apply it to each and every
dependency... when what you really want to say is "see this dependency
here? well that is the same as X, Y and Z so don't pull them in from the
transitive dependencies"
If you like that is the second use case in my proposal.
The first use case is less relevant from the "better excludes" point of
view, but I believe it to be more useful.
The third use case is for when you don't trust upstream to get things
right. Keep in mind that if, as a general principle, you don't trust
upstream to express things mostly correct then what the hell are you doing
trusting them to express their transitive dependencies correctly?
Tooling would allow us to validate a pom's claims of equivalence. But
ultimately I think we need this concept to deal with te fragmentation of
API jars and implementations of APIs that we see taking place in central.
On 20 June 2014 15:27, Paul Benedict <pb...@apache.org> wrote:
> I am having trouble understanding how grouping together artifacts that are
> "equivalent" gain me anything in my project building. I am already doing
> this in my POM today by excluding what's equivalent. Is this proposal
> simply about adding semantic information to the POM so it's more apparent
> what the intent is?
>
> Furthermore, whoever defines the equivalence needs to be extremely careful.
> Not all "equivalent" jars are actually equivalent. Most people have
> learned, for example, that the javaee-api:6.0 jar in Maven is all stubs and
> can't be used for unit testing; so people go to find EE jars from JBoss or
> GlassFish, to get the full functionality. So I am not sure I'd ever want
> someone telling me what's equivalent during consuming.
>
>
> Cheers,
> Paul
>
>
> On Fri, Jun 20, 2014 at 7:51 AM, Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
> > "supplies" concept proposal
> > ===========================
> >
> > Introduction
> > ------------
> >
> > The following is a proposal for Maven in a post-modelVersion-4.0.0 era.
> The
> > aim of this proposal is to simplify the management of dependency trees in
> > the decentralised era of artifact production that we find ourselves in.
> >
> > The core issue is that different organisations can produce artifacts that
> > may overlap. The easiest example is the servlet-api. If we restrict
> > ourselves to version 2.5 of the servlet specification there are quite a
> few
> > artifacts that all deliver the exact same content:
> >
> > * `jetty:servlet-api:2.5-6.0.2`
> > * `org.daisy.libs:servlet-api:2.5.0`
> > * `org.mortbay.jetty:servlet-api-2.5:6.1.14`
> > * `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final`
> > * etc
> >
> > **Note:** this is a generic problem that is not restricted to the
> > servlet-api, the servlet-api just provides the example that will be most
> > familiar to everyone.
> >
> > So where these multiple artifacts supplying the equivalent content
> becomes
> > a problem is when the dependency tree is being calculated. If you have
> two
> > dependencies each declaring transitive dependencies on different
> artifacts
> > that supply equivalent content, then you end up with two copies of the
> same
> > JAR file in your classpath.
> >
> > In the case of the servlet-api, the hack most people use is to declare
> the
> > servlet-api with scope `provided` thus preventing it from being
> transitive.
> > This is, however, a hack. In a more ideal world it would be better to let
> > the servlet-api be transitive and only when we get to the WAR module
> would
> > we declare that a specific servlet-api is to be provided in the
> containers
> > that the WAR is targets for deployment into.
> >
> > We can take a second example that does not have the luxury of a *de
> facto*
> > hack.
> >
> > * `javax.faces:jsf-api:2.1`
> > * `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final`
> > * `org.apache.myfaces.core:myfaces-api:2.1.13`
> >
> > Now in the case of the JSF API, you are supposed to bundle the JSF API in
> > your WAR file. So if I use three JSF component libraries, I could very
> well
> > end up with three different but equivalent JSF API jars in my WAR file.
> >
> > Ideally what we want is some way of telling Maven that these artifacts
> are
> > equivalent.
> >
> > Proposal
> > --------
> >
> > Introduce the concept of "supplies" to the project model. The concept
> needs
> > three changes to the project model:
> >
> > 1. An explicit top level construct for a project to explicitly declare
> > up-front artifacts that it knows - at the time the project is being
> > authored - to contain equivalent content to at least a subset of the
> > project's content. Declarations could include a claim from: `subset`,
> > `superset`, `disjoint` and `equivalent` with the default being
> `disjoint`.
> > 2. An explicit sub-element of the `dependency` construct to allow
> consumers
> > to *post-facto* declare a specific dependency as supplying equivalent
> > content for other dependencies
> > 3. An extension to the `dependency/excludes/exclude` construct to allow
> > consumers to remove claims a dependency makes with respect to supplying
> > equivalent content
> >
> > By way of illustration, here are some examples of these constructs mapped
> > to a Model Version 4.0.0 like XML schema. As the post-modelVersion-4.0.0
> > schema is not yet known, this represents the best way to illustrate how
> the
> > concept will work, but note that this proposal does not suggest a schema
> > for this concept.
> >
> > ### Example 1
> >
> > This illustrates how we would want, say, the `myfaces-api` project model
> to
> > look.
> >
> > ```
> > <project>
> > <groupId>org.apache.myfaces.core</groupId>
> > <artifactId>myfaces-api</artifactId>
> > <version>2.1.3</version>
> > ...
> > <supplyManagement>
> > <supplies>
> > <groupId>javax.faces</groupId>
> > <artifactId>jsf-api</artifactId>
> > <version>[2.1,2.2)</version>
> > <claim>superset</claim>
> > <supplies>
> > <supplies>
> > <groupId>org.jboss.spec.javax.faces</groupId>
> > <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> > <claim>equivalent</claim>
> > <supplies>
> > </supplyManagement>
> > ...
> > </project>
> > ```
> >
> > This indicates that the `myfaces-api` artifact is intended to be useable
> as
> > a drop-in replacement for either the `javax.faces:jsf-api` artifact
> within
> > a bounded range or for any version of the
> > `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you get
> a
> > supplier conflict in your classpath, then Maven should fail the build.
> >
> > For example if somebody forked `myfaces-api` but did not list
> `myfaces-api`
> > in the fork's supplyManagement and you end up with both `myfaces-api` and
> > `myfaces-fork-api` in your classpath. Maven can detect that there are two
> > dependencies that both claim to supply `javax.faces:jsf-api` and fail the
> > build, thereby letting the user add either exclusions or additional
> > supplies information to one of the artifacts and preventing duplicate
> > artifacts on the classpath. The build need not be failed if the supplies
> > claims provide a resolution. e.g. if the claim is `equivalent` then that
> > implies that there is a 1:1 mapping and hence the artifacts are drop-in
> > replacements. However where the claim is `superset` we cannot know that
> the
> > extra content in our artifact is the same as the extra content in another
> > artifact which has a superset of `javax.faces:jsf-api`.
> >
> > ### Example 2
> >
> > This illustrates a JSF component library that is working with the
> existing
> > JSF APIs
> >
> > ```
> > <project>
> > ...
> > <dependencies>
> > <dependency>
> > <groupId>javax.faces</groupId>
> > <artifactId>jsf-api</artifactId>
> > <version>2.1</version>
> > <supplyManagement>
> > <supplies>
> > <groupId>org.jboss.spec.javax.faces</groupId>
> > <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> > <claim>equivalent</claim>
> > <supplies>
> > <supplies>
> > <groupId>org.apache.myfaces.core</groupId>
> > <artifactId>myfaces-api</artifactId>
> > <version>[2.1,2.2)</version>
> > <claim>equivalent</claim>
> > </supplies>
> > </supplyManagement>
> > <dependency>
> > </dependencies>
> > ...
> > </project>
> > ```
> >
> > In this case we are publishing a transitive dependency with additional
> > supplyManagement injected. Consumers of this project would thus gain the
> > benefit of collapsing their transitive dependencies for any of these
> three
> > artifacts. As all artifacts are declared with `equivalent` claim, thus
> the
> > nearest of those three artifacts to the project will win as per the
> > standard dependency resolution rules when dealing with conflicting
> version
> > requirements in the transitive dependency tree.
> >
> > ### Example 3
> >
> > Finally, there is the case where you need to correct an incorrect claim
> of
> > supply
> >
> >
> > ```
> > <project>
> > ...
> > <dependencies>
> > <dependency>
> > <groupId>javax.faces</groupId>
> > <artifactId>jsf-api</artifactId>
> > <version>2.1</version>
> > <exclusions>
> > <exclusion>
> > <groupId>org.jboss.spec.javax.faces</groupId>
> > <artifactId>jboss-jsf-api_2.2_spec</artifactId>
> > <scope>supplies</scope>
> > <exclusion>
> > </exclusions>
> > <dependency>
> > </dependencies>
> > ...
> > </project>
> > ```
> >
> > This would typically be coupled with adding back in a correct supplies
> > definition, but we need to allow for people to correct the graph after
> the
> > fact of their dependencies being deployed to the remote repository.
> >
> > ### Claim conflict resolution
> >
> > The four classes of claim can be resolved using the following matrix
> >
> > ```
> > +---------------------------------------------------+
> > | A |
> > +------------+------------+------------+------------+
> > | subset | equivalent | superset | disjoint |
> > +---+------------+------------+------------+------------+------------+
> > | | subset | conflict | A wins | A wins | conflict |
> > | +------------+------------+------------+------------+------------+
> > | | equivalent | B wins | A or B | A wins | conflict |
> > | B +------------+------------+------------+------------+------------+
> > | | superset | B wins | B wins | conflict | conflict |
> > | +------------+------------+------------+------------+------------+
> > | | disjoint | conflict | conflict | conflict | conflict |
> > +---+------------+------------+------------+------------+------------+
> > ```
> >
> > The default unspecified claim is `disjoint` which indicates that some of
> > the content is reproduced, but not all and there is additional content
> > added. With such a claim there will always be conflict and the build
> should
> > fail until the Project Model is updated to either remove some of the
> claims
> > or resolve the dependency clash.
> >
> > The ideal claim is `equivalent` which indicates that the two artifacts
> are
> > bi-directionally substitutable. This does not mean that the contents are
> > identical. It does mean that they both deliver on the same contract in an
> > equivalent fashion.
> >
> > The `subset` and `superset` claims are for aggregation APIs. So for
> example
> > the Java EE Web Profile API is a superset of the various spec APIs that
> > make up the Java EE Web Profile and a subset of the full Java EE
> > specification. The use of the `subset` claim should be reserved to those
> > cases that are strict subsets. If anything is added that is not in the
> > supplied artifact then the correct claim is `disjoint`.
> >
> > ### Validation of supplies claims
> >
> > We do not want to introduce Java bias with this feature. As a result the
> > validation of claims and supplies directives will be left to plugins. For
> > the Java case we should probably provide either/both an enforcer rule or
> a
> > maven hosted plugin to assist in checking JAR projects against the
> declared
> > supplies declarations, but Maven core should not be concerned with
> solving
> > the validation problem.
> >
> > Similarly, while there may be advantages in a more fine grained API
> > contract negotiation between dependencies, to bind such a concept into
> the
> > project model would significantly taint the Maven project model with more
> > Java-centric concepts. Given that current software development typically
> > uses at least two core languages: Java and JavaScript, we should be
> aiming
> > to reduce Java-centric constructs from Maven rather than increase them.
> >
> > ### Backporting
> >
> > It will not be possible to fully express this concept in a modelVersion
> > 4.0.0 project model. Thus if generating 4.0.0 compatible project models,
> > the aim should be to fully resolve the dependencies of the project using
> > all available information and express that as the transitive
> dependencies.
> >
> > Thus we will not expose the "supplies" information to modelVersion 4.0.0
> > parsers but we will expose the end results of that and present the final
> > effective flattened dependency tree.
> >
> > modelVersion 4.0.0 consumers will thus be no worse off than they already
> > are and those consumers understanding newer modelVersions can get the
> > enhanced tree resolution that they would not get otherwise.
> >
>
Re: "supplies" concept proposal (was "provides" could be "proffers")
Posted by Paul Benedict <pb...@apache.org>.
I am having trouble understanding how grouping together artifacts that are
"equivalent" gain me anything in my project building. I am already doing
this in my POM today by excluding what's equivalent. Is this proposal
simply about adding semantic information to the POM so it's more apparent
what the intent is?
Furthermore, whoever defines the equivalence needs to be extremely careful.
Not all "equivalent" jars are actually equivalent. Most people have
learned, for example, that the javaee-api:6.0 jar in Maven is all stubs and
can't be used for unit testing; so people go to find EE jars from JBoss or
GlassFish, to get the full functionality. So I am not sure I'd ever want
someone telling me what's equivalent during consuming.
Cheers,
Paul
On Fri, Jun 20, 2014 at 7:51 AM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> "supplies" concept proposal
> ===========================
>
> Introduction
> ------------
>
> The following is a proposal for Maven in a post-modelVersion-4.0.0 era. The
> aim of this proposal is to simplify the management of dependency trees in
> the decentralised era of artifact production that we find ourselves in.
>
> The core issue is that different organisations can produce artifacts that
> may overlap. The easiest example is the servlet-api. If we restrict
> ourselves to version 2.5 of the servlet specification there are quite a few
> artifacts that all deliver the exact same content:
>
> * `jetty:servlet-api:2.5-6.0.2`
> * `org.daisy.libs:servlet-api:2.5.0`
> * `org.mortbay.jetty:servlet-api-2.5:6.1.14`
> * `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final`
> * etc
>
> **Note:** this is a generic problem that is not restricted to the
> servlet-api, the servlet-api just provides the example that will be most
> familiar to everyone.
>
> So where these multiple artifacts supplying the equivalent content becomes
> a problem is when the dependency tree is being calculated. If you have two
> dependencies each declaring transitive dependencies on different artifacts
> that supply equivalent content, then you end up with two copies of the same
> JAR file in your classpath.
>
> In the case of the servlet-api, the hack most people use is to declare the
> servlet-api with scope `provided` thus preventing it from being transitive.
> This is, however, a hack. In a more ideal world it would be better to let
> the servlet-api be transitive and only when we get to the WAR module would
> we declare that a specific servlet-api is to be provided in the containers
> that the WAR is targets for deployment into.
>
> We can take a second example that does not have the luxury of a *de facto*
> hack.
>
> * `javax.faces:jsf-api:2.1`
> * `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final`
> * `org.apache.myfaces.core:myfaces-api:2.1.13`
>
> Now in the case of the JSF API, you are supposed to bundle the JSF API in
> your WAR file. So if I use three JSF component libraries, I could very well
> end up with three different but equivalent JSF API jars in my WAR file.
>
> Ideally what we want is some way of telling Maven that these artifacts are
> equivalent.
>
> Proposal
> --------
>
> Introduce the concept of "supplies" to the project model. The concept needs
> three changes to the project model:
>
> 1. An explicit top level construct for a project to explicitly declare
> up-front artifacts that it knows - at the time the project is being
> authored - to contain equivalent content to at least a subset of the
> project's content. Declarations could include a claim from: `subset`,
> `superset`, `disjoint` and `equivalent` with the default being `disjoint`.
> 2. An explicit sub-element of the `dependency` construct to allow consumers
> to *post-facto* declare a specific dependency as supplying equivalent
> content for other dependencies
> 3. An extension to the `dependency/excludes/exclude` construct to allow
> consumers to remove claims a dependency makes with respect to supplying
> equivalent content
>
> By way of illustration, here are some examples of these constructs mapped
> to a Model Version 4.0.0 like XML schema. As the post-modelVersion-4.0.0
> schema is not yet known, this represents the best way to illustrate how the
> concept will work, but note that this proposal does not suggest a schema
> for this concept.
>
> ### Example 1
>
> This illustrates how we would want, say, the `myfaces-api` project model to
> look.
>
> ```
> <project>
> <groupId>org.apache.myfaces.core</groupId>
> <artifactId>myfaces-api</artifactId>
> <version>2.1.3</version>
> ...
> <supplyManagement>
> <supplies>
> <groupId>javax.faces</groupId>
> <artifactId>jsf-api</artifactId>
> <version>[2.1,2.2)</version>
> <claim>superset</claim>
> <supplies>
> <supplies>
> <groupId>org.jboss.spec.javax.faces</groupId>
> <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> <claim>equivalent</claim>
> <supplies>
> </supplyManagement>
> ...
> </project>
> ```
>
> This indicates that the `myfaces-api` artifact is intended to be useable as
> a drop-in replacement for either the `javax.faces:jsf-api` artifact within
> a bounded range or for any version of the
> `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you get a
> supplier conflict in your classpath, then Maven should fail the build.
>
> For example if somebody forked `myfaces-api` but did not list `myfaces-api`
> in the fork's supplyManagement and you end up with both `myfaces-api` and
> `myfaces-fork-api` in your classpath. Maven can detect that there are two
> dependencies that both claim to supply `javax.faces:jsf-api` and fail the
> build, thereby letting the user add either exclusions or additional
> supplies information to one of the artifacts and preventing duplicate
> artifacts on the classpath. The build need not be failed if the supplies
> claims provide a resolution. e.g. if the claim is `equivalent` then that
> implies that there is a 1:1 mapping and hence the artifacts are drop-in
> replacements. However where the claim is `superset` we cannot know that the
> extra content in our artifact is the same as the extra content in another
> artifact which has a superset of `javax.faces:jsf-api`.
>
> ### Example 2
>
> This illustrates a JSF component library that is working with the existing
> JSF APIs
>
> ```
> <project>
> ...
> <dependencies>
> <dependency>
> <groupId>javax.faces</groupId>
> <artifactId>jsf-api</artifactId>
> <version>2.1</version>
> <supplyManagement>
> <supplies>
> <groupId>org.jboss.spec.javax.faces</groupId>
> <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> <claim>equivalent</claim>
> <supplies>
> <supplies>
> <groupId>org.apache.myfaces.core</groupId>
> <artifactId>myfaces-api</artifactId>
> <version>[2.1,2.2)</version>
> <claim>equivalent</claim>
> </supplies>
> </supplyManagement>
> <dependency>
> </dependencies>
> ...
> </project>
> ```
>
> In this case we are publishing a transitive dependency with additional
> supplyManagement injected. Consumers of this project would thus gain the
> benefit of collapsing their transitive dependencies for any of these three
> artifacts. As all artifacts are declared with `equivalent` claim, thus the
> nearest of those three artifacts to the project will win as per the
> standard dependency resolution rules when dealing with conflicting version
> requirements in the transitive dependency tree.
>
> ### Example 3
>
> Finally, there is the case where you need to correct an incorrect claim of
> supply
>
>
> ```
> <project>
> ...
> <dependencies>
> <dependency>
> <groupId>javax.faces</groupId>
> <artifactId>jsf-api</artifactId>
> <version>2.1</version>
> <exclusions>
> <exclusion>
> <groupId>org.jboss.spec.javax.faces</groupId>
> <artifactId>jboss-jsf-api_2.2_spec</artifactId>
> <scope>supplies</scope>
> <exclusion>
> </exclusions>
> <dependency>
> </dependencies>
> ...
> </project>
> ```
>
> This would typically be coupled with adding back in a correct supplies
> definition, but we need to allow for people to correct the graph after the
> fact of their dependencies being deployed to the remote repository.
>
> ### Claim conflict resolution
>
> The four classes of claim can be resolved using the following matrix
>
> ```
> +---------------------------------------------------+
> | A |
> +------------+------------+------------+------------+
> | subset | equivalent | superset | disjoint |
> +---+------------+------------+------------+------------+------------+
> | | subset | conflict | A wins | A wins | conflict |
> | +------------+------------+------------+------------+------------+
> | | equivalent | B wins | A or B | A wins | conflict |
> | B +------------+------------+------------+------------+------------+
> | | superset | B wins | B wins | conflict | conflict |
> | +------------+------------+------------+------------+------------+
> | | disjoint | conflict | conflict | conflict | conflict |
> +---+------------+------------+------------+------------+------------+
> ```
>
> The default unspecified claim is `disjoint` which indicates that some of
> the content is reproduced, but not all and there is additional content
> added. With such a claim there will always be conflict and the build should
> fail until the Project Model is updated to either remove some of the claims
> or resolve the dependency clash.
>
> The ideal claim is `equivalent` which indicates that the two artifacts are
> bi-directionally substitutable. This does not mean that the contents are
> identical. It does mean that they both deliver on the same contract in an
> equivalent fashion.
>
> The `subset` and `superset` claims are for aggregation APIs. So for example
> the Java EE Web Profile API is a superset of the various spec APIs that
> make up the Java EE Web Profile and a subset of the full Java EE
> specification. The use of the `subset` claim should be reserved to those
> cases that are strict subsets. If anything is added that is not in the
> supplied artifact then the correct claim is `disjoint`.
>
> ### Validation of supplies claims
>
> We do not want to introduce Java bias with this feature. As a result the
> validation of claims and supplies directives will be left to plugins. For
> the Java case we should probably provide either/both an enforcer rule or a
> maven hosted plugin to assist in checking JAR projects against the declared
> supplies declarations, but Maven core should not be concerned with solving
> the validation problem.
>
> Similarly, while there may be advantages in a more fine grained API
> contract negotiation between dependencies, to bind such a concept into the
> project model would significantly taint the Maven project model with more
> Java-centric concepts. Given that current software development typically
> uses at least two core languages: Java and JavaScript, we should be aiming
> to reduce Java-centric constructs from Maven rather than increase them.
>
> ### Backporting
>
> It will not be possible to fully express this concept in a modelVersion
> 4.0.0 project model. Thus if generating 4.0.0 compatible project models,
> the aim should be to fully resolve the dependencies of the project using
> all available information and express that as the transitive dependencies.
>
> Thus we will not expose the "supplies" information to modelVersion 4.0.0
> parsers but we will expose the end results of that and present the final
> effective flattened dependency tree.
>
> modelVersion 4.0.0 consumers will thus be no worse off than they already
> are and those consumers understanding newer modelVersions can get the
> enhanced tree resolution that they would not get otherwise.
>