You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Oliver Lietz <ap...@oliverlietz.de> on 2016/06/03 09:20:36 UTC
Re: Dependency Management
On Monday 30 May 2016 15:28:04 Robert Munteanu wrote:
> On Fri, 2016-05-27 at 11:14 +0200, Oliver Lietz wrote:
> > I don't think there is support in Maven 3 for unit tests using a
> > different
> > version than for compile. Not sure if other build tools can handle
> > different
> > versions of dependencies for compile and test.
>
> Maven 3 is unable to do that. I can't say anything for other build
> systems.
Looks like Gradle can do (but I haven't tested):
https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plugin_and_dependency_management
> > So my preferred answers to eliminate outdated dependencies in tests:
> >
> > 1. use minimal version for compile and current version for test
> > (how?)
> > 2. use current version for compile and test
> > 3. use current version for compile and test and carefully expand
> > import range
>
> We seem to be discussing 'test' but I think we need to separate unit
> tests stored in the same Maven projects and integration tests.
"I don't think there is support in Maven 3 for *unit* tests using a different
version than for compile." - it's about unit tests, sorry if it wasn't clear
enough.
> a) Integration tests should use the same dependencies as the launchpad.
> After all, the launchpad is what we ship and things should work fine in
> that setup. If other parties want different combinations, they're free
> to define those combinations and test it themselves.
We have different setups for ITs so that is not always given (pure Pax Exam,
based on Sling Start, external Launchpad, Pax Exam Karaf, ...).
And yes, current Launchpad should be the benchmark for new releases. Other
parties are free to backport new releases for older systems.
> b) Unit tests may aim to use the same dependencies as the launchpad. I
> am not thoroughly convinced of the value, given that unit tests only
> touch the surface of an API. Chances of not finding out issues due to a
> missed upgrade of commons-lang are way smaller that missing an upgrade
> to oak-core ( for instance ).
AFAIR we are moving ITs closer to the code under test (Bertrand?) and we
already have some heavy unit tests for some modules which do not have
equivalent ITs, e.g. Event, Discovery, Distribution, Engine. Please do not
underrate our unit tests.
Commons Collections changed behavior from 3.2.1 to 3.2.2 due to unsafe
serialization. Similar changes can happen in other libraries for what ever
reason. How will we notice we are affected? Reports from users?
> Also, it's quite troublesome to keep all versions of all modules in
> sync with the launchpad. All that manual work is bound to get mistakes
> for - again, IMO - little gain.
>
> So I would say that if we want to align unit test dependencies with the
> launchpad versions we should have:
>
> 1. An automatic way of defining dependency versions
> 2. A way of separating unit test from compile dependencies
I don't think we need to keep all modules in sync. But we should do a moderate
modernization of (3rd party) libraries, instead of keeping backwards
compatibility forever for the sake of itself. This leads to ugly code and we
may miss incompatibilities with Launchpad. Again, other parties are free to
backport new releases for older systems or upgrade libraries also (AEM 6.1
works well with Commons Lang 3.4).
Regards,
O.
> Thanks,
>
> Robert
Re: Dependency Management
Posted by Konrad Windszus <ko...@gmx.de>.
> On 07 Jun 2016, at 10:32, Konrad Windszus <ko...@gmx.de> wrote:
>
> Hi,
>> On 07 Jun 2016, at 10:12, Oliver Lietz <ap...@oliverlietz.de> wrote:
>>
>> On Monday 06 June 2016 14:57:16 Konrad Windszus wrote:
>>> I would like to wrap things up here:
>>>
>>> So it seems the common agreement is:
>>
>> Really?
>>
>>> 1) Depend on the lowest possible version of a dependency (to let bnd
>>> generate import package ranges which are as broad as possible to support as
>>> many different systems as possible (even old ones)). A module should only
>>> be upgraded to a newer dependency in case this is really necessary (because
>>> of a feature being used which is not available in earlier version or if the
>>> newer version contains a bugfix from which the consumer bundle is
>>> affected).
>>
>> I don't see all topics discussed and your definition is very vague. It can be
>> taken as "never upgrade". This prevents modernization of our code base
>> (example given).
> I agree that this is rather vague. We definitely need an official way of upgrading dependencies. But in the cases of upgrading commons-lang and commons-io I don't see any convincing reason to upgrade to the newest dependency for the modules validation or health check (we don't actually leverage new API there yet). Only if we do, we can and should discuss again.
>
>>
>>> 2) For ITs and the Maven Launchpad depend on the same version
>>> which is reasonably new (this is only possible if the IT is in a dedicated
>>> Maven bundle).
>>
>> Pax Exam can pull in any version of a dependency even when running in the same
>> module. The limitation to one version is only true for our current setup of
>> unit tests with Maven 3.
> Not all ITs are based on Pax Exam. So the ones leveraging a real Sling instance and being located in the same bundle as the code itself (Transporter rule) are not that easy to decouple.
>>
>>> 3) Since Maven does not completely separate compile and test
>>> classpath for the unit test we must depend on the lowest possible version
>>> as well.
>>>
>>> If everyone is fine with that, I would ask Oliver Lietz to basically revert
>>> some changes being done in https://issues.apache.org/jira/browse/SLING-5603 <https://issues.apache.org/jira/browse/SLING-5603>
>>> and https://issues.apache.org/jira/browse/SLING-5685 <https://issues.apache.org/jira/browse/SLING-5685>. Thanks for your
>>> input,
>>
>> To make clear you are the driving force behind this approach please go ahead
>> and revert yourself (and don't forget to adjust fixed versions).
> Ok, I will do so for Healthcheck Core in https://issues.apache.org/jira/browse/SLING-5685 <https://issues.apache.org/jira/browse/SLING-5685>.
Actually it is unclear why the dependency to Commons IO was introduced at all for HC Core. Please also check https://issues.apache.org/jira/browse/SLING-5685?focusedCommentId=15318210&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15318210.
@Oli: Can you quickly comment on that? Thanks
>>
>> Regards,
>> O.
>>
>>> Konrad
>>>
>>>> On 03 Jun 2016, at 13:32, Oliver Lietz <ap...@oliverlietz.de> wrote:
>>>>
>>>> On Friday 03 June 2016 20:13:29 Steven Walters wrote:
>>>>> On Fri, Jun 3, 2016 at 6:20 PM, Oliver Lietz <ap...@oliverlietz.de>
>> wrote:
>>>>>> On Monday 30 May 2016 15:28:04 Robert Munteanu wrote:
>>>>>>> On Fri, 2016-05-27 at 11:14 +0200, Oliver Lietz wrote:
>>>>>>>> I don't think there is support in Maven 3 for unit tests using a
>>>>>>>> different
>>>>>>>> version than for compile. Not sure if other build tools can handle
>>>>>>>> different
>>>>>>>> versions of dependencies for compile and test.
>>>>>>>
>>>>>>> Maven 3 is unable to do that. I can't say anything for other build
>>>>>>> systems.
>>>>>>
>>>>>> Looks like Gradle can do (but I haven't tested):
>>>>>> https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plug
>>>>>> in
>>>>>> _and_dependency_management
>>>>>
>>>>> Yes, Gradle can do this.
>>>>> 'compile' and 'testRuntime' configurations are the standard
>>>>> configuration names that are relevant here and there is a natural
>>>>> inheritance chain (such that the default is that 'testRuntime'
>>>>> eventually inherits everything in 'compile'),
>>>>> but the versions used by 'testCompile/'testRuntime' can be overridden
>>>>> to be different (usually newer) ones than what is used by
>>>>> 'compile'/'runtime'
>>>>>
>>>>> I've done this plenty in the past when working with AEM to use newer
>>>>> APIs/frameworks in the tests than what the code is compiled against.
>>>>> such as compiling for AEM 5.6, but due to using newer versions of the
>>>>> Sling mock frameworks in test cases, having newer Sling jars pulled in
>>>>> as a result for test case compilation + execution.
>>>>> Maven can't do this, so it makes writing tests for the same code more
>>>>> difficult when using Maven when compared to Gradle under these
>>>>> circumstances.
>>>>>
>>>>> However, I don't particularly see this being enough reason alone to
>>>>> warrant the effort of having Sling switching to Gradle.
>>>>> There are some other possible benefits to using Gradle, such as being
>>>>> able to merge separate IT modules back into modules they are testing,
>>>>> therefore reducing the number of modules/build complexity.
>>>>> But even so, I still don't see Sling leaving Maven.
>>>>
>>>> Same here. That would be a lot of work and I don't think the project can
>>>> perform such a big task right now.
>>>>
>>>> O.
Re: Dependency Management
Posted by Konrad Windszus <ko...@gmx.de>.
Hi,
> On 07 Jun 2016, at 10:12, Oliver Lietz <ap...@oliverlietz.de> wrote:
>
> On Monday 06 June 2016 14:57:16 Konrad Windszus wrote:
>> I would like to wrap things up here:
>>
>> So it seems the common agreement is:
>
> Really?
>
>> 1) Depend on the lowest possible version of a dependency (to let bnd
>> generate import package ranges which are as broad as possible to support as
>> many different systems as possible (even old ones)). A module should only
>> be upgraded to a newer dependency in case this is really necessary (because
>> of a feature being used which is not available in earlier version or if the
>> newer version contains a bugfix from which the consumer bundle is
>> affected).
>
> I don't see all topics discussed and your definition is very vague. It can be
> taken as "never upgrade". This prevents modernization of our code base
> (example given).
I agree that this is rather vague. We definitely need an official way of upgrading dependencies. But in the cases of upgrading commons-lang and commons-io I don't see any convincing reason to upgrade to the newest dependency for the modules validation or health check (we don't actually leverage new API there yet). Only if we do, we can and should discuss again.
>
>> 2) For ITs and the Maven Launchpad depend on the same version
>> which is reasonably new (this is only possible if the IT is in a dedicated
>> Maven bundle).
>
> Pax Exam can pull in any version of a dependency even when running in the same
> module. The limitation to one version is only true for our current setup of
> unit tests with Maven 3.
Not all ITs are based on Pax Exam. So the ones leveraging a real Sling instance and being located in the same bundle as the code itself (Transporter rule) are not that easy to decouple.
>
>> 3) Since Maven does not completely separate compile and test
>> classpath for the unit test we must depend on the lowest possible version
>> as well.
>>
>> If everyone is fine with that, I would ask Oliver Lietz to basically revert
>> some changes being done in https://issues.apache.org/jira/browse/SLING-5603
>> and https://issues.apache.org/jira/browse/SLING-5685. Thanks for your
>> input,
>
> To make clear you are the driving force behind this approach please go ahead
> and revert yourself (and don't forget to adjust fixed versions).
Ok, I will do so for Healthcheck Core in https://issues.apache.org/jira/browse/SLING-5685.
>
> Regards,
> O.
>
>> Konrad
>>
>>> On 03 Jun 2016, at 13:32, Oliver Lietz <ap...@oliverlietz.de> wrote:
>>>
>>> On Friday 03 June 2016 20:13:29 Steven Walters wrote:
>>>> On Fri, Jun 3, 2016 at 6:20 PM, Oliver Lietz <ap...@oliverlietz.de>
> wrote:
>>>>> On Monday 30 May 2016 15:28:04 Robert Munteanu wrote:
>>>>>> On Fri, 2016-05-27 at 11:14 +0200, Oliver Lietz wrote:
>>>>>>> I don't think there is support in Maven 3 for unit tests using a
>>>>>>> different
>>>>>>> version than for compile. Not sure if other build tools can handle
>>>>>>> different
>>>>>>> versions of dependencies for compile and test.
>>>>>>
>>>>>> Maven 3 is unable to do that. I can't say anything for other build
>>>>>> systems.
>>>>>
>>>>> Looks like Gradle can do (but I haven't tested):
>>>>> https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plug
>>>>> in
>>>>> _and_dependency_management
>>>>
>>>> Yes, Gradle can do this.
>>>> 'compile' and 'testRuntime' configurations are the standard
>>>> configuration names that are relevant here and there is a natural
>>>> inheritance chain (such that the default is that 'testRuntime'
>>>> eventually inherits everything in 'compile'),
>>>> but the versions used by 'testCompile/'testRuntime' can be overridden
>>>> to be different (usually newer) ones than what is used by
>>>> 'compile'/'runtime'
>>>>
>>>> I've done this plenty in the past when working with AEM to use newer
>>>> APIs/frameworks in the tests than what the code is compiled against.
>>>> such as compiling for AEM 5.6, but due to using newer versions of the
>>>> Sling mock frameworks in test cases, having newer Sling jars pulled in
>>>> as a result for test case compilation + execution.
>>>> Maven can't do this, so it makes writing tests for the same code more
>>>> difficult when using Maven when compared to Gradle under these
>>>> circumstances.
>>>>
>>>> However, I don't particularly see this being enough reason alone to
>>>> warrant the effort of having Sling switching to Gradle.
>>>> There are some other possible benefits to using Gradle, such as being
>>>> able to merge separate IT modules back into modules they are testing,
>>>> therefore reducing the number of modules/build complexity.
>>>> But even so, I still don't see Sling leaving Maven.
>>>
>>> Same here. That would be a lot of work and I don't think the project can
>>> perform such a big task right now.
>>>
>>> O.
>
Re: Dependency Management
Posted by Oliver Lietz <ap...@oliverlietz.de>.
On Monday 06 June 2016 14:57:16 Konrad Windszus wrote:
> I would like to wrap things up here:
>
> So it seems the common agreement is:
Really?
> 1) Depend on the lowest possible version of a dependency (to let bnd
> generate import package ranges which are as broad as possible to support as
> many different systems as possible (even old ones)). A module should only
> be upgraded to a newer dependency in case this is really necessary (because
> of a feature being used which is not available in earlier version or if the
> newer version contains a bugfix from which the consumer bundle is
> affected).
I don't see all topics discussed and your definition is very vague. It can be
taken as "never upgrade". This prevents modernization of our code base
(example given).
> 2) For ITs and the Maven Launchpad depend on the same version
> which is reasonably new (this is only possible if the IT is in a dedicated
> Maven bundle).
Pax Exam can pull in any version of a dependency even when running in the same
module. The limitation to one version is only true for our current setup of
unit tests with Maven 3.
> 3) Since Maven does not completely separate compile and test
> classpath for the unit test we must depend on the lowest possible version
> as well.
>
> If everyone is fine with that, I would ask Oliver Lietz to basically revert
> some changes being done in https://issues.apache.org/jira/browse/SLING-5603
> and https://issues.apache.org/jira/browse/SLING-5685. Thanks for your
> input,
To make clear you are the driving force behind this approach please go ahead
and revert yourself (and don't forget to adjust fixed versions).
Regards,
O.
> Konrad
>
> > On 03 Jun 2016, at 13:32, Oliver Lietz <ap...@oliverlietz.de> wrote:
> >
> > On Friday 03 June 2016 20:13:29 Steven Walters wrote:
> >> On Fri, Jun 3, 2016 at 6:20 PM, Oliver Lietz <ap...@oliverlietz.de>
wrote:
> >>> On Monday 30 May 2016 15:28:04 Robert Munteanu wrote:
> >>>> On Fri, 2016-05-27 at 11:14 +0200, Oliver Lietz wrote:
> >>>>> I don't think there is support in Maven 3 for unit tests using a
> >>>>> different
> >>>>> version than for compile. Not sure if other build tools can handle
> >>>>> different
> >>>>> versions of dependencies for compile and test.
> >>>>
> >>>> Maven 3 is unable to do that. I can't say anything for other build
> >>>> systems.
> >>>
> >>> Looks like Gradle can do (but I haven't tested):
> >>> https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plug
> >>> in
> >>> _and_dependency_management
> >>
> >> Yes, Gradle can do this.
> >> 'compile' and 'testRuntime' configurations are the standard
> >> configuration names that are relevant here and there is a natural
> >> inheritance chain (such that the default is that 'testRuntime'
> >> eventually inherits everything in 'compile'),
> >> but the versions used by 'testCompile/'testRuntime' can be overridden
> >> to be different (usually newer) ones than what is used by
> >> 'compile'/'runtime'
> >>
> >> I've done this plenty in the past when working with AEM to use newer
> >> APIs/frameworks in the tests than what the code is compiled against.
> >> such as compiling for AEM 5.6, but due to using newer versions of the
> >> Sling mock frameworks in test cases, having newer Sling jars pulled in
> >> as a result for test case compilation + execution.
> >> Maven can't do this, so it makes writing tests for the same code more
> >> difficult when using Maven when compared to Gradle under these
> >> circumstances.
> >>
> >> However, I don't particularly see this being enough reason alone to
> >> warrant the effort of having Sling switching to Gradle.
> >> There are some other possible benefits to using Gradle, such as being
> >> able to merge separate IT modules back into modules they are testing,
> >> therefore reducing the number of modules/build complexity.
> >> But even so, I still don't see Sling leaving Maven.
> >
> > Same here. That would be a lot of work and I don't think the project can
> > perform such a big task right now.
> >
> > O.
Re: Dependency Management
Posted by Konrad Windszus <ko...@gmx.de>.
I would like to wrap things up here:
So it seems the common agreement is:
1) Depend on the lowest possible version of a dependency (to let bnd generate import package ranges which are as broad as possible to support as many different systems as possible (even old ones)).
A module should only be upgraded to a newer dependency in case this is really necessary (because of a feature being used which is not available in earlier version or if the newer version contains a bugfix from which the consumer bundle is affected).
2) For ITs and the Maven Launchpad depend on the same version which is reasonably new (this is only possible if the IT is in a dedicated Maven bundle).
3) Since Maven does not completely separate compile and test classpath for the unit test we must depend on the lowest possible version as well.
If everyone is fine with that, I would ask Oliver Lietz to basically revert some changes being done in https://issues.apache.org/jira/browse/SLING-5603 and https://issues.apache.org/jira/browse/SLING-5685.
Thanks for your input,
Konrad
> On 03 Jun 2016, at 13:32, Oliver Lietz <ap...@oliverlietz.de> wrote:
>
> On Friday 03 June 2016 20:13:29 Steven Walters wrote:
>> On Fri, Jun 3, 2016 at 6:20 PM, Oliver Lietz <ap...@oliverlietz.de> wrote:
>>> On Monday 30 May 2016 15:28:04 Robert Munteanu wrote:
>>>> On Fri, 2016-05-27 at 11:14 +0200, Oliver Lietz wrote:
>>>>> I don't think there is support in Maven 3 for unit tests using a
>>>>> different
>>>>> version than for compile. Not sure if other build tools can handle
>>>>> different
>>>>> versions of dependencies for compile and test.
>>>>
>>>> Maven 3 is unable to do that. I can't say anything for other build
>>>> systems.
>>>
>>> Looks like Gradle can do (but I haven't tested):
>>> https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plugin
>>> _and_dependency_management
>> Yes, Gradle can do this.
>> 'compile' and 'testRuntime' configurations are the standard
>> configuration names that are relevant here and there is a natural
>> inheritance chain (such that the default is that 'testRuntime'
>> eventually inherits everything in 'compile'),
>> but the versions used by 'testCompile/'testRuntime' can be overridden
>> to be different (usually newer) ones than what is used by
>> 'compile'/'runtime'
>>
>> I've done this plenty in the past when working with AEM to use newer
>> APIs/frameworks in the tests than what the code is compiled against.
>> such as compiling for AEM 5.6, but due to using newer versions of the
>> Sling mock frameworks in test cases, having newer Sling jars pulled in
>> as a result for test case compilation + execution.
>> Maven can't do this, so it makes writing tests for the same code more
>> difficult when using Maven when compared to Gradle under these
>> circumstances.
>>
>> However, I don't particularly see this being enough reason alone to
>> warrant the effort of having Sling switching to Gradle.
>> There are some other possible benefits to using Gradle, such as being
>> able to merge separate IT modules back into modules they are testing,
>> therefore reducing the number of modules/build complexity.
>> But even so, I still don't see Sling leaving Maven.
>
> Same here. That would be a lot of work and I don't think the project can
> perform such a big task right now.
>
> O.
>
Re: Dependency Management
Posted by Oliver Lietz <ap...@oliverlietz.de>.
On Friday 03 June 2016 20:13:29 Steven Walters wrote:
> On Fri, Jun 3, 2016 at 6:20 PM, Oliver Lietz <ap...@oliverlietz.de> wrote:
> > On Monday 30 May 2016 15:28:04 Robert Munteanu wrote:
> > > On Fri, 2016-05-27 at 11:14 +0200, Oliver Lietz wrote:
> > > > I don't think there is support in Maven 3 for unit tests using a
> > > > different
> > > > version than for compile. Not sure if other build tools can handle
> > > > different
> > > > versions of dependencies for compile and test.
> > >
> > > Maven 3 is unable to do that. I can't say anything for other build
> > > systems.
> >
> > Looks like Gradle can do (but I haven't tested):
> > https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plugin
> > _and_dependency_management
> Yes, Gradle can do this.
> 'compile' and 'testRuntime' configurations are the standard
> configuration names that are relevant here and there is a natural
> inheritance chain (such that the default is that 'testRuntime'
> eventually inherits everything in 'compile'),
> but the versions used by 'testCompile/'testRuntime' can be overridden
> to be different (usually newer) ones than what is used by
> 'compile'/'runtime'
>
> I've done this plenty in the past when working with AEM to use newer
> APIs/frameworks in the tests than what the code is compiled against.
> such as compiling for AEM 5.6, but due to using newer versions of the
> Sling mock frameworks in test cases, having newer Sling jars pulled in
> as a result for test case compilation + execution.
> Maven can't do this, so it makes writing tests for the same code more
> difficult when using Maven when compared to Gradle under these
> circumstances.
>
> However, I don't particularly see this being enough reason alone to
> warrant the effort of having Sling switching to Gradle.
> There are some other possible benefits to using Gradle, such as being
> able to merge separate IT modules back into modules they are testing,
> therefore reducing the number of modules/build complexity.
> But even so, I still don't see Sling leaving Maven.
Same here. That would be a lot of work and I don't think the project can
perform such a big task right now.
O.
Re: Dependency Management
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Fri, Jun 3, 2016 at 11:20 AM, Oliver Lietz <ap...@oliverlietz.de> wrote:
> ...AFAIR we are moving ITs closer to the code under test (Bertrand?)...
Yes, I've been doing that and encouraging people to do so.
OTOH the existing integration test suite in
launchpad/integration-tests works and there's no urgent need to
reorganize it IMO.
-Bertrand
Re: Dependency Management
Posted by Steven Walters <ke...@gmail.com>.
On Fri, Jun 3, 2016 at 6:20 PM, Oliver Lietz <ap...@oliverlietz.de> wrote:
>
> On Monday 30 May 2016 15:28:04 Robert Munteanu wrote:
> > On Fri, 2016-05-27 at 11:14 +0200, Oliver Lietz wrote:
> > > I don't think there is support in Maven 3 for unit tests using a
> > > different
> > > version than for compile. Not sure if other build tools can handle
> > > different
> > > versions of dependencies for compile and test.
> >
> > Maven 3 is unable to do that. I can't say anything for other build
> > systems.
>
> Looks like Gradle can do (but I haven't tested):
> https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plugin_and_dependency_management
Yes, Gradle can do this.
'compile' and 'testRuntime' configurations are the standard
configuration names that are relevant here and there is a natural
inheritance chain (such that the default is that 'testRuntime'
eventually inherits everything in 'compile'),
but the versions used by 'testCompile/'testRuntime' can be overridden
to be different (usually newer) ones than what is used by
'compile'/'runtime'
I've done this plenty in the past when working with AEM to use newer
APIs/frameworks in the tests than what the code is compiled against.
such as compiling for AEM 5.6, but due to using newer versions of the
Sling mock frameworks in test cases, having newer Sling jars pulled in
as a result for test case compilation + execution.
Maven can't do this, so it makes writing tests for the same code more
difficult when using Maven when compared to Gradle under these
circumstances.
However, I don't particularly see this being enough reason alone to
warrant the effort of having Sling switching to Gradle.
There are some other possible benefits to using Gradle, such as being
able to merge separate IT modules back into modules they are testing,
therefore reducing the number of modules/build complexity.
But even so, I still don't see Sling leaving Maven.