Re: dependency updates

[Jan Høydahl <ja...@cominvent.com>]

Hi Ben,

I started experimenting with gradle-versions-plugin, and it produces this report:
https://gist.github.com/janhoy/91acf9ebb856bf1a97c4cf1ff68b8c6e

Looks really useful. Perhaps you can comment on why it fails to find the version for org.apache.httpcomponents:httpmime
and also fails to determine lastest version for a whole lot dependencies like org.eclipse.jetty:jetty-http ?

We use the com.palantir.consistent-versions plugin, so versions are not declared in gradle files.


If we can get it working for all our dependencies, I plan to add it to our top build.gradle, and run it with Jenkins.
Would there be any benefit of triggering it with GitHub actions?

PR: https://github.com/apache/solr/pull/707 

Jan

> 18. mai 2021 kl. 01:36 skrev Benjamin Manes <be...@gmail.com>:
> 
> Hi Solr team,
> 
> I noticed that your project is pinned to an older version of Caffeine (v2.8.4, latest is v3.0.2). Both projects have moved to JDK11 in their latest major version. For Caffeine this allowed us to remove sun.misc.Unsafe in favor of VarHandles. I mention this because Andrzej Białecki specifically raised a concern about our use of Unsafe for Solr [1], but we could not address it then due to being on JDK8. For maintenance and support, it would be great if you can upgrade your dependencies regularly.
> 
> On that note, you might want to set up a periodic report to discover dependency updates. Keeping the build healthy can help avoid bugs and future incompatibility pains, but does risk suffering from newly introduced errors. Unfortunately Github's dependabot has poor Gradle support so the community continues to use the older gradle-versions-plugin [2], e.g. through a github action [3]. This plugin scripts Gradle's dependency management to generate a text or json report of possible upgrades based on the configured selection rules. Using this or something similar should let the team be more aware of possible upgrades and make the appropriate decisions. The plugin is agnostic to how you manage and store dependency versions, it merely reports based on what the build evaluates to.
> 
> Cheers,
> Ben
> 
> [1] https://github.com/ben-manes/caffeine/issues/273#issuecomment-557197399 <https://github.com/ben-manes/caffeine/issues/273#issuecomment-557197399>
> [2] https://github.com/ben-manes/gradle-versions-plugin <https://github.com/ben-manes/gradle-versions-plugin>
> [3] https://github.com/marketplace/actions/gradle-update-checker <https://github.com/marketplace/actions/gradle-update-checker>

Re: dependency updates

[Benjamin Manes <be...@gmail.com>]

Hi Jan,

This appears to be some interaction within your build, most likely by a
plugin like consistent-versions. The versions plugin copies the
configuration, rewrites the dependency to a dynamic query (+), and reports
on the differences. This relies on Gradle to perform the resolution.
Unfortunately other plugins have a habit of trying to constrain Gradle in
brittle ways, which can then lead to resolution failures. When using --info
the logs show errors like,

 - org.apache.hadoop:hadoop-common
     3.3.1
The exception that is the cause of unresolved state:
org.gradle.internal.resolve.ModuleVersionResolveException: Could not
resolve org.apache.hadoop:hadoop-common:+.
Required by:
    project :solr:modules:hadoop-auth
Caused by:
org.gradle.internal.component.AmbiguousConfigurationSelectionException: The
consumer was configured to find a usage of 'consistent-versions-usage' of a
library, preferably optimized for standard JVMs, and its dependencies
declared externally. However we cannot choose between the following
variants of org.apache.hadoop:hadoop-common:3.3.1:
  - compile
  - runtime

That leads me to believe that the consistent-versions plugin is not
handling the Gradle lifecycle correctly. Perhaps forgetting that
configurations can be added at runtime (detached, or in our case
configuration.copy()), judging from their manipulations (or it could be a
Gradle bug of not copying all of the configuration details). In these cases
the plugin that is interacting poorly needs to help investigate to
determine if it is their fix, Gradle's, or mine.

Their plugin seems to use dependency constraints to restrict version
resolution and therefore does not seem to handle dynamic ranges. In the
case of a `+` they should either skip their restriction or offer a means to
disable their check, e.g. so that you can disable constraints if the
dependencyUpdates task is on the gradle.taskGrapth. I think that you will
need to raise the issue with their team so that the plugin is compatible
with the ecosystem either by fixes or by offering an escape hatch setting.

Best,
Ben

On Fri, Feb 25, 2022 at 1:47 AM Jan Høydahl <ja...@cominvent.com>
wrote:

> Hi Ben,
>
> I started experimenting with gradle-versions-plugin, and it produces this
> report:
> https://gist.github.com/janhoy/91acf9ebb856bf1a97c4cf1ff68b8c6e
>
> Looks really useful. Perhaps you can comment on why it fails to find the
> version for org.apache.httpcomponents:httpmime
> and also fails to determine lastest version for a whole lot dependencies
> like org.eclipse.jetty:jetty-http ?
>
> We use the com.palantir.consistent-versions plugin, so versions are not
> declared in gradle files.
>
>
> If we can get it working for all our dependencies, I plan to add it to our
> top build.gradle, and run it with Jenkins.
> Would there be any benefit of triggering it with GitHub actions?
>
> PR: https://github.com/apache/solr/pull/707
>
> Jan
>
> 18. mai 2021 kl. 01:36 skrev Benjamin Manes <be...@gmail.com>:
>
> Hi Solr team,
>
> I noticed that your project is pinned to an older version of Caffeine
> (v2.8.4, latest is v3.0.2). Both projects have moved to JDK11 in their
> latest major version. For Caffeine this allowed us to remove
> sun.misc.Unsafe in favor of VarHandles. I mention this because Andrzej
> Białecki specifically raised a concern about our use of Unsafe for Solr
> [1], but we could not address it then due to being on JDK8. For maintenance
> and support, it would be great if you can upgrade your dependencies
> regularly.
>
> On that note, you might want to set up a periodic report to discover
> dependency updates. Keeping the build healthy can help avoid bugs and
> future incompatibility pains, but does risk suffering from newly introduced
> errors. Unfortunately Github's dependabot has poor Gradle support so the
> community continues to use the older gradle-versions-plugin [2], e.g.
> through a github action [3]. This plugin scripts Gradle's dependency
> management to generate a text or json report of possible upgrades based on
> the configured selection rules. Using this or something similar should let
> the team be more aware of possible upgrades and make the appropriate
> decisions. The plugin is agnostic to how you manage and store dependency
> versions, it merely reports based on what the build evaluates to.
>
> Cheers,
> Ben
>
> [1]
> https://github.com/ben-manes/caffeine/issues/273#issuecomment-557197399
> [2] https://github.com/ben-manes/gradle-versions-plugin
> [3] https://github.com/marketplace/actions/gradle-update-checker
>
>
>