You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Christian Frommeyer (Jira)" <ji...@apache.org> on 2022/07/08 09:16:00 UTC
[jira] [Updated] (MNG-7376) Maven artifact caching breaks in context of Jenkins Artifactory Plugin
[ https://issues.apache.org/jira/browse/MNG-7376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian Frommeyer updated MNG-7376:
-------------------------------------
Description:
We recently run into a build issue after a library update. While local builds would still work as expected, builds on our build machine would cause reproducible failures. The actual failure happens during site-reports are generated in the surefire report plugin but I believe the actual place is just a coincidence.
Unfortunately I wasn't able to find the trigger for the changed behavior. All we did was update the version of pact-jvm in the pom. Before the change no failure after the change every build fails. As this is proprietary code base I unfortunately currently cannot provide a sample project. However there is a couple of information I was able to find:
As there is little related logging even on debug level I had a look at the code. Using the stacktrace from the Jenkins log this pointed [here|https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/project/artifact/DefaultProjectArtifactsCache.java#L207]. Complaining about a re-caching of the project artifacts.
Looking at the calling code [here |https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/LifecycleDependencyResolver.java#L147] this doesn't seems to be possible. As the Cache-put is only called after the Cache-get returns {{{}null{}}}. This would only be the case if the cache was behaving incorrect (it's backed by a Java Concurrent HashMap, so pretty unlikely), or the key is not behaving as expected.
Looking at the [CacheKey|https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/project/artifact/DefaultProjectArtifactsCache.java#L58] reveals that this key in fact has been crafted to explicitly make it behave nicely. Most fields are immutable. However there is a small set of (potentially) mutable fields that would alter hashcode and equals result. If e.g. the RemoteRepositories stored in the {{repositories}} property would be altered this would change hashcode and equals. While the property is {{final}} and the constructor assures that it contains a fresh ArrayList, this is neither unmodifiable, nor are the repositories added to the list immutable.
To be sure I created a java-agent to monitor the behavior. I logged the key for both {{get}} and {{put}} and actually found different URLs for the repositories. Apparently the Jenkins-Artifactory-Plugin had changed them. And actually looking at the debug log there are log lines saying _Replacing resolution repository URL..._.
While this seems to happen very rarely it happens and if it does it's not fun debugging. I still don't know why it only happens rarely as I assume the plugin does rewrite the URLs always. But maybe there is only rare cases where the cache actually has data.
This are the updated artifacts:
{code:xml}
<dependencies>
<dependency>
<groupId>au.com.dius.pact.consumer</groupId>
<artifactId>junit5</artifactId>
<version>${pact.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>au.com.dius.pact.provider</groupId>
<artifactId>junit5spring</artifactId>
<version>${pact.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<plugins>
<plugin>
<groupId>au.com.dius.pact.provider</groupId>
<artifactId>maven</artifactId>
<version>${pact.version}</version>
</plugin>
</plugins>
{code}
The variable {{pact.version}} changed from 4.2.14 to 4.3.11
was:
We recently run into a build issue after a library update. While local builds would still work as expected, builds on our build machine would cause reproducible failures. The actual failure happens during site-reports are generated in the surefire report plugin but I believe the actual place is just a coincidence.
Unfortunately I wasn't able to find the trigger for the changed behavior. All we did was update the version of pact-jvm in the pom. Before the change no failure after the change every build fails. As this is proprietary code base I unfortunately currently cannot provide a sample project. However there is a couple of information I was able to find:
As there is little related logging even on debug level I had a look at the code. Using the stacktrace from the Jenkins log this pointed [here|https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/project/artifact/DefaultProjectArtifactsCache.java#L207]. Complaining about a re-caching of the project artifacts.
Looking at the calling code [here |https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/LifecycleDependencyResolver.java#L147] this doesn't seems to be possible. As the Cache-put is only called after the Cache-get returns {{{}null{}}}. This would only be the case if the cache was behaving incorrect (it's backed by a Java Concurrent HashMap, so pretty unlikely), or the key is not behaving as expected.
Looking at the [CacheKey|https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/project/artifact/DefaultProjectArtifactsCache.java#L58] reveals that this key in fact has been crafted to explicitly make it behave nicely. Most fields are immutable. However there is a small set of (potentially) mutable fields that would alter hashcode and equals result. If e.g. the RemoteRepositories stored in the {{repositories}} property would be altered this would change hashcode and equals. While the property is {{final}} and the constructor assures that it contains a fresh ArrayList, this is neither unmodifiable, nor are the repositories added to the list immutable.
To be sure I created a java-agent to monitor the behavior. I logged the key for both {{get}} and {{put}} and actually found different URLs for the repositories. Apparently the Jenkins-Artifactory-Plugin had changed them. And actually looking at the debug log there are log lines saying _Replacing resolution repository URL..._.
While this seems to happen very rarely it happens and if it does it's not fun debugging. I still don't know why it only happens rarely as I assume the plugin does rewrite the URLs always. But maybe there is only rare cases where the cache actually has data.
> Maven artifact caching breaks in context of Jenkins Artifactory Plugin
> ----------------------------------------------------------------------
>
> Key: MNG-7376
> URL: https://issues.apache.org/jira/browse/MNG-7376
> Project: Maven
> Issue Type: Bug
> Components: Core
> Affects Versions: 3.8.4
> Environment: Jenkins, Docker image based on upstream maven:3.8 image, JFrog Artifactory as artifact proxy
> Reporter: Christian Frommeyer
> Priority: Major
>
> We recently run into a build issue after a library update. While local builds would still work as expected, builds on our build machine would cause reproducible failures. The actual failure happens during site-reports are generated in the surefire report plugin but I believe the actual place is just a coincidence.
> Unfortunately I wasn't able to find the trigger for the changed behavior. All we did was update the version of pact-jvm in the pom. Before the change no failure after the change every build fails. As this is proprietary code base I unfortunately currently cannot provide a sample project. However there is a couple of information I was able to find:
> As there is little related logging even on debug level I had a look at the code. Using the stacktrace from the Jenkins log this pointed [here|https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/project/artifact/DefaultProjectArtifactsCache.java#L207]. Complaining about a re-caching of the project artifacts.
> Looking at the calling code [here |https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/LifecycleDependencyResolver.java#L147] this doesn't seems to be possible. As the Cache-put is only called after the Cache-get returns {{{}null{}}}. This would only be the case if the cache was behaving incorrect (it's backed by a Java Concurrent HashMap, so pretty unlikely), or the key is not behaving as expected.
> Looking at the [CacheKey|https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/project/artifact/DefaultProjectArtifactsCache.java#L58] reveals that this key in fact has been crafted to explicitly make it behave nicely. Most fields are immutable. However there is a small set of (potentially) mutable fields that would alter hashcode and equals result. If e.g. the RemoteRepositories stored in the {{repositories}} property would be altered this would change hashcode and equals. While the property is {{final}} and the constructor assures that it contains a fresh ArrayList, this is neither unmodifiable, nor are the repositories added to the list immutable.
> To be sure I created a java-agent to monitor the behavior. I logged the key for both {{get}} and {{put}} and actually found different URLs for the repositories. Apparently the Jenkins-Artifactory-Plugin had changed them. And actually looking at the debug log there are log lines saying _Replacing resolution repository URL..._.
> While this seems to happen very rarely it happens and if it does it's not fun debugging. I still don't know why it only happens rarely as I assume the plugin does rewrite the URLs always. But maybe there is only rare cases where the cache actually has data.
> This are the updated artifacts:
> {code:xml}
> <dependencies>
> <dependency>
> <groupId>au.com.dius.pact.consumer</groupId>
> <artifactId>junit5</artifactId>
> <version>${pact.version}</version>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>au.com.dius.pact.provider</groupId>
> <artifactId>junit5spring</artifactId>
> <version>${pact.version}</version>
> <scope>test</scope>
> </dependency>
> </dependencies>
> <plugins>
> <plugin>
> <groupId>au.com.dius.pact.provider</groupId>
> <artifactId>maven</artifactId>
> <version>${pact.version}</version>
> </plugin>
> </plugins>
> {code}
> The variable {{pact.version}} changed from 4.2.14 to 4.3.11
--
This message was sent by Atlassian Jira
(v8.20.10#820010)