You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Jonathan Haber (Jira)" <ji...@apache.org> on 2022/11/16 21:56:00 UTC
[jira] [Comment Edited] (MNG-5896) Download dependency POMs in parallel
[ https://issues.apache.org/jira/browse/MNG-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634949#comment-17634949 ]
Jonathan Haber edited comment on MNG-5896 at 11/16/22 9:55 PM:
---------------------------------------------------------------
On one of our medium-large projects (24 modules, ~10,000 total dependencies) we're seeing mvn dependency:resolve take 111 seconds using DF impl, and 15 seconds using BF impl. And this is with the offline option. When you actually need to download/update artifacts, that gap only widens.
No issues with correctness or stability reported yet (using BF + 10 threads)
EDIT: I thought that using the DF option would perform similarly to previous versions of Maven, and that this would be a mostly valid performance comparison. However, when I run the same command on the same repo using Maven 3.8.6, it also finishes in 15 seconds. Which is confusing, because I thought Maven 3.8.6 is effectively using the df strategy?
was (Author: jhaber):
On one of our medium-large projects (24 modules, ~10,000 total dependencies) we're seeing mvn dependency:resolve take 111 seconds using DF impl, and 15 seconds using BF impl. And this is with the offline option. When you actually need to download/update artifacts, that gap only widens.
No issues with correctness or stability reported yet (using BF + 10 threads)
EDIT: I thought that using the DF option would perform similarly to previous versions of Maven, and that this would be a mostly valid performance comparison. However, when I run the same command on the same repo using Maven 3.8.6, it finishes 15 seconds. Which is confusing, because I thought Maven 3.8.6 is effectively using the df strategy?
> Download dependency POMs in parallel
> ------------------------------------
>
> Key: MNG-5896
> URL: https://issues.apache.org/jira/browse/MNG-5896
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
> Affects Versions: 3.3.3
> Reporter: Harald Wellmann
> Priority: Major
>
> h3. Background
> When building a project with dependencies not yet available in the local repository, I noticed that Maven 3.3.3 first downloads the dependency POMs _sequentially_ and then proceeds downloading the dependency JARs with up to 5 threads _in parallel_.
> Due to this, when first building a project with a large number of dependencies, downloading a large number of small POMs may take a lot longer than downloading the much larger JARs, or even longer than building the project itself, especially when a repository manager is used which increases the download latency.
> h3. Enhancement
> Download POMs of (transitive) dependencies in parallel to significantly speed up initial builds of large projects.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)