[jira] [Comment Edited] (MNG-5896) Download dependency POMs in parallel

["Jonathan Haber (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MNG-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682767#comment-17682767 ] 

Jonathan Haber edited comment on MNG-5896 at 1/31/23 9:29 PM:
--------------------------------------------------------------

Sorry for the lengthy delay, I put together a repository that I believe reproduces the issue:

[https://github.com/jhaber/maven-resolver-test]

There are some directions in the readme. I also uploaded the full Maven folder to the repo in case it's an issue with how I built Maven 3.9.0 from source.

On my laptop, Maven 3.8.7 can run dependency:resolve in ~16 seconds. Maven 3.9.0 (with maven-resolver 1.9.4) takes ~31 seconds to run the same command. Running Maven 3.9.0 with BF mode brings the performance back to ~16 seconds.

I haven't dug into the root cause, but I would expect Maven 3.9.0 in DF mode to take ~16 seconds (ie, same performance as Maven 3.8.7), and BF mode to improve upon that.


was (Author: jhaber):
Sorry for the lengthy delay, I put together a repository that I believe reproduces the issue:

[https://github.com/jhaber/maven-resolver-test]

There are some directions in the readme. I also uploaded the full Maven folder to the repo in case it's an issue with how I built Maven 3.9.0 from source.

On my laptop, Maven 3.8.7 can run dependency:resolve in ~16 seconds. Maven 3.9.0 (with maven-resolver 1.9.0) takes ~31 seconds to run the same command. Running Maven 3.9.0 with BF mode brings the performance back to ~16 seconds.

I haven't dug into the root cause, but I would expect Maven 3.9.0 in DF mode to take ~16 seconds, and BF mode to improve upon that.

> Download dependency POMs in parallel
> ------------------------------------
>
>                 Key: MNG-5896
>                 URL: https://issues.apache.org/jira/browse/MNG-5896
>             Project: Maven
>          Issue Type: Improvement
>          Components: Dependencies
>    Affects Versions: 3.3.3
>            Reporter: Harald Wellmann
>            Priority: Major
>
> h3. Background
> When building a project with dependencies not yet available in the local repository, I noticed that Maven 3.3.3 first downloads the dependency POMs _sequentially_ and then proceeds downloading the dependency JARs with up to 5 threads _in parallel_.
> Due to this, when first building a project with a large number of dependencies, downloading a large number of small POMs may take a lot longer than downloading the much larger JARs, or even longer than building the project itself, especially when a repository manager is used which increases the download latency.
> h3. Enhancement
> Download POMs of (transitive) dependencies in parallel to significantly speed up initial builds of large projects.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)