[jira] [Comment Edited] (MRESOLVER-7) Download dependency POMs in parallel

["Palash Das (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MRESOLVER-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273947#comment-17273947 ] 

Palash Das edited comment on MRESOLVER-7 at 1/28/21, 7:24 PM:
--------------------------------------------------------------

You people keep fighting over decency of a PR or deadlocks of multi threaded downloads, but in reality this forces us to pay free money to the cloud providers for CI time. 
When we have 

 # sufficient network bandwidth
 #  High speed SSD
 # And good amount of cpu

 

*we still have to pay 3 hrs of CI time just to download 3.4GBs of jar files.*
Great collaboration skills guys. Great appreciations towards the OSS efforts that people show on their PRs. Approve the PR, merge it and revert it within 1 month. Awesome reward received. 

Time will come when we will switch the build tool instead of a dependency plugin. Cheers to gradle / bazel. Those comment fights will be remembered!


was (Author: ppanda-beta):
You people keep fighting over decency of a PR or deadlocks of multi threaded downloads, but in reality this forces us to pay free money to the cloud providers for CI time. 
When we have 
 # sufficient network bandwidth

 #  

High speed SSD

 # And good amount of cpu

 

*we still have to pay 3 hrs of CI time just to download 3.4GBs of jar files.*
Great collaboration skills guys. Great appreciations towards the OSS efforts that people show on their PRs. Approve the PR, merge it and revert it within 1 month. Awesome reward received. 

Time will come when we will switch the build tool instead of a dependency plugin. Cheers to gradle / bazel. Those comment fights will be remembered!

> Download dependency POMs in parallel
> ------------------------------------
>
>                 Key: MRESOLVER-7
>                 URL: https://issues.apache.org/jira/browse/MRESOLVER-7
>             Project: Maven Resolver
>          Issue Type: Improvement
>    Affects Versions: Aether 1.0.2
>            Reporter: Harald Wellmann
>            Priority: Major
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> h3. Background
> When building a project with dependencies not yet available in the local repository, I noticed that Maven 3.3.9/Aether 1.0.2 first downloads the dependency POMs _sequentially_ and then proceeds downloading the dependency JARs with up to 5 threads _in parallel_.
> Due to this, when first building a project with a large number of dependencies, downloading a large number of small POMs may take a lot longer than downloading the much larger JARs, or even longer than building the project itself, especially when a repository manager is used which increases the download latency.
> h3. Enhancement
> Download POMs of (transitive) dependencies in parallel to significantly speed up initial builds of large projects.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)