[GitHub] [maven-resolver] caiwei-ebay commented on a diff in pull request #178: [MRESOLVER-7] download poms in parallel

[GitBox <gi...@apache.org>]

caiwei-ebay commented on code in PR #178:
URL: https://github.com/apache/maven-resolver/pull/178#discussion_r941047977


##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/collect/bf/BfDependencyCollector.java:
##########
@@ -365,6 +477,64 @@ else if ( descriptorResult == DataPool.NO_DESCRIPTOR )
         return descriptorResult;
     }
 
+    static class ParallelDescriptorResolver
+    {
+        final ExecutorService executorService;

Review Comment:
   Uploaded a diagram to explain how the dependency resolution works.
   ![here](https://issues.apache.org/jira/secure/attachment/13047919/resolve_deps.png)
   
   In a Maven build:
   - DependencyCollector focus on resolve dependency graph by downloading and parsing metadata.xml and pom.xmls which is running in sequence as of now.
   
   - MojoExecutor calls ensureDependencyResolved which download jars in parallel based on the dependency graph calculated by DependencyCollector.
   
   Using the former (fully unitize get <List, List>) in DependencyCollector seems not possible,  I think we need to use a separate thread pool for dependency resolution in DependencyCollector which includes 2 steps:
   
   - Resolve version range (which resort to VersionRangeResolver -> MetadataResolver -> RepositoryConnector)
   - Resolve descriptor (download or pick local poms and then parse poms, which resort to  ArtifactDescriptorReader -> ArtifactResolver -> RepositoryConnector)
   
   By running the above 2 steps in a separate thread pool can achieve better parallelism.
   Like the MetadataResolver, it is also using a separate ThreadPool to download metadata.xml one by one from multiple repositories.
   
   May I know any concerns that we should not use another thread pool for dependency resolution here?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org