[jira] [Commented] (MNG-7509) Huge memory cost when parent pom widely used in a big project for dependencyManagement

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MNG-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573510#comment-17573510 ] 

ASF GitHub Bot commented on MNG-7509:
-------------------------------------

Yougoss commented on code in PR #768:
URL: https://github.com/apache/maven/pull/768#discussion_r934074703


##########
maven-resolver-provider/src/test/java/org/apache/maven/repository/internal/ArtifactDescriptorReaderDelegateTest.java:
##########
@@ -0,0 +1,79 @@
+package org.apache.maven.repository.internal;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.apache.maven.model.Dependency;
+import org.apache.maven.model.DependencyManagement;
+import org.apache.maven.model.Model;
+import org.eclipse.aether.artifact.DefaultArtifact;
+import org.eclipse.aether.resolution.ArtifactDescriptorRequest;
+import org.eclipse.aether.resolution.ArtifactDescriptorResult;
+
+public class ArtifactDescriptorReaderDelegateTest extends AbstractRepositoryTestCase{
+    public void testMNG7509()

Review Comment:
   Hi @michael-o , I'd like to double confirm to make sure my understanding is correct. Since I create this testcase refer to an existing one org.apache.maven.repository.internal.DefaultArtifactDescriptorReaderTest, I'm not sure whether the format doesn't follow your rules means the class and method name is not quite meaningful.
   If it is, I will change it. If not, please let me know what should I do.
   Thanks.





> Huge memory cost when parent pom widely used in a big project for dependencyManagement
> --------------------------------------------------------------------------------------
>
>                 Key: MNG-7509
>                 URL: https://issues.apache.org/jira/browse/MNG-7509
>             Project: Maven
>          Issue Type: Improvement
>          Components: Performance
>            Reporter: Xiong Luyao
>            Priority: Critical
>             Fix For: 3.9.0-candidate, 4.0.x-candidate
>
>         Attachments: image-2022-07-09-09-37-53-823.png, image-2022-07-09-09-38-26-354.png, image-2022-07-09-10-27-12-668.png, image-2022-07-09-10-27-56-437.png, image-2022-07-09-10-28-05-706.png, image-2022-07-09-10-28-22-864.png, image-2022-07-09-10-28-35-341.png, image-2022-07-09-10-28-40-612.png, image-2022-07-09-10-29-04-045.png, image-2022-07-09-10-29-15-822.png, image-2022-07-09-10-29-21-991.png, image-2022-07-09-10-29-46-216.png, image-2022-07-09-10-29-51-456.png
>
>
> When maven try to resolve dependency relationship, it will create many instances of dependency / artifact, even the dependency/artifact content is totally same, but just in different pom models. It cost huge memory if there is a parent pom with dependencyManagement which managed a lot of dependencies, and this parent pom are implemented by many project libraries.
> (libraries_count * managedDependency_count) dependency instances will be created. For example, if there are 3000 libraries, and all the library introduce same parent pom which managed 3000 dependencies version. There will be 3000*3000 =9,000,000 dependency instances will be created. But most of them are same, in fact, we only need one instance for each dependency in parent pom (3000 dependency instances).
>  
> I'm from eBay, and here is a real case in enterprise level project. We have about 3000 business domain libraries, there are dependency relationship between them. We need to build all libraries in one release to keep all the libraries in same release are based on same code. So we used a parent pom as a central management to manage all the version for a release, and introduced by those libraries.  As below picture, when the release start, it will calculate and start with the library which doesn't depend on others, then start the library which dependency libraries are already built. Keep this process until all libraries are built.
> With current maven resolve logic, it costs huge memory in above ways to built libraries. And even the libraries have been released, if the project which contains a lot of above libraries, it also cost huge memory when building project.
> So current now, we have to specify version in each library pom files instead of using parent pom. We think we can make some enhancement for this case.
>  
> !image-2022-07-09-09-37-53-823.png|width=493,height=226!
>  
> Here is a thread dump when building a real project which depends on about 1000 above libraries. The top 5 objects are all related to org.eclipse.aether.graph.Dependency.
> !image-2022-07-09-09-38-26-354.png|width=510,height=199!
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)