You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@netbeans.apache.org by GitBox <gi...@apache.org> on 2020/07/13 05:49:02 UTC
[GitHub] [netbeans] jlahoda opened a new pull request #2252: Speeding up build, fixing some dependencies as needed.
jlahoda opened a new pull request #2252:
URL: https://github.com/apache/netbeans/pull/2252
Currently, even incremental build is fairly slow. This is an attempt to improve the speed of the build - it reduces the build time by about 4 minutes on my computer, from ~16mins to ~12mins for a full build and ~10mins to 6mins for an incremental build (which is still a lot, of course).
The basic idea is simple: currently, when doing a build, we build every cluster separately in an ant instance. And every module has (generated) target like "all-java.source.base" depends="all-openide.util.lookup,...". Note that the module depends directly on modules from different clusters. So, when building the "java" cluster, we also do a build of "all-openide.util.lookup", even if we have already built the "platform" cluster. This, of course, does only a timestamp check, but takes some time anyway. And we do this for every cluster, and many more modules, accumulating to a significant chunk of time.
The proposed solution is to generate the dependencies so that "all-<module>" depends on a target that ensures all needed *clusters* are built, and on all-<dependent-module> targets for modules from the current cluster. This avoids the unnecessary timestamp checks and improves build time. The code was actually expected to mostly work like this, but there was a small coding mistake, which prevented it from doing that.
Sadly, the current repository contains several cases where modules have dependencies on modules from other clusters, but there is no sensible dependency between the clusters. Namely:
-extide/gradle depends on java/java.platform, but "java" depends on "extide", so this is a dependency in the wrong direction. I have moved gradle to the java cluster for the purpose of this patch.
-java/debugger.jpda.trufflenode depends on webcommon/javascript.nodejs, but the "java" cluster does not depend on the "webcommon" cluster. I made "java" cluster depend on "webcommon" for the purpose of this patch.
-nb/ko4j.debugging depends on java/maven, but the "nb" cluster does not (and should not) depend on the "java" cluster. I've moved the ko4j.debugging module into the "java" cluster for the purposed of this patch.
I'd be happy to resolve the issues in other ways, as desired, but it would be awesome if we could speed up the build and make our dependencies cleaner!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lkishalmi commented on pull request #2252: Speeding up build, fixing some dependencies as needed.
Posted by GitBox <gi...@apache.org>.
lkishalmi commented on pull request #2252:
URL: https://github.com/apache/netbeans/pull/2252#issuecomment-657630312
Good catch. Thanks. Maybe I should remove the java.platform dependency from the Gradle module instead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] jlahoda commented on pull request #2252: Speeding up build, fixing some dependencies as needed.
Posted by GitBox <gi...@apache.org>.
jlahoda commented on pull request #2252:
URL: https://github.com/apache/netbeans/pull/2252#issuecomment-657390076
Regarding gradle - the gradle module depends on "java.platform" from the "java" cluster. Normally java cluster depends on extide, but here is a module from extide depending on a module from java - that is a dependency in the wrong direction. Moving gradle is probably not the only solution, so if you have a better one, great!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lkishalmi commented on pull request #2252: Speeding up build, fixing some dependencies as needed.
Posted by GitBox <gi...@apache.org>.
lkishalmi commented on pull request #2252:
URL: https://github.com/apache/netbeans/pull/2252#issuecomment-657388073
One Question. Why the gradle and libs.gradle ended up in the java cluster?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lkishalmi commented on pull request #2252: Speeding up build, fixing some dependencies as needed.
Posted by GitBox <gi...@apache.org>.
lkishalmi commented on pull request #2252:
URL: https://github.com/apache/netbeans/pull/2252#issuecomment-659026727
Gradle project is now independent from Java Platform.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] neilcsmith-net commented on pull request #2252: Speeding up build, fixing some dependencies as needed.
Posted by GitBox <gi...@apache.org>.
neilcsmith-net commented on pull request #2252:
URL: https://github.com/apache/netbeans/pull/2252#issuecomment-660920967
Should we merge this now?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] JaroslavTulach merged pull request #2252: Speeding up build, fixing some dependencies as needed.
Posted by GitBox <gi...@apache.org>.
JaroslavTulach merged pull request #2252:
URL: https://github.com/apache/netbeans/pull/2252
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] JaroslavTulach commented on a change in pull request #2252: Speeding up build, fixing some dependencies as needed.
Posted by GitBox <gi...@apache.org>.
JaroslavTulach commented on a change in pull request #2252:
URL: https://github.com/apache/netbeans/pull/2252#discussion_r453451304
##########
File path: nbbuild/antsrc/org/netbeans/nbbuild/InsertModuleAllTargets.java
##########
@@ -136,8 +136,29 @@ public void setUseClusters(boolean b) {
continue;
}
String[] prereqsAsCnb = entry.getBuildPrerequisites();
- StringBuffer namedDeps = new StringBuffer("init");
- String myCluster = clustersOfModules.get(path);
+ StringBuilder namedDeps = new StringBuilder("init");
+ String myCluster = clustersOfModules.get(entry.getNetbeansOrgId());
+ if (myCluster != null) {
+ String clusterDep = "all-cluster-" + myCluster;
+ if (!prj.getTargets().keySet().contains(clusterDep)) {
Review comment:
I see, this is the fix then.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists