[GitHub] timboudreau opened a new pull request #537: Maven Indexing Optimizations

[GitBox <gi...@apache.org>]

timboudreau opened a new pull request #537: Maven Indexing Optimizations
URL: https://github.com/apache/incubator-netbeans/pull/537
 
 
   A set of optimizations which improve performance and reduce memory consumption of local maven indexing.
   
   Comparing performance before and after, gets the following numbers, indexing my local Maven repository (~2700 JAR files):
   ```
   Unoptimized:  48.489 seconds
   Optimized:    35.559 seconds
   ```
   The more important difference is likely to be in memory consumption and its effect on the runtime, since this is background activity.  With the old and new code patched to frequently record `Math.min(Runtime.getRuntime().getFreeMemory(), previousValue)`, and identical starting conditions of available memory of 245844424:
   ```
   Optimized - minimum free memory during local repository indexing   52679616
   Unoptimized - minimum free memory during local repository indexing  5351880
   ```
   That's a 10x reduction in the memory required for indexing, and GC thrashing is likely a significant part of indexing overhead.
   
   ### Specific Optimizations
   
    * Process class files incrementally as the JAR file is read, instead of caching each class file as a separate byte array and processing them later (the old code required 2-3x the uncompressed bytes of every class file in a JAR to be on the heap simultaneously)
    * Optimized directory scanning which only picks up files that are useful to index
    * Replaced several (unnecessary? tests pass without them) uses of TreeSet with HashSet
    * Reuse byte arrays for class data, rather than allocating and garbage collecting repeatedly
    * A few other minor optimizations

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists