[jira] [Commented] (LUCENE-10195) Gradle build speed improvement

["Jerome Prinet (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17434249#comment-17434249 ] 

Jerome Prinet commented on LUCENE-10195:
----------------------------------------

I mainly focussed on up-to-date checks and task caching, the most spectacular improvement happens when running a clean with a populated cache:

./gradlew clean build -Ptests.seed=deadbeef -Ptests.nightly=false -Ptests.neverUpToDate=false --scan

 

*Before changes*

[https://gradle.com/s/kxkbukaklyiz4]

1206 tasks executed in 40 projects *in 5m 39s, with 72 avoided tasks saving 16.359s*

*After changes*

[https://gradle.com/s/mfiwiheg4wxjq]

1206 tasks executed in 40 projects *in 26s, with 394 avoided tasks saving 30m 2.417s*

 

*Here is the detail of the changes:*
 * Declare outputs in your tasks to benefit from up-to-date checks (_CollectJarInfos_)
 * _validateSourcePatterns_ task should not take into account _.idea_, nor _.gradle_ files
 * Annotate tasks Cacheable to benefit from cache (_EcjLint, ValidateSourcePatterns, RatTask, RenderJavadoc, checkBrokenLinks_)
 * Use valid outputs rather than dummy ones (_EcjLint_)
 * Do not use string representation for task inputs being collection of files or directories (_ie. resources, scriptResources_) as you can’t benefit from caching when relocating workspace to a different folder
 * Minimize direct usage of system properties which are location or OS dependent, as they are part of the cache entry key
 * Do not set location or OS related information in the _MANIFEST.MF (X-Build-OS)_

 

*Here some advices for future improvements:*
 * Fixing _tests.seed_ obviously help to benefit from up-to-date check, I get the point about randomization, but this is a trade-off with expensive cost of resources
 * Use the standard _Gradle wrapper_
 * Setup local _gradle.properties_ in Gradle home folder rather than having an automatic generation from _gradle/generation/local-settings.gradle_
 * Add to VCS gradle _.properties & Gradle wrapper_
 * Do not override Gradle daemon TTL to 15mn, this is way to short
 * Do not create / commit generated test files to src directory (_frenchArticles.txt, Top50KWiki.utf8, CambridgeMA.utf8, Latin-dont-break-on-hyphens.rbbi_)

=> _:lucene:analysis:common:compileTestJava_ is not cacheable due to overlapping outputs

> Gradle build speed improvement
> ------------------------------
>
>                 Key: LUCENE-10195
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10195
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Jerome Prinet
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Increase Gradle build speed with help of Gradle built-in features, mostly cache and up-to-date checks
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org