[jira] [Commented] (GROOVY-12402) Local `gradlew test` oversubscribes small machines

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/GROOVY-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18114691#comment-18114691 ] 

ASF GitHub Bot commented on GROOVY-12402:
-----------------------------------------

daniellansun opened a new pull request, #2925:
URL: https://github.com/apache/groovy/pull/2925

   https://issues.apache.org/jira/browse/GROOVY-12402




> Local `gradlew test` oversubscribes small machines
> --------------------------------------------------
>
>                 Key: GROOVY-12402
>                 URL: https://issues.apache.org/jira/browse/GROOVY-12402
>             Project: Groovy
>          Issue Type: Improvement
>            Reporter: Daniel Sun
>            Priority: Major
>
> h3. Problem
> On a 6-CPU developer box, {{./gradlew test}} runs two Test tasks with {{maxParallelForks = nproc/2}}, so six worker JVMs. Each is launched at {{-Xmx2g}} with no {{-XX:ActiveProcessorCount}}, and G1/C2 then size themselves for *all* six CPUs ({{ParallelGCThreads=6}}, {{CICompilerCount=3}} per JVM). Load averages of 16-21 make a full suite take roughly twice as long as a less contended run.
> Three modules ({{performance}}, {{tests-preview}}, {{groovy-osgi-test}}) do not apply {{org.apache.groovy-tested}}, so they sit outside {{ConcurrentExecutionControlBuildService}} and inherit the ergonomic heap (about one quarter of RAM). The indy parity tests then fork child JVMs with the same default.
> * 6 vCPU KVM guest, 23 GiB RAM, no swap
> * JDK 25 (Corretto 25.0.4), Gradle 9.7.1
> * {{master}} @ {{b142a89351}}, development environment ({{CI}} unset)
> h3. Expected
> Local test workers should not each assume they own every CPU. CI must stay conservative: {{maxParallelForks=1}}, two concurrent Test tasks, two groovyc forks.
> h3. Before
> {code}
> ./gradlew cleanTest test --no-build-cache
> {code}
> * Peak 6 Test Executor processes; {{-Xms512m -Xmx2g}}; no APC
> * Full suite: 25m 41s; loadavg mean 15.0, max 21.7
> * Five-module subset ({{performance}}, {{groovy-json}}, {{groovy-xml}}, {{groovy-templates}}, {{groovy-sql}}): 4m 11s; loadavg max 18.1
> * 23731 passed, 131 skipped, 0 failed
> h3. Approach
> Keep the CI caps. Locally, size forks from the machine and tell each worker how many CPUs it may use:
> * {{maxParallelForks = max(1, (nproc-2)/2)}} (2 on six CPUs)
> * {{-XX:ActiveProcessorCount = floor(nproc / peakWorkers)}} (1 on six CPUs)
> * {{groovyJUnit_ms=256m}} ({{groovyJUnit_mx}} stays 2g)
> * Enroll {{performance}}, {{tests-preview}}, and {{groovy-osgi-test}} in the existing Test limiter; cap their heaps at 512m; pin parity-test children to {{-Xmx256m -XX:ActiveProcessorCount=1}}
> Knobs live on {{SharedConfiguration}} so Test and groovyc share one formula.
> h3. After
> Same command, same revision, same 23731 passed / 131 skipped:
> ||Run||Before||After||
> |full cleanTest test --no-build-cache|25m 41s|22m 59s|
> |five-module subset|4m 11s|2m 52s|
> |peak Test JVMs|6|4|
> |loadavg mean (full)|15.0|8.3|
> |loadavg max (full)|21.7|15.1|



--
This message was sent by Atlassian Jira
(v8.20.10#820010)