[jira] [Commented] (LUCENE-9861) speed up precommit

["Robert Muir (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306439#comment-17306439 ] 

Robert Muir commented on LUCENE-9861:
-------------------------------------

For renderJavadoc this is also 2x speedup:
Before:
{noformat}
Aggregate task times (possibly running in parallel!):
 357.87 sec.  renderJavadoc
 259.61 sec.  compileJava
   1.05 sec.  processResources
   0.28 sec.  clean
{noformat}

After:
{noformat}
Aggregate task times (possibly running in parallel!):
 182.77 sec.  renderJavadoc
 166.17 sec.  compileJava
   0.94 sec.  processResources
   0.16 sec.  clean
{noformat}

I haven't yet attacked {{javac}}, the reason you see tasks get faster there is just because C2 compiler threads from {{javadoc}} aren't eating up all my cpu and slowing them down.

> speed up precommit
> ------------------
>
>                 Key: LUCENE-9861
>                 URL: https://issues.apache.org/jira/browse/LUCENE-9861
>             Project: Lucene - Core
>          Issue Type: Task
>            Reporter: Robert Muir
>            Priority: Major
>
> A lot of the java tools for precommit aren't being called in efficient ways (compilation, linting, etc).
> For example ecjlint, it runs very slow:
> {noformat}
> Aggregate task times (possibly running in parallel!):
>  271.73 sec.  ecjLintMain
>  270.18 sec.  ecjLintTest
>  227.17 sec.  compileJava
>   12.07 sec.  compileTestJava
>    1.21 sec.  processResources
>    0.18 sec.  clean
> {noformat}
> Simplying adding a couple reasonable jvm arguments to the ecj linter ({{jvmArgs = [ '-XX:+UseParallelGC', '-XX:TieredStopAtLevel=1' ]}}) speeds it up significantly.
> Speedup for ecjLint is 3x for me:
> {noformat}
> Aggregate task times (possibly running in parallel!):
>  163.38 sec.  compileJava
>   84.57 sec.  ecjLintMain
>   83.12 sec.  ecjLintTest
>    6.11 sec.  compileTestJava
>    0.95 sec.  processResources
>    0.15 sec.  clean
> {noformat}
> I imagine same may be true for a lot of these tasks. We're currently tossing default jvm args at these short-lived subprocesses, which is very suboptimal.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org