[jira] [Updated] (LUCENE-9861) speed up precommit

["Robert Muir (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir updated LUCENE-9861:
--------------------------------
    Attachment: LUCENE-9861_hack.patch
        Status: Open  (was: Open)

My current hack patch. Still want to investigate {{javac}} and any other bad guys that are slow during precommit.

Also I think we need not hardcode the args? I already have this stuff set correctly in {{org.gradle.jvmargs}}, should we just plumb those to these subprocesses?

It will be a little annoying for some things such as javadoc as we'll need to put {{-J}} in front of each arg. cc: [~dweiss]

> speed up precommit
> ------------------
>
>                 Key: LUCENE-9861
>                 URL: https://issues.apache.org/jira/browse/LUCENE-9861
>             Project: Lucene - Core
>          Issue Type: Task
>            Reporter: Robert Muir
>            Priority: Major
>         Attachments: LUCENE-9861_hack.patch
>
>
> A lot of the java tools for precommit aren't being called in efficient ways (compilation, linting, etc).
> For example ecjlint, it runs very slow:
> {noformat}
> Aggregate task times (possibly running in parallel!):
>  271.73 sec.  ecjLintMain
>  270.18 sec.  ecjLintTest
>  227.17 sec.  compileJava
>   12.07 sec.  compileTestJava
>    1.21 sec.  processResources
>    0.18 sec.  clean
> {noformat}
> Simplying adding a couple reasonable jvm arguments to the ecj linter ({{jvmArgs = [ '-XX:+UseParallelGC', '-XX:TieredStopAtLevel=1' ]}}) speeds it up significantly.
> Speedup for ecjLint is 3x for me:
> {noformat}
> Aggregate task times (possibly running in parallel!):
>  163.38 sec.  compileJava
>   84.57 sec.  ecjLintMain
>   83.12 sec.  ecjLintTest
>    6.11 sec.  compileTestJava
>    0.95 sec.  processResources
>    0.15 sec.  clean
> {noformat}
> I imagine same may be true for a lot of these tasks. We're currently tossing default jvm args at these short-lived subprocesses, which is very suboptimal.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org