[GitHub] [james-project] quantranhong1999 commented on pull request #1077: [BUILD] Try to fasten the build on CI

[GitBox <gi...@apache.org>]

quantranhong1999 commented on PR #1077:
URL: https://github.com/apache/james-project/pull/1077#issuecomment-1200667675

   > Can you please summarize your appoach (parallel execution of maven projects instead of relying on surefire fork counts, if I followed well) and its achievment both in term of build time and stability.
   
   My idea is to increase the parallelism by adding parallel executions in Maven parallel builds (this works with surefire fork counts, not replace it). 
   Maven surefire plugin doc (https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html#forked-test-execution):
   ```
   You can use the place holder ${surefire.forkNumber} within argLine, or within the system properties (both those specified via mvn test -D... and via systemPropertyVariables). Before executing the tests, the surefire plugin replaces that place holder by the number of the actually executing process, counting from 1 to the effective value of forkCount times the maximum number of parallel executions in Maven parallel builds, i.e. the effective value of the -T command line argument of Maven core.
   ```
   
   So this idea will actually create more forks.
   E.g before: forkCount = 1 => 1 fork
         after: mvn test -T 8 ... with fork count = 1 => 8 forks.
   
   
   Achievement until now:
   - Build time: decrease much (stable test before 4h -> after ~ 1.5h)
   - Stability: sadly decreased. It creates more unstable tests, especially some docker tests (LDAP as I observed).
   
   Our current test suite is likely not sustainable that much concurrency. I did try to reduce concurrency by using fewer CI worker CPU cores but still did not archive a green build. 
   Specifying `threadCount = 1` on docker tests module may help, but I am not sure it will works.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org