You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Alexander Ashitkin (Jira)" <ji...@apache.org> on 2022/10/26 00:14:00 UTC
[jira] [Updated] (MBUILDCACHE-27) Improve efficiency of cache with plugin and source type aware processing
[ https://issues.apache.org/jira/browse/MBUILDCACHE-27?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Ashitkin updated MBUILDCACHE-27:
------------------------------------------
Description:
Currently build cache has implementation which checks source code and plugins and if it detects mismatch, whole project is rerun. This is is redundant and could be optimized in following cases:
* Test code changed. In that case optimal behavior is to rerun test in an affected module and reuse cached artifacts in case of success. Actual behavior: change in a test in upstream module invalidates all downstream projects
* Upstream dependency changed. In that case any plugin which execution is not affected by upstream dependency, could be skip. Typically, it could be any of linter plugins - like checkstyle, which checks project source code only. Current behavior: plugin will rerun
With granular plugin and sources demarcation, it is possible to significantly optimize cached build time.
To address this problem need following changes:
* changes in build configuration to demarcate plugins as private (no need rebuild on dependencies change)
* split checksum into at least 3 parts - test sources checksum, dependencies checksum and production checksum
* Take into account changes in every checksum when decision of plugin execution is taken. If only dependencies changed, `source-code-only` plugins could be skipped
* When calculating dependencies between reactor projects, take into account "production" checksum only. In that case changes in test code will not affect downstream projects
This behavior could be made opt-in
was:
Currently build cache has implementation which checks source code and plugins and if it detects mismatch, whole project is rerun. This is is redundant and could be optimized in following cases:
* Test code changed. In that case optimal behavior is to rerun test in an affected module and reuse cached artifacts in case of success. Actual behavior: change in a test in upstream module invalidates all downstream projects
* Upstream dependency changed. In that case any plugin which execution is not affected by upstream dependency, could be skip. Typically, it could be any of linter plugins - like checkstyle, which checks project source code only. Current behavior: plugin will rerun
With granular plugin and sources demarcation, it is possible to significantly optimize cached build time.
To address this problem need following changes:
* changes in build configuration to demarcate plugins as private (no need rebuild on dependencies change)
* split checksum into at least 3 parts - test sources checksum, dependencies checksum and production checksum
* Take into account changes in every checksum when decision of plugin execution is taken. If only dependencies changed, `source-code-only` plugins could be skipped
* When calculating dependencies between reactor projects, take into account "production" checksum only. In that case changes in test code will not affect downstream projects
> Improve efficiency of cache with plugin and source type aware processing
> ------------------------------------------------------------------------
>
> Key: MBUILDCACHE-27
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-27
> Project: Maven Build Cache Extension
> Issue Type: Improvement
> Reporter: Alexander Ashitkin
> Priority: Major
>
> Currently build cache has implementation which checks source code and plugins and if it detects mismatch, whole project is rerun. This is is redundant and could be optimized in following cases:
> * Test code changed. In that case optimal behavior is to rerun test in an affected module and reuse cached artifacts in case of success. Actual behavior: change in a test in upstream module invalidates all downstream projects
> * Upstream dependency changed. In that case any plugin which execution is not affected by upstream dependency, could be skip. Typically, it could be any of linter plugins - like checkstyle, which checks project source code only. Current behavior: plugin will rerun
> With granular plugin and sources demarcation, it is possible to significantly optimize cached build time.
> To address this problem need following changes:
> * changes in build configuration to demarcate plugins as private (no need rebuild on dependencies change)
> * split checksum into at least 3 parts - test sources checksum, dependencies checksum and production checksum
> * Take into account changes in every checksum when decision of plugin execution is taken. If only dependencies changed, `source-code-only` plugins could be skipped
> * When calculating dependencies between reactor projects, take into account "production" checksum only. In that case changes in test code will not affect downstream projects
>
> This behavior could be made opt-in
--
This message was sent by Atlassian Jira
(v8.20.10#820010)