[jira] [Commented] (MBUILDCACHE-27) Improve efficiency of cache with plugin and source type aware processing

["Maximilian Novikov (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MBUILDCACHE-27?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633979#comment-17633979 ] 

Maximilian Novikov commented on MBUILDCACHE-27:
-----------------------------------------------

>Additional challenge is to rerun-tests-only mode - it obviously relies on compiled project, but probably that could be worked around by restoring cached jar

agree, that's the risky part, and you might have to restore whole target folder

> Improve efficiency of cache with plugin and source type aware processing
> ------------------------------------------------------------------------
>
>                 Key: MBUILDCACHE-27
>                 URL: https://issues.apache.org/jira/browse/MBUILDCACHE-27
>             Project: Maven Build Cache Extension
>          Issue Type: Improvement
>            Reporter: Alexander Ashitkin
>            Priority: Major
>
> Currently build cache has implementation which checks source code and plugins and if it detects mismatch, whole project is rerun. This is is redundant and could be optimized in following cases:
>  * Test code changed. In that case optimal behavior is to rerun test in an affected module and reuse cached artifacts in case of success. Actual behavior: change in a test in upstream module invalidates all downstream projects
>  * Variation of the situation above is a test library project (aka test-jar) - though by convention modification is in production code, because the project is used only in test scope - modification of upstream test jar should cause just tests rerun 
>  * Upstream dependency changed. In that case any plugin which execution is not affected by upstream dependency, could be skip. Typically, it could be any of linter plugins - like checkstyle, which checks project source code only. Current behavior: plugin will rerun
> With such granular plugin and sources demarcation, it is possible to significantly optimize cached build time.
> To address this problem need following changes:
>  * changes in build configuration to demarcate plugins as private (no need rebuild on dependencies change)
>  * split checksum into at least 3 parts - test sources checksum, dependencies checksum and  production checksum
>  * Take into account changes in every checksum when decision of plugin execution is taken. If only dependencies changed, `source-code-only` plugins could be skipped
>  * When calculating dependencies between reactor projects, take into account "production" checksum only. In that case changes in test code will not affect downstream projects
> Additional challenge is to rerun-tests-only mode - it obviously relies on compiled project, but probably that could be worked around by restoring cached jar
> Summarizing the above and applying ideas of ABI to maven build, need to distinguish 3 categories of project inputs:
>  * implementation sources - part of the code resembling functionality shared within reactor (src/main and similar)
>  * private sources - part which is not shared (unit test and similar)
>  * dependencies - they change functionality but do not change code
> Having this 3 categories correctly represented by a separate checksums, it is possibly to implement optimization mentioned above.
> In target state, following behavior is desirable:
>  
> ||Change||Changed project||Downstream Project||
> |Test sources|rerun tests + later plugins|Reuse cache|
> |Test dependency |rerun tests + later plugins except "source-only" plugins|Reuse cache|
> |Test library project|rerun everything|rerun tests|
> |Prod sources|rerun everything|Rerun everything except "source only plugins"|
> |Prod dependency|rerun everything except "source-only" plugins|Rerun everything except "source only plugins"|
> This behavior could be made opt-in
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)