[GitHub] [calcite] akshayabd commented on pull request #3272: [CALCITE-5785] Update fmppMain tasks to use relative paths for input hashes so they work with remote build caching

["akshayabd (via GitHub)" <gi...@apache.org>]

akshayabd commented on PR #3272:
URL: https://github.com/apache/calcite/pull/3272#issuecomment-1595940325

   Currently you can reproduce this issue by doing the following:
   1. Clone the project in 2 different directories
   2. Disable local caching for both clones and ensure remote caching is enabled
   3. Execute a build in the first clone
   4. Execute a build in the second clone at the same commit
   
   You will notice that most tasks that are cacheable are not executed in the 2nd build, the outputs are fetched from the remote cache. However the fmppMain tasks, though they are marked as cacheable, will re-execute and the outputs are not fetched from the remote cache.
   
   This is because the input cache key uses the absolute paths of the codegen directory, which will be different for the 2 clones. The cache key should use the relative paths. You can read more about this below in the Gradle docs:
   
   https://docs.gradle.org/current/userguide/build_cache.html#header
   "Wrongly declared task inputs can lead to cache misses especially when containing volatile data or absolute paths."
   
   And here:
   https://docs.gradle.org/current/userguide/build_cache.html#using_annotations
   "However, input files are identified by default by their absolute path. So if the cache needs to be shared between several developers or machines using different paths, that won’t work as expected. So we also need to set the [path sensitivity](https://docs.gradle.org/current/userguide/build_cache_concepts.html#relocatability). In this case, the relative path of the input files can be used to identify them."
   
   I've also attached a couple of screenshots from build scans in the JIRA issue that show the input cache key differences:
   https://issues.apache.org/jira/browse/CALCITE-5785
   
   --
   
   At the end of the day this PR impacts build performance only. The fmppMain tasks will not be executed if the outputs can be gotten from the cache - which will now correctly happen because of how the input cache key is computed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org