[GitHub] [spark] ben-manes commented on a change in pull request #31517: [WIP][SPARK-34309][BUILD][CORE][SQL] [K8S]Use Caffeine instead of Guava Cache

[GitBox <gi...@apache.org>]

ben-manes commented on a change in pull request #31517:
URL: https://github.com/apache/spark/pull/31517#discussion_r599720425



##########
File path: core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala
##########
@@ -58,24 +58,26 @@ private[history] class ApplicationCache(
 
   }
 
-  private val removalListener = new RemovalListener[CacheKey, CacheEntry] {
+  private val cacheWriter = new CacheWriter[CacheKey, CacheEntry] {

Review comment:
       @LuciferYang This can be accomplished by using `cache.asMap().computeIfPresent` instead of `invalidate`. Then you may callback manually within the atomic operation and return `null` to remove the entry.
   
   `CacheWriter` tried to intercept but it was confusing by missing some cases. For example should a `invalidate(key)` call the writer if the entry is absent? It does not, but in a multi-layer cache one might expect to propagate the removal even if absent. These quirks led to replacing it with explicit callbacks via compute for manual operations and `evictionListener` for automatic ones. An invalidation is a manual operation, so it requires a map compute.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org