[GitHub] [spark] shuwang21 commented on a diff in pull request #41821: [SPARK-44272][YARN] Path Inconsistency when Operating statCache within Yarn Client

["shuwang21 (via GitHub)" <gi...@apache.org>]

shuwang21 commented on code in PR #41821:
URL: https://github.com/apache/spark/pull/41821#discussion_r1252563553


##########
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientDistributedCacheManagerSuite.scala:
##########
@@ -45,6 +45,63 @@ class ClientDistributedCacheManagerSuite extends SparkFunSuite with MockitoSugar
     }
   }
 
+  /*
+  * This this a customized ClientDistributedCacheManager to test statCache.
+  * */
+  class CustomizedClientDistributedCacheManager extends ClientDistributedCacheManager {
+    override def addResource(fs: FileSystem, conf: Configuration, destPath: Path,
+      localResources: HashMap[String, LocalResource], resourceType: LocalResourceType,
+      link: String, statCache: Map[URI, FileStatus], appMasterOnly: Boolean): Unit = {
+      statCache.getOrElseUpdate(destPath.toUri(), fs.getFileStatus(destPath))
+    }
+
+    /*
+    * This simulates the isPublic method. It will return true if the result is cached in the
+    *  statCache.
+    * */
+    def isPublicResultCached(uri: URI, statCache: Map[URI, FileStatus]): Boolean = {
+      statCache.get(uri) match {
+        case Some(_) => true
+        case None => false
+      }

Review Comment:
   Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org