[GitHub] [spark] vicennial commented on a diff in pull request #42069: [WIP] Fix class loading problem caused by stub user classes not found on the server classpath

["vicennial (via GitHub)" <gi...@apache.org>]

vicennial commented on code in PR #42069:
URL: https://github.com/apache/spark/pull/42069#discussion_r1267720733


##########
core/src/main/scala/org/apache/spark/executor/Executor.scala:
##########
@@ -1102,12 +1117,19 @@ private[spark] class Executor(
           val url = new File(root, localName).toURI.toURL
           if (!state.urlClassLoader.getURLs().contains(url)) {
             logInfo(s"Adding $url to class loader")
-            state.urlClassLoader.addURL(url)
+            // TODO: make use of the session cache for the class loader.
+            // Currently we invalidate all when adding a new url to always clear the stubbed
+            // classes in the current repl class loader.
+            // This is not always needed if the newly added jar does not contains any stubbed
+            // classes.
+            isolatedSessionCache.invalidateAll()

Review Comment:
   Shouldn't we only invalidate the `isolatedSession` that is being updated? Each session has its own individual `URLClassLoader`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org