[GitHub] [spark] LuciferYang commented on a change in pull request #34368: [WIP][SPARK-37072][CORE][TEST] Pass all UTs in `repl` with Java 17

[GitBox <gi...@apache.org>]

LuciferYang commented on a change in pull request #34368:
URL: https://github.com/apache/spark/pull/34368#discussion_r734931107



##########
File path: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala
##########
@@ -407,6 +417,24 @@ private[spark] object ClosureCleaner extends Logging {
     }
   }
 
+  /**
+   * This method is used to get the final modifier field when use Java 17.
+   */
+  private def getFinalModifiersFieldForJava17(field: Field): Option[Field] = {
+    if (SystemUtils.isJavaVersionAtLeast(JavaVersion.JAVA_17) &&
+      Modifier.isFinal(field.getModifiers)) {
+      val methodGetDeclaredFields0 = classOf[Class[_]]
+        .getDeclaredMethod("getDeclaredFields0", classOf[Boolean])
+      methodGetDeclaredFields0.setAccessible(true)
+      val fields = methodGetDeclaredFields0.invoke(classOf[Field], false.asInstanceOf[Object])
+        .asInstanceOf[Array[Field]]
+      val modifiersFieldOption = fields.find(field => "modifiers".equals(field.getName))
+      assert(modifiersFieldOption.isDefined)

Review comment:
       ok， will fix it later

##########
File path: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala
##########
@@ -394,8 +395,17 @@ private[spark] object ClosureCleaner extends Logging {
             parent = null, outerThis, capturingClass, accessedFields)
 
           val outerField = func.getClass.getDeclaredField("arg$1")
+          // SPARK-37072: When Java 17 is used and `outerField` is read-only,
+          // the content of `outerField` cannot be set by reflect api directly.
+          // But We can remove the `final` modifier of `outerField` before set value

Review comment:
       I don't know what the result will be, but I can do an experiment later
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org