[GitHub] [spark] ulysses-you commented on a diff in pull request #38779: [SPARK-41244][UI] Introducing a Protobuf serializer for UI data on KV store

[GitBox <gi...@apache.org>]

ulysses-you commented on code in PR #38779:
URL: https://github.com/apache/spark/pull/38779#discussion_r1044175147


##########
core/pom.xml:
##########
@@ -616,6 +621,48 @@
           </execution>
         </executions>
       </plugin>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-shade-plugin</artifactId>

Review Comment:
   @gengliangwang  It actually conflicts with parent pom.xml which causes the jetty can not be shaded into spark-core. The current master branch can not start Spark UI, see error message:
   
   ```
   Setting default log level to "WARN".
   To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
   22/12/09 14:58:54 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   java.lang.NoClassDefFoundError: org/eclipse/jetty/util/thread/Scheduler
     at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:67)
     at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:223)
     at org.apache.spark.SparkContext.<init>(SparkContext.scala:484)
     at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2739)
     at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:978)
     at scala.Option.getOrElse(Option.scala:189)
     at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:972)
     at org.apache.spark.repl.Main$.createSparkSession(Main.scala:106)
     ... 55 elided
   Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.util.thread.Scheduler
     at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
     ... 63 more
   <console>:14: error: not found: value spark
          import spark.implicits._
                 ^
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org