You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@livy.apache.org by gg...@apache.org on 2026/10/20 16:53:20 UTC

(livy) branch master updated: [LIVY-1090] Upgrade SLF4J 1.7.36 -> 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl), replace jetty-runner

This is an automated email from the ASF dual-hosted git repository.

gyogal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/livy.git


The following commit(s) were added to refs/heads/master by this push:
     new bc234453 [LIVY-1090] Upgrade SLF4J 1.7.36 -> 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl), replace jetty-runner
bc234453 is described below

commit bc234453011e55138767f1b0ad3de9d4f77f93e8
Author: Gabor Roczei <19...@users.noreply.github.com>
AuthorDate: Tue Oct 20 18:53:15 2026 +0200

    [LIVY-1090] Upgrade SLF4J 1.7.36 -> 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl), replace jetty-runner
    
    ## What changes were proposed in this pull request?
    
    SLF4J 2.0.x replaced the 1.x StaticLoggerBinder lookup with a ServiceLoader-based
    SLF4JServiceProvider, so the Log4j 2 binding must be swapped from log4j-slf4j-impl
    (SLF4J 1.x) to log4j-slf4j2-impl (SLF4J 2.x).
    
    - pom.xml: slf4j.version 1.7.36 -> 2.0.17
    - Swap log4j-slf4j-impl -> log4j-slf4j2-impl in the binding declarations:
      root dependencyManagement, core, repl (provided), rsc (provided)
    - thriftserver/client: exclude log4j-slf4j-impl from hive-jdbc and hive-beeline
      (hive-jdbc:3.0.0 and hive-beeline -> hive-common still bring log4j-slf4j-impl:2.10.0)
      and declare log4j-slf4j2-impl explicitly
    - integration-test: additionally exclude log4j-slf4j2-impl from the test-scoped
      hive-jdbc (next to the existing log4j-slf4j-impl exclusion)
    - thriftserver/server: replace the jetty-runner uber-jar (which bundles a shaded
      SLF4J 1.x that shadowed slf4j-api 2.x and caused a "Failed to load class
      org.slf4j.impl.StaticLoggerBinder" NOP fallback at server startup) with the
      individual jetty-server, jetty-servlet and jetty-webapp modules, and drop the
      now-obsolete jetty-util/jetty-xml/jetty-webapp excludeArtifactIds that only
      existed to avoid duplicating jetty-runner's bundled copies. The existing
      log4j-slf4j-impl exclusion on hive-jdbc is kept.
    - Root pom: manage jetty-webapp, jetty-xml and jetty-client at ${jetty.version} in
      dependencyManagement so all Jetty modules resolve to the same version as
      livy-server, overriding Hive's older Jetty (LIVY-543). The jetty-webapp declaration
      in thriftserver/server now inherits its version from the root pom.
    
    No Log4j bump needed (already on 2.26.0, >= the 2.23.0 minimum for log4j-slf4j2-impl).
    Livy has no org.slf4j.impl / StaticLoggerBinder usage, so no source changes are required.
    
    JIRA: https://issues.apache.org/jira/browse/LIVY-1090
    
    ## How was this patch tested?
    
    Unit tests.
    - core/src/test/scala/org/apache/livy/LoggingSuite.scala: verifies a single
      SLF4JServiceProvider is registered and resolves to Log4j 2 (org.apache.logging.slf4j),
      and that messages logged through the Logging trait are routed to Log4j 2. Passes 2/2
      on Scala 2.13 and test-compiles under Scala 2.12.
    - thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala:
      asserts exactly one org.slf4j.LoggerFactory is on the classpath (fails if a shaded
      SLF4J such as jetty-runner is reintroduced) and that SLF4J binds to Log4j 2 rather
      than falling back to the NOP logger.
    
    Also verified via mvn dependency:tree on core, repl, rsc, server, and thriftserver/server:
    slf4j-api resolves to 2.0.17, log4j-slf4j2-impl is present, and no log4j-slf4j-impl
    remains on the compile/runtime classpath (including under hive-jdbc and hive-beeline).
    Started livy-server locally and confirmed the StaticLoggerBinder NOP warning is gone and
    logging works.
    
    ## Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (Claude Opus 4.8)
---
 core/pom.xml                                       |  2 +-
 .../test/scala/org/apache/livy/LoggingSuite.scala  | 99 ++++++++++++++++++++++
 integration-test/pom.xml                           |  4 +
 pom.xml                                            | 22 ++++-
 repl/pom.xml                                       |  2 +-
 rsc/pom.xml                                        |  2 +-
 thriftserver/client/pom.xml                        | 12 +++
 thriftserver/server/pom.xml                        | 28 ++++--
 .../thriftserver/TestThriftServerLogging.scala     | 57 +++++++++++++
 9 files changed, 214 insertions(+), 14 deletions(-)

diff --git a/core/pom.xml b/core/pom.xml
index 8bbe4b7e..a0fd8231 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -39,7 +39,7 @@
 
     <dependency>
       <groupId>org.apache.logging.log4j</groupId>
-      <artifactId>log4j-slf4j-impl</artifactId>
+      <artifactId>log4j-slf4j2-impl</artifactId>
     </dependency>
     <dependency>
       <groupId>org.apache.logging.log4j</groupId>
diff --git a/core/src/test/scala/org/apache/livy/LoggingSuite.scala b/core/src/test/scala/org/apache/livy/LoggingSuite.scala
new file mode 100644
index 00000000..aa8bfb28
--- /dev/null
+++ b/core/src/test/scala/org/apache/livy/LoggingSuite.scala
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy
+
+import java.util.ServiceLoader
+import java.util.concurrent.ConcurrentLinkedQueue
+
+import scala.collection.JavaConverters._
+
+import org.apache.logging.log4j.{Level, LogManager}
+import org.apache.logging.log4j.core.{LogEvent, LoggerContext}
+import org.apache.logging.log4j.core.appender.AbstractAppender
+import org.apache.logging.log4j.core.config.Property
+import org.apache.logging.log4j.core.layout.PatternLayout
+import org.scalatest.funsuite.AnyFunSuite
+import org.slf4j.LoggerFactory
+import org.slf4j.spi.SLF4JServiceProvider
+
+class LoggingSuite extends AnyFunSuite with LivyBaseUnitTestSuite {
+
+  test("SLF4J is bound to the Log4j 2 provider via log4j-slf4j2-impl") {
+    // The org.slf4j.spi.SLF4JServiceProvider interface only exists in SLF4J 2.x,
+    // and the ServiceLoader-based lookup replaces the SLF4J 1.x StaticLoggerBinder.
+    // log4j-slf4j-impl (the 1.x binding) registers no provider, so this check fails
+    // if the binding was not migrated to log4j-slf4j2-impl.
+    val providers = ServiceLoader.load(classOf[SLF4JServiceProvider]).asScala.toList
+    assert(providers.size === 1,
+      s"expected exactly one SLF4J provider, found: ${providers.map(_.getClass.getName)}")
+    assert(providers.head.getClass.getName === "org.apache.logging.slf4j.SLF4JServiceProvider")
+
+    // The binding actually wired into LoggerFactory must be the same Log4j 2 implementation.
+    assert(LoggerFactory.getILoggerFactory.getClass.getName ===
+      "org.apache.logging.slf4j.Log4jLoggerFactory")
+    val logger = LoggerFactory.getLogger(classOf[LoggingSuite])
+    assert(logger.getClass.getName === "org.apache.logging.slf4j.Log4jLogger")
+  }
+
+  test("messages logged through the Logging trait are routed to Log4j 2") {
+    val ctx = LogManager.getContext(false).asInstanceOf[LoggerContext]
+    val config = ctx.getConfiguration
+    val root = config.getRootLogger
+
+    val appender = new CapturingAppender("LoggingSuiteCapture")
+    appender.start()
+    config.addAppender(appender)
+    val previousLevel = root.getLevel
+    root.addAppender(appender, Level.TRACE, null)
+    root.setLevel(Level.TRACE)
+    ctx.updateLoggers()
+
+    try {
+      val subject = new Logging {}
+      val marker = "livy-slf4j2-logging-check"
+      subject.info(marker)
+      subject.warn(s"$marker-warn")
+
+      val captured = appender.formattedMessages
+      assert(captured.contains(marker),
+        s"info message not routed to Log4j 2; captured: $captured")
+      assert(captured.contains(s"$marker-warn"),
+        s"warn message not routed to Log4j 2; captured: $captured")
+    } finally {
+      root.removeAppender(appender.getName)
+      root.setLevel(previousLevel)
+      appender.stop()
+      ctx.updateLoggers()
+    }
+  }
+}
+
+/** In-memory Log4j 2 appender that records the events it receives. */
+private class CapturingAppender(name: String)
+  extends AbstractAppender(
+    name, null, PatternLayout.createDefaultLayout(), false, Property.EMPTY_ARRAY) {
+
+  private val events = new ConcurrentLinkedQueue[LogEvent]()
+
+  override def append(event: LogEvent): Unit = {
+    events.add(event.toImmutable)
+  }
+
+  def formattedMessages: Seq[String] =
+    events.asScala.map(_.getMessage.getFormattedMessage).toSeq
+}
diff --git a/integration-test/pom.xml b/integration-test/pom.xml
index a71e2afd..6768062b 100644
--- a/integration-test/pom.xml
+++ b/integration-test/pom.xml
@@ -341,6 +341,10 @@
               <groupId>org.apache.logging.log4j</groupId>
               <artifactId>log4j-slf4j-impl</artifactId>
             </exclusion>
+            <exclusion>
+              <groupId>org.apache.logging.log4j</groupId>
+              <artifactId>log4j-slf4j2-impl</artifactId>
+            </exclusion>
             <exclusion>
               <groupId>log4j</groupId>
               <artifactId>log4j</artifactId>
diff --git a/pom.xml b/pom.xml
index 8c6f7518..d32b5735 100644
--- a/pom.xml
+++ b/pom.xml
@@ -80,7 +80,7 @@
   <properties>
     <asynchttpclient.version>2.10.1</asynchttpclient.version>
     <hadoop.scope>compile</hadoop.scope>
-    <slf4j.version>1.7.36</slf4j.version>
+    <slf4j.version>2.0.17</slf4j.version>
     <spark.version>4.1.2</spark.version>
     <kubernetes.client.version>6.8.1</kubernetes.client.version>
     <hive.version>3.0.0</hive.version>
@@ -306,7 +306,7 @@
 
       <dependency>
         <groupId>org.apache.logging.log4j</groupId>
-        <artifactId>log4j-slf4j-impl</artifactId>
+        <artifactId>log4j-slf4j2-impl</artifactId>
         <version>${log4j.version}</version>
       </dependency>
       <dependency>
@@ -890,6 +890,24 @@
         <version>${jetty.version}</version>
       </dependency>
 
+      <dependency>
+        <groupId>org.eclipse.jetty</groupId>
+        <artifactId>jetty-webapp</artifactId>
+        <version>${jetty.version}</version>
+      </dependency>
+
+      <dependency>
+        <groupId>org.eclipse.jetty</groupId>
+        <artifactId>jetty-xml</artifactId>
+        <version>${jetty.version}</version>
+      </dependency>
+
+      <dependency>
+        <groupId>org.eclipse.jetty</groupId>
+        <artifactId>jetty-client</artifactId>
+        <version>${jetty.version}</version>
+      </dependency>
+
       <dependency>
         <groupId>org.json4s</groupId>
         <artifactId>json4s_${scala.binary.version}</artifactId>
diff --git a/repl/pom.xml b/repl/pom.xml
index cd7f0ab0..dc66585e 100644
--- a/repl/pom.xml
+++ b/repl/pom.xml
@@ -92,7 +92,7 @@
 
     <dependency>
       <groupId>org.apache.logging.log4j</groupId>
-      <artifactId>log4j-slf4j-impl</artifactId>
+      <artifactId>log4j-slf4j2-impl</artifactId>
       <scope>provided</scope>
     </dependency>
     <dependency>
diff --git a/rsc/pom.xml b/rsc/pom.xml
index 4c2ab144..9cd4c8b9 100644
--- a/rsc/pom.xml
+++ b/rsc/pom.xml
@@ -88,7 +88,7 @@
     </dependency>
     <dependency>
       <groupId>org.apache.logging.log4j</groupId>
-      <artifactId>log4j-slf4j-impl</artifactId>
+      <artifactId>log4j-slf4j2-impl</artifactId>
       <scope>provided</scope>
     </dependency>
     <dependency>
diff --git a/thriftserver/client/pom.xml b/thriftserver/client/pom.xml
index de5c7b51..6cd22db0 100644
--- a/thriftserver/client/pom.xml
+++ b/thriftserver/client/pom.xml
@@ -51,6 +51,10 @@
           <groupId>org.slf4j</groupId>
           <artifactId>slf4j-log4j12</artifactId>
         </exclusion>
+        <exclusion>
+          <groupId>org.apache.logging.log4j</groupId>
+          <artifactId>log4j-slf4j-impl</artifactId>
+        </exclusion>
       </exclusions>
     </dependency>
     <dependency>
@@ -62,8 +66,16 @@
           <groupId>org.apache.hbase</groupId>
           <artifactId>*</artifactId>
         </exclusion>
+        <exclusion>
+          <groupId>org.apache.logging.log4j</groupId>
+          <artifactId>log4j-slf4j-impl</artifactId>
+        </exclusion>
       </exclusions>
     </dependency>
+    <dependency>
+      <groupId>org.apache.logging.log4j</groupId>
+      <artifactId>log4j-slf4j2-impl</artifactId>
+    </dependency>
   </dependencies>
   <build>
     <plugins>
diff --git a/thriftserver/server/pom.xml b/thriftserver/server/pom.xml
index 964e28a0..26bde875 100644
--- a/thriftserver/server/pom.xml
+++ b/thriftserver/server/pom.xml
@@ -116,10 +116,27 @@
       <version>2.25.1</version>
     </dependency>
 
+    <!--
+      Depend on the individual Jetty modules instead of the jetty-runner uber-jar.
+      jetty-runner bundles a shaded SLF4J 1.x (org.slf4j.*) that shadows
+      slf4j-api 2.x on the classpath and triggers the "Failed to load class
+      org.slf4j.impl.StaticLoggerBinder" / NOP warning. jetty-server and
+      jetty-servlet are directly used by ThriftHttpCLIService; jetty-webapp
+      overrides Hive's older Jetty (LIVY-543). All Jetty versions are managed in
+      the root pom so these jars match the ones shipped by livy-server.
+      None of these bundle SLF4J.
+    -->
     <dependency>
       <groupId>org.eclipse.jetty</groupId>
-      <artifactId>jetty-runner</artifactId>
-      <version>${jetty.version}</version>
+      <artifactId>jetty-server</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.eclipse.jetty</groupId>
+      <artifactId>jetty-servlet</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.eclipse.jetty</groupId>
+      <artifactId>jetty-webapp</artifactId>
     </dependency>
 
     <!-- needed for testing -->
@@ -211,13 +228,6 @@
             <goals>
               <goal>copy-dependencies</goal>
             </goals>
-            <configuration>
-              <excludeArtifactIds>
-                jetty-util,
-                jetty-xml,
-                jetty-webapp
-              </excludeArtifactIds>
-            </configuration>
           </execution>
         </executions>
       </plugin>
diff --git a/thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala b/thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala
new file mode 100644
index 00000000..b3d1cf08
--- /dev/null
+++ b/thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver
+
+import scala.collection.JavaConverters._
+
+import org.junit.Assert._
+import org.junit.Test
+import org.slf4j.LoggerFactory
+
+/**
+ * Guards the thriftserver classpath against a shaded SLF4J binding. The
+ * jetty-runner uber-jar used to bundle a copy of SLF4J 1.x (org.slf4j.*). On the
+ * server classpath that copy could shadow slf4j-api 2.x, which then looked for
+ * the removed 1.x org.slf4j.impl.StaticLoggerBinder and fell back to the no-op
+ * (NOP) logger -- silently dropping all logging. These tests fail if such a jar
+ * is reintroduced.
+ */
+class TestThriftServerLogging {
+
+  @Test
+  def exactlyOneLoggerFactoryOnClasspath(): Unit = {
+    val resources =
+      getClass.getClassLoader.getResources("org/slf4j/LoggerFactory.class").asScala.toList
+    assertEquals(
+      "Expected exactly one org.slf4j.LoggerFactory on the classpath. A shaded SLF4J " +
+        "(e.g. the jetty-runner uber-jar) adds a second copy that can shadow slf4j-api and " +
+        s"trigger the StaticLoggerBinder NOP fallback. Found: $resources",
+      1, resources.size)
+  }
+
+  @Test
+  def slf4jBindsToLog4j2NotNop(): Unit = {
+    val factory = LoggerFactory.getILoggerFactory.getClass.getName
+    assertEquals("SLF4J did not bind to Log4j 2 (log4j-slf4j2-impl)",
+      "org.apache.logging.slf4j.Log4jLoggerFactory", factory)
+
+    val logger = LoggerFactory.getLogger(classOf[TestThriftServerLogging]).getClass.getName
+    assertFalse(s"SLF4J fell back to a NOP logger implementation: $logger",
+      logger.toUpperCase.contains("NOP"))
+  }
+}