You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@livy.apache.org by gg...@apache.org on 2026/10/20 16:53:20 UTC
(livy) branch master updated: [LIVY-1090] Upgrade SLF4J 1.7.36 -> 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl), replace jetty-runner
This is an automated email from the ASF dual-hosted git repository.
gyogal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/livy.git
The following commit(s) were added to refs/heads/master by this push:
new bc234453 [LIVY-1090] Upgrade SLF4J 1.7.36 -> 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl), replace jetty-runner
bc234453 is described below
commit bc234453011e55138767f1b0ad3de9d4f77f93e8
Author: Gabor Roczei <19...@users.noreply.github.com>
AuthorDate: Tue Oct 20 18:53:15 2026 +0200
[LIVY-1090] Upgrade SLF4J 1.7.36 -> 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl), replace jetty-runner
## What changes were proposed in this pull request?
SLF4J 2.0.x replaced the 1.x StaticLoggerBinder lookup with a ServiceLoader-based
SLF4JServiceProvider, so the Log4j 2 binding must be swapped from log4j-slf4j-impl
(SLF4J 1.x) to log4j-slf4j2-impl (SLF4J 2.x).
- pom.xml: slf4j.version 1.7.36 -> 2.0.17
- Swap log4j-slf4j-impl -> log4j-slf4j2-impl in the binding declarations:
root dependencyManagement, core, repl (provided), rsc (provided)
- thriftserver/client: exclude log4j-slf4j-impl from hive-jdbc and hive-beeline
(hive-jdbc:3.0.0 and hive-beeline -> hive-common still bring log4j-slf4j-impl:2.10.0)
and declare log4j-slf4j2-impl explicitly
- integration-test: additionally exclude log4j-slf4j2-impl from the test-scoped
hive-jdbc (next to the existing log4j-slf4j-impl exclusion)
- thriftserver/server: replace the jetty-runner uber-jar (which bundles a shaded
SLF4J 1.x that shadowed slf4j-api 2.x and caused a "Failed to load class
org.slf4j.impl.StaticLoggerBinder" NOP fallback at server startup) with the
individual jetty-server, jetty-servlet and jetty-webapp modules, and drop the
now-obsolete jetty-util/jetty-xml/jetty-webapp excludeArtifactIds that only
existed to avoid duplicating jetty-runner's bundled copies. The existing
log4j-slf4j-impl exclusion on hive-jdbc is kept.
- Root pom: manage jetty-webapp, jetty-xml and jetty-client at ${jetty.version} in
dependencyManagement so all Jetty modules resolve to the same version as
livy-server, overriding Hive's older Jetty (LIVY-543). The jetty-webapp declaration
in thriftserver/server now inherits its version from the root pom.
No Log4j bump needed (already on 2.26.0, >= the 2.23.0 minimum for log4j-slf4j2-impl).
Livy has no org.slf4j.impl / StaticLoggerBinder usage, so no source changes are required.
JIRA: https://issues.apache.org/jira/browse/LIVY-1090
## How was this patch tested?
Unit tests.
- core/src/test/scala/org/apache/livy/LoggingSuite.scala: verifies a single
SLF4JServiceProvider is registered and resolves to Log4j 2 (org.apache.logging.slf4j),
and that messages logged through the Logging trait are routed to Log4j 2. Passes 2/2
on Scala 2.13 and test-compiles under Scala 2.12.
- thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala:
asserts exactly one org.slf4j.LoggerFactory is on the classpath (fails if a shaded
SLF4J such as jetty-runner is reintroduced) and that SLF4J binds to Log4j 2 rather
than falling back to the NOP logger.
Also verified via mvn dependency:tree on core, repl, rsc, server, and thriftserver/server:
slf4j-api resolves to 2.0.17, log4j-slf4j2-impl is present, and no log4j-slf4j-impl
remains on the compile/runtime classpath (including under hive-jdbc and hive-beeline).
Started livy-server locally and confirmed the StaticLoggerBinder NOP warning is gone and
logging works.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
---
core/pom.xml | 2 +-
.../test/scala/org/apache/livy/LoggingSuite.scala | 99 ++++++++++++++++++++++
integration-test/pom.xml | 4 +
pom.xml | 22 ++++-
repl/pom.xml | 2 +-
rsc/pom.xml | 2 +-
thriftserver/client/pom.xml | 12 +++
thriftserver/server/pom.xml | 28 ++++--
.../thriftserver/TestThriftServerLogging.scala | 57 +++++++++++++
9 files changed, 214 insertions(+), 14 deletions(-)
diff --git a/core/pom.xml b/core/pom.xml
index 8bbe4b7e..a0fd8231 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -39,7 +39,7 @@
<dependency>
<groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-slf4j-impl</artifactId>
+ <artifactId>log4j-slf4j2-impl</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
diff --git a/core/src/test/scala/org/apache/livy/LoggingSuite.scala b/core/src/test/scala/org/apache/livy/LoggingSuite.scala
new file mode 100644
index 00000000..aa8bfb28
--- /dev/null
+++ b/core/src/test/scala/org/apache/livy/LoggingSuite.scala
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy
+
+import java.util.ServiceLoader
+import java.util.concurrent.ConcurrentLinkedQueue
+
+import scala.collection.JavaConverters._
+
+import org.apache.logging.log4j.{Level, LogManager}
+import org.apache.logging.log4j.core.{LogEvent, LoggerContext}
+import org.apache.logging.log4j.core.appender.AbstractAppender
+import org.apache.logging.log4j.core.config.Property
+import org.apache.logging.log4j.core.layout.PatternLayout
+import org.scalatest.funsuite.AnyFunSuite
+import org.slf4j.LoggerFactory
+import org.slf4j.spi.SLF4JServiceProvider
+
+class LoggingSuite extends AnyFunSuite with LivyBaseUnitTestSuite {
+
+ test("SLF4J is bound to the Log4j 2 provider via log4j-slf4j2-impl") {
+ // The org.slf4j.spi.SLF4JServiceProvider interface only exists in SLF4J 2.x,
+ // and the ServiceLoader-based lookup replaces the SLF4J 1.x StaticLoggerBinder.
+ // log4j-slf4j-impl (the 1.x binding) registers no provider, so this check fails
+ // if the binding was not migrated to log4j-slf4j2-impl.
+ val providers = ServiceLoader.load(classOf[SLF4JServiceProvider]).asScala.toList
+ assert(providers.size === 1,
+ s"expected exactly one SLF4J provider, found: ${providers.map(_.getClass.getName)}")
+ assert(providers.head.getClass.getName === "org.apache.logging.slf4j.SLF4JServiceProvider")
+
+ // The binding actually wired into LoggerFactory must be the same Log4j 2 implementation.
+ assert(LoggerFactory.getILoggerFactory.getClass.getName ===
+ "org.apache.logging.slf4j.Log4jLoggerFactory")
+ val logger = LoggerFactory.getLogger(classOf[LoggingSuite])
+ assert(logger.getClass.getName === "org.apache.logging.slf4j.Log4jLogger")
+ }
+
+ test("messages logged through the Logging trait are routed to Log4j 2") {
+ val ctx = LogManager.getContext(false).asInstanceOf[LoggerContext]
+ val config = ctx.getConfiguration
+ val root = config.getRootLogger
+
+ val appender = new CapturingAppender("LoggingSuiteCapture")
+ appender.start()
+ config.addAppender(appender)
+ val previousLevel = root.getLevel
+ root.addAppender(appender, Level.TRACE, null)
+ root.setLevel(Level.TRACE)
+ ctx.updateLoggers()
+
+ try {
+ val subject = new Logging {}
+ val marker = "livy-slf4j2-logging-check"
+ subject.info(marker)
+ subject.warn(s"$marker-warn")
+
+ val captured = appender.formattedMessages
+ assert(captured.contains(marker),
+ s"info message not routed to Log4j 2; captured: $captured")
+ assert(captured.contains(s"$marker-warn"),
+ s"warn message not routed to Log4j 2; captured: $captured")
+ } finally {
+ root.removeAppender(appender.getName)
+ root.setLevel(previousLevel)
+ appender.stop()
+ ctx.updateLoggers()
+ }
+ }
+}
+
+/** In-memory Log4j 2 appender that records the events it receives. */
+private class CapturingAppender(name: String)
+ extends AbstractAppender(
+ name, null, PatternLayout.createDefaultLayout(), false, Property.EMPTY_ARRAY) {
+
+ private val events = new ConcurrentLinkedQueue[LogEvent]()
+
+ override def append(event: LogEvent): Unit = {
+ events.add(event.toImmutable)
+ }
+
+ def formattedMessages: Seq[String] =
+ events.asScala.map(_.getMessage.getFormattedMessage).toSeq
+}
diff --git a/integration-test/pom.xml b/integration-test/pom.xml
index a71e2afd..6768062b 100644
--- a/integration-test/pom.xml
+++ b/integration-test/pom.xml
@@ -341,6 +341,10 @@
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j2-impl</artifactId>
+ </exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
diff --git a/pom.xml b/pom.xml
index 8c6f7518..d32b5735 100644
--- a/pom.xml
+++ b/pom.xml
@@ -80,7 +80,7 @@
<properties>
<asynchttpclient.version>2.10.1</asynchttpclient.version>
<hadoop.scope>compile</hadoop.scope>
- <slf4j.version>1.7.36</slf4j.version>
+ <slf4j.version>2.0.17</slf4j.version>
<spark.version>4.1.2</spark.version>
<kubernetes.client.version>6.8.1</kubernetes.client.version>
<hive.version>3.0.0</hive.version>
@@ -306,7 +306,7 @@
<dependency>
<groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-slf4j-impl</artifactId>
+ <artifactId>log4j-slf4j2-impl</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
@@ -890,6 +890,24 @@
<version>${jetty.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.eclipse.jetty</groupId>
+ <artifactId>jetty-webapp</artifactId>
+ <version>${jetty.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.eclipse.jetty</groupId>
+ <artifactId>jetty-xml</artifactId>
+ <version>${jetty.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.eclipse.jetty</groupId>
+ <artifactId>jetty-client</artifactId>
+ <version>${jetty.version}</version>
+ </dependency>
+
<dependency>
<groupId>org.json4s</groupId>
<artifactId>json4s_${scala.binary.version}</artifactId>
diff --git a/repl/pom.xml b/repl/pom.xml
index cd7f0ab0..dc66585e 100644
--- a/repl/pom.xml
+++ b/repl/pom.xml
@@ -92,7 +92,7 @@
<dependency>
<groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-slf4j-impl</artifactId>
+ <artifactId>log4j-slf4j2-impl</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
diff --git a/rsc/pom.xml b/rsc/pom.xml
index 4c2ab144..9cd4c8b9 100644
--- a/rsc/pom.xml
+++ b/rsc/pom.xml
@@ -88,7 +88,7 @@
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-slf4j-impl</artifactId>
+ <artifactId>log4j-slf4j2-impl</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
diff --git a/thriftserver/client/pom.xml b/thriftserver/client/pom.xml
index de5c7b51..6cd22db0 100644
--- a/thriftserver/client/pom.xml
+++ b/thriftserver/client/pom.xml
@@ -51,6 +51,10 @@
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ </exclusion>
</exclusions>
</dependency>
<dependency>
@@ -62,8 +66,16 @@
<groupId>org.apache.hbase</groupId>
<artifactId>*</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ </exclusion>
</exclusions>
</dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j2-impl</artifactId>
+ </dependency>
</dependencies>
<build>
<plugins>
diff --git a/thriftserver/server/pom.xml b/thriftserver/server/pom.xml
index 964e28a0..26bde875 100644
--- a/thriftserver/server/pom.xml
+++ b/thriftserver/server/pom.xml
@@ -116,10 +116,27 @@
<version>2.25.1</version>
</dependency>
+ <!--
+ Depend on the individual Jetty modules instead of the jetty-runner uber-jar.
+ jetty-runner bundles a shaded SLF4J 1.x (org.slf4j.*) that shadows
+ slf4j-api 2.x on the classpath and triggers the "Failed to load class
+ org.slf4j.impl.StaticLoggerBinder" / NOP warning. jetty-server and
+ jetty-servlet are directly used by ThriftHttpCLIService; jetty-webapp
+ overrides Hive's older Jetty (LIVY-543). All Jetty versions are managed in
+ the root pom so these jars match the ones shipped by livy-server.
+ None of these bundle SLF4J.
+ -->
<dependency>
<groupId>org.eclipse.jetty</groupId>
- <artifactId>jetty-runner</artifactId>
- <version>${jetty.version}</version>
+ <artifactId>jetty-server</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.eclipse.jetty</groupId>
+ <artifactId>jetty-servlet</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.eclipse.jetty</groupId>
+ <artifactId>jetty-webapp</artifactId>
</dependency>
<!-- needed for testing -->
@@ -211,13 +228,6 @@
<goals>
<goal>copy-dependencies</goal>
</goals>
- <configuration>
- <excludeArtifactIds>
- jetty-util,
- jetty-xml,
- jetty-webapp
- </excludeArtifactIds>
- </configuration>
</execution>
</executions>
</plugin>
diff --git a/thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala b/thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala
new file mode 100644
index 00000000..b3d1cf08
--- /dev/null
+++ b/thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver
+
+import scala.collection.JavaConverters._
+
+import org.junit.Assert._
+import org.junit.Test
+import org.slf4j.LoggerFactory
+
+/**
+ * Guards the thriftserver classpath against a shaded SLF4J binding. The
+ * jetty-runner uber-jar used to bundle a copy of SLF4J 1.x (org.slf4j.*). On the
+ * server classpath that copy could shadow slf4j-api 2.x, which then looked for
+ * the removed 1.x org.slf4j.impl.StaticLoggerBinder and fell back to the no-op
+ * (NOP) logger -- silently dropping all logging. These tests fail if such a jar
+ * is reintroduced.
+ */
+class TestThriftServerLogging {
+
+ @Test
+ def exactlyOneLoggerFactoryOnClasspath(): Unit = {
+ val resources =
+ getClass.getClassLoader.getResources("org/slf4j/LoggerFactory.class").asScala.toList
+ assertEquals(
+ "Expected exactly one org.slf4j.LoggerFactory on the classpath. A shaded SLF4J " +
+ "(e.g. the jetty-runner uber-jar) adds a second copy that can shadow slf4j-api and " +
+ s"trigger the StaticLoggerBinder NOP fallback. Found: $resources",
+ 1, resources.size)
+ }
+
+ @Test
+ def slf4jBindsToLog4j2NotNop(): Unit = {
+ val factory = LoggerFactory.getILoggerFactory.getClass.getName
+ assertEquals("SLF4J did not bind to Log4j 2 (log4j-slf4j2-impl)",
+ "org.apache.logging.slf4j.Log4jLoggerFactory", factory)
+
+ val logger = LoggerFactory.getLogger(classOf[TestThriftServerLogging]).getClass.getName
+ assertFalse(s"SLF4J fell back to a NOP logger implementation: $logger",
+ logger.toUpperCase.contains("NOP"))
+ }
+}