[jira] [Updated] (LIVY-1090) Upgrade SLF4J 1.7.36 -> 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl), replace jetty-runner

["Gabor Roczei (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LIVY-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gabor Roczei updated LIVY-1090:
-------------------------------
    Summary: Upgrade SLF4J 1.7.36 -> 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl), replace jetty-runner  (was: Upgrade SLF4J from 1.7.36 to 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl))

> Upgrade SLF4J 1.7.36 -> 2.0.17 (log4j-slf4j-impl -> log4j-slf4j2-impl), replace jetty-runner
> --------------------------------------------------------------------------------------------
>
>                 Key: LIVY-1090
>                 URL: https://issues.apache.org/jira/browse/LIVY-1090
>             Project: Livy
>          Issue Type: Improvement
>    Affects Versions: 1.0.0
>            Reporter: Gabor Roczei
>            Assignee: Gabor Roczei
>            Priority: Major
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> h3. Problem
> Livy depends on SLF4J {{1.7.36}} and binds it to Log4j 2 via {{{}log4j-slf4j-impl{}}}, which targets the SLF4J 1.x API. SLF4J 2.0.x changed its provider-discovery mechanism: the old {{StaticLoggerBinder}} lookup was replaced by a {{{}ServiceLoader{}}}-based {{{}org.slf4j.spi.SLF4JServiceProvider{}}}. Running on SLF4J 2.x therefore requires the matching Log4j binding, {{{}log4j-slf4j2-impl{}}}.
> h3. Change
>  * {{{}pom.xml{}}}: {{slf4j.version}} {{1.7.36}} -> {{2.0.17}}
>  * Swap the Log4j binding artifact {{log4j-slf4j-impl}} -> {{log4j-slf4j2-impl}} in the direct declarations we own:
>  ** {{pom.xml}} ({{{}dependencyManagement{}}}; version stays {{{}$\{log4j.version{}}}})
>  ** {{core/pom.xml}}
>  ** {{repl/pom.xml}} ({{{}scope: provided{}}} unchanged)
>  ** {{rsc/pom.xml}} ({{{}scope: provided{}}} unchanged)
>  * Add a {{log4j-slf4j2-impl}} exclusion alongside the existing {{log4j-slf4j-impl}} exclusion on {{hive-jdbc}} in {{thriftserver/server/pom.xml}} and {{integration-test/pom.xml}} (exclude both bindings — see Notes).
>  * {{{}thriftserver/server/pom.xml{}}}: replace the {{jetty-runner}} uber-jar with the individual {{{}jetty-server{}}}, {{jetty-servlet}} and {{jetty-webapp}} modules, and drop the now-obsolete {{{}jetty-util{}}}/{{{}jetty-xml{}}}/{{{}jetty-webapp{}}} {{excludeArtifactIds}} from the {{copy-dependencies}} step (see Notes).
> h3. Notes
>  * No Log4j version bump is needed: Livy is already on Log4j {{{}2.26.0{}}}, which is newer than the {{2.23.0}} minimum required by {{{}log4j-slf4j2-impl{}}}.
>  * {{hive-jdbc:3.0.0}} transitively brings {{log4j-slf4j-impl:2.10.0}} (the SLF4J-1.x binding), so the existing {{log4j-slf4j-impl}} exclusion must stay. A {{log4j-slf4j2-impl}} exclusion is added next to it so neither binding can leak from Hive onto the (test) classpath, regardless of the Hive version — avoiding a dual-binding conflict with the explicit {{{}log4j-slf4j2-impl{}}}.
>  * The {{jetty-runner}} uber-jar bundles a shaded SLF4J 1.x ({{{}org.slf4j.*{}}}). On the Livy server classpath that copy shadowed {{{}slf4j-api 2.x{}}}, so SLF4J looked for the now-absent 1.x {{org.slf4j.impl.StaticLoggerBinder}} and fell back to the no-op (NOP) logger — printing {{SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder"}} / {{Defaulting to no-operation (NOP) logger implementation}} at server startup and silently dropping logging. Before this upgrade the 1.x binding ({{{}log4j-slf4j-impl{}}}) satisfied it, so the warning was hidden. {{jetty-runner}} is not referenced in Livy source; the thriftserver only uses {{{}jetty-server{}}}/{{{}jetty-servlet{}}}/{{{}jetty-util{}}}. Depending on the individual modules (at {{{}$\{jetty.version{}}}}, so Hive's older Jetty is still overridden per LIVY-543) removes the bundled SLF4J. The {{excludeArtifactIds}} only existed to avoid duplicating {{{}jetty-runner{}}}'s bundled jetty-util/xml/webapp and is no longer needed.
>  * Livy's own code has no references to {{org.slf4j.impl}} / {{{}StaticLoggerBinder{}}}.
> h3. Test
>  * {{core/src/test/scala/org/apache/livy/LoggingSuite.scala}} (ScalaTest {{{}AnyFunSuite{}}}, cross-compiles under Scala 2.12 and 2.13):
>  ** SLF4J is bound to the Log4j 2 provider via log4j-slf4j2-impl — loads {{org.slf4j.spi.SLF4JServiceProvider}} via {{ServiceLoader}} and asserts exactly one provider is registered, that it is {{{}org.apache.logging.slf4j.SLF4JServiceProvider{}}}, and that {{{}LoggerFactory{}}}'s active factory/logger are the Log4j 2 implementations. Decisive migration check: the old {{log4j-slf4j-impl}} registers no {{SLF4JServiceProvider}} and the {{org.slf4j.spi}} interface exists only in SLF4J 2.x.
>  ** messages logged through the Logging trait are routed to Log4j 2 — attaches an in-memory Log4j 2 {{AbstractAppender}} to the root logger, logs {{{}info{}}}/{{{}warn{}}} through Livy's {{Logging}} trait, asserts the messages are received, and restores the prior appender/level afterwards.
>  * {{thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala}} (JUnit, runs in the surefire {{test}} phase): asserts exactly one {{org.slf4j.LoggerFactory}} is on the classpath (fails if a shaded SLF4J such as {{jetty-runner}} is reintroduced) and that SLF4J binds to Log4j 2 rather than falling back to the NOP logger.
> h3. Verification
> {{mvn dependency:tree}} on {{{}core{}}}, {{{}repl{}}}, {{{}rsc{}}}, {{{}server{}}}, and {{thriftserver/server}} confirms {{slf4j-api}} resolves to {{{}2.0.17{}}}, {{log4j-slf4j2-impl}} is present, and no {{log4j-slf4j-impl}} remains on the compile/runtime classpath (including under {{{}hive-jdbc{}}}). On the assembled server classpath only {{slf4j-api-2.0.17.jar}} provides {{org.slf4j.LoggerFactory}} and no {{org.slf4j.impl.StaticLoggerBinder}} class is present. {{LoggingSuite}} (2/2) and {{TestThriftServerLogging}} (2/2) pass. Starting {{livy-server}} locally no longer prints the StaticLoggerBinder NOP warning and logging works.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)