[jira] [Commented] (SPARK-38502) Distribution with hadoop-provided is missing log4j2

["Emil Ejbyfeldt (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-38502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504749#comment-17504749 ] 

Emil Ejbyfeldt commented on SPARK-38502:
----------------------------------------

Duplicate of SPARK-38516

> Distribution with hadoop-provided is missing log4j2
> ---------------------------------------------------
>
>                 Key: SPARK-38502
>                 URL: https://issues.apache.org/jira/browse/SPARK-38502
>             Project: Spark
>          Issue Type: Bug
>          Components: Build
>    Affects Versions: 3.3.0
>            Reporter: Emil Ejbyfeldt
>            Priority: Major
>
> Currently building spark 3.3.0-SNAPSHOT using `./dev/make-distribution.sh --tgz --name hadoop-provided-test -Phadoop-provided -Pyarn` script will build a package that does not included log4j2. Trying to run spark-submit with the latest hadoop release 3.3.2 and this build will result in
> {code:java}
> $ spark-submit run-example org.apache.spark.examples.SparkPi
> Error: Unable to initialize main class org.apache.spark.deploy.SparkSubmit
> Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/core/Filter
> {code}
> Since log4j2 is not found. So I believe the maven build settings needs to be tweaked so that log4j2 is included in the spark distribution.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org