You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Amit Sharma <re...@gmail.com> on 2022/01/12 19:52:19 UTC
Log4j2 upgrade
Hello, everyone. I am replacing log4j with log4j2 in my spark streaming
application. When i deployed my application to spark cluster it is giving
me the below error .
" ERROR StatusLogger Log4j2 could not find a logging implementation. Please
add log4j-core to the classpath. Using SimpleLogger to log to the console "
I am including the core jar in my fat jar and core jar also included in the
jar. Although the application is running fine, I am doubtful the logs are
generated using log4j not log4j2 .
I am using sbt assembly jar and also noticed below messages in the build
Fully-qualified classname does not match jar entry:
jar entry: META-INF/versions/9/module-info.class
class name: module-info.class
Omitting META-INF/versions/9/module-info.class.
Fully-qualified classname does not match jar entry:
jar entry:
META-INF/versions/9/org/apache/logging/log4j/util/Base64Util.class
class name: org/apache/logging/log4j/util/Base64Util.class
Omitting META-INF/versions/9/org/apache/logging/log4j/util/Base64Util.class.
Fully-qualified classname does not match jar entry:
jar entry:
META-INF/versions/9/org/apache/logging/log4j/util/internal/DefaultObjectInputFilter.class
any idea how to resolve these.
Thanks
Amit
Re: Log4j2 upgrade
Posted by Jörn Franke <jo...@gmail.com>.
You cannot simply replace it - log4j2 has a slightly different API than log4j. The Spark source code needs to be changed in a couple of places
> Am 12.01.2022 um 20:53 schrieb Amit Sharma <re...@gmail.com>:
>
>
> Hello, everyone. I am replacing log4j with log4j2 in my spark streaming application. When i deployed my application to spark cluster it is giving me the below error .
>
> " ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console "
>
>
> I am including the core jar in my fat jar and core jar also included in the jar. Although the application is running fine, I am doubtful the logs are generated using log4j not log4j2 .
> I am using sbt assembly jar and also noticed below messages in the build
>
> Fully-qualified classname does not match jar entry:
> jar entry: META-INF/versions/9/module-info.class
>
>
> class name: module-info.class
> Omitting META-INF/versions/9/module-info.class.
> Fully-qualified classname does not match jar entry:
> jar entry: META-INF/versions/9/org/apache/logging/log4j/util/Base64Util.class
> class name: org/apache/logging/log4j/util/Base64Util.class
> Omitting META-INF/versions/9/org/apache/logging/log4j/util/Base64Util.class.
> Fully-qualified classname does not match jar entry:
> jar entry: META-INF/versions/9/org/apache/logging/log4j/util/internal/DefaultObjectInputFilter.class
>
>
> any idea how to resolve these.
>
>
> Thanks
> Amit
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org