You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/05/30 04:18:05 UTC
[GitHub] [spark] LuciferYang opened a new pull request, #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
LuciferYang opened a new pull request, #36721:
URL: https://github.com/apache/spark/pull/36721
### What changes were proposed in this pull request?
[HADOOP-18088 Replace log4j 1.x with reload4j](https://issues.apache.org/jira/browse/HADOOP-18088) , this pr change to exclude `slf4j-reload4j` instead of `slf4j-log4j12` for `hadoop-minikdc` to clean up waring message about `Class path contains multiple SLF4J bindings`.
### Why are the changes needed?
Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GA
- Manual test
run `mvn clean test -pl core`
**Before**
```
[INFO] Running test.org.apache.spark.Java8RDDAPISuite
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/xxx/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.2/log4j-slf4j-impl-2.17.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/xxx/.m2/repository/org/slf4j/slf4j-reload4j/1.7.36/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
```
**After**
no above warings
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #36721: [SPARK-39334][BUILD] Exclude `slf4j-reload4j` from `hadoop-minikdc` test dependency
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #36721:
URL: https://github.com/apache/spark/pull/36721#issuecomment-1140874241
Although this is a test-only patch and we have Apache Hadoop 3.3.3 on `master` branch, I'll land this patch to `branch-3.3` because the users can provide Hadoop 3.3.3 during Apache Spark 3.3.0 RC testings.
Merged to master/3.3. I manually verified this patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #36721: [SPARK-39334][BUILD] Exclude `slf4j-reload4j` from `hadoop-minikdc` test dependency
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #36721: [SPARK-39334][BUILD] Exclude `slf4j-reload4j` from `hadoop-minikdc` test dependency
URL: https://github.com/apache/spark/pull/36721
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #36721:
URL: https://github.com/apache/spark/pull/36721#issuecomment-1140709486
cc @viirya
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #36721: [SPARK-39334][BUILD] Exclude `slf4j-reload4j` from `hadoop-minikdc` test dependency
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #36721:
URL: https://github.com/apache/spark/pull/36721#issuecomment-1140875643
thanks all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #36721:
URL: https://github.com/apache/spark/pull/36721#issuecomment-1140677005
cc @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #36721: [SPARK-39334][BUILD] Exclude `slf4j-reload4j` from `hadoop-minikdc` test dependency
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #36721:
URL: https://github.com/apache/spark/pull/36721#issuecomment-1140784995
> Could you update the following PR description according to the latest change, @LuciferYang ? We exclude both ones.
>
> > this pr change to exclude slf4j-reload4j instead of slf4j-log4j12
>
> For the code change, +1, LGTM.
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a diff in pull request #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on code in PR #36721:
URL: https://github.com/apache/spark/pull/36721#discussion_r884425677
##########
pom.xml:
##########
@@ -1418,7 +1418,7 @@
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <artifactId>slf4j-reload4j</artifactId>
Review Comment:
ok ~ although Hadoop 3.3.3 no longer relies on `slf4j-log4j12 `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a diff in pull request #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
Posted by GitBox <gi...@apache.org>.
wangyum commented on code in PR #36721:
URL: https://github.com/apache/spark/pull/36721#discussion_r884420659
##########
pom.xml:
##########
@@ -1418,7 +1418,7 @@
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <artifactId>slf4j-reload4j</artifactId>
Review Comment:
Could we keep `slf4j-log4j12` and also add `slf4j-reload4j` to exclusions list?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #36721: [SPARK-39334][BUILD] Exclude `slf4j-reload4j` from `hadoop-minikdc` test dependency
Posted by GitBox <gi...@apache.org>.
sunchao commented on PR #36721:
URL: https://github.com/apache/spark/pull/36721#issuecomment-1141704120
LGTM too, thanks @LuciferYang !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a diff in pull request #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on code in PR #36721:
URL: https://github.com/apache/spark/pull/36721#discussion_r884431050
##########
pom.xml:
##########
@@ -1418,7 +1418,7 @@
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <artifactId>slf4j-reload4j</artifactId>
Review Comment:
> User may build and test with Hadoop 3.3.2 by specify the exact version of Hadoop.
Got it ~ thanks @wangyum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #36721:
URL: https://github.com/apache/spark/pull/36721#issuecomment-1140678889
Looks like a change introduced by hadoop 3.3.3
https://github.com/apache/hadoop/compare/rel/release-3.3.2...rel/release-3.3.3
<img width="1384" alt="image" src="https://user-images.githubusercontent.com/1475305/170916543-a6fd40a3-e141-415b-8951-fcebd030b379.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a diff in pull request #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
Posted by GitBox <gi...@apache.org>.
wangyum commented on code in PR #36721:
URL: https://github.com/apache/spark/pull/36721#discussion_r884430447
##########
pom.xml:
##########
@@ -1418,7 +1418,7 @@
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <artifactId>slf4j-reload4j</artifactId>
Review Comment:
User may build and test with Hadoop 3.3.2 by specify the exact version of Hadoop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a diff in pull request #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
Posted by GitBox <gi...@apache.org>.
wangyum commented on code in PR #36721:
URL: https://github.com/apache/spark/pull/36721#discussion_r884430447
##########
pom.xml:
##########
@@ -1418,7 +1418,7 @@
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <artifactId>slf4j-reload4j</artifactId>
Review Comment:
User may build and test with Hadoop 3.3.2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a diff in pull request #36721: [SPARK-39334][BUILD] Cleanup `Class path contains multiple SLF4J bindings` waring when run UTs
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on code in PR #36721:
URL: https://github.com/apache/spark/pull/36721#discussion_r884426484
##########
pom.xml:
##########
@@ -1418,7 +1418,7 @@
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
+ <artifactId>slf4j-reload4j</artifactId>
Review Comment:
[be2e8cb](https://github.com/apache/spark/pull/36721/commits/be2e8cb8f55cc40b98722349da31fef698fb3c34) fix this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org