You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@celeborn.apache.org by et...@apache.org on 2024/05/09 09:06:36 UTC
(celeborn) branch main updated: [CELEBORN-1006][FOLLOWUP] Dependency hadoop-client should exclude hadoop-mapreduce-client dependencies for Hadoop 2
This is an automated email from the ASF dual-hosted git repository.
ethanfeng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new a7862c952 [CELEBORN-1006][FOLLOWUP] Dependency hadoop-client should exclude hadoop-mapreduce-client dependencies for Hadoop 2
a7862c952 is described below
commit a7862c95231ef40ce481a21aac79ddd1053469d6
Author: SteNicholas <pr...@163.com>
AuthorDate: Thu May 9 17:06:16 2024 +0800
[CELEBORN-1006][FOLLOWUP] Dependency hadoop-client should exclude hadoop-mapreduce-client dependencies for Hadoop 2
### What changes were proposed in this pull request?
Dependency `hadoop-client` should exclude `hadoop-mapreduce-client` dependencies for Hadoop 2.
### Why are the changes needed?
`hadoop-mapreduce-client` dependencies of `hadoop-client` are unneccessary. Meanwhile, `hadoop-mapreduce-client` dependencies have dependency `javax.servlet:javax.servlet-api` which causes the compilation error of service module after http server refine in 0.5.0 version.
```
$ mvn clean install -Dhadoop.version=2.10.0 -pl service -DskipTests -Dcheckstyle.skip=true -Drat.skip=true -Dspotless.check.skip=true
[INFO] --- scala-maven-plugin:4.7.2:testCompile (scala-test-compile-first) celeborn-service_2.12 ---
[INFO] Using incremental compilation using Mixed compile order
[INFO] Compiler bridge file: /Users/nicholas/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.7.1-bin_2.12.18__55.0-1.7.1_20220712T022208.jar
[INFO] compiler plugin: BasicArtifact(com.github.ghik,silencer-plugin_2.12.18,1.7.13,null)
[INFO] compiling 2 Scala sources and 1 Java source to /Users/nicholas/Github/celeborn/service/target/test-classes ...
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:33: value readEntity is not a member of javax.ws.rs.core.Response
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:77: value readEntity is not a member of javax.ws.rs.core.Response
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:84: value readEntity is not a member of javax.ws.rs.core.Response
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:91: value readEntity is not a member of javax.ws.rs.core.Response
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:97: value readEntity is not a member of javax.ws.rs.core.Response
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual test.
```
$ mvn clean install -Dhadoop.version=2.10.0 -pl service -DskipTests -Dcheckstyle.skip=true -Drat.skip=true -Dspotless.check.skip=true
[INFO] --- maven-install-plugin:2.5.2:install (default-install) celeborn-service_2.12 ---
[INFO] Installing /Users/nicholas/Github/celeborn/service/target/celeborn-service_2.12-0.5.0-SNAPSHOT.jar to /Users/nicholas/.m2/repository/org/apache/celeborn/celeborn-service_2.12/0.5.0-SNAPSHOT/celeborn-service_2.12-0.5.0-SNAPSHOT.jar
[INFO] Installing /Users/nicholas/Github/celeborn/service/.flattened-pom.xml to /Users/nicholas/.m2/repository/org/apache/celeborn/celeborn-service_2.12/0.5.0-SNAPSHOT/celeborn-service_2.12-0.5.0-SNAPSHOT.pom
[INFO] Installing /Users/nicholas/Github/celeborn/service/target/celeborn-service_2.12-0.5.0-SNAPSHOT-tests.jar to /Users/nicholas/.m2/repository/org/apache/celeborn/celeborn-service_2.12/0.5.0-SNAPSHOT/celeborn-service_2.12-0.5.0-SNAPSHOT-tests.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13.928 s
[INFO] Finished at: 2024-05-09T14:59:23+08:00
[INFO] ------------------------------------------------------------------------
```
Closes #2497 from SteNicholas/CELEBORN-1006.
Authored-by: SteNicholas <pr...@163.com>
Signed-off-by: mingji <fe...@alibaba-inc.com>
---
pom.xml | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/pom.xml b/pom.xml
index 357bb1839..f9a824a2f 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1186,6 +1186,18 @@
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-annotations</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-mapreduce-client-app</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-mapreduce-client-core</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-mapreduce-client-jobclient</artifactId>
+ </exclusion>
<exclusion>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-client</artifactId>