You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/06/26 22:56:17 UTC
[spark] branch master updated: [SPARK-44197][BUILD] Upgrade Hadoop to 3.3.6
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new cf3820bf48f [SPARK-44197][BUILD] Upgrade Hadoop to 3.3.6
cf3820bf48f is described below
commit cf3820bf48f0050f33cbf4525e6f3bf2fc76033f
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Mon Jun 26 15:56:07 2023 -0700
[SPARK-44197][BUILD] Upgrade Hadoop to 3.3.6
### What changes were proposed in this pull request?
This PR aims to upgrade Apache Hadoop dependency to 3.3.6.
### Why are the changes needed?
To bring the latest bug fixes for Apache Spark 3.5.0 and use the artifacts with SBOM
- NEWS: https://hadoop.apache.org/release/3.3.6.html
> It contains 117 bug fixes, improvements and enhancements since 3.3.5.
- DOCS: https://hadoop.apache.org/docs/r3.3.6/index.html
- CHANGELOG: https://hadoop.apache.org/docs/r3.3.6/hadoop-project-dist/hadoop-common/release/3.3.6/CHANGELOG.3.3.6.html
### Does this PR introduce _any_ user-facing change?
This is a dependency change.
### How was this patch tested?
Pass the CIs.
Closes #41744 from dongjoon-hyun/SPARK-44197.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
dev/deps/spark-deps-hadoop-3-hive-2.3 | 22 +++++++++++-----------
pom.xml | 2 +-
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 3c08d3da9fc..8d8fb88cb69 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -24,7 +24,7 @@ audience-annotations/0.5.0//audience-annotations-0.5.0.jar
avro-ipc/1.11.1//avro-ipc-1.11.1.jar
avro-mapred/1.11.1//avro-mapred-1.11.1.jar
avro/1.11.1//avro-1.11.1.jar
-aws-java-sdk-bundle/1.12.316//aws-java-sdk-bundle-1.12.316.jar
+aws-java-sdk-bundle/1.12.367//aws-java-sdk-bundle-1.12.367.jar
azure-data-lake-store-sdk/2.3.9//azure-data-lake-store-sdk-2.3.9.jar
azure-keyvault-core/1.0.0//azure-keyvault-core-1.0.0.jar
azure-storage/7.0.1//azure-storage-7.0.1.jar
@@ -66,16 +66,16 @@ gcs-connector/hadoop3-2.2.14/shaded/gcs-connector-hadoop3-2.2.14-shaded.jar
gmetric4j/1.0.10//gmetric4j-1.0.10.jar
gson/2.2.4//gson-2.2.4.jar
guava/14.0.1//guava-14.0.1.jar
-hadoop-aliyun/3.3.5//hadoop-aliyun-3.3.5.jar
-hadoop-annotations/3.3.5//hadoop-annotations-3.3.5.jar
-hadoop-aws/3.3.5//hadoop-aws-3.3.5.jar
-hadoop-azure-datalake/3.3.5//hadoop-azure-datalake-3.3.5.jar
-hadoop-azure/3.3.5//hadoop-azure-3.3.5.jar
-hadoop-client-api/3.3.5//hadoop-client-api-3.3.5.jar
-hadoop-client-runtime/3.3.5//hadoop-client-runtime-3.3.5.jar
-hadoop-cloud-storage/3.3.5//hadoop-cloud-storage-3.3.5.jar
+hadoop-aliyun/3.3.6//hadoop-aliyun-3.3.6.jar
+hadoop-annotations/3.3.6//hadoop-annotations-3.3.6.jar
+hadoop-aws/3.3.6//hadoop-aws-3.3.6.jar
+hadoop-azure-datalake/3.3.6//hadoop-azure-datalake-3.3.6.jar
+hadoop-azure/3.3.6//hadoop-azure-3.3.6.jar
+hadoop-client-api/3.3.6//hadoop-client-api-3.3.6.jar
+hadoop-client-runtime/3.3.6//hadoop-client-runtime-3.3.6.jar
+hadoop-cloud-storage/3.3.6//hadoop-cloud-storage-3.3.6.jar
hadoop-shaded-guava/1.1.1//hadoop-shaded-guava-1.1.1.jar
-hadoop-yarn-server-web-proxy/3.3.5//hadoop-yarn-server-web-proxy-3.3.5.jar
+hadoop-yarn-server-web-proxy/3.3.6//hadoop-yarn-server-web-proxy-3.3.6.jar
hive-beeline/2.3.9//hive-beeline-2.3.9.jar
hive-cli/2.3.9//hive-cli-2.3.9.jar
hive-common/2.3.9//hive-common-2.3.9.jar
@@ -125,7 +125,7 @@ jersey-container-servlet-core/2.36//jersey-container-servlet-core-2.36.jar
jersey-container-servlet/2.36//jersey-container-servlet-2.36.jar
jersey-hk2/2.36//jersey-hk2-2.36.jar
jersey-server/2.36//jersey-server-2.36.jar
-jettison/1.5.3//jettison-1.5.3.jar
+jettison/1.5.4//jettison-1.5.4.jar
jetty-util-ajax/9.4.51.v20230217//jetty-util-ajax-9.4.51.v20230217.jar
jetty-util/9.4.51.v20230217//jetty-util-9.4.51.v20230217.jar
jline/2.14.6//jline-2.14.6.jar
diff --git a/pom.xml b/pom.xml
index 592766b17ca..89bb7e89cca 100644
--- a/pom.xml
+++ b/pom.xml
@@ -122,7 +122,7 @@
<slf4j.version>2.0.7</slf4j.version>
<log4j.version>2.20.0</log4j.version>
<!-- make sure to update IsolatedClientLoader whenever this version is changed -->
- <hadoop.version>3.3.5</hadoop.version>
+ <hadoop.version>3.3.6</hadoop.version>
<!-- SPARK-41247: When updating `protobuf.version`, also need to update `protoVersion` in `SparkBuild.scala` -->
<protobuf.version>3.23.2</protobuf.version>
<protoc-jar-maven-plugin.version>3.11.4</protoc-jar-maven-plugin.version>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org