You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2021/02/24 02:37:04 UTC
[spark] branch branch-3.1 updated: [MINOR][DOCS][K8S] Use
hadoop-aws 3.2.0 in K8s example
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.1 by this push:
new a377b0b [MINOR][DOCS][K8S] Use hadoop-aws 3.2.0 in K8s example
a377b0b is described below
commit a377b0b9cd73f58361913939d333bc7b7c5307cb
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Wed Feb 24 11:34:52 2021 +0900
[MINOR][DOCS][K8S] Use hadoop-aws 3.2.0 in K8s example
### What changes were proposed in this pull request?
This PR aims to update `Hadoop` dependency in K8S doc example.
### Why are the changes needed?
Apache Spark 3.1.1 is using Apache Hadoop 3.2.0 by default.
### Does this PR introduce _any_ user-facing change?
No. This is a doc-only change.
### How was this patch tested?
N/A
Closes #31629 from dongjoon-hyun/minor-doc-3.1.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: HyukjinKwon <gu...@apache.org>
---
docs/running-on-kubernetes.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md
index cf5c016..b9a018a 100644
--- a/docs/running-on-kubernetes.md
+++ b/docs/running-on-kubernetes.md
@@ -212,7 +212,7 @@ A typical example of this using S3 is via passing the following options:
```
...
---packages com.amazonaws:aws-java-sdk:1.7.4,org.apache.hadoop:hadoop-aws:2.7.6
+--packages org.apache.hadoop:hadoop-aws:3.2.0
--conf spark.kubernetes.file.upload.path=s3a://<s3-bucket>/path
--conf spark.hadoop.fs.s3a.access.key=...
--conf spark.hadoop.fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org