You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by pn...@apache.org on 2026/06/03 06:17:13 UTC
(flink) 02/02: [FLINK-39811][s3] Document adjusted part size for s5cmd
This is an automated email from the ASF dual-hosted git repository.
pnowojski pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 8ca97a33d25080bf06db74fb45b2df1fa921c4d6
Author: Efrat Levitan <el...@confluent.io>
AuthorDate: Mon Jun 1 15:51:37 2026 +0300
[FLINK-39811][s3] Document adjusted part size for s5cmd
---
docs/content.zh/docs/deployment/filesystems/s3.md | 2 ++
docs/content/docs/deployment/filesystems/s3.md | 2 ++
2 files changed, 4 insertions(+)
diff --git a/docs/content.zh/docs/deployment/filesystems/s3.md b/docs/content.zh/docs/deployment/filesystems/s3.md
index 79637524fcc..44e6dde5c58 100644
--- a/docs/content.zh/docs/deployment/filesystems/s3.md
+++ b/docs/content.zh/docs/deployment/filesystems/s3.md
@@ -294,6 +294,8 @@ s3.s5cmd.args: -r 0
s3.s5cmd.batch.max-size: 1024mb
# Maximum number of files that will be uploaded via a single s5cmd call.
s3.s5cmd.batch.max-files: 100
+# Use fileSize/concurrency for part size instead of the static default (50MiB)
+s3.s5cmd.adjust-part-size: true
```
Both `s3.s5cmd.batch.max-size` and `s3.s5cmd.batch.max-files` control resource usage of the `s5cmd` binary to prevent it from overloading the task manager.
diff --git a/docs/content/docs/deployment/filesystems/s3.md b/docs/content/docs/deployment/filesystems/s3.md
index 79420f1e663..95e8f33d082 100644
--- a/docs/content/docs/deployment/filesystems/s3.md
+++ b/docs/content/docs/deployment/filesystems/s3.md
@@ -294,6 +294,8 @@ s3.s5cmd.args: -r 0
s3.s5cmd.batch.max-size: 1024mb
# Maximum number of files that will be uploaded via a single s5cmd call.
s3.s5cmd.batch.max-files: 100
+# Use fileSize/concurrency for part size instead of the static default (50MiB)
+s3.s5cmd.adjust-part-size: true
```
Both `s3.s5cmd.batch.max-size` and `s3.s5cmd.batch.max-files` control resource usage of the `s5cmd` binary to prevent it from overloading the task manager.