You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2026/08/06 13:45:23 UTC
[XS] Change in asterixdb[master]: [NO ISSUE][EXT] Cap COPY TO max-file-size at 5GB
From Rithwik Koul <ri...@couchbase.com>:
Rithwik Koul has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21498?usp=email )
Change subject: [NO ISSUE][EXT] Cap COPY TO max-file-size at 5GB
......................................................................
[NO ISSUE][EXT] Cap COPY TO max-file-size at 5GB
- user model changes: yes
- storage format changes: no
- interface changes: no
Details:
Lower the upper bound of the max-file-size WITH option from 5TB to 5GB,
the largest object a single-part upload accepts. The previous bound
tracked the maximum object size of a multipart upload, which is not
what the option is for: a cap that large leaves each file too big to be
consumed comfortably by downstream readers, and no comparable system
allows one. Redshift caps MAXFILESIZE at 6.2GB and Snowflake caps
MAX_FILE_SIZE at 5GB.
The lower bound of 5MB is unchanged.
Ext-ref: MB-70477
Change-Id: I6acc84ee6dae890dc5266b68678ef8d6a6e9e1e0
---
M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/util/WriterMaxFileSizeValidationTest.java
2 files changed, 3 insertions(+), 3 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/98/21498/1
diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
index d115e2f..2b9e29d 100644
--- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
+++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
@@ -388,7 +388,7 @@
public static final long WRITER_MAX_FILE_SIZE_MINIMUM =
StorageUtil.getLongSizeInBytes(5, StorageUtil.StorageUnit.MEGABYTE);
public static final long WRITER_MAX_FILE_SIZE_MAXIMUM =
- StorageUtil.getLongSizeInBytes(5, StorageUtil.StorageUnit.TERABYTE);
+ StorageUtil.getLongSizeInBytes(5, StorageUtil.StorageUnit.GIGABYTE);
public static final Set<String> WRITER_SUPPORTED_FORMATS;
public static final Set<String> WRITER_SUPPORTED_ADAPTERS;
public static final Set<String> TEXTUAL_WRITER_SUPPORTED_COMPRESSION;
diff --git a/asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/util/WriterMaxFileSizeValidationTest.java b/asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/util/WriterMaxFileSizeValidationTest.java
index 13966af..0c9f9ee 100644
--- a/asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/util/WriterMaxFileSizeValidationTest.java
+++ b/asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/util/WriterMaxFileSizeValidationTest.java
@@ -44,8 +44,8 @@
validate("5MB");
validate("128MB");
validate("1GB");
- validate("8.5GB");
- validate("5TB");
+ validate("4.5GB");
+ validate("5GB");
validate("10485760");
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21498?usp=email
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email
Gerrit-MessageType: newchange
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I6acc84ee6dae890dc5266b68678ef8d6a6e9e1e0
Gerrit-Change-Number: 21498
Gerrit-PatchSet: 1
Gerrit-Owner: Rithwik Koul <ri...@couchbase.com>