[GitHub] [flink-table-store] LadyForest commented on a diff in pull request #165: [FLINK-27696] Add bin-pack strategy to split the whole bucket data files into several small splits

[GitBox <gi...@apache.org>]

LadyForest commented on code in PR #165:
URL: https://github.com/apache/flink-table-store/pull/165#discussion_r901383071


##########
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/FileStoreOptions.java:
##########
@@ -141,6 +141,20 @@ public class FileStoreOptions implements Serializable {
                                     .list(formatEnumOption(WriteMode.CHANGE_LOG))
                                     .build());
 
+    public static final ConfigOption<MemorySize> SOURCE_SPLIT_TARGET_SIZE =
+            ConfigOptions.key("source.split.target-size")
+                    .memoryType()
+                    .defaultValue(MemorySize.ofMebiBytes(128))
+                    .withDescription("Target size of a source split when scanning a bucket.");
+
+    public static final ConfigOption<MemorySize> SOURCE_SPLIT_OPEN_FILE_COST =
+            ConfigOptions.key("source.split.open-file-cost")
+                    .memoryType()
+                    .defaultValue(MemorySize.ofMebiBytes(4))
+                    .withDescription(
+                            "Open file cost of a source file. It is used to avoid reading"
+                                    + " too many files with a source split, which can be very slow.");
+
     private final Configuration options;
 
     public static Set<ConfigOption<?>> allOptions() {

Review Comment:
   add `SOURCE_SPLIT_TARGET_SIZE` and `SOURCE_SPLIT_OPEN_FILE_COST` to `allOptions`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org