[GitHub] [incubator-paimon] zhangjun0x01 commented on a diff in pull request #584: [FLINK-31338] support infer parallelism for flink table store

["zhangjun0x01 (via GitHub)" <gi...@apache.org>]

zhangjun0x01 commented on code in PR #584:
URL: https://github.com/apache/incubator-paimon/pull/584#discussion_r1138732118


##########
flink-table-store-flink/flink-table-store-flink-common/src/main/java/org/apache/flink/table/store/connector/FlinkConnectorOptions.java:
##########
@@ -82,6 +82,20 @@ public class FlinkConnectorOptions {
                                     + "By default, if this option is not defined, the planner will derive the parallelism "
                                     + "for each statement individually by also considering the global configuration.");
 
+    public static final ConfigOption<Boolean> INFER_SCAN_PARALLELISM =
+            ConfigOptions.key("scan.infer-parallelism")
+                    .booleanType()
+                    .defaultValue(false)
+                    .withDescription(
+                            "If it is false, parallelism of source are set by scan.parallelism. "
+                                    + "If it is true, source parallelism is inferred according to splits number (batch mode) or bucket number(streaming mode).");
+
+    public static final ConfigOption<Integer> INFER_SCAN_PARALLELISM_MAX =
+            ConfigOptions.key("scan.infer-parallelism.max")

Review Comment:
   Do you mean we delete the max parallelism  and let users adjust the number of split by `source.split.target-size` and `source.split.open-file-cost`? and  then get the parallelism by the splits number?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org