[GitHub] [hudi] f7753 commented on a diff in pull request #6218: [HUDI-4477] Add an option for bulk insert tasks to adjust parallelism

[GitBox <gi...@apache.org>]

f7753 commented on code in PR #6218:
URL: https://github.com/apache/hudi/pull/6218#discussion_r934107862


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java:
##########
@@ -487,6 +487,13 @@ private FlinkOptions() {
       .defaultValue(true) // default true for MOR write
       .withDescription("Schedule the compaction plan, enabled by default for MOR");
 
+  public static final ConfigOption<Integer> WRITE_BULK_INSERT_MAX_PARALLELISM = ConfigOptions
+          .key("write.bulk_insert.max.parallelism")
+          .intType()
+          .defaultValue(StreamGraphGenerator.DEFAULT_LOWER_BOUND_MAX_PARALLELISM)
+          .withDescription("The maximum number of flink write tasks, since the original " +

Review Comment:
   i see, how about `org.apache.flink.runtime.state.KeyGroupRangeAssignment.UPPER_BOUND_MAX_PARALLELISM`, which is hardcoded to 32768? i guess it will work for almost all the workloads.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org