Re: [PR] [flink] Support BUCKET_LOAD_BALANCE shuffle strategy for table with bucket kyes. [fluss]

["leonardBang (via GitHub)" <gi...@apache.org>]

leonardBang commented on code in PR #4450:
URL: https://github.com/apache/fluss/pull/4450#discussion_r4217692864


##########
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/FlinkConnectorOptions.java:
##########
@@ -232,6 +232,7 @@ public class FlinkConnectorOptions {
                     .defaultValue(DistributionMode.AUTO)
                     .withDescription(
                             "Defines the distribution mode for writing data to the sink. Available options are:\n"
+                                    + "- BUCKET_LOAD_BALANCE: Shuffle data by bucket key with even load distribution across all downstream subtasks using LCM-based logical slot assignment. Records with the same bucket key always route to the same subtask. Requires 'bucket.key' to be defined. Suitable for tables where intra-bucket ordering is not required but even load distribution matters. Not supported for tables with the aggregation merge engine, because Undo Recovery requires each bucket to be written by exactly one subtask.\n"

Review Comment:
   Could we make the guarantee of `BUCKET_LOAD_BALANCE` slightly more precise here? 
   
   BUCKET_LOAD_BALANCE: Shuffles records by bucket key using an LCM-based logical-slot assignment. This improves sink-subtask utilization and expected load distribution, particularly when the bucket count and sink parallelism are not evenly divisible. Records with the same bucket key always route to the same subtask, while different bucket keys in the same physical bucket may be processed by different subtasks whose logical-slot ranges overlap that bucket. This mode does not guarantee that every physical bucket is distributed across all subtasks; when the bucket count is an exact multiple of the sink parallelism, it behaves like BUCKET mode. It requires bucket keys and does not preserve intra-bucket ordering. It is not supported for tables using the aggregation merge engine because Undo Recovery requires each physical bucket to be written by exactly one subtask.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org