[I] Introduce a new sink distribution mode `BUCKET_LOAD_BALANCE` for the Flink connector. [fluss]

["loserwang1024 (via GitHub)" <gi...@apache.org>]

loserwang1024 opened a new issue, #4449:
URL: https://github.com/apache/fluss/issues/4449

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
   
   
   ### Motivation
   
   For tables with bucket keys, `BUCKET` mode provides good locality and ordering guarantees, but
   it can suffer from severe data skew when the number of buckets is not evenly divisible by the
   sink parallelism, or when some buckets are hotter than others. `NONE` mode avoids skew but
   breaks merge-engine semantics because records with the same key may be processed by different
   subtasks.
   
   
   Unlike the existing `BUCKET` mode (which maps each bucket to exactly one subtask and may leave
   subtasks idle), `BUCKET_LOAD_BALANCE` distributes records by bucket key using an LCM-based
   logical slot assignment so that every subtask receives traffic while still guaranteeing that
   records with the same bucket key route to the same subtask.
   
   ### Solution
   
   
   - Records with the same bucket key always route to the same subtask.
   - All subtasks receive data when `numBuckets` and `numChannels` are not evenly divisible.
   - Intra-bucket ordering is **not** preserved (records with different bucket keys in the same
     bucket may be processed by different subtasks).
   
   ### Anything else?
   
   _No response_
   
   ### Willingness to contribute
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Introduce a new sink distribution mode `BUCKET_LOAD_BALANCE` for the Flink connector. [fluss]

["loserwang1024 (via GitHub)" <gi...@apache.org>]

loserwang1024 closed issue #4449: Introduce a new sink distribution mode `BUCKET_LOAD_BALANCE` for the Flink connector.
URL: https://github.com/apache/fluss/issues/4449


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org