[GitHub] [kafka] C0urante commented on a change in pull request #10315: KAFKA-12463: Update default sink task partition assignor to cooperative sticky assignor

[GitBox <gi...@apache.org>]

C0urante commented on a change in pull request #10315:
URL: https://github.com/apache/kafka/pull/10315#discussion_r597034374



##########
File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java
##########
@@ -698,6 +700,8 @@ private WorkerTask buildWorkerTask(ClusterConfigState configState,
         consumerProps.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
         consumerProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.ByteArrayDeserializer");
         consumerProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.ByteArrayDeserializer");
+        consumerProps.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
+            CooperativeStickyAssignor.class.getName() + "," + RangeAssignor.class.getName());

Review comment:
       Yeah, a comment is definitely warranted, good call.
   
   We might use the `RoundRobinAssignor` but don't think we have to settle for that just yet. There's a couple of bugs/improvements in the way, but at least one of them is being actively worked on (https://github.com/apache/kafka/pull/10345) and I'm happy to take care of the other.
   
   I converted this to a draft since it's definitely not ready for merge yet and it's unclear which path forward we might want to take. We might want to reason through this carefully since we only get one shot to do this kind of automated upgrade before the next one gets more complicated (the list of assignors will grow from one to two this time around; it'll either have to grow from two to three the next, or we'll have to risk breaking changes for users who skip an upgrade step).
   
   I'll try to do an analysis of the pros/cons of the `CooperativeStickyAssignor` vs the `RoundRobinAssignor` with regards to sink connectors on the Jira ticket; I'm actually leaning a tiny bit toward the `RoundRobinAssignor` but I might be suffering from tunnel vision due to recent misadventures with other assignors.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org