[jira] [Created] (KAFKA-12959) Prioritize assigning standby tasks to threads without any active tasks

["Ravi Bhardwaj (Jira)" <ji...@apache.org>]

Ravi Bhardwaj created KAFKA-12959:
-------------------------------------

             Summary: Prioritize assigning standby tasks to threads without any active tasks
                 Key: KAFKA-12959
                 URL: https://issues.apache.org/jira/browse/KAFKA-12959
             Project: Kafka
          Issue Type: Improvement
          Components: streams
    Affects Versions: 2.8.0
            Reporter: Ravi Bhardwaj
         Attachments: app1.log, app2.log

Currently while distributing the standby tasks streams does not check if there are threads without any tasks or with less number of tasks. This can lead to few threads getting assigned both active and standby tasks when are threads within the same instance without any tasks assigned.

Example:
{code:java}
App 1:
[wordcount-lambda-example-client-StreamThread-1] Handle new assignment with:
 New active tasks: [0_1]
 New standby tasks: [1_0]
 Existing active tasks: []
 Existing standby tasks: [1_0]
[wordcount-lambda-example-client-StreamThread-2] Handle new assignment with:
 New active tasks: [1_1]
 New standby tasks: []
 Existing active tasks: [1_1]
 Existing standby tasks: []
[wordcount-lambda-example-client-StreamThread-3] Handle new assignment with:
 New active tasks: []
 New standby tasks: []
 Existing active tasks: []
 Existing standby tasks: []
[wordcount-lambda-example-client-StreamThread-4] Handle new assignment with:
 New active tasks: []
 New standby tasks: []
 Existing active tasks: []
 Existing standby tasks: []
{code}
 
{code:java}
App2:
[wordcount-lambda-example-client-StreamThread-1] Handle new assignment with:
 New active tasks: [1_0]
 New standby tasks: [1_1]
 Existing active tasks: []
 Existing standby tasks: [1_0, 1_1]
[wordcount-lambda-example-client-StreamThread-2] Handle new assignment with:
 New active tasks: [0_0]
 New standby tasks: []
 Existing active tasks: []
 Existing standby tasks: []
[wordcount-lambda-example-client-StreamThread-3] Handle new assignment with:
 New active tasks: []
 New standby tasks: []
 Existing active tasks: []
 Existing standby tasks: []
[wordcount-lambda-example-client-StreamThread-4] Handle new assignment with:
 New active tasks: []
 New standby tasks: []
 Existing active tasks: []
 Existing standby tasks: []
{code}
 The standby tasks in both apps is assigned to Thread-1 even though it has an active tasks when Thread-3 and Thread-4 didn't have any tasks assigned.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)