You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Bharath Kumarasubramanian (JIRA)" <ji...@apache.org> on 2019/01/24 23:59:00 UTC

[jira] [Created] (SAMZA-2087) Use separate thread pools for AsyncStreamTaskAdapter and AsyncRunLoop

Bharath Kumarasubramanian created SAMZA-2087:
------------------------------------------------

             Summary: Use separate thread pools for AsyncStreamTaskAdapter and AsyncRunLoop
                 Key: SAMZA-2087
                 URL: https://issues.apache.org/jira/browse/SAMZA-2087
             Project: Samza
          Issue Type: Bug
          Components: container
            Reporter: Bharath Kumarasubramanian
            Assignee: Bharath Kumarasubramanian


Currently, AsyncStreamTaskAdapter and AsyncRunLoop share thread pools and the thread pool size is governed by {{job.container.thread.pool.size}}. This introduces disparity in the semantics of {{task.max.concurrency}} between async stream task vs adapted async stream task.

task.max.concurrency governs the parallelism within a task. In case of applications that directly implement AsyncStreamTask, this corresponds to maximum number of inflight messages for a task. However, in the case of applications that get adapted to AsyncStreamTask, the maximum number of inflight messages are bounded by the job.container.thread.pool.size. In fact, all the tasks within a container share the thread pool which results in even lesser parallelism within a task even though applications configure task.max.concurrency to a much greater number than job.container.thread.pool.size

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)