[jira] [Commented] (TEZ-1187) Share Thread pools between different tasks

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15928679#comment-15928679 ] 

Siddharth Seth commented on TEZ-1187:
-------------------------------------

[~harishjp] - How will this be used in Inputs/Outputs/Processors. What is the interface on the ProcessorContext, InputContext, OuptutContext for the user code to actually get a handle of an ExecutorServce.

In terms of the thread pools, and deadlocks. One option is to make sure that a single task (or more likely every P, I, O) has a minimum number of threads available to it. After this, the additional threads can be shared across tasks. We can enforce some amount of fairness as tasks start giving up threads.

> Share Thread pools between different tasks
> ------------------------------------------
>
>                 Key: TEZ-1187
>                 URL: https://issues.apache.org/jira/browse/TEZ-1187
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Siddharth Seth
>            Assignee: Harish Jaiprakash
>         Attachments: TEZ-1187.WIP.01.patch
>
>
> Thread pools are used all over for fetchers, sort etc. When running a single task - this is already a problem, and gets worse when running multiple tasks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)