You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tez.apache.org by "Gurleen S Dhody (Jira)" <ji...@apache.org> on 2020/06/09 01:46:00 UTC

[jira] [Created] (TEZ-4193) Async DIspatcher drained property

Gurleen S Dhody created TEZ-4193:
------------------------------------

             Summary: Async DIspatcher drained property
                 Key: TEZ-4193
                 URL: https://issues.apache.org/jira/browse/TEZ-4193
             Project: Apache Tez
          Issue Type: Bug
    Affects Versions: 0.9.3
            Reporter: Gurleen S Dhody
             Fix For: 0.9.3


[https://github.com/apache/tez/blob/cf111b1c896db94d40ea92fc6da2a6fcce356114/tez-common/src/main/java/org/apache/tez/common/AsyncDispatcher.java#L318]
 The drained property should be set after all dispatchers in the handle() [https://github.com/apache/tez/blob/cf111b1c896db94d40ea92fc6da2a6fcce356114/tez-common/src/main/java/org/apache/tez/common/AsyncDispatcher.java#L311]

function are checked for.

We should set 
{code:java}
// drained = false;
{code}
 

[https://github.com/apache/tez/blob/cf111b1c896db94d40ea92fc6da2a6fcce356114/tez-common/src/main/java/org/apache/tez/common/AsyncDispatcher.java#L332 |https://github.com/apache/tez/blob/cf111b1c896db94d40ea92fc6da2a6fcce356114/tez-common/src/main/java/org/apache/tez/common/AsyncDispatcher.java#L332]here.


Reason is that the thread on the event queue might already be empty and waiting on

[https://github.com/apache/tez/blob/cf111b1c896db94d40ea92fc6da2a6fcce356114/tez-common/src/main/java/org/apache/tez/common/AsyncDispatcher.java#L106 |https://github.com/apache/tez/blob/cf111b1c896db94d40ea92fc6da2a6fcce356114/tez-common/src/main/java/org/apache/tez/common/AsyncDispatcher.java#L106]new event.

Setting drained as false but passing the event to another dispatcher will cause issue when stopping this dispatcher service, when drainEventsOnStop is set true. The thread will be stuck in queue and drained property will be false causing a deadlock here 

[https://github.com/apache/tez/blob/cf111b1c896db94d40ea92fc6da2a6fcce356114/tez-common/src/main/java/org/apache/tez/common/AsyncDispatcher.java#L146]


Needs to be addressed in [https://github.com/apache/tez/blob/cf111b1c896db94d40ea92fc6da2a6fcce356114/tez-common/src/main/java/org/apache/tez/common/AsyncDispatcherConcurrent.java] in AsyncConcurrecyDispatcher as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)