[jira] [Updated] (TEZ-3245) Race condition between addKnowInput and clearAndGetOnepartition of InputHost

["Zhiyuan Yang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhiyuan Yang updated TEZ-3245:
------------------------------
    Summary: Race condition between addKnowInput and clearAndGetOnepartition of InputHost  (was: Data race between addKnowInput and clearAndGetOnepartition of InputHost)

> Race condition between addKnowInput and clearAndGetOnepartition of InputHost
> ----------------------------------------------------------------------------
>
>                 Key: TEZ-3245
>                 URL: https://issues.apache.org/jira/browse/TEZ-3245
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.9.0
>            Reporter: Zhiyuan Yang
>            Assignee: Zhiyuan Yang
>             Fix For: 0.9.0
>
>         Attachments: TEZ-3245.1.patch
>
>
> In ShuffleManager::addKnownInput, InputHost::addKnowInput is invoked without locking. At the same time, InputHost::clearAndGetOnePartition can be invoked with lock(ShuffleManager.RunShuffleCallable::callInternal => ShuffleManager::constructFetcherForHost => InputHosst::clearAndGetOnePartition). The method clearAndGetOnePartition does two things: drain input attempt of a host and then delete the entry for this host. If addKnownInput happens between these two operation, the added input will be deleted without being handled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)