[jira] [Comment Edited] (MESOS-9172) Fetcher deadlock with duplicated URIs.

["James Peach (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-9172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598998#comment-16598998 ] 

James Peach edited comment on MESOS-9172 at 8/31/18 4:46 PM:
-------------------------------------------------------------

| [r/68587|https://reviews.apache.org/r/68587] | Fixed fetcher deadlock with duplicate URIs. |
| [r/68586|https://reviews.apache.org/r/68586] | Add the output file to the hash on CommandInfo::URI. |


was (Author: jamespeach):
| [r/68587|https://reviews.apache.org/*r/68587] | Fixed fetcher deadlock with duplicate URIs. |
| [r/68586|https://reviews.apache.org/*r/68586] | Add the output file to the hash on CommandInfo::URI. |

> Fetcher deadlock with duplicated URIs.
> --------------------------------------
>
>                 Key: MESOS-9172
>                 URL: https://issues.apache.org/jira/browse/MESOS-9172
>             Project: Mesos
>          Issue Type: Bug
>          Components: fetcher
>            Reporter: James Peach
>            Assignee: James Peach
>            Priority: Major
>
> If the fetcher cache is empty and you launch a task that contains duplicate URIs, the fetcher deadlocks waiting for the futures in {{FetcherProcess::_fetch}}.
> What happens is that when the fetcher is setting up the initial match of cache lookup futures in {{FetcherProcess::fetch}}, the duplicate URIs cause cache hits on the placeholder cache entries. This code is assuming that there is already an operation in flight that will populate the cache entry. However, the cache is currently empty - the placeholder entry is caused by a the duplicate in the task's URIs.
> When we await the futures in {{FetcherProcess::_fetch}}, we end up waiting for the future that indicated the cache entry becomes populated, but that won't ever happen because we need to make progress on the current fetching batch in order to populate the cache entry. At this point we are live-locked.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)