Re: Review Request 68587: Fixed fetcher deadlock with duplicate URIs.

[Jiang Yan Xu <ya...@jxu.me>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68587/#review209704
-----------------------------------------------------------


Fix it, then Ship it!





src/slave/containerizer/fetcher.cpp
Lines 409-411 (patched)
<https://reviews.apache.org/r/68587/#comment294254>

    The crux here is that URIs with the same `value` but  are different in other fields like `executable` or `extract` can lead to multiple items in `entries` pointing to the same cache entry. Add it to the comment?



src/slave/containerizer/fetcher.cpp
Lines 421 (patched)
<https://reviews.apache.org/r/68587/#comment294252>

    Using `newEntries.at()` is more idiomatic even though we did check that the entry exists.


- Jiang Yan Xu


On Aug. 31, 2018, 9:47 a.m., James Peach wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68587/
> -----------------------------------------------------------
> 
> (Updated Aug. 31, 2018, 9:47 a.m.)
> 
> 
> Review request for mesos, Gilbert Song and Joseph Wu.
> 
> 
> Bugs: MESOS-9172
>     https://issues.apache.org/jira/browse/MESOS-9172
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> If a fetch request contains duplicate URIs that are not already
> in the cache, the fetcher would erroneously expect that some other
> fetch process is going to download that cache entry. It will then
> wait for a Future that will never complete.
> 
> The fix is to track whether the cache entry was created in this
> fetch, and in that case to simply allow the duplicate URI. In
> the fetcher, we check the cache before downloading so that a URIs
> can be fetched to distinct output files without being downloaded
> multiple times.
> 
> 
> Diffs
> -----
> 
>   src/launcher/fetcher.cpp ef8d7ebdcf6d5d15459fd3392cb99774f078347a 
>   src/slave/containerizer/fetcher.cpp 17f5388200c8341936cb4d7f8da67b5f286b727d 
>   src/tests/fetcher_tests.cpp f3ea7092635c88b9dddcc2998a4c7350fb56110c 
> 
> 
> Diff: https://reviews.apache.org/r/68587/diff/1/
> 
> 
> Testing
> -------
> 
> make check (Fedora 28)
> 
> 
> Thanks,
> 
> James Peach
> 
>