[jira] [Created] (MESOS-3918) Unified and pluggable URI fetching support.

["Jie Yu (JIRA)" <ji...@apache.org>]

Jie Yu created MESOS-3918:
-----------------------------

             Summary: Unified and pluggable URI fetching support.
                 Key: MESOS-3918
                 URL: https://issues.apache.org/jira/browse/MESOS-3918
             Project: Mesos
          Issue Type: Epic
            Reporter: Jie Yu


Fetcher was originally designed to fetch CommandInfo::URIs (e.g., executor binary) for executors/tasks. A recent refactor (MESOS-336) added caching support to the fetcher. The recent work on filesystem isolation/unified containerizer (MESOS-2840) requires Mesos to fetch filesystem images (e.g., APPC/DOCKER images) as well. The natural question is: can we leverage the fetcher to fetch those filesystem images (and cache them accordingly)? Unfortunately, the existing fetcher interface is tightly coupled with CommandInfo::URIs for executors/tasks, making it very hard to be re used to fetch/cache filesystem images.

Another motivation for the refactor is that we want to extend the fetcher to support more types of schemes. For instance, we want to support magnet URI to enable p2p fetching. This is in fact quite important for operating a large cluster (MESOS-3596). The goal here is to allow fetcher to be extended (e.g., using modules) so that operators can add custom fetching support.

The main design goal here is to decouple artifacts fetching from artifacts cache management. We can make artifacts fetching extensible (e.g. to support p2p fetching), and solve the cache management part later.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)