Persistent resource volumes

[Maxime Brugidou <ma...@gmail.com>]

Hi,

We have been talking more and more about the long-awaited feature at
https://issues.apache.org/jira/browse/MESOS-1554

I started looking at what have been done in the APIs and as far as I
understand framework will be able to reserved and unreserve resources with
DiskInfo. Each DiskInfo has a persistent id and a volume. The host_path in
DiskInfo is ignored (only used for containerInfo).

Question: how could can we tell the mesos slave which disks to allocate and
how do we identify disks? My slaves have multiple disks mounted, with
different characteristics and sizes. Some are more appropriate for HDFS use
and some for Cassandra. Right now everything is done out of band but I
wonder how this could be done with persistent resources.

Same question with dynamic reservations: how can I reserve a disk and not
just some disk space quota? I clearly don't want my applications to all
work on the sane disk (really bad for IO).

Maybe I missed the design doc or there is a more detailed description of
the future feature somewhere?

Thanks for your help,
Maxime

Re: Persistent resource volumes

[Maxime Brugidou <ma...@gmail.com>]

Thanks i didn't know about MESOS-191. I think that it is indeed not easy to
manage the disk resource. The simplest solution would be in addition to
offer "space" on the workspace volume, to simply dedicate other
volumes/disks/spindles (configured in the slave) to some frameworks. The
DiskInfo would identify the disk by mount point?

The longer-term solution would be to:
* attach metadata to the DiskInfo for disk type and speed (ssd, 5krpm,
15krpm...) In addition to size, this could be easy to standardize and add
in the API
* allow frameworks to take over the full disk or just part of it with I/O
isolation
* allow frameworks to use a mounted and formatted disk partition or
directly the device (some frameworks could prefer to format the device the
way they prefer for special filesystems)
On Apr 1, 2015 10:18 AM, "Adam Bordelon" <ad...@mesosphere.io> wrote:

> So far the "disk" resource reported by Mesos refers to the disk where
> sandboxes will be created. By default, persistent volumes will be created
> under the same slave work_dir. We could pretty easily provide a separate
> flag so that all persistent volumes can go on a different disk than all
> sandboxes, but the real answer would be proper multi-disk support in the
> resource offers, and then each framework can choose which disk to use for
> its sandbox and/or volumes. Multi-disk support is on the roadmap, but I
> don't know if there's any design/doc for it yet besides MESOS-191
> <https://issues.apache.org/jira/browse/MESOS-191>.
>
> You might also be interested in MESOS-2406
> <https://issues.apache.org/jira/browse/MESOS-2406> for migrating
> pre-existing data in Mesos persistent volumes.
>
> On Tue, Mar 31, 2015 at 11:35 PM, Maxime Brugidou <
> maxime.brugidou@gmail.com> wrote:
>
>> Hi,
>>
>> We have been talking more and more about the long-awaited feature at
>> https://issues.apache.org/jira/browse/MESOS-1554
>>
>> I started looking at what have been done in the APIs and as far as I
>> understand framework will be able to reserved and unreserve resources with
>> DiskInfo. Each DiskInfo has a persistent id and a volume. The host_path in
>> DiskInfo is ignored (only used for containerInfo).
>>
>> Question: how could can we tell the mesos slave which disks to allocate
>> and how do we identify disks? My slaves have multiple disks mounted, with
>> different characteristics and sizes. Some are more appropriate for HDFS use
>> and some for Cassandra. Right now everything is done out of band but I
>> wonder how this could be done with persistent resources.
>>
>> Same question with dynamic reservations: how can I reserve a disk and not
>> just some disk space quota? I clearly don't want my applications to all
>> work on the sane disk (really bad for IO).
>>
>> Maybe I missed the design doc or there is a more detailed description of
>> the future feature somewhere?
>>
>> Thanks for your help,
>> Maxime
>>
>
>

Re: Persistent resource volumes

[Adam Bordelon <ad...@mesosphere.io>]

So far the "disk" resource reported by Mesos refers to the disk where
sandboxes will be created. By default, persistent volumes will be created
under the same slave work_dir. We could pretty easily provide a separate
flag so that all persistent volumes can go on a different disk than all
sandboxes, but the real answer would be proper multi-disk support in the
resource offers, and then each framework can choose which disk to use for
its sandbox and/or volumes. Multi-disk support is on the roadmap, but I
don't know if there's any design/doc for it yet besides MESOS-191
<https://issues.apache.org/jira/browse/MESOS-191>.

You might also be interested in MESOS-2406
<https://issues.apache.org/jira/browse/MESOS-2406> for migrating
pre-existing data in Mesos persistent volumes.

On Tue, Mar 31, 2015 at 11:35 PM, Maxime Brugidou <maxime.brugidou@gmail.com
> wrote:

> Hi,
>
> We have been talking more and more about the long-awaited feature at
> https://issues.apache.org/jira/browse/MESOS-1554
>
> I started looking at what have been done in the APIs and as far as I
> understand framework will be able to reserved and unreserve resources with
> DiskInfo. Each DiskInfo has a persistent id and a volume. The host_path in
> DiskInfo is ignored (only used for containerInfo).
>
> Question: how could can we tell the mesos slave which disks to allocate
> and how do we identify disks? My slaves have multiple disks mounted, with
> different characteristics and sizes. Some are more appropriate for HDFS use
> and some for Cassandra. Right now everything is done out of band but I
> wonder how this could be done with persistent resources.
>
> Same question with dynamic reservations: how can I reserve a disk and not
> just some disk space quota? I clearly don't want my applications to all
> work on the sane disk (really bad for IO).
>
> Maybe I missed the design doc or there is a more detailed description of
> the future feature somewhere?
>
> Thanks for your help,
> Maxime
>