You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by "Khanduja, Vaibhav" <va...@emc.com> on 2015/06/05 18:56:20 UTC
Mesos/Marathon support for Docker extension
Recently docker pulled in code for supporting docker volume extensions.
The docker now (1.9 & above) through CLI can be specified volume plugin
name, which docker daemon connects get the actual storage. The plugin has
today reasonable hooks for maintaining and cleaning the storage. I was
wondering if there is any analysis done on the support of this in the
code? What I understand the slave code which calls in docker cli would
need this additional parameter to be passed into.
Thx
Re: Mesos/Marathon support for Docker extension
Posted by "Khanduja, Vaibhav" <va...@emc.com>.
Hi,
I managed to get this working for a plugin, though with a bit of an issue.
The Docker cli now accepts volume name without a ³/³, this is done as the
name is not absolute path but a name to be used by plugin. The plugin
would then return the mount point, to the Docker daemon. The slave Docker
code not like and throws an error. The condition check in docker.cpp,
ŠŠŠŠŠŠŠŠŠŠŠ.
string volumeConfig = volume.container_path();
if (volume.has_host_path()) {
if (!strings::startsWith(volume.host_path(), "/")) {
// Support mapping relative paths from the sandbox.
volumeConfig =
path::join(sandboxDirectory, volume.host_path()) + ":" +
volumeConfig;
} else {
ŠŠŠŠŠŠŠ..
requires a modification. The workaround for now is to use pass volume
name, as key value pair, but I suppose this won¹t fit well with persistent
volumes implementation.
Any suggestions? I can work on this if a ³Shepherd² can spend time with me.
Thanks
On 6/6/15, 8:44 PM, "Timothy Chen" <tn...@gmail.com> wrote:
>Yes.
>
>Tim
>
>Sent from my iPhone
>
>> On Jun 6, 2015, at 8:38 PM, Khanduja, Vaibhav
>><va...@emc.com> wrote:
>>
>> Hi Tim
>>
>> Are you referring to following pull request
>>
>> https://github.com/mesosphere/marathon/pull/798
>>
>> Thanks
>>
>>
>>> On 6/6/15, 8:21 PM, "Timothy Chen" <tn...@gmail.com> wrote:
>>>
>>> Hi Khadijah/Shuai,
>>>
>>> Mesos slave actually does invoke the docker cli directly for its
>>> integration.
>>>
>>> To support various options that docker will be adding we allowed
>>> arbitrary flags to be passed when launching a docker task (Params
>>>field I
>>> believe).
>>>
>>> Therefore if you know you have latest docker installed you can pass the
>>> Extra volume option and it should work.
>>>
>>> Tim
>>>
>>>> On Jun 6, 2015, at 7:51 PM, Shuai Lin <li...@gmail.com> wrote:
>>>>
>>>> Hi Khanduja,
>>>>
>>>> What I understand the slave code which calls in docker cli would need
>>>> this
>>>>> additional parameter to be passed into.
>>>>
>>>>
>>>> I don't think mesos slave invokes docker cli directly. It calls the
>>>> docker
>>>> api instead.
>>>>
>>>> Also Is there any official documentation/introduction to docker plugin
>>>> or
>>>> volume extensions? I found the following two pages when googling, but
>>>> non
>>>> of them has an official introduction to volume extension.
>>>>
>>>> https://clusterhq.com/2014/12/08/docker-extensions/
>>>> https://github.com/docker/docker/pull/13161
>>>>
>>>> Best Regards,
>>>> Shuai
>>>>
>>>>
>>>> On Sat, Jun 6, 2015 at 12:56 AM, Khanduja, Vaibhav
>>>> <vaibhav.khanduja@emc.com
>>>>> wrote:
>>>>
>>>>> Recently docker pulled in code for supporting docker volume
>>>>>extensions.
>>>>> The docker now (1.9 & above) through CLI can be specified volume
>>>>>plugin
>>>>> name, which docker daemon connects get the actual storage. The plugin
>>>>> has
>>>>> today reasonable hooks for maintaining and cleaning the storage. I
>>>>>was
>>>>> wondering if there is any analysis done on the support of this in the
>>>>> code? What I understand the slave code which calls in docker cli
>>>>>would
>>>>> need this additional parameter to be passed into.
>>>>>
>>>>> Thx
>>
Re: Mesos/Marathon support for Docker extension
Posted by Timothy Chen <tn...@gmail.com>.
Yes.
Tim
Sent from my iPhone
> On Jun 6, 2015, at 8:38 PM, Khanduja, Vaibhav <va...@emc.com> wrote:
>
> Hi Tim
>
> Are you referring to following pull request
>
> https://github.com/mesosphere/marathon/pull/798
>
> Thanks
>
>
>> On 6/6/15, 8:21 PM, "Timothy Chen" <tn...@gmail.com> wrote:
>>
>> Hi Khadijah/Shuai,
>>
>> Mesos slave actually does invoke the docker cli directly for its
>> integration.
>>
>> To support various options that docker will be adding we allowed
>> arbitrary flags to be passed when launching a docker task (Params field I
>> believe).
>>
>> Therefore if you know you have latest docker installed you can pass the
>> Extra volume option and it should work.
>>
>> Tim
>>
>>> On Jun 6, 2015, at 7:51 PM, Shuai Lin <li...@gmail.com> wrote:
>>>
>>> Hi Khanduja,
>>>
>>> What I understand the slave code which calls in docker cli would need
>>> this
>>>> additional parameter to be passed into.
>>>
>>>
>>> I don't think mesos slave invokes docker cli directly. It calls the
>>> docker
>>> api instead.
>>>
>>> Also Is there any official documentation/introduction to docker plugin
>>> or
>>> volume extensions? I found the following two pages when googling, but
>>> non
>>> of them has an official introduction to volume extension.
>>>
>>> https://clusterhq.com/2014/12/08/docker-extensions/
>>> https://github.com/docker/docker/pull/13161
>>>
>>> Best Regards,
>>> Shuai
>>>
>>>
>>> On Sat, Jun 6, 2015 at 12:56 AM, Khanduja, Vaibhav
>>> <vaibhav.khanduja@emc.com
>>>> wrote:
>>>
>>>> Recently docker pulled in code for supporting docker volume extensions.
>>>> The docker now (1.9 & above) through CLI can be specified volume plugin
>>>> name, which docker daemon connects get the actual storage. The plugin
>>>> has
>>>> today reasonable hooks for maintaining and cleaning the storage. I was
>>>> wondering if there is any analysis done on the support of this in the
>>>> code? What I understand the slave code which calls in docker cli would
>>>> need this additional parameter to be passed into.
>>>>
>>>> Thx
>
Re: Mesos/Marathon support for Docker extension
Posted by "Khanduja, Vaibhav" <va...@emc.com>.
Hi Tim
Are you referring to following pull request
https://github.com/mesosphere/marathon/pull/798
Thanks
On 6/6/15, 8:21 PM, "Timothy Chen" <tn...@gmail.com> wrote:
>Hi Khadijah/Shuai,
>
>Mesos slave actually does invoke the docker cli directly for its
>integration.
>
>To support various options that docker will be adding we allowed
>arbitrary flags to be passed when launching a docker task (Params field I
>believe).
>
>Therefore if you know you have latest docker installed you can pass the
>Extra volume option and it should work.
>
>Tim
>
>> On Jun 6, 2015, at 7:51 PM, Shuai Lin <li...@gmail.com> wrote:
>>
>> Hi Khanduja,
>>
>> What I understand the slave code which calls in docker cli would need
>>this
>>> additional parameter to be passed into.
>>
>>
>> I don't think mesos slave invokes docker cli directly. It calls the
>>docker
>> api instead.
>>
>> Also Is there any official documentation/introduction to docker plugin
>>or
>> volume extensions? I found the following two pages when googling, but
>>non
>> of them has an official introduction to volume extension.
>>
>> https://clusterhq.com/2014/12/08/docker-extensions/
>> https://github.com/docker/docker/pull/13161
>>
>> Best Regards,
>> Shuai
>>
>>
>> On Sat, Jun 6, 2015 at 12:56 AM, Khanduja, Vaibhav
>><vaibhav.khanduja@emc.com
>>> wrote:
>>
>>> Recently docker pulled in code for supporting docker volume extensions.
>>> The docker now (1.9 & above) through CLI can be specified volume plugin
>>> name, which docker daemon connects get the actual storage. The plugin
>>>has
>>> today reasonable hooks for maintaining and cleaning the storage. I was
>>> wondering if there is any analysis done on the support of this in the
>>> code? What I understand the slave code which calls in docker cli would
>>> need this additional parameter to be passed into.
>>>
>>> Thx
>>>
>>>
Re: Mesos/Marathon support for Docker extension
Posted by Timothy Chen <tn...@gmail.com>.
Hi Khadijah/Shuai,
Mesos slave actually does invoke the docker cli directly for its integration.
To support various options that docker will be adding we allowed arbitrary flags to be passed when launching a docker task (Params field I believe).
Therefore if you know you have latest docker installed you can pass the
Extra volume option and it should work.
Tim
> On Jun 6, 2015, at 7:51 PM, Shuai Lin <li...@gmail.com> wrote:
>
> Hi Khanduja,
>
> What I understand the slave code which calls in docker cli would need this
>> additional parameter to be passed into.
>
>
> I don't think mesos slave invokes docker cli directly. It calls the docker
> api instead.
>
> Also Is there any official documentation/introduction to docker plugin or
> volume extensions? I found the following two pages when googling, but non
> of them has an official introduction to volume extension.
>
> https://clusterhq.com/2014/12/08/docker-extensions/
> https://github.com/docker/docker/pull/13161
>
> Best Regards,
> Shuai
>
>
> On Sat, Jun 6, 2015 at 12:56 AM, Khanduja, Vaibhav <vaibhav.khanduja@emc.com
>> wrote:
>
>> Recently docker pulled in code for supporting docker volume extensions.
>> The docker now (1.9 & above) through CLI can be specified volume plugin
>> name, which docker daemon connects get the actual storage. The plugin has
>> today reasonable hooks for maintaining and cleaning the storage. I was
>> wondering if there is any analysis done on the support of this in the
>> code? What I understand the slave code which calls in docker cli would
>> need this additional parameter to be passed into.
>>
>> Thx
>>
>>
Re: Mesos/Marathon support for Docker extension
Posted by "Khanduja, Vaibhav" <va...@emc.com>.
Hi Shuai
>From the link below, I pulled out changes needed while invoking docker cli or docker api. This documentation is quite comprehensive and should tell what changes are needed.
https://github.com/docker/docker/pull/13161
……..
$ docker run -ti -v volumename:/data --volume-driver=flocker busy box sh
There is similar api change needed ..
…………..
Adding this plugin into Mesos would give all necessary storage extensions needed for Mesos/Marathon. Can we discuss to support this out?
Thanks
On 6/6/15, 7:51 PM, "Shuai Lin" <li...@gmail.com>> wrote:
Hi Khanduja,
What I understand the slave code which calls in docker cli would need this
additional parameter to be passed into.
I don't think mesos slave invokes docker cli directly. It calls the docker
api instead.
Also Is there any official documentation/introduction to docker plugin or
volume extensions? I found the following two pages when googling, but non
of them has an official introduction to volume extension.
https://clusterhq.com/2014/12/08/docker-extensions/
https://github.com/docker/docker/pull/13161
Best Regards,
Shuai
On Sat, Jun 6, 2015 at 12:56 AM, Khanduja, Vaibhav <va...@emc.com>
wrote:
Recently docker pulled in code for supporting docker volume extensions.
The docker now (1.9 & above) through CLI can be specified volume plugin
name, which docker daemon connects get the actual storage. The plugin has
today reasonable hooks for maintaining and cleaning the storage. I was
wondering if there is any analysis done on the support of this in the
code? What I understand the slave code which calls in docker cli would
need this additional parameter to be passed into.
Thx
Re: Mesos/Marathon support for Docker extension
Posted by Shuai Lin <li...@gmail.com>.
Hi Khanduja,
What I understand the slave code which calls in docker cli would need this
> additional parameter to be passed into.
I don't think mesos slave invokes docker cli directly. It calls the docker
api instead.
Also Is there any official documentation/introduction to docker plugin or
volume extensions? I found the following two pages when googling, but non
of them has an official introduction to volume extension.
https://clusterhq.com/2014/12/08/docker-extensions/
https://github.com/docker/docker/pull/13161
Best Regards,
Shuai
On Sat, Jun 6, 2015 at 12:56 AM, Khanduja, Vaibhav <vaibhav.khanduja@emc.com
> wrote:
> Recently docker pulled in code for supporting docker volume extensions.
> The docker now (1.9 & above) through CLI can be specified volume plugin
> name, which docker daemon connects get the actual storage. The plugin has
> today reasonable hooks for maintaining and cleaning the storage. I was
> wondering if there is any analysis done on the support of this in the
> code? What I understand the slave code which calls in docker cli would
> need this additional parameter to be passed into.
>
> Thx
>
>