You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Scott Rankin <sr...@motus.com> on 2016/01/11 21:12:24 UTC
Linking mesos tasks to Docker containers
Hi all,
I’m trying to put together a monitoring system for our Marathon/Mesos/Docker app infrastructure, and I’m having some trouble linking between Mesos and Docker. Mesos has a task ID (like app_auth.6fb24412-af26-11e5-9900-02d62c7c9807), but the only place that shows up in the Docker container is in the MESOS_TASK_ID environment variable passed to the Docker containers. I see that the Docker containers are created using the format mesos-<slaveid>-<something>, but I cannot find the <something> value anywhere in the metadata from the master or the slave.
Am I missing something, or is there any way that I can configure Mesos to add a label or something to the Docker container so that our monitoring scripts can tie the Docker container to the Mesos task?
Thanks!
Scott
SCOTT RANKIN
VP, Technology
Motus, LLC
Two Financial Center, 60 South Street, Boston, MA 02111
617.467.1931 (W) | srankin@motus.com<ma...@motus.com>
Follow us on LinkedIn<https://www.linkedin.com/company/motus-llc/> | Visit us at motus.com<http://www.motus.com/>
This email message contains information that Motus, LLC considers confidential and/or proprietary, or may later designate as confidential and proprietary. It is intended only for use of the individual or entity named above and should not be forwarded to any other persons or entities without the express consent of Motus, LLC, nor should it be used for any purpose other than in the course of any potential or actual business relationship with Motus, LLC. If the reader of this message is not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify sender immediately and destroy the original message.
Internal Revenue Service regulations require that certain types of written advice include a disclaimer. To the extent the preceding message contains advice relating to a Federal tax issue, unless expressly stated otherwise the advice is not intended or written to be used, and it cannot be used by the recipient or any other taxpayer, for the purpose of avoiding Federal tax penalties, and was not written to support the promotion or marketing of any transaction or matter discussed herein.
Re: Linking mesos tasks to Docker containers
Posted by Benjamin Mahler <bm...@apache.org>.
+tim, jie
On Mon, Jan 11, 2016 at 12:12 PM, Scott Rankin <sr...@motus.com> wrote:
> Hi all,
>
> I’m trying to put together a monitoring system for our
> Marathon/Mesos/Docker app infrastructure, and I’m having some trouble
> linking between Mesos and Docker. Mesos has a task ID
> (like app_auth.6fb24412-af26-11e5-9900-02d62c7c9807), but the only place
> that shows up in the Docker container is in the MESOS_TASK_ID environment
> variable passed to the Docker containers. I see that the Docker containers
> are created using the format mesos-<slaveid>-<something>, but I cannot find
> the <something> value anywhere in the metadata from the master or the
> slave.
>
> Am I missing something, or is there any way that I can configure Mesos to
> add a label or something to the Docker container so that our monitoring
> scripts can tie the Docker container to the Mesos task?
>
> Thanks!
> Scott
>
> SCOTT *RANKIN*
> VP, Technology
>
> *Motus, LLC*
> Two Financial Center, 60 South Street, Boston, MA 02111
> 617.467.1931 (W) | srankin@motus.com <rc...@motus.com>
>
>
>
> Follow us on LinkedIn <https://www.linkedin.com/company/motus-llc/> |
> Visit us at motus.com <http://www.motus.com/>
>
>
>
> This email message contains information that Motus, LLC considers
> confidential and/or proprietary, or may later designate as confidential and
> proprietary. It is intended only for use of the individual or entity named
> above and should not be forwarded to any other persons or entities without
> the express consent of Motus, LLC, nor should it be used for any purpose
> other than in the course of any potential or actual business relationship
> with Motus, LLC. If the reader of this message is not the intended
> recipient, or the employee or agent responsible to deliver it to the
> intended recipient, you are hereby notified that any dissemination,
> distribution, or copying of this communication is strictly prohibited. If
> you have received this communication in error, please notify sender
> immediately and destroy the original message.
>
> Internal Revenue Service regulations require that certain types of written
> advice include a disclaimer. To the extent the preceding message contains
> advice relating to a Federal tax issue, unless expressly stated otherwise
> the advice is not intended or written to be used, and it cannot be used by
> the recipient or any other taxpayer, for the purpose of avoiding Federal
> tax penalties, and was not written to support the promotion or marketing of
> any transaction or matter discussed herein.
>
Re: Linking mesos tasks to Docker containers
Posted by Kevin Klues <kl...@gmail.com>.
Not that /state.json will be deprecated soon. Use /state instead.
On Tue, Jan 12, 2016 at 4:21 AM, Scott Rankin <sr...@motus.com> wrote:
> Got it. Thanks Qian, haosdent – I had been looking at
> /monitor/statistics.json on the slave, but not state.json.
>
> Scott
>
> From: Qian Zhang <zh...@gmail.com>
> Reply-To: "user@mesos.apache.org" <us...@mesos.apache.org>
> Date: Monday, January 11, 2016 at 9:54 PM
> To: "user@mesos.apache.org" <us...@mesos.apache.org>
> Subject: Re: Linking mesos tasks to Docker containers
>
> I think slave state endpoint has the info you need:
> ...
> "executors": [
> {
> "container": "b824a7ad-35c6-48ab-9378-b180ed431fb5",
> <----- this is <something>
> ...
> "tasks": [
> {
> "executor_id": "",
> "framework_id":
> "83ced7f5-69b3-409b-abe5-a582a5d278cd-0000",
> "id":
> "app-docker1.f94cba69-b85e-11e5-bdf1-0242497320ff",
> "name": "app-docker1",
> ...
>
> So you can use task ID to locate the task, and each task must be under an
> executor, and you can get <something> from executors.container, then it can
> link to the Docker container name.
>
>
> 2016-01-12 10:44 GMT+08:00 haosdent <ha...@gmail.com>:
>>
>> could you found it in slave state http endpoint? I remember the hierarchy
>> should be framework->executors->container.
>>
>> On Tue, Jan 12, 2016 at 10:41 AM, haosdent <ha...@gmail.com> wrote:
>>>
>>> <something> is containerid here.
>>>
>>> On Tue, Jan 12, 2016 at 10:36 AM, Shuai Lin <li...@gmail.com>
>>> wrote:
>>>>
>>>> Take a look at
>>>> https://github.com/scrapinghub/marathon-apps-collectd-plugin, basically it
>>>> works like this:
>>>>
>>>> - there is a custom collectd (https://github.com/collectd/collectd)
>>>> plugin, written in python
>>>> - the plugin uses docker inspect api to query the MESOS_TASK_ID and
>>>> MARATHON_APP_ID environment variable for each container
>>>> - the plugin queries cpu/mem/network stats from docker stats api
>>>> - collectd sends the (marathon app, task id, stats) data to a storage
>>>> backend, e.g. graphite, elasticsearch, influxdb, etc.
>>>>
>>>> Hope that helps.
>>>>
>>>> Shuai
>>>>
>>>> On Tue, Jan 12, 2016 at 4:12 AM, Scott Rankin <sr...@motus.com> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I’m trying to put together a monitoring system for our
>>>>> Marathon/Mesos/Docker app infrastructure, and I’m having some trouble
>>>>> linking between Mesos and Docker. Mesos has a task ID (like
>>>>> app_auth.6fb24412-af26-11e5-9900-02d62c7c9807), but the only place that
>>>>> shows up in the Docker container is in the MESOS_TASK_ID environment
>>>>> variable passed to the Docker containers. I see that the Docker containers
>>>>> are created using the format mesos-<slaveid>-<something>, but I cannot find
>>>>> the <something> value anywhere in the metadata from the master or the slave.
>>>>>
>>>>> Am I missing something, or is there any way that I can configure Mesos
>>>>> to add a label or something to the Docker container so that our monitoring
>>>>> scripts can tie the Docker container to the Mesos task?
>>>>>
>>>>> Thanks!
>>>>> Scott
>>>>>
>>>>> SCOTT RANKIN
>>>>> VP, Technology
>>>>>
>>>>> Motus, LLC
>>>>> Two Financial Center, 60 South Street, Boston, MA 02111
>>>>> 617.467.1931 (W) | srankin@motus.com
>>>>>
>>>>>
>>>>>
>>>>> Follow us on LinkedIn | Visit us at motus.com
>>>>>
>>>>>
>>>>>
>>>>> This email message contains information that Motus, LLC considers
>>>>> confidential and/or proprietary, or may later designate as confidential and
>>>>> proprietary. It is intended only for use of the individual or entity named
>>>>> above and should not be forwarded to any other persons or entities without
>>>>> the express consent of Motus, LLC, nor should it be used for any purpose
>>>>> other than in the course of any potential or actual business relationship
>>>>> with Motus, LLC. If the reader of this message is not the intended
>>>>> recipient, or the employee or agent responsible to deliver it to the
>>>>> intended recipient, you are hereby notified that any dissemination,
>>>>> distribution, or copying of this communication is strictly prohibited. If
>>>>> you have received this communication in error, please notify sender
>>>>> immediately and destroy the original message.
>>>>>
>>>>> Internal Revenue Service regulations require that certain types of
>>>>> written advice include a disclaimer. To the extent the preceding message
>>>>> contains advice relating to a Federal tax issue, unless expressly stated
>>>>> otherwise the advice is not intended or written to be used, and it cannot be
>>>>> used by the recipient or any other taxpayer, for the purpose of avoiding
>>>>> Federal tax penalties, and was not written to support the promotion or
>>>>> marketing of any transaction or matter discussed herein.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>
>
> This email message contains information that Motus, LLC considers
> confidential and/or proprietary, or may later designate as confidential and
> proprietary. It is intended only for use of the individual or entity named
> above and should not be forwarded to any other persons or entities without
> the express consent of Motus, LLC, nor should it be used for any purpose
> other than in the course of any potential or actual business relationship
> with Motus, LLC. If the reader of this message is not the intended
> recipient, or the employee or agent responsible to deliver it to the
> intended recipient, you are hereby notified that any dissemination,
> distribution, or copying of this communication is strictly prohibited. If
> you have received this communication in error, please notify sender
> immediately and destroy the original message.
>
> Internal Revenue Service regulations require that certain types of written
> advice include a disclaimer. To the extent the preceding message contains
> advice relating to a Federal tax issue, unless expressly stated otherwise
> the advice is not intended or written to be used, and it cannot be used by
> the recipient or any other taxpayer, for the purpose of avoiding Federal tax
> penalties, and was not written to support the promotion or marketing of any
> transaction or matter discussed herein.
--
~Kevin
Re: Linking mesos tasks to Docker containers
Posted by Mauricio Garavaglia <ma...@gmail.com>.
Hi,
Is there any possibility that this could be exposed in the mesos UI?
Mauricio
On Tue, Jan 12, 2016 at 9:21 AM, Scott Rankin <sr...@motus.com> wrote:
> Got it. Thanks Qian, haosdent – I had been looking at
> /monitor/statistics.json on the slave, but not state.json.
>
> Scott
>
> From: Qian Zhang <zh...@gmail.com>
> Reply-To: "user@mesos.apache.org" <us...@mesos.apache.org>
> Date: Monday, January 11, 2016 at 9:54 PM
> To: "user@mesos.apache.org" <us...@mesos.apache.org>
> Subject: Re: Linking mesos tasks to Docker containers
>
> I think slave state endpoint has the info you need:
> ...
> "executors": [
> {
> "container": "b824a7ad-35c6-48ab-9378-b180ed431fb5",
> <----- this is <something>
> ...
> "tasks": [
> {
> "executor_id": "",
> "framework_id":
> "83ced7f5-69b3-409b-abe5-a582a5d278cd-0000",
> "id":
> "app-docker1.f94cba69-b85e-11e5-bdf1-0242497320ff",
> "name": "app-docker1",
> ...
>
> So you can use task ID to locate the task, and each task must be under an
> executor, and you can get <something> from executors.container, then it can
> link to the Docker container name.
>
>
> 2016-01-12 10:44 GMT+08:00 haosdent <ha...@gmail.com>:
>
>> could you found it in slave state http endpoint? I remember the
>> hierarchy should be framework->executors->container.
>>
>> On Tue, Jan 12, 2016 at 10:41 AM, haosdent <ha...@gmail.com> wrote:
>>
>>> <something> is containerid here.
>>>
>>> On Tue, Jan 12, 2016 at 10:36 AM, Shuai Lin <li...@gmail.com>
>>> wrote:
>>>
>>>> Take a look at
>>>> https://github.com/scrapinghub/marathon-apps-collectd-plugin,
>>>> basically it works like this:
>>>>
>>>> - there is a custom collectd (https://github.com/collectd/collectd)
>>>> plugin, written in python
>>>> - the plugin uses docker inspect api to query the MESOS_TASK_ID and
>>>> MARATHON_APP_ID environment variable for each container
>>>> - the plugin queries cpu/mem/network stats from docker stats api
>>>> - collectd sends the (marathon app, task id, stats) data to a storage
>>>> backend, e.g. graphite, elasticsearch, influxdb, etc.
>>>>
>>>> Hope that helps.
>>>>
>>>> Shuai
>>>>
>>>> On Tue, Jan 12, 2016 at 4:12 AM, Scott Rankin <sr...@motus.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I’m trying to put together a monitoring system for our
>>>>> Marathon/Mesos/Docker app infrastructure, and I’m having some trouble
>>>>> linking between Mesos and Docker. Mesos has a task ID
>>>>> (like app_auth.6fb24412-af26-11e5-9900-02d62c7c9807), but the only place
>>>>> that shows up in the Docker container is in the MESOS_TASK_ID environment
>>>>> variable passed to the Docker containers. I see that the Docker containers
>>>>> are created using the format mesos-<slaveid>-<something>, but I cannot find
>>>>> the <something> value anywhere in the metadata from the master or the
>>>>> slave.
>>>>>
>>>>> Am I missing something, or is there any way that I can configure Mesos
>>>>> to add a label or something to the Docker container so that our monitoring
>>>>> scripts can tie the Docker container to the Mesos task?
>>>>>
>>>>> Thanks!
>>>>> Scott
>>>>>
>>>>> SCOTT *RANKIN*
>>>>> VP, Technology
>>>>>
>>>>> *Motus, LLC*
>>>>> Two Financial Center, 60 South Street, Boston, MA 02111
>>>>> 617.467.1931 (W) | srankin@motus.com <rc...@motus.com>
>>>>>
>>>>>
>>>>>
>>>>> Follow us on LinkedIn <https://www.linkedin.com/company/motus-llc/> |
>>>>> Visit us at motus.com <http://www.motus.com/>
>>>>>
>>>>>
>>>>>
>>>>> This email message contains information that Motus, LLC considers
>>>>> confidential and/or proprietary, or may later designate as confidential and
>>>>> proprietary. It is intended only for use of the individual or entity named
>>>>> above and should not be forwarded to any other persons or entities without
>>>>> the express consent of Motus, LLC, nor should it be used for any purpose
>>>>> other than in the course of any potential or actual business relationship
>>>>> with Motus, LLC. If the reader of this message is not the intended
>>>>> recipient, or the employee or agent responsible to deliver it to the
>>>>> intended recipient, you are hereby notified that any dissemination,
>>>>> distribution, or copying of this communication is strictly prohibited. If
>>>>> you have received this communication in error, please notify sender
>>>>> immediately and destroy the original message.
>>>>>
>>>>> Internal Revenue Service regulations require that certain types of
>>>>> written advice include a disclaimer. To the extent the preceding message
>>>>> contains advice relating to a Federal tax issue, unless expressly stated
>>>>> otherwise the advice is not intended or written to be used, and it cannot
>>>>> be used by the recipient or any other taxpayer, for the purpose of avoiding
>>>>> Federal tax penalties, and was not written to support the promotion or
>>>>> marketing of any transaction or matter discussed herein.
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
> This email message contains information that Motus, LLC considers
> confidential and/or proprietary, or may later designate as confidential and
> proprietary. It is intended only for use of the individual or entity named
> above and should not be forwarded to any other persons or entities without
> the express consent of Motus, LLC, nor should it be used for any purpose
> other than in the course of any potential or actual business relationship
> with Motus, LLC. If the reader of this message is not the intended
> recipient, or the employee or agent responsible to deliver it to the
> intended recipient, you are hereby notified that any dissemination,
> distribution, or copying of this communication is strictly prohibited. If
> you have received this communication in error, please notify sender
> immediately and destroy the original message.
>
> Internal Revenue Service regulations require that certain types of written
> advice include a disclaimer. To the extent the preceding message contains
> advice relating to a Federal tax issue, unless expressly stated otherwise
> the advice is not intended or written to be used, and it cannot be used by
> the recipient or any other taxpayer, for the purpose of avoiding Federal
> tax penalties, and was not written to support the promotion or marketing of
> any transaction or matter discussed herein.
>
Re: Linking mesos tasks to Docker containers
Posted by Scott Rankin <sr...@motus.com>.
Got it. Thanks Qian, haosdent – I had been looking at /monitor/statistics.json on the slave, but not state.json.
Scott
From: Qian Zhang <zh...@gmail.com>>
Reply-To: "user@mesos.apache.org<ma...@mesos.apache.org>" <us...@mesos.apache.org>>
Date: Monday, January 11, 2016 at 9:54 PM
To: "user@mesos.apache.org<ma...@mesos.apache.org>" <us...@mesos.apache.org>>
Subject: Re: Linking mesos tasks to Docker containers
I think slave state endpoint has the info you need:
...
"executors": [
{
"container": "b824a7ad-35c6-48ab-9378-b180ed431fb5", <----- this is <something>
...
"tasks": [
{
"executor_id": "",
"framework_id": "83ced7f5-69b3-409b-abe5-a582a5d278cd-0000",
"id": "app-docker1.f94cba69-b85e-11e5-bdf1-0242497320ff",
"name": "app-docker1",
...
So you can use task ID to locate the task, and each task must be under an executor, and you can get <something> from executors.container, then it can link to the Docker container name.
2016-01-12 10:44 GMT+08:00 haosdent <ha...@gmail.com>>:
could you found it in slave state http endpoint? I remember the hierarchy should be framework->executors->container.
On Tue, Jan 12, 2016 at 10:41 AM, haosdent <ha...@gmail.com>> wrote:
<something> is containerid here.
On Tue, Jan 12, 2016 at 10:36 AM, Shuai Lin <li...@gmail.com>> wrote:
Take a look at https://github.com/scrapinghub/marathon-apps-collectd-plugin, basically it works like this:
- there is a custom collectd (https://github.com/collectd/collectd) plugin, written in python
- the plugin uses docker inspect api to query the MESOS_TASK_ID and MARATHON_APP_ID environment variable for each container
- the plugin queries cpu/mem/network stats from docker stats api
- collectd sends the (marathon app, task id, stats) data to a storage backend, e.g. graphite, elasticsearch, influxdb, etc.
Hope that helps.
Shuai
On Tue, Jan 12, 2016 at 4:12 AM, Scott Rankin <sr...@motus.com>> wrote:
Hi all,
I’m trying to put together a monitoring system for our Marathon/Mesos/Docker app infrastructure, and I’m having some trouble linking between Mesos and Docker. Mesos has a task ID (like app_auth.6fb24412-af26-11e5-9900-02d62c7c9807), but the only place that shows up in the Docker container is in the MESOS_TASK_ID environment variable passed to the Docker containers. I see that the Docker containers are created using the format mesos-<slaveid>-<something>, but I cannot find the <something> value anywhere in the metadata from the master or the slave.
Am I missing something, or is there any way that I can configure Mesos to add a label or something to the Docker container so that our monitoring scripts can tie the Docker container to the Mesos task?
Thanks!
Scott
SCOTT RANKIN
VP, Technology
Motus, LLC
Two Financial Center, 60 South Street, Boston, MA 02111
617.467.1931<tel:617.467.1931> (W) | srankin@motus.com<ma...@motus.com>
Follow us on LinkedIn<https://www.linkedin.com/company/motus-llc/> | Visit us at motus.com<http://www.motus.com/>
This email message contains information that Motus, LLC considers confidential and/or proprietary, or may later designate as confidential and proprietary. It is intended only for use of the individual or entity named above and should not be forwarded to any other persons or entities without the express consent of Motus, LLC, nor should it be used for any purpose other than in the course of any potential or actual business relationship with Motus, LLC. If the reader of this message is not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify sender immediately and destroy the original message.
Internal Revenue Service regulations require that certain types of written advice include a disclaimer. To the extent the preceding message contains advice relating to a Federal tax issue, unless expressly stated otherwise the advice is not intended or written to be used, and it cannot be used by the recipient or any other taxpayer, for the purpose of avoiding Federal tax penalties, and was not written to support the promotion or marketing of any transaction or matter discussed herein.
--
Best Regards,
Haosdent Huang
--
Best Regards,
Haosdent Huang
This email message contains information that Motus, LLC considers confidential and/or proprietary, or may later designate as confidential and proprietary. It is intended only for use of the individual or entity named above and should not be forwarded to any other persons or entities without the express consent of Motus, LLC, nor should it be used for any purpose other than in the course of any potential or actual business relationship with Motus, LLC. If the reader of this message is not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify sender immediately and destroy the original message.
Internal Revenue Service regulations require that certain types of written advice include a disclaimer. To the extent the preceding message contains advice relating to a Federal tax issue, unless expressly stated otherwise the advice is not intended or written to be used, and it cannot be used by the recipient or any other taxpayer, for the purpose of avoiding Federal tax penalties, and was not written to support the promotion or marketing of any transaction or matter discussed herein.
Re: Linking mesos tasks to Docker containers
Posted by Qian Zhang <zh...@gmail.com>.
I think slave state endpoint has the info you need:
...
"executors": [
{
"container": "b824a7ad-35c6-48ab-9378-b180ed431fb5",
<----- this is <something>
...
"tasks": [
{
"executor_id": "",
"framework_id":
"83ced7f5-69b3-409b-abe5-a582a5d278cd-0000",
"id":
"app-docker1.f94cba69-b85e-11e5-bdf1-0242497320ff",
"name": "app-docker1",
...
So you can use task ID to locate the task, and each task must be under an
executor, and you can get <something> from executors.container, then it can
link to the Docker container name.
2016-01-12 10:44 GMT+08:00 haosdent <ha...@gmail.com>:
> could you found it in slave state http endpoint? I remember the
> hierarchy should be framework->executors->container.
>
> On Tue, Jan 12, 2016 at 10:41 AM, haosdent <ha...@gmail.com> wrote:
>
>> <something> is containerid here.
>>
>> On Tue, Jan 12, 2016 at 10:36 AM, Shuai Lin <li...@gmail.com>
>> wrote:
>>
>>> Take a look at
>>> https://github.com/scrapinghub/marathon-apps-collectd-plugin, basically
>>> it works like this:
>>>
>>> - there is a custom collectd (https://github.com/collectd/collectd)
>>> plugin, written in python
>>> - the plugin uses docker inspect api to query the MESOS_TASK_ID and
>>> MARATHON_APP_ID environment variable for each container
>>> - the plugin queries cpu/mem/network stats from docker stats api
>>> - collectd sends the (marathon app, task id, stats) data to a storage
>>> backend, e.g. graphite, elasticsearch, influxdb, etc.
>>>
>>> Hope that helps.
>>>
>>> Shuai
>>>
>>> On Tue, Jan 12, 2016 at 4:12 AM, Scott Rankin <sr...@motus.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I’m trying to put together a monitoring system for our
>>>> Marathon/Mesos/Docker app infrastructure, and I’m having some trouble
>>>> linking between Mesos and Docker. Mesos has a task ID
>>>> (like app_auth.6fb24412-af26-11e5-9900-02d62c7c9807), but the only place
>>>> that shows up in the Docker container is in the MESOS_TASK_ID environment
>>>> variable passed to the Docker containers. I see that the Docker containers
>>>> are created using the format mesos-<slaveid>-<something>, but I cannot find
>>>> the <something> value anywhere in the metadata from the master or the
>>>> slave.
>>>>
>>>> Am I missing something, or is there any way that I can configure Mesos
>>>> to add a label or something to the Docker container so that our monitoring
>>>> scripts can tie the Docker container to the Mesos task?
>>>>
>>>> Thanks!
>>>> Scott
>>>>
>>>> SCOTT *RANKIN*
>>>> VP, Technology
>>>>
>>>> *Motus, LLC*
>>>> Two Financial Center, 60 South Street, Boston, MA 02111
>>>> 617.467.1931 (W) | srankin@motus.com <rc...@motus.com>
>>>>
>>>>
>>>>
>>>> Follow us on LinkedIn <https://www.linkedin.com/company/motus-llc/> |
>>>> Visit us at motus.com <http://www.motus.com/>
>>>>
>>>>
>>>>
>>>> This email message contains information that Motus, LLC considers
>>>> confidential and/or proprietary, or may later designate as confidential and
>>>> proprietary. It is intended only for use of the individual or entity named
>>>> above and should not be forwarded to any other persons or entities without
>>>> the express consent of Motus, LLC, nor should it be used for any purpose
>>>> other than in the course of any potential or actual business relationship
>>>> with Motus, LLC. If the reader of this message is not the intended
>>>> recipient, or the employee or agent responsible to deliver it to the
>>>> intended recipient, you are hereby notified that any dissemination,
>>>> distribution, or copying of this communication is strictly prohibited. If
>>>> you have received this communication in error, please notify sender
>>>> immediately and destroy the original message.
>>>>
>>>> Internal Revenue Service regulations require that certain types of
>>>> written advice include a disclaimer. To the extent the preceding message
>>>> contains advice relating to a Federal tax issue, unless expressly stated
>>>> otherwise the advice is not intended or written to be used, and it cannot
>>>> be used by the recipient or any other taxpayer, for the purpose of avoiding
>>>> Federal tax penalties, and was not written to support the promotion or
>>>> marketing of any transaction or matter discussed herein.
>>>>
>>>
>>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>
Re: Linking mesos tasks to Docker containers
Posted by haosdent <ha...@gmail.com>.
could you found it in slave state http endpoint? I remember the
hierarchy should be framework->executors->container.
On Tue, Jan 12, 2016 at 10:41 AM, haosdent <ha...@gmail.com> wrote:
> <something> is containerid here.
>
> On Tue, Jan 12, 2016 at 10:36 AM, Shuai Lin <li...@gmail.com>
> wrote:
>
>> Take a look at
>> https://github.com/scrapinghub/marathon-apps-collectd-plugin, basically
>> it works like this:
>>
>> - there is a custom collectd (https://github.com/collectd/collectd)
>> plugin, written in python
>> - the plugin uses docker inspect api to query the MESOS_TASK_ID and
>> MARATHON_APP_ID environment variable for each container
>> - the plugin queries cpu/mem/network stats from docker stats api
>> - collectd sends the (marathon app, task id, stats) data to a storage
>> backend, e.g. graphite, elasticsearch, influxdb, etc.
>>
>> Hope that helps.
>>
>> Shuai
>>
>> On Tue, Jan 12, 2016 at 4:12 AM, Scott Rankin <sr...@motus.com> wrote:
>>
>>> Hi all,
>>>
>>> I’m trying to put together a monitoring system for our
>>> Marathon/Mesos/Docker app infrastructure, and I’m having some trouble
>>> linking between Mesos and Docker. Mesos has a task ID
>>> (like app_auth.6fb24412-af26-11e5-9900-02d62c7c9807), but the only place
>>> that shows up in the Docker container is in the MESOS_TASK_ID environment
>>> variable passed to the Docker containers. I see that the Docker containers
>>> are created using the format mesos-<slaveid>-<something>, but I cannot find
>>> the <something> value anywhere in the metadata from the master or the
>>> slave.
>>>
>>> Am I missing something, or is there any way that I can configure Mesos
>>> to add a label or something to the Docker container so that our monitoring
>>> scripts can tie the Docker container to the Mesos task?
>>>
>>> Thanks!
>>> Scott
>>>
>>> SCOTT *RANKIN*
>>> VP, Technology
>>>
>>> *Motus, LLC*
>>> Two Financial Center, 60 South Street, Boston, MA 02111
>>> 617.467.1931 (W) | srankin@motus.com <rc...@motus.com>
>>>
>>>
>>>
>>> Follow us on LinkedIn <https://www.linkedin.com/company/motus-llc/> |
>>> Visit us at motus.com <http://www.motus.com/>
>>>
>>>
>>>
>>> This email message contains information that Motus, LLC considers
>>> confidential and/or proprietary, or may later designate as confidential and
>>> proprietary. It is intended only for use of the individual or entity named
>>> above and should not be forwarded to any other persons or entities without
>>> the express consent of Motus, LLC, nor should it be used for any purpose
>>> other than in the course of any potential or actual business relationship
>>> with Motus, LLC. If the reader of this message is not the intended
>>> recipient, or the employee or agent responsible to deliver it to the
>>> intended recipient, you are hereby notified that any dissemination,
>>> distribution, or copying of this communication is strictly prohibited. If
>>> you have received this communication in error, please notify sender
>>> immediately and destroy the original message.
>>>
>>> Internal Revenue Service regulations require that certain types of
>>> written advice include a disclaimer. To the extent the preceding message
>>> contains advice relating to a Federal tax issue, unless expressly stated
>>> otherwise the advice is not intended or written to be used, and it cannot
>>> be used by the recipient or any other taxpayer, for the purpose of avoiding
>>> Federal tax penalties, and was not written to support the promotion or
>>> marketing of any transaction or matter discussed herein.
>>>
>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>
--
Best Regards,
Haosdent Huang
Re: Linking mesos tasks to Docker containers
Posted by haosdent <ha...@gmail.com>.
<something> is containerid here.
On Tue, Jan 12, 2016 at 10:36 AM, Shuai Lin <li...@gmail.com> wrote:
> Take a look at
> https://github.com/scrapinghub/marathon-apps-collectd-plugin, basically
> it works like this:
>
> - there is a custom collectd (https://github.com/collectd/collectd)
> plugin, written in python
> - the plugin uses docker inspect api to query the MESOS_TASK_ID and
> MARATHON_APP_ID environment variable for each container
> - the plugin queries cpu/mem/network stats from docker stats api
> - collectd sends the (marathon app, task id, stats) data to a storage
> backend, e.g. graphite, elasticsearch, influxdb, etc.
>
> Hope that helps.
>
> Shuai
>
> On Tue, Jan 12, 2016 at 4:12 AM, Scott Rankin <sr...@motus.com> wrote:
>
>> Hi all,
>>
>> I’m trying to put together a monitoring system for our
>> Marathon/Mesos/Docker app infrastructure, and I’m having some trouble
>> linking between Mesos and Docker. Mesos has a task ID
>> (like app_auth.6fb24412-af26-11e5-9900-02d62c7c9807), but the only place
>> that shows up in the Docker container is in the MESOS_TASK_ID environment
>> variable passed to the Docker containers. I see that the Docker containers
>> are created using the format mesos-<slaveid>-<something>, but I cannot find
>> the <something> value anywhere in the metadata from the master or the
>> slave.
>>
>> Am I missing something, or is there any way that I can configure Mesos to
>> add a label or something to the Docker container so that our monitoring
>> scripts can tie the Docker container to the Mesos task?
>>
>> Thanks!
>> Scott
>>
>> SCOTT *RANKIN*
>> VP, Technology
>>
>> *Motus, LLC*
>> Two Financial Center, 60 South Street, Boston, MA 02111
>> 617.467.1931 (W) | srankin@motus.com <rc...@motus.com>
>>
>>
>>
>> Follow us on LinkedIn <https://www.linkedin.com/company/motus-llc/> |
>> Visit us at motus.com <http://www.motus.com/>
>>
>>
>>
>> This email message contains information that Motus, LLC considers
>> confidential and/or proprietary, or may later designate as confidential and
>> proprietary. It is intended only for use of the individual or entity named
>> above and should not be forwarded to any other persons or entities without
>> the express consent of Motus, LLC, nor should it be used for any purpose
>> other than in the course of any potential or actual business relationship
>> with Motus, LLC. If the reader of this message is not the intended
>> recipient, or the employee or agent responsible to deliver it to the
>> intended recipient, you are hereby notified that any dissemination,
>> distribution, or copying of this communication is strictly prohibited. If
>> you have received this communication in error, please notify sender
>> immediately and destroy the original message.
>>
>> Internal Revenue Service regulations require that certain types of
>> written advice include a disclaimer. To the extent the preceding message
>> contains advice relating to a Federal tax issue, unless expressly stated
>> otherwise the advice is not intended or written to be used, and it cannot
>> be used by the recipient or any other taxpayer, for the purpose of avoiding
>> Federal tax penalties, and was not written to support the promotion or
>> marketing of any transaction or matter discussed herein.
>>
>
>
--
Best Regards,
Haosdent Huang
Re: Linking mesos tasks to Docker containers
Posted by Shuai Lin <li...@gmail.com>.
Take a look at https://github.com/scrapinghub/marathon-apps-collectd-plugin,
basically it works like this:
- there is a custom collectd (https://github.com/collectd/collectd) plugin,
written in python
- the plugin uses docker inspect api to query the MESOS_TASK_ID and
MARATHON_APP_ID environment variable for each container
- the plugin queries cpu/mem/network stats from docker stats api
- collectd sends the (marathon app, task id, stats) data to a storage
backend, e.g. graphite, elasticsearch, influxdb, etc.
Hope that helps.
Shuai
On Tue, Jan 12, 2016 at 4:12 AM, Scott Rankin <sr...@motus.com> wrote:
> Hi all,
>
> I’m trying to put together a monitoring system for our
> Marathon/Mesos/Docker app infrastructure, and I’m having some trouble
> linking between Mesos and Docker. Mesos has a task ID
> (like app_auth.6fb24412-af26-11e5-9900-02d62c7c9807), but the only place
> that shows up in the Docker container is in the MESOS_TASK_ID environment
> variable passed to the Docker containers. I see that the Docker containers
> are created using the format mesos-<slaveid>-<something>, but I cannot find
> the <something> value anywhere in the metadata from the master or the
> slave.
>
> Am I missing something, or is there any way that I can configure Mesos to
> add a label or something to the Docker container so that our monitoring
> scripts can tie the Docker container to the Mesos task?
>
> Thanks!
> Scott
>
> SCOTT *RANKIN*
> VP, Technology
>
> *Motus, LLC*
> Two Financial Center, 60 South Street, Boston, MA 02111
> 617.467.1931 (W) | srankin@motus.com <rc...@motus.com>
>
>
>
> Follow us on LinkedIn <https://www.linkedin.com/company/motus-llc/> |
> Visit us at motus.com <http://www.motus.com/>
>
>
>
> This email message contains information that Motus, LLC considers
> confidential and/or proprietary, or may later designate as confidential and
> proprietary. It is intended only for use of the individual or entity named
> above and should not be forwarded to any other persons or entities without
> the express consent of Motus, LLC, nor should it be used for any purpose
> other than in the course of any potential or actual business relationship
> with Motus, LLC. If the reader of this message is not the intended
> recipient, or the employee or agent responsible to deliver it to the
> intended recipient, you are hereby notified that any dissemination,
> distribution, or copying of this communication is strictly prohibited. If
> you have received this communication in error, please notify sender
> immediately and destroy the original message.
>
> Internal Revenue Service regulations require that certain types of written
> advice include a disclaimer. To the extent the preceding message contains
> advice relating to a Federal tax issue, unless expressly stated otherwise
> the advice is not intended or written to be used, and it cannot be used by
> the recipient or any other taxpayer, for the purpose of avoiding Federal
> tax penalties, and was not written to support the promotion or marketing of
> any transaction or matter discussed herein.
>