You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Jie Yu (JIRA)" <ji...@apache.org> on 2018/08/13 18:20:00 UTC
[jira] [Assigned] (MESOS-9142) CNI detach might fail due to missing
network config file.
[ https://issues.apache.org/jira/browse/MESOS-9142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jie Yu reassigned MESOS-9142:
-----------------------------
Assignee: Jie Yu
> CNI detach might fail due to missing network config file.
> ---------------------------------------------------------
>
> Key: MESOS-9142
> URL: https://issues.apache.org/jira/browse/MESOS-9142
> Project: Mesos
> Issue Type: Bug
> Affects Versions: 1.4.1, 1.5.1, 1.6.1, 1.7.0
> Reporter: Jie Yu
> Assignee: Jie Yu
> Priority: Major
>
> Observed this in one of the scale testing:
> If the container is destroyed while in PREPARING state, the network config file won't be there. The CNI detach in that case should return success because attach is not called yet.
> {code}
> Aug 08 23:52:03 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:52:03.762208 4045 slave.cpp:3562] Launching container 278d88a8-2bd1-4359-8c70-25dc90879cdc for executor 'testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a' of framework 4397555f-b8fd-478c-907a-7532e2f5e32e-0001
> Aug 08 23:53:03 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:53:03.232110 4047 containerizer.cpp:1211] Starting container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:53:03 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:53:03.249105 4049 provisioner.cpp:546] Provisioning image rootfs '/var/lib/mesos/slave/provisioner/containers/278d88a8-2bd1-4359-8c70-25dc90879cdc/backends/overlay/rootfses/25dc9ff3-a4ad-4b59-bdf7-02f7e741c729' for container 278d88a8-2bd1-4359-8c70-25dc90879cdc using overlay backend
> Aug 08 23:53:31 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:53:31.367316 4047 containerizer.cpp:2981] Transitioning the state of container 278d88a8-2bd1-4359-8c70-25dc90879cdc from PROVISIONING to PREPARING
> Aug 08 23:55:22 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:55:22.988091 4047 memory.cpp:479] Started listening for OOM events for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:55:23 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:55:23.196844 4047 memory.cpp:591] Started listening on 'low' memory pressure events for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:55:23 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:55:23.277029 4047 memory.cpp:591] Started listening on 'medium' memory pressure events for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:55:23 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:55:23.407688 4047 memory.cpp:591] Started listening on 'critical' memory pressure events for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:57:31 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:57:31.760859 4043 memory.cpp:199] Updated 'memory.soft_limit_in_bytes' to 64MB for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:57:31 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:57:31.878435 4043 memory.cpp:228] Updated 'memory.limit_in_bytes' to 64MB for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:57:35 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:57:35.892071 4044 cpu.cpp:101] Updated 'cpu.shares' to 112 (cpus 0.11) for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:57:36 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0808 23:57:36.132098 4044 cpu.cpp:121] Updated 'cpu.cfs_period_us' to 100ms and 'cpu.cfs_quota_us' to 11ms (cpus 0.11) for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:01.791821 4047 isolator_module.cpp:68] Container prepare: container_id[value: "278d88a8-2bd1-4359-8c70-25dc90879cdc"] container_config[directory: "/var/lib/mesos/slave/slaves/4397555f-b8fd-478c-907a-7532e2f5e32e-S0/frameworks/4397555f-b8fd-478c-907a-7532e2f5e32e-0001/executors/testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a/runs/278d88a8-2bd1-4359-8c70-25dc90879cdc" user: "root" rootfs: "/var/lib/mesos/slave/provisioner/containers/278d88a8-2bd1-4359-8c70-25dc90879cdc/backends/overlay/rootfses/25dc9ff3-a4ad-4b59-bdf7-02f7e741c729" docker { manifest { id: "d7066268059eb2499922833bd058e9e0b9da8365bcae4b85ee22296b02fa0489" parent: "f18ee96f0b1656cab52554b270f19e8df5046d307296d2146539c04565d67747" created: "2018-07-06T14:14:06.393355914Z" container: "b449414436d7bc63cc85dcd24abde14bcec1d92f54f2316212059ee2ed7f3a65" container_config { Hostname: "b449414436d7" Env: "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" User: "" Cmd: "/bin/sh" Cmd: "-c" Cmd: "#(nop) " Cmd: "CMD [\"/bin/sh\"]" WorkingDir: "" Image: "sha256:9025f4d6338c3425933e4734e8a27fb615d56ca8862e78bb8c4e2426f2db78bd" } docker_version: "17.06.2-ce" config { Hostname: "" Env: "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" User: "" Cmd: "/bin/sh" WorkingDir: "" Image: "sha256:9025f4d6338c3425933e4734e8a27fb615d56ca8862e78bb8c4e2426f2db78bd" } architecture: "amd64" os: "linux" } } executor_info { executor_id { value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } resources { name: "cpus" type: SCALAR scalar { value: 0.11 } allocation_info { role: "slave_public" } } resources { name: "mem" type: SCALAR scalar { value: 64 } allocation_info { role: "slave_public" } } resources { name: "ports" type: RANGES ranges { range { begin: 19471 end: 19471 } } allocation_info { role: "slave_public" } } command { environment { variables { name: "MARATHON_APP_VERSION" value: "2018-08-08T23:45:27.052Z" } variables { name: "HOST" value: "10.0.3.181" } variables { name: "MARATHON_APP_RESOURCE_CPUS" value: "0.01" } variables { name: "MARATHON_APP_RESOURCE_GPUS" value: "0" } variables { name: "MESOS_TASK_ID" value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } variables { name: "PORT" value: "19471" } variables { name: "MARATHON_APP_RESOURCE_MEM" value: "32.0" } variables { name: "PORTS" value: "19471" } variables { name: "MARATHON_APP_RESOURCE_DISK" value: "0.0" } variables { name: "PORT_80" value: "19471" } variables { name: "MARATHON_APP_LABELS" value: "" } variables { name: "MARATHON_APP_ID" value: "/testcni2" } variables { name: "PORT0" value: "19471" } } value: "/opt/mesosphere/packages/mesos--39041edf914496bb882949ade1e78043d53dadd0/libexec/mesos/mesos-executor" shell: false arguments: "mesos-executor" arguments: "--launcher_dir=/opt/mesosphere/active/mesos/libexec/mesos" arguments: "--sandbox_directory=/mnt/mesos/sandbox" arguments: "--user=root" } framework_id { value: "4397555f-b8fd-478c-907a-7532e2f5e32e-0001" } name: "Command Executor (Task: testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a) (Command: sh -c \'sleep 10000\')" source: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" container { type: MESOS mesos { image { type: DOCKER docker { name: "alpine" } cached: true } } network_infos { labels { labels { key: "DCOS_SPACE" value: "/testcni2" } } ip_addresses { } name: "mesos-bridge" port_mappings { host_port: 19471 container_port: 80 protocol: "tcp" } } } discovery { visibility: FRAMEWORK name: "testcni2" ports { ports { number: 19471 protocol: "tcp" labels { labels { key: "network-scope" value: "host" } } } } } labels { labels { key: "DCOS_SPACE" value: "/testcni2" } } } task_info { name: "testcni2" task_id { value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } slave_id { value: "4397555f-b8fd-478c-907a-7532e2f5e32e-S0" } resources { name: "cpus" type: SCALAR scalar { value: 0.01 } allocation_info { role: "slave_public" } } resources { name: "mem" type: SCALAR scalar { value: 32 } allocation_info { role: "slave_public" } } resources { name: "ports" type: RANGES ranges { range { begin: 19471 end: 19471 } } allocation_info { role: "slave_public" } } command { environment { variables { name: "MARATHON_APP_VERSION" value: "2018-08-08T23:45:27.052Z" } variables { name: "HOST" value: "10.0.3.181" } variables { name: "MARATHON_APP_RESOURCE_CPUS" value: "0.01" } variables { name: "MARATHON_APP_RESOURCE_GPUS" value: "0" } variables { name: "MESOS_TASK_ID" value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } variables { name: "PORT" value: "19471" } variables { name: "MARATHON_APP_RESOURCE_MEM" value: "32.0" } variables { name: "PORTS" value: "19471" } variables { name: "MARATHON_APP_RESOURCE_DISK" value: "0.0" } variables { name: "PORT_80" value: "19471" } variables { name: "MARATHON_APP_LABELS" value: "" } variables { name: "MARATHON_APP_ID" value: "/testcni2" } variables { name: "PORT0" value: "19471" } } value: "sleep 10000" } container { type: MESOS mesos { image { type: DOCKER docker { name: "alpine" } cached: true } } network_infos { labels { labels { key: "DCOS_SPACE" value: "/testcni2" } } ip_addresses { } name: "mesos-bridge" port_mappings { host_port: 19471 container_port: 80 protocol: "tcp" } } } labels { labels { key: "DCOS_SPACE" value: "/testcni2" } } discovery { visibility: FRAMEWORK name: "testcni2" ports { ports { number: 19471 protocol: "tcp" labels { labels { key: "network-scope" value: "host" } } } } } } command_info { environment { variables { name: "MARATHON_APP_VERSION" value: "2018-08-08T23:45:27.052Z" } variables { name: "HOST" value: "10.0.3.181" } variables { name: "MARATHON_APP_RESOURCE_CPUS" value: "0.01" } variables { name: "MARATHON_APP_RESOURCE_GPUS" value: "0" } variables { name: "MESOS_TASK_ID" value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } variables { name: "PORT" value: "19471" } variables { name: "MARATHON_APP_RESOURCE_MEM" value: "32.0" } variables { name: "PORTS" value: "19471" } variables { name: "MARATHON_APP_RESOURCE_DISK" value: "0.0" } variables { name: "PORT_80" value: "19471" } variables { name: "MARATHON_APP_LABELS" value: "" } variables { name: "MARATHON_APP_ID" value: "/testcni2" } variables { name: "PORT0" value: "19471" } } value: "/opt/mesosphere/packages/mesos--39041edf914496bb882949ade1e78043d53dadd0/libexec/mesos/mesos-executor" shell: false arguments: "mesos-executor" arguments: "--launcher_dir=/opt/mesosphere/active/mesos/libexec/mesos" arguments: "--sandbox_directory=/mnt/mesos/sandbox" arguments: "--user=root" } container_info { type: MESOS mesos { image { type: DOCKER docker { name: "alpine" } cached: true } } network_infos { labels { labels { key: "DCOS_SPACE" value: "/testcni2" } } ip_addresses { } name: "mesos-bridge" port_mappings { host_port: 19471 container_port: 80 protocol: "tcp" } } } resources { name: "cpus" type: SCALAR scalar { value: 0.11 } allocation_info { role: "slave_public" } } resources { name: "mem" type: SCALAR scalar { value: 64 } allocation_info { role: "slave_public" } } resources { name: "ports" type: RANGES ranges { range { begin: 19471 end: 19471 } } allocation_info { role: "slave_public" } }]
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:01.792201 4047 container_assigner.cpp:64] Registering and retrieving endpoint for container_id[value: "278d88a8-2bd1-4359-8c70-25dc90879cdc"] executor_info[executor_id { value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } resources { name: "cpus" type: SCALAR scalar { value: 0.11 } allocation_info { role: "slave_public" } } resources { name: "mem" type: SCALAR scalar { value: 64 } allocation_info { role: "slave_public" } } resources { name: "ports" type: RANGES ranges { range { begin: 19471 end: 19471 } } allocation_info { role: "slave_public" } } command { environment { variables { name: "MARATHON_APP_VERSION" value: "2018-08-08T23:45:27.052Z" } variables { name: "HOST" value: "10.0.3.181" } variables { name: "MARATHON_APP_RESOURCE_CPUS" value: "0.01" } variables { name: "MARATHON_APP_RESOURCE_GPUS" value: "0" } variables { name: "MESOS_TASK_ID" value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } variables { name: "PORT" value: "19471" } variables { name: "MARATHON_APP_RESOURCE_MEM" value: "32.0" } variables { name: "PORTS" value: "19471" } variables { name: "MARATHON_APP_RESOURCE_DISK" value: "0.0" } variables { name: "PORT_80" value: "19471" } variables { name: "MARATHON_APP_LABELS" value: "" } variables { name: "MARATHON_APP_ID" value: "/testcni2" } variables { name: "PORT0" value: "19471" } } value: "/opt/mesosphere/packages/mesos--39041edf914496bb882949ade1e78043d53dadd0/libexec/mesos/mesos-executor" shell: false arguments: "mesos-executor" arguments: "--launcher_dir=/opt/mesosphere/active/mesos/libexec/mesos" arguments: "--sandbox_directory=/mnt/mesos/sandbox" arguments: "--user=root" } framework_id { value: "4397555f-b8fd-478c-907a-7532e2f5e32e-0001" } name: "Command Executor (Task: testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a) (Command: sh -c \'sleep 10000\')" source: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" container { type: MESOS mesos { image { type: DOCKER docker { name: "alpine" } cached: true } } network_infos { labels { labels { key: "DCOS_SPACE" value: "/testcni2" } } ip_addresses { } name: "mesos-bridge" port_mappings { host_port: 19471 container_port: 80 protocol: "tcp" } } } discovery { visibility: FRAMEWORK name: "testcni2" ports { ports { number: 19471 protocol: "tcp" labels { labels { key: "network-scope" value: "host" } } } } } labels { labels { key: "DCOS_SPACE" value: "/testcni2" } }].
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:01.822054 4195 container_assigner_strategy.cpp:156] New ephemeral-port reader for container[value: "278d88a8-2bd1-4359-8c70-25dc90879cdc"] created at endpoint[198.51.100.1:47454].
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:01.822118 4195 container_state_cache_impl.cpp:134] Writing container file[/run/dcos/mesos/isolators/com_mesosphere_MetricsIsolatorModule/containers/278d88a8-2bd1-4359-8c70-25dc90879cdc] with endpoint[198.51.100.1:47454]
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:01.828858 4045 secret.cpp:296] 0 secrets have been resolved for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 09 00:02:03 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:03.230397 4047 switchboard.cpp:317] Container logger module finished preparing container 278d88a8-2bd1-4359-8c70-25dc90879cdc; IOSwitchboard server is not required
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:04.510732 4045 containerizer.cpp:2367] Destroying container 278d88a8-2bd1-4359-8c70-25dc90879cdc in PREPARING state
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:04.511010 4045 containerizer.cpp:2981] Transitioning the state of container 278d88a8-2bd1-4359-8c70-25dc90879cdc from PREPARING to DESTROYING
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:04.745018 4042 container_assigner.cpp:101] Unregistering container_id[value: "278d88a8-2bd1-4359-8c70-25dc90879cdc"].
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:04.746515 4195 container_assigner_strategy.cpp:202] Closing ephemeral-port reader for container[value: "278d88a8-2bd1-4359-8c70-25dc90879cdc"] at endpoint[198.51.100.1:47454].
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:02:04.749131 4195 container_state_cache_impl.cpp:145] Removing container file[/run/dcos/mesos/isolators/com_mesosphere_MetricsIsolatorModule/containers/278d88a8-2bd1-4359-8c70-25dc90879cdc]
> Aug 09 00:02:05 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: E0809 00:02:05.153556 4048 slave.cpp:6201] Container '278d88a8-2bd1-4359-8c70-25dc90879cdc' for executor 'testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a' of framework 4397555f-b8fd-478c-907a-7532e2f5e32e-0001 failed to start: Container is being destroyed during preparing
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: E0809 00:07:08.151012 4045 slave.cpp:6308] Termination of executor 'testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a' of framework 4397555f-b8fd-478c-907a-7532e2f5e32e-0001 failed: Failed to clean up an isolator when destroying container: Failed to parse CNI network configuration file: '/run/mesos/isolators/network/cni/278d88a8-2bd1-4359-8c70-25dc90879cdc/mesos-bridge/network.conf': Failed to read CNI network configuration file: '/run/mesos/isolators/network/cni/278d88a8-2bd1-4359-8c70-25dc90879cdc/mesos-bridge/network.conf': Failed to open file: No such file or directory
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: E0809 00:07:08.151798 4046 slave.cpp:5639] Failed to update resources for container 278d88a8-2bd1-4359-8c70-25dc90879cdc of executor 'testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a' running task testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a on status update for terminal task, destroying container: Container not found
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:08.152240 4043 composing.cpp:600] Attempted to destroy unknown container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:07:08.164916 4047 gc.cpp:95] Scheduling '/var/lib/mesos/slave/slaves/4397555f-b8fd-478c-907a-7532e2f5e32e-S0/frameworks/4397555f-b8fd-478c-907a-7532e2f5e32e-0001/executors/testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a/runs/278d88a8-2bd1-4359-8c70-25dc90879cdc' for gc 1.99999809405037days in the future
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: I0809 00:07:08.165079 4047 gc.cpp:95] Scheduling '/var/lib/mesos/slave/meta/slaves/4397555f-b8fd-478c-907a-7532e2f5e32e-S0/frameworks/4397555f-b8fd-478c-907a-7532e2f5e32e-0001/executors/testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a/runs/278d88a8-2bd1-4359-8c70-25dc90879cdc' for gc 1.99999809211852days in the future
> Aug 09 00:07:26 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:26.571424 4047 cgroups.cpp:737] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to get usage for subsystem 'memory': Unknown container
> Aug 09 00:07:26 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:26.571480 4047 cgroups.cpp:737] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read 'cpu.stat': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid cgroup
> Aug 09 00:07:26 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:26.571648 4047 containerizer.cpp:2221] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Unknown container
> Aug 09 00:07:27 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:27.801781 4043 cgroups.cpp:737] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read from 'blkio.time': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid cgroup
> Aug 09 00:07:27 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:27.801839 4043 cgroups.cpp:737] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to get usage for subsystem 'memory': Unknown container
> Aug 09 00:07:27 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:27.801862 4043 cgroups.cpp:737] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read 'cpu.stat': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid cgroup
> Aug 09 00:07:27 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:27.801961 4043 containerizer.cpp:2221] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Unknown container
> Aug 09 00:07:29 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:29.182669 4048 cgroups.cpp:737] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read from 'blkio.time': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid cgroup
> Aug 09 00:07:29 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:29.182729 4048 cgroups.cpp:737] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to get usage for subsystem 'memory': Unknown container
> Aug 09 00:07:29 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:29.182752 4048 cgroups.cpp:737] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read 'cpu.stat': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid cgroup
> Aug 09 00:07:29 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]: W0809 00:07:29.182870 4046 containerizer.cpp:2221] Skipping resource statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Unknown container
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)