You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Joseph Wu (JIRA)" <ji...@apache.org> on 2018/11/07 01:50:00 UTC
[jira] [Commented] (MESOS-8723)
ROOT_HealthCheckUsingPersistentVolume is flaky.
[ https://issues.apache.org/jira/browse/MESOS-8723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677515#comment-16677515 ]
Joseph Wu commented on MESOS-8723:
----------------------------------
Another bad run, on the 1.6.x branch (Ubuntu 16)
{code}
[ RUN ] LauncherAndIsolationParam/PersistentVolumeDefaultExecutor.ROOT_HealthCheckUsingPersistentVolume/1
I1106 20:15:34.354775 32499 cluster.cpp:172] Creating default 'local' authorizer
I1106 20:15:34.355837 22262 master.cpp:463] Master ee3a72ac-f1ea-4572-ab7a-424ecc6e517c (ip-172-16-10-158.ec2.internal) started on 172.16.10.158:46488
I1106 20:15:34.355865 22262 master.cpp:466] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="hierarchical" --authenticate_agents="true" --authenticate_frameworks="true" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authentication_v0_timeout="15secs" --authenticators="crammd5" --authorizers="local" --credentials="/tmp/yQNdFw/credentials" --filter_gpu_resources="true" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --max_unreachable_tasks_per_framework="1000" --memory_profiling="false" --min_allocatable_resources="cpus:0.01|mem:32" --port="5050" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memory" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --require_agent_domain="false" --role_sorter="drf" --root_submissions="true" --version="false" --webui_dir="/usr/local/share/mesos/webui" --work_dir="/tmp/yQNdFw/master" --zk_session_timeout="10secs"
I1106 20:15:34.356046 22262 master.cpp:515] Master only allowing authenticated frameworks to register
I1106 20:15:34.356058 22262 master.cpp:521] Master only allowing authenticated agents to register
I1106 20:15:34.356146 22262 master.cpp:527] Master only allowing authenticated HTTP frameworks to register
I1106 20:15:34.356154 22262 credentials.hpp:37] Loading credentials for authentication from '/tmp/yQNdFw/credentials'
I1106 20:15:34.356290 22262 master.cpp:571] Using default 'crammd5' authenticator
I1106 20:15:34.356390 22262 http.cpp:959] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readonly'
I1106 20:15:34.356514 22262 http.cpp:959] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readwrite'
I1106 20:15:34.356560 22262 http.cpp:959] Creating default 'basic' HTTP authenticator for realm 'mesos-master-scheduler'
I1106 20:15:34.356707 22262 master.cpp:652] Authorization enabled
I1106 20:15:34.356874 22263 hierarchical.cpp:177] Initialized hierarchical allocator process
I1106 20:15:34.356999 22263 whitelist_watcher.cpp:77] No whitelist given
I1106 20:15:34.357659 22263 master.cpp:2162] Elected as the leading master!
I1106 20:15:34.357681 22263 master.cpp:1717] Recovering from registrar
I1106 20:15:34.357723 22263 registrar.cpp:339] Recovering registrar
I1106 20:15:34.357962 22257 registrar.cpp:383] Successfully fetched the registry (0B) in 184832ns
I1106 20:15:34.358003 22257 registrar.cpp:487] Applied 1 operations in 7418ns; attempting to update the registry
I1106 20:15:34.358218 22262 registrar.cpp:544] Successfully updated the registry in 129792ns
I1106 20:15:34.358259 22262 registrar.cpp:416] Successfully recovered registrar
I1106 20:15:34.358475 22262 master.cpp:1831] Recovered 0 agents from the registry (176B); allowing 10mins for agents to reregister
I1106 20:15:34.358522 22262 hierarchical.cpp:215] Skipping recovery of hierarchical allocator: nothing to recover
I1106 20:15:34.359434 32499 containerizer.cpp:296] Using isolation { environment_secret, network/cni, filesystem/posix, volume/sandbox_path }
I1106 20:15:34.361624 32499 linux_launcher.cpp:147] Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
I1106 20:15:34.362061 32499 provisioner.cpp:299] Using default backend 'overlay'
W1106 20:15:34.363476 32499 process.cpp:2829] Attempted to spawn already running process files@172.16.10.158:46488
I1106 20:15:34.363585 32499 cluster.cpp:460] Creating default 'local' authorizer
I1106 20:15:34.364131 22257 slave.cpp:265] Mesos agent started on (1090)@172.16.10.158:46488
I1106 20:15:34.364296 22257 slave.cpp:266] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="false" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authentication_timeout_max="1mins" --authentication_timeout_min="5secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/credential" --default_role="*" --disallow_sharing_agent_pid_namespace="false" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_reregistration_timeout="2secs" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/fetch" --fetcher_cache_size="2GB" --fetcher_stall_timeout="1mins" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_command_executor="false" --http_credentials="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/http_credentials" --http_heartbeat_interval="30secs" --initialize_driver_logging="true" --isolation="volume/sandbox_path" --launcher="linux" --launcher_dir="/home/ubuntu/workspace/mesos/Mesos_CI-build/FLAG/Plain/label/mesos-ec2-ubuntu-16.04/mesos/build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --memory_profiling="false" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --reconfiguration_policy="equal" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX" --zk_session_timeout="10secs"
I1106 20:15:34.364569 22257 credentials.hpp:86] Loading credential for authentication from '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/credential'
I1106 20:15:34.364682 22257 slave.cpp:298] Agent using credential for: test-principal
I1106 20:15:34.364763 22257 credentials.hpp:37] Loading credentials for authentication from '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/http_credentials'
I1106 20:15:34.364459 32499 scheduler.cpp:189] Version: 1.6.2
I1106 20:15:34.364928 22257 http.cpp:959] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
I1106 20:15:34.365077 22257 disk_profile_adaptor.cpp:80] Creating default disk profile adaptor module
I1106 20:15:34.365506 22261 scheduler.cpp:355] Using default 'basic' HTTP authenticatee
I1106 20:15:34.365697 22261 scheduler.cpp:538] New master detected at master@172.16.10.158:46488
I1106 20:15:34.365722 22261 scheduler.cpp:547] Waiting for 0ns before initiating a re-(connection) attempt with the master
I1106 20:15:34.365808 22257 slave.cpp:613] Agent resources: [{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}]
I1106 20:15:34.365932 22257 slave.cpp:621] Agent attributes: [ ]
I1106 20:15:34.365942 22257 slave.cpp:630] Agent hostname: ip-172-16-10-158.ec2.internal
I1106 20:15:34.366056 22263 task_status_update_manager.cpp:181] Pausing sending task status updates
I1106 20:15:34.366300 22257 state.cpp:66] Recovering state from '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/meta'
I1106 20:15:34.366366 22257 slave.cpp:6928] Finished recovering checkpointed state from '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/meta', beginning agent recovery
I1106 20:15:34.366427 22261 task_status_update_manager.cpp:207] Recovering task status update manager
I1106 20:15:34.366487 22263 containerizer.cpp:712] Recovering Mesos containers
I1106 20:15:34.366566 22263 linux_launcher.cpp:300] Recovering Linux launcher
I1106 20:15:34.367132 22263 containerizer.cpp:1019] Recovering isolators
I1106 20:15:34.367535 22264 containerizer.cpp:1058] Recovering provisioner
I1106 20:15:34.367668 22264 provisioner.cpp:495] Provisioner recovery complete
I1106 20:15:34.367774 22264 slave.cpp:7157] Recovering executors
I1106 20:15:34.367808 22264 slave.cpp:7310] Finished recovery
I1106 20:15:34.368073 22261 slave.cpp:1273] New master detected at master@172.16.10.158:46488
I1106 20:15:34.368291 22261 slave.cpp:1338] Detecting new master
I1106 20:15:34.368099 22263 task_status_update_manager.cpp:181] Pausing sending task status updates
I1106 20:15:34.368311 22262 scheduler.cpp:429] Connected with the master at http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.368762 22257 scheduler.cpp:248] Sending SUBSCRIBE call to http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.369148 22262 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master/api/v1/scheduler'
I1106 20:15:34.369537 22262 http.cpp:1099] HTTP POST for /master/api/v1/scheduler from 172.16.10.158:55292
I1106 20:15:34.369619 22262 master.cpp:2579] Received subscription request for HTTP framework 'default'
I1106 20:15:34.369673 22262 master.cpp:2234] Authorizing framework principal 'test-principal' to receive offers for roles '{ default-role }'
I1106 20:15:34.369825 22259 master.cpp:2714] Subscribing framework 'default' with checkpointing disabled and capabilities [ MULTI_ROLE, RESERVATION_REFINEMENT ]
I1106 20:15:34.370122 22259 master.cpp:9837] Adding framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default) with roles { } suppressed
I1106 20:15:34.370306 22263 hierarchical.cpp:299] Added framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.370420 22263 hierarchical.cpp:1520] Performed allocation for 0 agents in 10390ns
I1106 20:15:34.370815 22261 scheduler.cpp:845] Enqueuing event SUBSCRIBED received from http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.371007 22264 scheduler.cpp:845] Enqueuing event HEARTBEAT received from http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.372133 22261 slave.cpp:1365] Authenticating with master master@172.16.10.158:46488
I1106 20:15:34.372160 22261 slave.cpp:1374] Using default CRAM-MD5 authenticatee
I1106 20:15:34.372229 22261 authenticatee.cpp:121] Creating new client SASL connection
I1106 20:15:34.372637 22261 master.cpp:9607] Authenticating slave(1090)@172.16.10.158:46488
I1106 20:15:34.372691 22261 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(1899)@172.16.10.158:46488
I1106 20:15:34.372758 22261 authenticator.cpp:98] Creating new server SASL connection
I1106 20:15:34.373095 22261 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
I1106 20:15:34.373118 22261 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
I1106 20:15:34.373152 22261 authenticator.cpp:204] Received SASL authentication start
I1106 20:15:34.373193 22261 authenticator.cpp:326] Authentication requires more steps
I1106 20:15:34.373234 22261 authenticatee.cpp:259] Received SASL authentication step
I1106 20:15:34.373296 22261 authenticator.cpp:232] Received SASL authentication step
I1106 20:15:34.373333 22261 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'ip-172-16-10-158.ec2.internal' server FQDN: 'ip-172-16-10-158.ec2.internal' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
I1106 20:15:34.373354 22261 auxprop.cpp:181] Looking up auxiliary property '*userPassword'
I1106 20:15:34.373366 22261 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5'
I1106 20:15:34.373376 22261 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'ip-172-16-10-158.ec2.internal' server FQDN: 'ip-172-16-10-158.ec2.internal' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
I1106 20:15:34.373383 22261 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true
I1106 20:15:34.373389 22261 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
I1106 20:15:34.373404 22261 authenticator.cpp:318] Authentication success
I1106 20:15:34.373445 22261 authenticatee.cpp:299] Authentication success
I1106 20:15:34.373520 22261 master.cpp:9639] Successfully authenticated principal 'test-principal' at slave(1090)@172.16.10.158:46488
I1106 20:15:34.373567 22261 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(1899)@172.16.10.158:46488
I1106 20:15:34.373663 22261 slave.cpp:1465] Successfully authenticated with master master@172.16.10.158:46488
I1106 20:15:34.373793 22261 slave.cpp:1896] Will retry registration in 6.172179ms if necessary
I1106 20:15:34.373915 22259 master.cpp:6596] Received register agent message from slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.373999 22259 master.cpp:3896] Authorizing agent providing resources 'cpus:2; mem:1024; disk:1024; ports:[31000-32000]' with principal 'test-principal'
I1106 20:15:34.374158 22259 master.cpp:6663] Authorized registration of agent at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.374199 22259 master.cpp:6778] Registering agent at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal) with id ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0
I1106 20:15:34.374311 22259 registrar.cpp:487] Applied 1 operations in 35637ns; attempting to update the registry
I1106 20:15:34.374452 22262 registrar.cpp:544] Successfully updated the registry in 113152ns
I1106 20:15:34.374583 22257 master.cpp:6826] Admitted agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.374799 22263 hierarchical.cpp:576] Added agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 (ip-172-16-10-158.ec2.internal) with cpus:2; mem:1024; disk:1024; ports:[31000-32000] (allocated: {})
I1106 20:15:34.374996 22263 hierarchical.cpp:1520] Performed allocation for 1 agents in 140536ns
I1106 20:15:34.375049 22257 master.cpp:6871] Registered agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal) with cpus:2; mem:1024; disk:1024; ports:[31000-32000]
I1106 20:15:34.375183 22257 master.cpp:9439] Sending 1 offers to framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default)
I1106 20:15:34.375349 22257 slave.cpp:1498] Registered with master master@172.16.10.158:46488; given agent ID ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0
I1106 20:15:34.375527 22262 task_status_update_manager.cpp:188] Resuming sending task status updates
I1106 20:15:34.375576 22257 slave.cpp:1518] Checkpointing SlaveInfo to '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/meta/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/slave.info'
I1106 20:15:34.375730 22263 scheduler.cpp:845] Enqueuing event OFFERS received from http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.376322 22257 slave.cpp:1567] Forwarding agent update {"operations":{},"resource_version_uuid":{"value":"M1q84jWRTE6I2ROZ5MTpFA=="},"slave_id":{"value":"ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0"},"update_oversubscribed_resources":true}
I1106 20:15:34.376456 22257 master.cpp:7821] Received update of agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal) with total oversubscribed resources {}
I1106 20:15:34.376837 22257 master.cpp:7930] Ignoring update on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal) as it reports no changes
I1106 20:15:34.377053 22261 scheduler.cpp:248] Sending ACCEPT call to http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.377487 22261 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master/api/v1/scheduler'
I1106 20:15:34.377794 22258 http.cpp:1099] HTTP POST for /master/api/v1/scheduler from 172.16.10.158:55290
I1106 20:15:34.377987 22258 master.cpp:11400] Removing offer ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-O0
I1106 20:15:34.378183 22258 master.cpp:4399] Processing ACCEPT call for offers: [ ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-O0 ] on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal) for framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default)
I1106 20:15:34.378245 22258 master.cpp:3639] Authorizing principal 'test-principal' to reserve resources 'cpus(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):32'
I1106 20:15:34.378412 22258 master.cpp:3774] Authorizing principal 'test-principal' to create volumes '[{"allocation_info":{"role":"default-role"},"disk":{"persistence":{"id":"id1","principal":"test-principal"},"volume":{"container_path":"task_volume_path","mode":"RW"}},"name":"disk","reservations":[{"principal":"test-principal","role":"default-role","type":"DYNAMIC"}],"scalar":{"value":1.0},"type":"SCALAR"}]'
I1106 20:15:34.378548 22258 master.cpp:3571] Authorizing framework principal 'test-principal' to launch task 6317b43b-bde1-4c4f-a7c8-2187062839db
I1106 20:15:34.379128 22258 master.cpp:4762] Applying RESERVE operation for resources [{"allocation_info":{"role":"default-role"},"name":"cpus","reservations":[{"principal":"test-principal","role":"default-role","type":"DYNAMIC"}],"scalar":{"value":0.1},"type":"SCALAR"},{"allocation_info":{"role":"default-role"},"name":"mem","reservations":[{"principal":"test-principal","role":"default-role","type":"DYNAMIC"}],"scalar":{"value":32.0},"type":"SCALAR"},{"allocation_info":{"role":"default-role"},"name":"disk","reservations":[{"principal":"test-principal","role":"default-role","type":"DYNAMIC"}],"scalar":{"value":32.0},"type":"SCALAR"}] from framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default) to agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.379442 22258 master.cpp:11298] Sending operation '' (uuid: 16e569ee-a765-4cc1-accc-b1ae91a0eecf) to agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.379925 22258 master.cpp:4905] Applying CREATE operation for volumes [{"allocation_info":{"role":"default-role"},"disk":{"persistence":{"id":"id1","principal":"test-principal"},"volume":{"container_path":"task_volume_path","mode":"RW"}},"name":"disk","reservations":[{"principal":"test-principal","role":"default-role","type":"DYNAMIC"}],"scalar":{"value":1.0},"type":"SCALAR"}] from framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default) to agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.380125 22258 master.cpp:11298] Sending operation '' (uuid: 4750abda-927c-443c-b387-8018ec5ceeb9) to agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.380765 22258 master.cpp:12120] Adding task 6317b43b-bde1-4c4f-a7c8-2187062839db with resources cpus(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):31; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1 on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.380888 22261 slave.cpp:4219] Updated checkpointed resources from {} to cpus(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(reservations: [(DYNAMIC,default-role,test-principal)]):32
I1106 20:15:34.380897 22258 master.cpp:5586] Launching task group { 6317b43b-bde1-4c4f-a7c8-2187062839db } of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default) with resources disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):31; mem(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):32; cpus(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):0.1 on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal) on new executor
I1106 20:15:34.381080 22261 slave.cpp:7989] Updating the state of operation with no ID (uuid: 16e569ee-a765-4cc1-accc-b1ae91a0eecf) for framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (latest state: OPERATION_FINISHED, status update state: OPERATION_FINISHED)
I1106 20:15:34.381837 22261 slave.cpp:4219] Updated checkpointed resources from cpus(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(reservations: [(DYNAMIC,default-role,test-principal)]):32 to cpus(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(reservations: [(DYNAMIC,default-role,test-principal)]):31; disk(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1
I1106 20:15:34.381966 22264 hierarchical.cpp:928] Updated allocation of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 from cpus(allocated: default-role):2; mem(allocated: default-role):1024; disk(allocated: default-role):1024; ports(allocated: default-role):[31000-32000] to cpus(allocated: default-role):1.9; mem(allocated: default-role):992; disk(allocated: default-role):992; ports(allocated: default-role):[31000-32000]; cpus(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):31; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1
I1106 20:15:34.382098 22261 slave.cpp:7989] Updating the state of operation with no ID (uuid: 4750abda-927c-443c-b387-8018ec5ceeb9) for framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (latest state: OPERATION_FINISHED, status update state: OPERATION_FINISHED)
I1106 20:15:34.382292 22260 hierarchical.cpp:1195] Recovered cpus(allocated: default-role):1.8; mem(allocated: default-role):960; disk(allocated: default-role):960; ports(allocated: default-role):[31000-32000] (total: cpus:1.9; mem:992; disk:992; ports:[31000-32000]; cpus(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(reservations: [(DYNAMIC,default-role,test-principal)]):31; disk(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1, allocated: cpus(allocated: default-role):0.1; mem(allocated: default-role):32; disk(allocated: default-role):32; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1; cpus(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):31) on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 from framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.382364 22260 hierarchical.cpp:1241] Framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 filtered agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 for 5secs
I1106 20:15:34.382483 22258 master.cpp:11048] Updating the state of operation '' (uuid: 16e569ee-a765-4cc1-accc-b1ae91a0eecf) for framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (latest state: OPERATION_PENDING, status update state: OPERATION_FINISHED)
I1106 20:15:34.382550 22258 master.cpp:11048] Updating the state of operation '' (uuid: 4750abda-927c-443c-b387-8018ec5ceeb9) for framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (latest state: OPERATION_PENDING, status update state: OPERATION_FINISHED)
I1106 20:15:34.382673 22261 slave.cpp:2033] Got assigned task group containing tasks [ 6317b43b-bde1-4c4f-a7c8-2187062839db ] for framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.382992 22261 slave.cpp:2407] Authorizing task group containing tasks [ 6317b43b-bde1-4c4f-a7c8-2187062839db ] for framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.383049 22261 slave.cpp:8453] Authorizing framework principal 'test-principal' to launch task 6317b43b-bde1-4c4f-a7c8-2187062839db
I1106 20:15:34.383651 22264 slave.cpp:2850] Launching task group containing tasks [ 6317b43b-bde1-4c4f-a7c8-2187062839db ] for framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.383707 22264 paths.cpp:745] Creating sandbox '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39' for user 'root'
I1106 20:15:34.384938 22264 slave.cpp:8984] Launching executor 'default' of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 with resources [{"allocation_info":{"role":"default-role"},"name":"cpus","scalar":{"value":0.1},"type":"SCALAR"},{"allocation_info":{"role":"default-role"},"name":"mem","scalar":{"value":32.0},"type":"SCALAR"},{"allocation_info":{"role":"default-role"},"name":"disk","scalar":{"value":32.0},"type":"SCALAR"}] in work directory '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39'
I1106 20:15:34.385169 22264 slave.cpp:3526] Launching container 77a94bf0-25b5-476e-be40-8442c774cc39 for executor 'default' of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.385301 22264 slave.cpp:3047] Queued task group containing tasks [ 6317b43b-bde1-4c4f-a7c8-2187062839db ] for executor 'default' of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.385360 22264 slave.cpp:1007] Successfully attached '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39' to virtual path '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/latest'
I1106 20:15:34.385426 22264 slave.cpp:1007] Successfully attached '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39' to virtual path '/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/latest'
I1106 20:15:34.385465 22264 containerizer.cpp:1246] Starting container 77a94bf0-25b5-476e-be40-8442c774cc39
I1106 20:15:34.385586 22260 slave.cpp:1007] Successfully attached '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39' to virtual path '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39'
I1106 20:15:34.385800 22264 containerizer.cpp:1412] Checkpointed ContainerConfig at '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/containers/77a94bf0-25b5-476e-be40-8442c774cc39/config'
I1106 20:15:34.385823 22264 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39 from PROVISIONING to PREPARING
I1106 20:15:34.386288 22264 switchboard.cpp:321] Container logger module finished preparing container 77a94bf0-25b5-476e-be40-8442c774cc39; IOSwitchboard server is not required
I1106 20:15:34.386696 22260 containerizer.cpp:1890] Launching 'mesos-containerizer' with flags '--help="false" --launch_info="{"command":{"arguments":["mesos-default-executor","--launcher_dir=\/home\/ubuntu\/workspace\/mesos\/Mesos_CI-build\/FLAG\/Plain\/label\/mesos-ec2-ubuntu-16.04\/mesos\/build\/src"],"shell":false,"value":"\/home\/ubuntu\/workspace\/mesos\/Mesos_CI-build\/FLAG\/Plain\/label\/mesos-ec2-ubuntu-16.04\/mesos\/build\/src\/mesos-default-executor"},"environment":{"variables":[{"name":"LIBPROCESS_PORT","type":"VALUE","value":"0"},{"name":"MESOS_AGENT_ENDPOINT","type":"VALUE","value":"172.16.10.158:46488"},{"name":"MESOS_CHECKPOINT","type":"VALUE","value":"0"},{"name":"MESOS_DIRECTORY","type":"VALUE","value":"\/tmp\/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX\/slaves\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0\/frameworks\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000\/executors\/default\/runs\/77a94bf0-25b5-476e-be40-8442c774cc39"},{"name":"MESOS_EXECUTOR_ID","type":"VALUE","value":"default"},{"name":"MESOS_EXECUTOR_SHUTDOWN_GRACE_PERIOD","type":"VALUE","value":"5secs"},{"name":"MESOS_FRAMEWORK_ID","type":"VALUE","value":"ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000"},{"name":"MESOS_HTTP_COMMAND_EXECUTOR","type":"VALUE","value":"0"},{"name":"MESOS_SLAVE_ID","type":"VALUE","value":"ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0"},{"name":"MESOS_SLAVE_PID","type":"VALUE","value":"slave(1090)@172.16.10.158:46488"},{"name":"MESOS_SANDBOX","type":"VALUE","value":"\/tmp\/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX\/slaves\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0\/frameworks\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000\/executors\/default\/runs\/77a94bf0-25b5-476e-be40-8442c774cc39"}]},"task_environment":{},"user":"root","working_directory":"\/tmp\/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX\/slaves\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0\/frameworks\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000\/executors\/default\/runs\/77a94bf0-25b5-476e-be40-8442c774cc39"}" --pipe_read="58" --pipe_write="61" --runtime_directory="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/containers/77a94bf0-25b5-476e-be40-8442c774cc39" --unshare_namespace_mnt="false"'
I1106 20:15:34.386991 22258 linux_launcher.cpp:506] Launching container 77a94bf0-25b5-476e-be40-8442c774cc39 and cloning with namespaces
I1106 20:15:34.417224 22260 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39 from PREPARING to ISOLATING
I1106 20:15:34.417551 22260 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39 from ISOLATING to FETCHING
I1106 20:15:34.417611 22260 fetcher.cpp:369] Starting to fetch URIs for container: 77a94bf0-25b5-476e-be40-8442c774cc39, directory: /tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39
I1106 20:15:34.417856 22260 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39 from FETCHING to RUNNING
I1106 20:15:34.522084 22263 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1/executor'
I1106 20:15:34.522440 22258 http.cpp:1099] HTTP POST for /slave(1090)/api/v1/executor from 172.16.10.158:55294
I1106 20:15:34.522531 22258 slave.cpp:4624] Received Subscribe request for HTTP executor 'default' of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.523564 22261 posix.cpp:213] Changing the ownership of the persistent volume at '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/volumes/roles/default-role/id1' with uid 0 and gid 0
I1106 20:15:34.523624 22261 posix.cpp:259] Adding symlink from '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/volumes/roles/default-role/id1' to '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/task_volume_path' for persistent volume disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1 of container 77a94bf0-25b5-476e-be40-8442c774cc39
I1106 20:15:34.523921 22259 slave.cpp:3299] Sending queued task group containing tasks [ 6317b43b-bde1-4c4f-a7c8-2187062839db ] to executor 'default' of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (via HTTP)
I1106 20:15:34.524410 22262 slave.cpp:1007] Successfully attached '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/task_volume_path' to virtual path '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/tasks/6317b43b-bde1-4c4f-a7c8-2187062839db/task_volume_path'
I1106 20:15:34.527220 22261 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1/executor'
I1106 20:15:34.527297 22261 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1'
I1106 20:15:34.527575 22263 http.cpp:1099] HTTP POST for /slave(1090)/api/v1 from 172.16.10.158:55298
I1106 20:15:34.527664 22263 http.cpp:1099] HTTP POST for /slave(1090)/api/v1/executor from 172.16.10.158:55296
I1106 20:15:34.527741 22263 slave.cpp:5286] Handling status update TASK_STARTING (Status UUID: 1ae4ed04-77e1-4458-8353-5c91876184bd) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.527896 22263 http.cpp:2402] Processing LAUNCH_NESTED_CONTAINER call for container '77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d'
I1106 20:15:34.528245 22263 task_status_update_manager.cpp:328] Received task status update TASK_STARTING (Status UUID: 1ae4ed04-77e1-4458-8353-5c91876184bd) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.528280 22263 task_status_update_manager.cpp:507] Creating StatusUpdate stream for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.528635 22262 containerizer.cpp:1208] Creating sandbox '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d' for user 'root'
I1106 20:15:34.528827 22262 containerizer.cpp:1246] Starting container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d
I1106 20:15:34.529183 22262 containerizer.cpp:1412] Checkpointed ContainerConfig at '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/containers/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/config'
I1106 20:15:34.529206 22262 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d from PROVISIONING to PREPARING
I1106 20:15:34.529367 22262 sandbox_path.cpp:384] Linking SANDBOX_PATH volume from '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/task_volume_path' to '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/task_volume_path' for container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d
I1106 20:15:34.529649 22262 switchboard.cpp:321] Container logger module finished preparing container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d; IOSwitchboard server is not required
I1106 20:15:34.529981 22262 containerizer.cpp:1890] Launching 'mesos-containerizer' with flags '--help="false" --launch_info="{"command":{"shell":true,"value":"echo abc > task_volume_path\/file && sleep 31337"},"environment":{"variables":[{"name":"MESOS_SANDBOX","type":"VALUE","value":"\/tmp\/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX\/slaves\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0\/frameworks\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000\/executors\/default\/runs\/77a94bf0-25b5-476e-be40-8442c774cc39\/containers\/a8144d7f-ccc1-4476-b9d3-acb86132c45d"},{"name":"MESOS_CONTAINER_IP","type":"VALUE","value":"172.16.10.158"}]},"task_environment":{},"user":"root","working_directory":"\/tmp\/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX\/slaves\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0\/frameworks\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000\/executors\/default\/runs\/77a94bf0-25b5-476e-be40-8442c774cc39\/containers\/a8144d7f-ccc1-4476-b9d3-acb86132c45d"}" --pipe_read="62" --pipe_write="63" --runtime_directory="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/containers/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d" --unshare_namespace_mnt="false"'
I1106 20:15:34.528475 22263 task_status_update_manager.cpp:383] Forwarding task status update TASK_STARTING (Status UUID: 1ae4ed04-77e1-4458-8353-5c91876184bd) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 to the agent
I1106 20:15:34.530247 22263 linux_launcher.cpp:506] Launching nested container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d and cloning with namespaces
I1106 20:15:34.534224 22261 slave.cpp:5778] Forwarding the update TASK_STARTING (Status UUID: 1ae4ed04-77e1-4458-8353-5c91876184bd) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 to master@172.16.10.158:46488
I1106 20:15:34.534623 22260 master.cpp:8366] Status update TASK_STARTING (Status UUID: 1ae4ed04-77e1-4458-8353-5c91876184bd) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 from agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.534653 22260 master.cpp:8423] Forwarding status update TASK_STARTING (Status UUID: 1ae4ed04-77e1-4458-8353-5c91876184bd) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.534842 22260 master.cpp:10876] Updating the state of task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (latest state: TASK_STARTING, status update state: TASK_STARTING)
I1106 20:15:34.535869 22260 scheduler.cpp:845] Enqueuing event UPDATE received from http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.535969 22261 slave.cpp:5671] Task status update manager successfully handled status update TASK_STARTING (Status UUID: 1ae4ed04-77e1-4458-8353-5c91876184bd) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.536615 22260 scheduler.cpp:248] Sending ACKNOWLEDGE call to http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.537305 22260 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master/api/v1/scheduler'
I1106 20:15:34.561110 22262 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d from PREPARING to ISOLATING
I1106 20:15:34.561483 22262 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d from ISOLATING to FETCHING
I1106 20:15:34.561547 22262 fetcher.cpp:369] Starting to fetch URIs for container: 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d, directory: /tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d
I1106 20:15:34.561866 22262 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d from FETCHING to RUNNING
I1106 20:15:34.569880 22260 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1'
I1106 20:15:34.570107 22260 http.cpp:1099] HTTP POST for /slave(1090)/api/v1 from 172.16.10.158:55300
I1106 20:15:34.570458 22263 http.cpp:3355] Processing LAUNCH_NESTED_CONTAINER_SESSION call for container '77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf'
I1106 20:15:34.570739 22263 containerizer.cpp:1208] Creating sandbox '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/containers/check-7773a21d-20ed-4d8c-82d8-30c119b283cf' for user 'root'
I1106 20:15:34.570946 22263 containerizer.cpp:1246] Starting container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf
I1106 20:15:34.571488 22263 containerizer.cpp:1412] Checkpointed ContainerConfig at '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/containers/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/containers/check-7773a21d-20ed-4d8c-82d8-30c119b283cf/config'
I1106 20:15:34.571518 22263 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf from PROVISIONING to PREPARING
I1106 20:15:34.572028 22257 switchboard.cpp:321] Container logger module finished preparing container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf; IOSwitchboard server is required
I1106 20:15:34.572235 22257 switchboard.cpp:574] Launching 'mesos-io-switchboard' with flags '--heartbeat_interval="30secs" --help="false" --socket_address="/tmp/mesos-io-switchboard-4a3e90da-917e-4158-96d9-c9a7fe53eaeb" --stderr_from_fd="71" --stderr_to_fd="2" --stdin_to_fd="64" --stdout_from_fd="66" --stdout_to_fd="1" --tty="false" --wait_for_connection="true"' for container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf
I1106 20:15:34.577131 22261 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1/executor'
I1106 20:15:34.578238 22258 http.cpp:1099] HTTP POST for /master/api/v1/scheduler from 172.16.10.158:55290
I1106 20:15:34.578327 22258 master.cpp:6232] Processing ACKNOWLEDGE call for status 1ae4ed04-77e1-4458-8353-5c91876184bd for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default) on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0
I1106 20:15:34.578702 22258 task_status_update_manager.cpp:401] Received task status update acknowledgement (UUID: 1ae4ed04-77e1-4458-8353-5c91876184bd) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.584892 22259 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1'
I1106 20:15:34.585093 22259 http.cpp:1099] HTTP POST for /slave(1090)/api/v1 from 172.16.10.158:55302
I1106 20:15:34.585203 22259 http.cpp:2637] Processing WAIT_NESTED_CONTAINER call for container '77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d'
I1106 20:15:34.585602 22257 switchboard.cpp:616] Created I/O switchboard server (pid: 9437) listening on socket file '/tmp/mesos-io-switchboard-4a3e90da-917e-4158-96d9-c9a7fe53eaeb' for container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf
I1106 20:15:34.585664 22258 slave.cpp:4522] Task status update manager successfully handled status update acknowledgement (UUID: 1ae4ed04-77e1-4458-8353-5c91876184bd) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.586483 22262 containerizer.cpp:1890] Launching 'mesos-containerizer' with flags '--help="false" --launch_info="{"command":{"shell":true,"value":"cat task_volume_path\/file"},"environment":{"variables":[{"name":"MESOS_SANDBOX","type":"VALUE","value":"\/tmp\/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX\/slaves\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0\/frameworks\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000\/executors\/default\/runs\/77a94bf0-25b5-476e-be40-8442c774cc39\/containers\/a8144d7f-ccc1-4476-b9d3-acb86132c45d"},{"name":"MESOS_CONTAINER_IP","type":"VALUE","value":"172.16.10.158"}]},"task_environment":{},"user":"root","working_directory":"\/tmp\/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX\/slaves\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0\/frameworks\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000\/executors\/default\/runs\/77a94bf0-25b5-476e-be40-8442c774cc39\/containers\/a8144d7f-ccc1-4476-b9d3-acb86132c45d"}" --pipe_read="61" --pipe_write="64" --runtime_directory="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/containers/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/containers/check-7773a21d-20ed-4d8c-82d8-30c119b283cf" --unshare_namespace_mnt="false"'
I1106 20:15:34.586753 22263 linux_launcher.cpp:506] Launching nested container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf and cloning with namespaces
I1106 20:15:34.613453 22258 http.cpp:1099] HTTP POST for /slave(1090)/api/v1/executor from 172.16.10.158:55296
I1106 20:15:34.613602 22258 slave.cpp:5286] Handling status update TASK_RUNNING (Status UUID: 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.632884 22262 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf from PREPARING to ISOLATING
I1106 20:15:34.633610 22262 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf from ISOLATING to FETCHING
I1106 20:15:34.634084 22262 fetcher.cpp:369] Starting to fetch URIs for container: 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf, directory: /tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/containers/check-7773a21d-20ed-4d8c-82d8-30c119b283cf
I1106 20:15:34.634488 22262 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf from FETCHING to RUNNING
I1106 20:15:34.634764 22262 http.cpp:3600] Processing ATTACH_CONTAINER_OUTPUT call for container '77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf'
I1106 20:15:34.633929 22257 task_status_update_manager.cpp:328] Received task status update TASK_RUNNING (Status UUID: 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.635417 22257 task_status_update_manager.cpp:383] Forwarding task status update TASK_RUNNING (Status UUID: 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 to the agent
I1106 20:15:34.635660 22257 slave.cpp:5778] Forwarding the update TASK_RUNNING (Status UUID: 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 to master@172.16.10.158:46488
I1106 20:15:34.635900 22257 slave.cpp:5671] Task status update manager successfully handled status update TASK_RUNNING (Status UUID: 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.636101 22259 master.cpp:8366] Status update TASK_RUNNING (Status UUID: 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 from agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.637038 22259 master.cpp:8423] Forwarding status update TASK_RUNNING (Status UUID: 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.637336 22259 master.cpp:10876] Updating the state of task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (latest state: TASK_RUNNING, status update state: TASK_RUNNING)
I1106 20:15:34.640879 22261 scheduler.cpp:845] Enqueuing event UPDATE received from http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.641306 22261 scheduler.cpp:248] Sending ACKNOWLEDGE call to http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.642101 22260 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master/api/v1/scheduler'
I1106 20:15:34.681025 22259 http.cpp:1099] HTTP POST for /master/api/v1/scheduler from 172.16.10.158:55290
I1106 20:15:34.681118 22259 master.cpp:6232] Processing ACKNOWLEDGE call for status 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default) on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0
I1106 20:15:34.681432 22259 task_status_update_manager.cpp:401] Received task status update acknowledgement (UUID: 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.681677 22259 slave.cpp:4522] Task status update manager successfully handled status update acknowledgement (UUID: 9fdf5be9-7eff-48d4-8c1c-a97a3c50751a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.715781 22260 http.cpp:3448] Received EOF attach response for 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf
I1106 20:15:34.716044 22260 containerizer.cpp:2403] Destroying container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf in RUNNING state
I1106 20:15:34.716073 22260 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf from RUNNING to DESTROYING
W1106 20:15:34.716150 22260 http.cpp:3465] Launch nested container session connection for container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf closed
I1106 20:15:34.716217 22260 linux_launcher.cpp:594] Asked to destroy container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf
I1106 20:15:34.717622 22260 linux_launcher.cpp:641] Destroying cgroup '/sys/fs/cgroup/freezer/mesos/77a94bf0-25b5-476e-be40-8442c774cc39/mesos/a8144d7f-ccc1-4476-b9d3-acb86132c45d/mesos/check-7773a21d-20ed-4d8c-82d8-30c119b283cf'
I1106 20:15:34.718843 22260 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1'
I1106 20:15:34.718888 22261 cgroups.cpp:3056] Freezing cgroup /sys/fs/cgroup/freezer/mesos/77a94bf0-25b5-476e-be40-8442c774cc39/mesos/a8144d7f-ccc1-4476-b9d3-acb86132c45d/mesos/check-7773a21d-20ed-4d8c-82d8-30c119b283cf
I1106 20:15:34.719025 22260 http.cpp:1099] HTTP POST for /slave(1090)/api/v1 from 172.16.10.158:55304
I1106 20:15:34.719130 22260 http.cpp:2637] Processing WAIT_NESTED_CONTAINER call for container '77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf'
I1106 20:15:34.719889 22261 cgroups.cpp:1412] Successfully froze cgroup /sys/fs/cgroup/freezer/mesos/77a94bf0-25b5-476e-be40-8442c774cc39/mesos/a8144d7f-ccc1-4476-b9d3-acb86132c45d/mesos/check-7773a21d-20ed-4d8c-82d8-30c119b283cf after 974080ns
I1106 20:15:34.720407 22261 cgroups.cpp:3074] Thawing cgroup /sys/fs/cgroup/freezer/mesos/77a94bf0-25b5-476e-be40-8442c774cc39/mesos/a8144d7f-ccc1-4476-b9d3-acb86132c45d/mesos/check-7773a21d-20ed-4d8c-82d8-30c119b283cf
I1106 20:15:34.721302 22261 cgroups.cpp:1441] Successfully thawed cgroup /sys/fs/cgroup/freezer/mesos/77a94bf0-25b5-476e-be40-8442c774cc39/mesos/a8144d7f-ccc1-4476-b9d3-acb86132c45d/mesos/check-7773a21d-20ed-4d8c-82d8-30c119b283cf after 860160ns
I1106 20:15:34.745442 22257 switchboard.cpp:925] I/O switchboard server process for container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf has terminated (status=0)
I1106 20:15:34.745597 22260 containerizer.cpp:2860] Container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf has exited
I1106 20:15:34.746192 22262 provisioner.cpp:598] Ignoring destroy request for unknown container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf
I1106 20:15:34.746263 22262 containerizer.cpp:2693] Checkpointing termination state to nested container's runtime directory '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/containers/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/containers/check-7773a21d-20ed-4d8c-82d8-30c119b283cf/termination'
I1106 20:15:34.748060 22258 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1/executor'
I1106 20:15:34.748703 22257 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1'
I1106 20:15:34.748981 22259 http.cpp:1099] HTTP POST for /slave(1090)/api/v1 from 172.16.10.158:55306
I1106 20:15:34.749110 22259 http.cpp:2923] Processing REMOVE_NESTED_CONTAINER call for container '77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-7773a21d-20ed-4d8c-82d8-30c119b283cf'
I1106 20:15:34.757517 22257 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1'
I1106 20:15:34.757721 22259 http.cpp:1099] HTTP POST for /slave(1090)/api/v1 from 172.16.10.158:55308
I1106 20:15:34.757843 22259 http.cpp:3355] Processing LAUNCH_NESTED_CONTAINER_SESSION call for container '77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac'
I1106 20:15:34.758065 22264 containerizer.cpp:1208] Creating sandbox '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/containers/check-2f466d67-e1b7-4721-9e60-1565886735ac' for user 'root'
I1106 20:15:34.758241 22264 containerizer.cpp:1246] Starting container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac
I1106 20:15:34.758637 22264 containerizer.cpp:1412] Checkpointed ContainerConfig at '/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/containers/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/containers/check-2f466d67-e1b7-4721-9e60-1565886735ac/config'
I1106 20:15:34.758661 22264 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac from PROVISIONING to PREPARING
I1106 20:15:34.759080 22263 switchboard.cpp:321] Container logger module finished preparing container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac; IOSwitchboard server is required
I1106 20:15:34.759268 22263 switchboard.cpp:574] Launching 'mesos-io-switchboard' with flags '--heartbeat_interval="30secs" --help="false" --socket_address="/tmp/mesos-io-switchboard-e0a0510f-5a6e-4a0c-8b64-c38a673412ae" --stderr_from_fd="67" --stderr_to_fd="2" --stdin_to_fd="63" --stdout_from_fd="64" --stdout_to_fd="1" --tty="false" --wait_for_connection="true"' for container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac
I1106 20:15:34.763309 22263 switchboard.cpp:616] Created I/O switchboard server (pid: 9487) listening on socket file '/tmp/mesos-io-switchboard-e0a0510f-5a6e-4a0c-8b64-c38a673412ae' for container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac
I1106 20:15:34.765789 22258 containerizer.cpp:1890] Launching 'mesos-containerizer' with flags '--help="false" --launch_info="{"command":{"shell":true,"value":"cat task_volume_path\/file"},"environment":{"variables":[{"name":"MESOS_SANDBOX","type":"VALUE","value":"\/tmp\/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX\/slaves\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0\/frameworks\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000\/executors\/default\/runs\/77a94bf0-25b5-476e-be40-8442c774cc39\/containers\/a8144d7f-ccc1-4476-b9d3-acb86132c45d"},{"name":"MESOS_CONTAINER_IP","type":"VALUE","value":"172.16.10.158"}]},"task_environment":{},"user":"root","working_directory":"\/tmp\/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX\/slaves\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0\/frameworks\/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000\/executors\/default\/runs\/77a94bf0-25b5-476e-be40-8442c774cc39\/containers\/a8144d7f-ccc1-4476-b9d3-acb86132c45d"}" --pipe_read="63" --pipe_write="64" --runtime_directory="/tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_9fXuUo/containers/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/containers/check-2f466d67-e1b7-4721-9e60-1565886735ac" --unshare_namespace_mnt="false"'
I1106 20:15:34.766279 22257 linux_launcher.cpp:506] Launching nested container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac and cloning with namespaces
I1106 20:15:34.786149 22261 http.cpp:1099] HTTP POST for /slave(1090)/api/v1/executor from 172.16.10.158:55296
I1106 20:15:34.786257 22261 slave.cpp:5286] Handling status update TASK_RUNNING (Status UUID: bb203ec6-f769-4029-a027-754959189c8a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db in health state healthy of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.800918 22258 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac from PREPARING to ISOLATING
I1106 20:15:34.801584 22258 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac from ISOLATING to FETCHING
I1106 20:15:34.801753 22258 fetcher.cpp:369] Starting to fetch URIs for container: 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac, directory: /tmp/LauncherAndIsolationParam_PersistentVolumeDefaultExecutor_ROOT_HealthCheckUsingPersistentVolume_1_2FgRMX/slaves/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0/frameworks/ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000/executors/default/runs/77a94bf0-25b5-476e-be40-8442c774cc39/containers/a8144d7f-ccc1-4476-b9d3-acb86132c45d/containers/check-2f466d67-e1b7-4721-9e60-1565886735ac
I1106 20:15:34.802112 22258 containerizer.cpp:3021] Transitioning the state of container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac from FETCHING to RUNNING
I1106 20:15:34.802254 22258 task_status_update_manager.cpp:328] Received task status update TASK_RUNNING (Status UUID: bb203ec6-f769-4029-a027-754959189c8a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db in health state healthy of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.802321 22258 task_status_update_manager.cpp:383] Forwarding task status update TASK_RUNNING (Status UUID: bb203ec6-f769-4029-a027-754959189c8a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db in health state healthy of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 to the agent
I1106 20:15:34.802428 22258 http.cpp:3600] Processing ATTACH_CONTAINER_OUTPUT call for container '77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d.check-2f466d67-e1b7-4721-9e60-1565886735ac'
I1106 20:15:34.802537 22258 slave.cpp:5778] Forwarding the update TASK_RUNNING (Status UUID: bb203ec6-f769-4029-a027-754959189c8a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db in health state healthy of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 to master@172.16.10.158:46488
I1106 20:15:34.802647 22258 slave.cpp:5671] Task status update manager successfully handled status update TASK_RUNNING (Status UUID: bb203ec6-f769-4029-a027-754959189c8a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db in health state healthy of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.802881 22258 master.cpp:8366] Status update TASK_RUNNING (Status UUID: bb203ec6-f769-4029-a027-754959189c8a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db in health state healthy of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 from agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.802917 22258 master.cpp:8423] Forwarding status update TASK_RUNNING (Status UUID: bb203ec6-f769-4029-a027-754959189c8a) for task 6317b43b-bde1-4c4f-a7c8-2187062839db in health state healthy of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.803084 22258 master.cpp:10876] Updating the state of task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (latest state: TASK_RUNNING, status update state: TASK_RUNNING)
I1106 20:15:34.804291 22262 scheduler.cpp:845] Enqueuing event UPDATE received from http://172.16.10.158:46488/master/api/v1/scheduler
I1106 20:15:34.809274 22259 master.cpp:1445] Framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default) disconnected
I1106 20:15:34.809304 22259 master.cpp:3303] Deactivating framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default)
I1106 20:15:34.809332 22259 master.cpp:3280] Disconnecting framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default)
I1106 20:15:34.809345 22259 master.cpp:1460] Giving framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default) 0ns to failover
I1106 20:15:34.809411 22259 hierarchical.cpp:407] Deactivated framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
*** Aborted at 1541535334 (unix time) try "date -d @1541535334" if you are using GNU date ***
PC: @ 0x7f8b9b4e1023 mesos::v1::scheduler::Mesos::send()
I1106 20:15:34.813431 22259 master.cpp:9252] Framework failover timeout, removing framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default)
I1106 20:15:34.813457 22259 master.cpp:10151] Removing framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (default)
I1106 20:15:34.813496 22259 master.cpp:10876] Updating the state of task 6317b43b-bde1-4c4f-a7c8-2187062839db of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (latest state: TASK_KILLED, status update state: TASK_KILLED)
I1106 20:15:34.813709 22259 master.cpp:10975] Removing task 6317b43b-bde1-4c4f-a7c8-2187062839db with resources cpus(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):31; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1 of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.813871 22259 master.cpp:11006] Removing executor 'default' with resources [{"allocation_info":{"role":"default-role"},"name":"cpus","scalar":{"value":0.1},"type":"SCALAR"},{"allocation_info":{"role":"default-role"},"name":"mem","scalar":{"value":32.0},"type":"SCALAR"},{"allocation_info":{"role":"default-role"},"name":"disk","scalar":{"value":32.0},"type":"SCALAR"}] of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 at slave(1090)@172.16.10.158:46488 (ip-172-16-10-158.ec2.internal)
I1106 20:15:34.814097 22259 slave.cpp:3913] Asked to shut down framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 by master@172.16.10.158:46488
I1106 20:15:34.814124 22259 slave.cpp:3938] Shutting down framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.814136 22259 slave.cpp:6659] Shutting down executor 'default' of framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000 (via HTTP)
I1106 20:15:34.814668 22259 hierarchical.cpp:1195] Recovered cpus(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)]):31; disk(allocated: default-role)(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1 (total: cpus:1.9; mem:992; disk:992; ports:[31000-32000]; cpus(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(reservations: [(DYNAMIC,default-role,test-principal)]):31; disk(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1, allocated: cpus(allocated: default-role):0.1; mem(allocated: default-role):32; disk(allocated: default-role):32) on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 from framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.814868 22259 hierarchical.cpp:1195] Recovered cpus(allocated: default-role):0.1; mem(allocated: default-role):32; disk(allocated: default-role):32 (total: cpus:1.9; mem:992; disk:992; ports:[31000-32000]; cpus(reservations: [(DYNAMIC,default-role,test-principal)]):0.1; mem(reservations: [(DYNAMIC,default-role,test-principal)]):32; disk(reservations: [(DYNAMIC,default-role,test-principal)]):31; disk(reservations: [(DYNAMIC,default-role,test-principal)])[id1:task_volume_path]:1, allocated: {}) on agent ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-S0 from framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.815014 22259 hierarchical.cpp:346] Removed framework ee3a72ac-f1ea-4572-ab7a-424ecc6e517c-0000
I1106 20:15:34.816267 22259 process.cpp:3588] Handling HTTP event for process 'slave(1090)' with path: '/slave(1090)/api/v1'
I1106 20:15:34.816480 22259 http.cpp:1099] HTTP POST for /slave(1090)/api/v1 from 172.16.10.158:55310
I1106 20:15:34.816624 22259 http.cpp:2802] Processing KILL_NESTED_CONTAINER call for container '77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d'
I1106 20:15:34.816866 22259 containerizer.cpp:2737] Sending Terminated to container 77a94bf0-25b5-476e-be40-8442c774cc39.a8144d7f-ccc1-4476-b9d3-acb86132c45d in RUNNING state
*** SIGSEGV (@0x0) received by PID 32499 (TID 0x7f8b8e76d700) from PID 0; stack trace: ***
@ 0x7f8b6590b3a2 (unknown)
@ 0x7f8b6590f8f9 (unknown)
@ 0x7f8b65902fb8 (unknown)
@ 0x7f8b989b5390 (unknown)
@ 0x7f8b9b4e1023 mesos::v1::scheduler::Mesos::send()
@ 0x55f895654ec6 _ZNK5mesos8internal5tests2v19scheduler23SendAcknowledgeActionP2INS_2v111FrameworkIDENS5_7AgentIDEE10gmock_ImplIFvPNS5_9scheduler5MesosERKNSA_12Event_UpdateEEE17gmock_PerformImplISC_SF_N7testing8internal12ExcessiveArgESL_SL_SL_SL_SL_SL_SL_EEvRKSt5tupleIJSC_SF_EET_T0_T1_T2_T3_T4_T5_T6_T7_T8_
@ 0x55f895655040 _ZN5mesos8internal5tests2v19scheduler23SendAcknowledgeActionP2INS_2v111FrameworkIDENS5_7AgentIDEE10gmock_ImplIFvPNS5_9scheduler5MesosERKNSA_12Event_UpdateEEE7PerformERKSt5tupleIJSC_SF_EE
@ 0x55f89553f0be _ZN7testing8internal12DoBothActionI17PromiseArgActionPILi1EPN7process7PromiseIN5mesos2v19scheduler12Event_UpdateEEEENS5_8internal5tests2v19scheduler23SendAcknowledgeActionP2INS6_11FrameworkIDENS6_7AgentIDEEEE4ImplIFvPNS7_5MesosERKS8_EE7PerformERKSt5tupleIJSN_SP_EE
@ 0x55f895574417 testing::internal::FunctionMockerBase<>::UntypedPerformAction()
@ 0x55f896872059 testing::internal::UntypedFunctionMockerBase::UntypedInvokeWith()
@ 0x55f895657e1a mesos::internal::tests::scheduler::MockHTTPScheduler<>::events()
@ 0x55f8955d45e6 std::_Function_handler<>::_M_invoke()
@ 0x7f8b9b4e6128 process::AsyncExecutorProcess::execute<>()
@ 0x7f8b9b4f0dad _ZNO6lambda12CallableOnceIFvPN7process11ProcessBaseEEE10CallableFnINS_8internal7PartialIZNS1_8dispatchI7NothingNS1_20AsyncExecutorProcessERKSt8functionIFvRKSt5queueIN5mesos2v19scheduler5EventESt5dequeISH_SaISH_EEEEESL_SR_RSL_EENS1_6FutureIT_EERKNS1_3PIDIT0_EEMSX_FSU_T1_T2_EOT3_OT4_EUlSt10unique_ptrINS1_7PromiseISA_EESt14default_deleteIS1B_EEOSP_OSL_S3_E_IS1E_SP_SL_St12_PlaceholderILi1EEEEEEclEOS3_
@ 0x7f8b9c1907d1 process::ProcessBase::consume()
@ 0x7f8b9c1b2b9a process::ProcessManager::resume()
@ 0x7f8b9c1b68e6 _ZNSt6thread5_ImplISt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvEUlvE_vEEE6_M_runEv
@ 0x7f8b99198c80 (unknown)
@ 0x7f8b989ab6ba start_thread
@ 0x7f8b986e141d (unknown)
{code}
> ROOT_HealthCheckUsingPersistentVolume is flaky.
> -----------------------------------------------
>
> Key: MESOS-8723
> URL: https://issues.apache.org/jira/browse/MESOS-8723
> Project: Mesos
> Issue Type: Bug
> Components: test
> Affects Versions: 1.5.0
> Environment: ec2's CentOS 7 with SSL
> Reporter: Alexander Rukletsov
> Priority: Major
> Labels: flaky-test
> Attachments: ROOT_HealthCheckUsingPersistentVolume-badrun.txt
>
>
> {noformat}
> ../../src/tests/cluster.cpp:660: Failure
> Failed to wait 15secs for destroy
> I0321 19:45:11.676262 8064 master.cpp:1137] Master terminating
> I0321 19:45:11.676625 27242 hierarchical.cpp:609] Removed agent b7675b9a-d9e9-4c97-a5c2-d50fc6101301-S0
> {noformat}
> Full log attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)