You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Alexander Rukletsov (JIRA)" <ji...@apache.org> on 2017/10/17 19:49:02 UTC
[jira] [Updated] (MESOS-6792)
MasterSlaveReconciliationTest.ReconcileLostTask test is flaky
[ https://issues.apache.org/jira/browse/MESOS-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Rukletsov updated MESOS-6792:
---------------------------------------
Description:
The test {{MasterSlaveReconciliationTest.ReconcileLostTask}} is flaky for me as of {{e99ea9ce8b1de01dd8b3cac6675337edb6320f38}},
{code}
Repeating all tests (iteration 912) . . .
Note: Google Test filter = <...>
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from MasterSlaveReconciliationTest
[ RUN ] MasterSlaveReconciliationTest.SlaveReregisterTerminatedExecutor
I1214 04:41:11.559672 2005 cluster.cpp:160] Creating default 'local' authorizer
I1214 04:41:11.560848 2045 master.cpp:380] Master 87dd8179-dd7d-4270-ace2-ea771b57371c (gru1.hw.ca1.mesosphere.com) started on 192.99.40.208:37659
I1214 04:41:11.560878 2045 master.cpp:382] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="true" --authenticate_frameworks="true" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticators="crammd5" --authorizers="local" --credentials="/tmp/cXHI89/credentials" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memory" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/home/bbannier/src/mesos/build/P/share/mesos/webui" --work_dir="/tmp/cXHI89/master" --zk_session_timeout="10secs"
I1214 04:41:11.561079 2045 master.cpp:432] Master only allowing authenticated frameworks to register
I1214 04:41:11.561089 2045 master.cpp:446] Master only allowing authenticated agents to register
I1214 04:41:11.561095 2045 master.cpp:459] Master only allowing authenticated HTTP frameworks to register
I1214 04:41:11.561101 2045 credentials.hpp:39] Loading credentials for authentication from '/tmp/cXHI89/credentials'
I1214 04:41:11.561194 2045 master.cpp:504] Using default 'crammd5' authenticator
I1214 04:41:11.561236 2045 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-master-readonly'
I1214 04:41:11.561274 2045 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-master-readwrite'
I1214 04:41:11.561301 2045 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-master-scheduler'
I1214 04:41:11.561326 2045 master.cpp:584] Authorization enabled
I1214 04:41:11.562155 2039 master.cpp:2045] Elected as the leading master!
I1214 04:41:11.562173 2039 master.cpp:1568] Recovering from registrar
I1214 04:41:11.562347 2045 registrar.cpp:362] Successfully fetched the registry (0B) in 114944ns
I1214 04:41:11.562441 2045 registrar.cpp:461] Applied 1 operations in 7920ns; attempting to update the registry
I1214 04:41:11.562621 2048 registrar.cpp:506] Successfully updated the registry in 155136ns
I1214 04:41:11.562664 2048 registrar.cpp:392] Successfully recovered registrar
I1214 04:41:11.562832 2044 master.cpp:1684] Recovered 0 agents from the registry (166B); allowing 10mins for agents to re-register
I1214 04:41:11.568444 2005 cluster.cpp:446] Creating default 'local' authorizer
I1214 04:41:11.569344 2005 sched.cpp:232] Version: 1.2.0
I1214 04:41:11.569842 2035 slave.cpp:209] Mesos agent started on (912)@192.99.40.208:37659
I1214 04:41:11.570080 2040 sched.cpp:336] New master detected at master@192.99.40.208:37659
I1214 04:41:11.570117 2040 sched.cpp:402] Authenticating with master master@192.99.40.208:37659
I1214 04:41:11.570127 2040 sched.cpp:409] Using default CRAM-MD5 authenticatee
I1214 04:41:11.570220 2040 authenticatee.cpp:121] Creating new client SASL connection
I1214 04:41:11.570000 2035 slave.cpp:210] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/http_credentials" --http_heartbeat_interval="30secs" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/home/bbannier/src/mesos/build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc"
I1214 04:41:11.570297 2035 credentials.hpp:88] Loading credential for authentication from '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/credential'
I1214 04:41:11.570307 2040 master.cpp:6748] Authenticating scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.570351 2035 slave.cpp:352] Agent using credential for: test-principal
I1214 04:41:11.570363 2035 credentials.hpp:39] Loading credentials for authentication from '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/http_credentials'
I1214 04:41:11.570405 2040 authenticator.cpp:98] Creating new server SASL connection
I1214 04:41:11.570439 2035 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
I1214 04:41:11.570461 2040 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
I1214 04:41:11.570478 2035 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
I1214 04:41:11.570752 2035 slave.cpp:539] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I1214 04:41:11.570791 2035 slave.cpp:547] Agent attributes: [ ]
I1214 04:41:11.570801 2035 slave.cpp:552] Agent hostname: gru1.hw.ca1.mesosphere.com
I1214 04:41:11.571265 2035 state.cpp:57] Recovering state from '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc/meta'
I1214 04:41:11.571367 2035 status_update_manager.cpp:203] Recovering status update manager
I1214 04:41:11.571449 2035 slave.cpp:5420] Finished recovery
I1214 04:41:11.571684 2035 slave.cpp:924] New master detected at master@192.99.40.208:37659
I1214 04:41:11.571702 2035 slave.cpp:983] Authenticating with master master@192.99.40.208:37659
I1214 04:41:11.571720 2035 slave.cpp:994] Using default CRAM-MD5 authenticatee
I1214 04:41:11.571751 2035 slave.cpp:956] Detecting new master
I1214 04:41:11.571807 2035 status_update_manager.cpp:177] Pausing sending status updates
I1214 04:41:11.571841 2035 authenticatee.cpp:121] Creating new client SASL connection
I1214 04:41:11.571909 2035 master.cpp:6748] Authenticating slave(912)@192.99.40.208:37659
I1214 04:41:11.571982 2035 authenticator.cpp:98] Creating new server SASL connection
I1214 04:41:11.572031 2035 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
I1214 04:41:11.572052 2035 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
I1214 04:41:11.572078 2035 authenticator.cpp:204] Received SASL authentication start
I1214 04:41:11.572108 2035 authenticator.cpp:326] Authentication requires more steps
I1214 04:41:11.572135 2035 authenticatee.cpp:259] Received SASL authentication step
I1214 04:41:11.572171 2035 authenticator.cpp:232] Received SASL authentication step
I1214 04:41:11.572201 2035 authenticator.cpp:318] Authentication success
I1214 04:41:11.572234 2035 authenticatee.cpp:299] Authentication success
I1214 04:41:11.572257 2035 master.cpp:6778] Successfully authenticated principal 'test-principal' at slave(912)@192.99.40.208:37659
I1214 04:41:11.572331 2035 slave.cpp:1078] Successfully authenticated with master master@192.99.40.208:37659
I1214 04:41:11.572430 2035 master.cpp:5161] Registering agent at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with id 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.572541 2035 registrar.cpp:461] Applied 1 operations in 18957ns; attempting to update the registry
I1214 04:41:11.572929 2035 registrar.cpp:506] Successfully updated the registry in 363008ns
I1214 04:41:11.573122 2035 master.cpp:5232] Registered agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I1214 04:41:11.573240 2035 hierarchical.cpp:490] Added agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (allocated: {})
I1214 04:41:11.573312 2035 slave.cpp:1124] Registered with master master@192.99.40.208:37659; given agent ID 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.573545 2035 slave.cpp:1184] Forwarding total oversubscribed resources {}
I1214 04:41:11.573599 2035 status_update_manager.cpp:184] Resuming sending status updates
I1214 04:41:11.573631 2035 master.cpp:5633] Received update of agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with total oversubscribed resources {}
I1214 04:41:11.573696 2035 hierarchical.cpp:560] Agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com) updated with oversubscribed resources {} (total: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000], allocated: {})
I1214 04:41:11.570480 2040 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
I1214 04:41:11.573894 2040 authenticator.cpp:204] Received SASL authentication start
I1214 04:41:11.573921 2040 authenticator.cpp:326] Authentication requires more steps
I1214 04:41:11.573951 2040 authenticatee.cpp:259] Received SASL authentication step
I1214 04:41:11.573988 2040 authenticator.cpp:232] Received SASL authentication step
I1214 04:41:11.574015 2040 authenticator.cpp:318] Authentication success
I1214 04:41:11.574054 2040 authenticatee.cpp:299] Authentication success
I1214 04:41:11.574079 2040 master.cpp:6778] Successfully authenticated principal 'test-principal' at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.574154 2040 sched.cpp:508] Successfully authenticated with master master@192.99.40.208:37659
I1214 04:41:11.574247 2040 master.cpp:2633] Received SUBSCRIBE call for framework 'default' at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.574265 2040 master.cpp:2081] Authorizing framework principal 'test-principal' to receive offers for role '*'
I1214 04:41:11.574359 2040 master.cpp:2709] Subscribing framework default with checkpointing disabled and capabilities [ ]
I1214 04:41:11.574496 2040 hierarchical.cpp:276] Added framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.574750 2040 sched.cpp:749] Framework registered with 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.574884 2040 master.cpp:6577] Sending 1 offers to framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.575361 2040 master.cpp:3588] Processing ACCEPT call for offers: [ 87dd8179-dd7d-4270-ace2-ea771b57371c-O0 ] on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.575392 2040 master.cpp:3175] Authorizing framework principal 'test-principal' to launch task 0
W1214 04:41:11.575858 2040 validation.cpp:1001] Executor 'default' for task '0' uses less CPUs (None) than the minimum required (0.01). Please update your executor, as this will be mandatory in future releases.
W1214 04:41:11.575880 2040 validation.cpp:1013] Executor 'default' for task '0' uses less memory (None) than the minimum required (32MB). Please update your executor, as this will be mandatory in future releases.
I1214 04:41:11.575949 2040 master.cpp:8501] Adding task 0 with resources cpus(*):1; mem(*):512 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.575989 2040 master.cpp:4240] Launching task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659 with resources cpus(*):1; mem(*):512 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.576154 2040 slave.cpp:1556] Got assigned task '0' for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.576323 2040 slave.cpp:1718] Launching task '0' for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.576510 2040 paths.cpp:530] Trying to chown '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc/slaves/87dd8179-dd7d-4270-ace2-ea771b57371c-S0/frameworks/87dd8179-dd7d-4270-ace2-ea771b57371c-0000/executors/default/runs/d6293832-3a4c-4731-b4de-da91f64c1c09' to user 'bbannier'
I1214 04:41:11.586915 2040 slave.cpp:6347] Launching executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 with resources {} in work directory '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc/slaves/87dd8179-dd7d-4270-ace2-ea771b57371c-S0/frameworks/87dd8179-dd7d-4270-ace2-ea771b57371c-0000/executors/default/runs/d6293832-3a4c-4731-b4de-da91f64c1c09'
I1214 04:41:11.587187 2040 slave.cpp:2040] Queued task '0' for executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.588081 2034 exec.cpp:162] Version: 1.2.0
I1214 04:41:11.588315 2035 slave.cpp:3314] Got registration for executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from executor(912)@192.99.40.208:37659
I1214 04:41:11.588631 2046 exec.cpp:237] Executor registered on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.588687 2035 slave.cpp:2256] Sending queued task '0' to executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 at executor(912)@192.99.40.208:37659
I1214 04:41:11.588961 2034 slave.cpp:3749] Handling status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from executor(912)@192.99.40.208:37659
I1214 04:41:11.589154 2044 status_update_manager.cpp:323] Received status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.589452 2044 slave.cpp:4190] Forwarding the update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to master@192.99.40.208:37659
I1214 04:41:11.589516 2044 slave.cpp:4100] Sending acknowledgement for status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to executor(912)@192.99.40.208:37659
I1214 04:41:11.589596 2044 master.cpp:5769] Status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.589625 2044 master.cpp:5831] Forwarding status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.589680 2044 master.cpp:7867] Updating the state of task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (latest state: TASK_RUNNING, status update state: TASK_RUNNING)
I1214 04:41:11.589999 2034 master.cpp:4877] Processing ACKNOWLEDGE call c9738162-8aa4-40f7-9be8-ec1d4877a94c for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.590206 2045 status_update_manager.cpp:395] Received status update acknowledgement (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.591419 2043 slave.cpp:3749] Handling status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from executor(912)@192.99.40.208:37659
I1214 04:41:11.591619 2043 status_update_manager.cpp:323] Received status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.591711 2043 slave.cpp:4190] Forwarding the update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to master@192.99.40.208:37659
I1214 04:41:11.591760 2043 slave.cpp:4100] Sending acknowledgement for status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to executor(912)@192.99.40.208:37659
I1214 04:41:11.592257 2043 slave.cpp:924] New master detected at master@192.99.40.208:37659
I1214 04:41:11.592280 2043 slave.cpp:983] Authenticating with master master@192.99.40.208:37659
I1214 04:41:11.592298 2043 slave.cpp:994] Using default CRAM-MD5 authenticatee
I1214 04:41:11.592313 2039 status_update_manager.cpp:177] Pausing sending status updates
I1214 04:41:11.592341 2043 slave.cpp:956] Detecting new master
I1214 04:41:11.592363 2039 authenticatee.cpp:121] Creating new client SASL connection
I1214 04:41:11.592383 2043 slave.cpp:4318] Got exited event for executor(912)@192.99.40.208:37659
I1214 04:41:11.592438 2039 master.cpp:6748] Authenticating slave(912)@192.99.40.208:37659
I1214 04:41:11.592550 2039 authenticator.cpp:98] Creating new server SASL connection
I1214 04:41:11.592614 2039 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
I1214 04:41:11.592635 2039 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
I1214 04:41:11.592672 2039 authenticator.cpp:204] Received SASL authentication start
I1214 04:41:11.592707 2039 authenticator.cpp:326] Authentication requires more steps
I1214 04:41:11.592741 2039 authenticatee.cpp:259] Received SASL authentication step
I1214 04:41:11.592779 2039 authenticator.cpp:232] Received SASL authentication step
I1214 04:41:11.592819 2039 authenticator.cpp:318] Authentication success
I1214 04:41:11.592850 2039 authenticatee.cpp:299] Authentication success
I1214 04:41:11.592870 2039 master.cpp:6778] Successfully authenticated principal 'test-principal' at slave(912)@192.99.40.208:37659
I1214 04:41:11.592921 2039 slave.cpp:1078] Successfully authenticated with master master@192.99.40.208:37659
I1214 04:41:11.593102 2039 master.cpp:5311] Re-registering agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.593189 2039 master.cpp:5572] Sending updated checkpointed resources {} to agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.593250 2039 slave.cpp:1226] Re-registered with master master@192.99.40.208:37659
I1214 04:41:11.593283 2039 slave.cpp:1262] Forwarding total oversubscribed resources {}
I1214 04:41:11.593329 2039 status_update_manager.cpp:184] Resuming sending status updates
W1214 04:41:11.593340 2039 status_update_manager.cpp:191] Resending status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.593410 2039 master.cpp:5633] Received update of agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with total oversubscribed resources {}
I1214 04:41:11.593451 2039 slave.cpp:4190] Forwarding the update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to master@192.99.40.208:37659
I1214 04:41:11.593497 2039 hierarchical.cpp:560] Agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com) updated with oversubscribed resources {} (total: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000], allocated: cpus(*):1; mem(*):512)
I1214 04:41:11.593605 2039 master.cpp:5769] Status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.593619 2039 master.cpp:5831] Forwarding status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.593657 2039 master.cpp:7867] Updating the state of task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (latest state: TASK_FINISHED, status update state: TASK_FINISHED)
I1214 04:41:11.593978 2041 master.cpp:4877] Processing ACKNOWLEDGE call 79f94437-6066-412e-9de6-6d9a1f256539 for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.594013 2041 master.cpp:7963] Removing task 0 with resources cpus(*):1; mem(*):512 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.594048 2005 sched.cpp:2008] Asked to stop the driver
I1214 04:41:11.594158 2048 sched.cpp:1193] Stopping framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594192 2041 status_update_manager.cpp:395] Received status update acknowledgement (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594251 2048 master.cpp:7287] Processing TEARDOWN call for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.594272 2048 master.cpp:7299] Removing framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.594336 2048 master.cpp:7992] Removing executor 'default' with resources {} of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.594383 2041 hierarchical.cpp:391] Deactivated framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594450 2041 hierarchical.cpp:342] Removed framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594570 2042 slave.cpp:2609] Shutting down framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594593 2042 slave.cpp:4999] Shutting down executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 at executor(912)@192.99.40.208:37659
I1214 04:41:11.594758 2040 slave.cpp:4681] Executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 exited with status 0
I1214 04:41:11.594861 2040 slave.cpp:4785] Cleaning up executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 at executor(912)@192.99.40.208:37659
W1214 04:41:11.594920 2041 master.cpp:5887] Ignoring unknown exited executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
../../src/tests/master_slave_reconciliation_tests.cpp:145: Failure
Actual function call count doesn't match EXPECT_CALL(sched, executorLost(&driver, createExecutorInfo("default", "exit 1").executor_id(), _, _))...
Expected: to be called once
Actual: never called - unsatisfied and active
*** Aborted at 1481708471 (unix time) try "date -d @1481708471" if you are using GNU date ***
I1214 04:41:11.595078 2040 slave.cpp:4873] Cleaning up framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
PC: @ 0x1d5c445 testing::UnitTest::AddTestPartResult()
*** SIGSEGV (@0x0) received by PID 2005 (TID 0x7f04125ff8c0) from PID 0; stack trace: ***
@ 0x7f040d5c95c0 (unknown)
@ 0x1d5c445 testing::UnitTest::AddTestPartResult()
@ 0x1d5c0af testing::internal::AssertHelper::operator=()
@ 0x1d8b61f testing::internal::GoogleTestFailureReporter::ReportFailure()
@ 0x1d80511 testing::internal::UntypedFunctionMockerBase::VerifyAndClearExpectationsLocked()
@ 0x1136031 testing::internal::FunctionMockerBase<>::~FunctionMockerBase()
@ 0x112e023 mesos::internal::tests::MockScheduler::~MockScheduler()
@ 0xf66780 mesos::internal::tests::MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Test::TestBody()
@ 0x1d84a46 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x1d6450d testing::Test::Run()
@ 0x1d65520 testing::TestInfo::Run()
@ 0x1d65c23 testing::TestCase::Run()
@ 0x1d6d8b9 testing::internal::UnitTestImpl::RunAllTests()
@ 0x1d8539f testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x1d6d55f testing::UnitTest::Run()
@ 0xd5d8bb main
@ 0x7f040c348401 __libc_start_main
@ 0x51a81a _start
{code}
was:
The test {{MasterSlaveReconciliationTest.ReconcileLostTask}} is flaky for me as of {{e99ea9ce8b1de01dd8b3cac6675337edb6320f38}},
{code}
Repeating all tests (iteration 912) . . .
Note: Google Test filter = MasterSlaveReconciliationTest.SlaveReregisterTerminatedExecutor-ContainerLoggerTest.LOGROTATE_RotateInSandbox:ContainerLoggerTest.LOGROTATE_CustomRotateOptions:ContainerLoggerTest.LOGROTATE_ModuleFDOwnership:FetcherTest.ROOT_RootProtectedFileURI:GarbageCollectorIntegrationTest.ROOT_BusyMountPoint:HealthCheckTest.ROOT_HealthyTaskWithContainerImage:HealthCheckTest.ROOT_DOCKER_DockerHealthyTask:HealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange:HealthCheckTest.ROOT_INTERNET_CURL_HealthyTaskViaHTTPWithContainerImage:HealthCheckTest.ROOT_INTERNET_CURL_HealthyTaskViaHTTPSWithContainerImage:HealthCheckTest.ROOT_INTERNET_CURL_HealthyTaskViaTCPWithContainerImage:HealthCheckTest.ROOT_DOCKER_DockerHealthyTaskViaHTTP:HealthCheckTest.ROOT_DOCKER_DockerHealthyTaskViaHTTPS:HealthCheckTest.ROOT_DOCKER_DockerHealthyTaskViaTCP:HookTest.ROOT_DOCKER_VerifySlavePreLaunchDockerTaskExecutorDecorator:HookTest.ROOT_DOCKER_VerifySlavePreLaunchDockerValidator:HookTest.ROOT_DOCKER_VerifySlavePreLaunchDockerHook:HookTest.ROOT_DOCKER_VerifySlavePostFetchHook:Resources_Filter_BENCHMARK_Test.Filters:SlaveRecoveryTest/0.DISABLED_ROOT_CGROUPS_ReconnectDefaultExecutor:MesosContainerizerSlaveRecoveryTest.CGROUPS_ROOT_PidNamespaceForward:MesosContainerizerSlaveRecoveryTest.CGROUPS_ROOT_PidNamespaceBackward:SlaveTest.ROOT_LaunchTaskInfoWithContainerInfo:SlaveTest.ROOT_RunTaskWithCommandInfoWithoutUser:SlaveTest.DISABLED_ROOT_RunTaskWithCommandInfoWithUser:DockerContainerizerTest.ROOT_DOCKER_Launch_Executor:DockerContainerizerTest.DISABLED_ROOT_DOCKER_Launch_Executor_Bridged:DockerContainerizerTest.ROOT_DOCKER_Launch:DockerContainerizerTest.ROOT_DOCKER_Kill:DockerContainerizerTest.ROOT_DOCKER_TaskKillingCapability:DockerContainerizerTest.ROOT_DOCKER_Usage:DockerContainerizerTest.ROOT_DOCKER_Update:DockerContainerizerTest.ROOT_DOCKER_Recover:DockerContainerizerTest.ROOT_DOCKER_KillOrphanContainers:DockerContainerizerTest.ROOT_DOCKER_SkipRecoverNonDocker:DockerContainerizerTest.ROOT_DOCKER_SkipRecoverMalformedUUID:DockerContainerizerTest.ROOT_DOCKER_LaunchWithPersistentVolumes:DockerContainerizerTest.ROOT_DOCKER_RecoverPersistentVolumes:DockerContainerizerTest.ROOT_DOCKER_RecoverOrphanedPersistentVolumes:DockerContainerizerTest.ROOT_DOCKER_Logs:DockerContainerizerTest.ROOT_DOCKER_Default_CMD:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Override:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Args:DockerContainerizerTest.ROOT_DOCKER_SlaveRecoveryTaskContainer:DockerContainerizerTest.DISABLED_ROOT_DOCKER_SlaveRecoveryExecutorContainer:DockerContainerizerTest.ROOT_DOCKER_NC_PortMapping:DockerContainerizerTest.ROOT_DOCKER_LaunchSandboxWithColon:DockerContainerizerTest.ROOT_DOCKER_DestroyWhileFetching:DockerContainerizerTest.ROOT_DOCKER_DestroyWhilePulling:DockerContainerizerTest.ROOT_DOCKER_DestroyUnknownContainer:DockerContainerizerTest.ROOT_DOCKER_ExecutorCleanupWhenLaunchFailed:DockerContainerizerTest.ROOT_DOCKER_FetchFailure:DockerContainerizerTest.ROOT_DOCKER_DockerPullFailure:DockerContainerizerTest.ROOT_DOCKER_DockerInspectDiscard:DockerContainerizerTest.ROOT_DOCKER_WaitUnknownContainer:DockerContainerizerTest.ROOT_DOCKER_NoTransitionFromKillingToRunning:DockerTest.ROOT_DOCKER_interface:DockerTest.ROOT_DOCKER_kill:DockerTest.ROOT_DOCKER_Version:DockerTest.ROOT_DOCKER_CheckCommandWithShell:DockerTest.ROOT_DOCKER_CheckPortResource:DockerTest.ROOT_DOCKER_CancelPull:DockerTest.ROOT_DOCKER_MountRelativeHostPath:DockerTest.ROOT_DOCKER_MountAbsoluteHostPath:DockerTest.ROOT_DOCKER_MountRelativeContainerPath:DockerTest.ROOT_DOCKER_MountRelativeHostPathRelativeContainerPath:DockerTest.ROOT_DOCKER_NVIDIA_GPU_DeviceAllow:DockerTest.ROOT_DOCKER_NVIDIA_GPU_InspectDevices:DockerTest.ROOT_DOCKER_ConflictingVolumeDriversInMultipleVolumes:DockerTest.ROOT_DOCKER_ConflictingVolumeDrivers:NamespacesIsolatorTest.ROOT_PidNamespace:NamespacesIsolatorTest.ROOT_IPCNamespace:MesosContainerizerExecuteTest.ROOT_SandboxFileOwnership:ProvisionerAppcTest.ROOT_Provision:ProvisionerAppcTest.ROOT_ProvisionNestedContainer:AppcProvisionerIntegrationTest.ROOT_SimpleLinuxImageTest:OverlayBackendTest.ROOT_OVERLAYFS_OverlayFSBackend:OverlayBackendTest.ROOT_OVERLAYFS_OverlayFSBackendWithManyLayers:BindBackendTest.ROOT_BindBackend:AufsBackendTest.ROOT_AUFS_AufsBackend:CopyBackendTest.ROOT_CopyBackend:ProvisionerDockerPullerTest.ROOT_LocalPullerSimpleCommand:ProvisionerDockerPullerTest.ROOT_INTERNET_CURL_SimpleCommand:ProvisionerDockerPullerTest.ROOT_INTERNET_CURL_Normalize:ProvisionerDockerPullerTest.ROOT_INTERNET_CURL_ScratchImage:CapabilitiesTest.ROOT_PingWithNoNetRawCaps:CapabilitiesTest.ROOT_PingWithNoNetRawCapsChangeUser:CapabilitiesTest.ROOT_PingWithJustNetRawSysAdminCap:ContainerizerTest.ROOT_CGROUPS_BalloonFramework:CgroupsIsolatorTest.ROOT_CGROUPS_PERF_NET_CLS_UserCgroup:CgroupsIsolatorTest.ROOT_CGROUPS_RevocableCpu:CgroupsIsolatorTest.ROOT_CGROUPS_CFS_EnableCfs:CgroupsIsolatorTest.ROOT_CGROUPS_PidsAndTids:CgroupsIsolatorTest.ROOT_CGROUPS_NET_CLS_Isolate:CgroupsIsolatorTest.ROOT_CGROUPS_NET_CLS_ContainerStatus:CgroupsIsolatorTest.ROOT_CGROUPS_PERF_Sample:CgroupsIsolatorTest.ROOT_CGROUPS_PERF_PerfForward:CgroupsIsolatorTest.ROOT_CGROUPS_MemoryForward:CgroupsIsolatorTest.ROOT_CGROUPS_MemoryBackward:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Enabled:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Subsystems:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Mounted:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Get:CgroupsAnyHierarchyTest.ROOT_CGROUPS_NestedCgroups:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Tasks:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Read:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Write:CgroupsAnyHierarchyTest.ROOT_CGROUPS_CFS_Big_Quota:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_Busy:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_SubsystemsHierarchy:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_FindCgroupSubsystems:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_MountedSubsystems:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_CreateRemove:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_Listen:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_FreezeNonFreezer:CgroupsNoHierarchyTest.ROOT_CGROUPS_NOHIERARCHY_MountUnmountHierarchy:CgroupsAnyHierarchyWithCpuAcctMemoryTest.ROOT_CGROUPS_Stat:CgroupsAnyHierarchyWithCpuAcctMemoryTest.ROOT_CGROUPS_CpuAcctsStats:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_Freeze:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_Kill:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_Destroy:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_AssignThreads:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_DestroyStoppedProcess:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_DestroyTracedProcess:CgroupsAnyHierarchyWithPerfEventTest.ROOT_CGROUPS_PERF_PerfTest:CgroupsAnyHierarchyMemoryPressureTest.ROOT_IncreaseRSS:CgroupsAnyHierarchyMemoryPressureTest.ROOT_IncreasePageCache:CgroupsAnyHierarchyDevicesTest.ROOT_CGROUPS_Devices:CniIsolatorTest.ROOT_INTERNET_CURL_LaunchCommandTask:CniIsolatorTest.ROOT_VerifyCheckpointedInfo:CniIsolatorTest.ROOT_FailedPlugin:CniIsolatorTest.ROOT_SlaveRecovery:CniIsolatorTest.ROOT_EnvironmentLibprocessIP:CniIsolatorTest.ROOT_INTERNET_CURL_LaunchContainerInHostNetwork:DockerVolumeIsolatorTest.ROOT_CommandTaskNoRootfsWithVolumes:DockerVolumeIsolatorTest.ROOT_CommandTaskNoRootfsFailedWithSameVolumes:DockerVolumeIsolatorTest.ROOT_CommandTaskNoRootfsSlaveRecovery:DockerVolumeIsolatorTest.ROOT_CommandTaskNoRootfsSingleVolumeMultipleContainers:DockerVolumeIsolatorTest.ROOT_INTERNET_CURL_CommandTaskRootfsWithAbsolutePathVolume:DockerVolumeIsolatorTest.ROOT_INTERNET_CURL_CommandTaskRootfsWithRelativeVolume:LinuxFilesystemIsolatorTest.ROOT_ChangeRootFilesystem:LinuxFilesystemIsolatorTest.ROOT_Metrics:LinuxFilesystemIsolatorTest.ROOT_VolumeFromSandbox:LinuxFilesystemIsolatorTest.ROOT_VolumeFromHost:LinuxFilesystemIsolatorTest.ROOT_FileVolumeFromHost:LinuxFilesystemIsolatorTest.ROOT_VolumeFromHostSandboxMountPoint:LinuxFilesystemIsolatorTest.ROOT_FileVolumeFromHostSandboxMountPoint:LinuxFilesystemIsolatorTest.ROOT_PersistentVolumeWithRootFilesystem:LinuxFilesystemIsolatorTest.ROOT_PersistentVolumeWithoutRootFilesystem:LinuxFilesystemIsolatorTest.ROOT_MultipleContainers:LinuxFilesystemIsolatorTest.ROOT_WorkDirMountNotNeeded:LinuxFilesystemIsolatorTest.ROOT_WorkDirMountNeeded:LinuxFilesystemIsolatorMesosTest.ROOT_ChangeRootFilesystemCommandExecutor:LinuxFilesystemIsolatorMesosTest.ROOT_ChangeRootFilesystemCommandExecutorWithHostVolumes:LinuxFilesystemIsolatorMesosTest.ROOT_ChangeRootFilesystemCommandExecutorPersistentVolume:LinuxFilesystemIsolatorMesosTest.ROOT_RecoverOrphanedPersistentVolume:LinuxFilesystemIsolatorMesosTest.ROOT_SandboxEnvironmentVariable:LinuxFilesystemIsolatorMesosTest.ROOT_VolumeUsageExceedsSandboxQuota:LinuxFilesystemIsolatorMesosTest.ROOT_WriteAccessSharedPersistentVolumeReadOnlyMode:FsTest.ROOT_SharedMount:FsTest.ROOT_SlaveMount:MemoryPressureMesosTest.CGROUPS_ROOT_Statistics:MemoryPressureMesosTest.CGROUPS_ROOT_SlaveRecovery:NestedMesosContainerizerTest.ROOT_CGROUPS_LaunchNested:NestedMesosContainerizerTest.ROOT_CGROUPS_LaunchNestedDebugCheckPidNamespace:NestedMesosContainerizerTest.ROOT_CGROUPS_CURL_INTERNET_LaunchNestedDebugCheckMntNamespace:NestedMesosContainerizerTest.ROOT_CGROUPS_DestroyDebugContainerOnRecover:NestedMesosContainerizerTest.ROOT_CGROUPS_DestroyNested:NestedMesosContainerizerTest.ROOT_CGROUPS_DestroyParent:NestedMesosContainerizerTest.ROOT_CGROUPS_ParentExit:NestedMesosContainerizerTest.ROOT_CGROUPS_ParentSigterm:NestedMesosContainerizerTest.ROOT_CGROUPS_RecoverNested:NestedMesosContainerizerTest.ROOT_CGROUPS_RecoverLauncherOrphans:NestedMesosContainerizerTest.ROOT_CGROUPS_RecoverNestedLauncherOrphans:NestedMesosContainerizerTest.ROOT_CGROUPS_RecoverLauncherOrphanAndSingleNestedLauncherOrphan:NestedMesosContainerizerTest.ROOT_CGROUPS_RecoverMultipleNestedLauncherOrphans:NestedMesosContainerizerTest.ROOT_CGROUPS_RecoverNestedContainersWithLauncherOrphans:NestedMesosContainerizerTest.ROOT_CGROUPS_RecoverLauncherOrphanAndMultipleNestedLauncherOrphans:NestedMesosContainerizerTest.ROOT_CGROUPS_WaitAfterDestroy:NestedMesosContainerizerTest.ROOT_CGROUPS_Environment:NestedMesosContainerizerTest.ROOT_CGROUPS_LaunchNestedThreeLevels:NsTest.ROOT_setns:NsTest.ROOT_setnsMultipleThreads:NsTest.ROOT_getns:NsTest.ROOT_destroy:NsTest.ROOT_clone:NvidiaGpuTest.ROOT_CGROUPS_NVIDIA_GPU_VerifyDeviceAccess:NvidiaGpuTest.ROOT_INTERNET_CURL_CGROUPS_NVIDIA_GPU_NvidiaDockerImage:NvidiaGpuTest.ROOT_CGROUPS_NVIDIA_GPU_FractionalResources:NvidiaGpuTest.NVIDIA_GPU_Discovery:NvidiaGpuTest.ROOT_CGROUPS_NVIDIA_GPU_FlagValidation:NvidiaGpuTest.NVIDIA_GPU_Allocator:NvidiaGpuTest.ROOT_NVIDIA_GPU_VolumeCreation:NvidiaGpuTest.ROOT_NVIDIA_GPU_VolumeShouldInject:PerfTest.ROOT_PERF_Events:PerfTest.ROOT_PERF_Sample:DockerArchiveTest.ROOT_CreateDockerLocalTar:DockerRuntimeIsolatorTest.ROOT_DockerDefaultCmdLocalPuller:DockerRuntimeIsolatorTest.ROOT_DockerDefaultEntryptLocalPuller:DockerRuntimeIsolatorTest.ROOT_CURL_INTERNET_DockerDefaultEntryptRegistryPuller:SchedTest.ROOT_PolicySelf:SchedTest.ROOT_PolicyChild:bool/UserContainerLoggerTest.ROOT_LOGROTATE_RotateWithSwitchUserTrueOrFalse/0:bool/UserContainerLoggerTest.ROOT_LOGROTATE_RotateWithSwitchUserTrueOrFalse/1:MesosContainerizer/DefaultExecutorTest.ROOT_ContainerStatusForTask/0:ROOT_DOCKER_DockerAndMesosContainerizers/DefaultExecutorTest.TaskRunning/0:ROOT_DOCKER_DockerAndMesosContainerizers/DefaultExecutorTest.KillTask/0:ROOT_DOCKER_DockerAndMesosContainerizers/DefaultExecutorTest.TaskUsesExecutor/0:ROOT_DOCKER_DockerAndMesosContainerizers/DefaultExecutorTest.ROOT_ContainerStatusForTask/0:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/0:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/1:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/2:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/3:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/4:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/5:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/6:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/7:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/8:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/9:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/10:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/11:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/12:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/13:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/14:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/15:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/16:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/17:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/18:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/19:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/20:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/21:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/22:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/23:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/24:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/25:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/26:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/27:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/28:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/29:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/30:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/31:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/32:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/33:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/34:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/35:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/36:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/37:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/38:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/39:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/40:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/41:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/42:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/43:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/44:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/45:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/46:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/47:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/0:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/1:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/2:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/3:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/4:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/5:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/6:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/7:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/8:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/9:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/10:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/11:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/12:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/13:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/14:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/15:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/16:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/17:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/18:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/19:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/20:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/21:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/22:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/23:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/24:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/25:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/26:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/27:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/28:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/29:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/30:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/31:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/32:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/33:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/34:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/35:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/36:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/37:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/38:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/39:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/40:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/41:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/42:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/43:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/44:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/45:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/46:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/47:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/0:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/1:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/2:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/3:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/4:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/5:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/6:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/7:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/8:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/9:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/10:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/11:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/12:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/13:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/14:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/15:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/16:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/17:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/18:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/19:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/20:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/21:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/22:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/23:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/24:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/25:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/26:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/27:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/28:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/29:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/30:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/31:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/32:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/33:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/34:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/35:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/36:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/37:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/38:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/39:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/40:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/41:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/42:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/43:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/44:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/45:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/46:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.ResourceLabels/47:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/0:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/1:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/2:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/3:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/4:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/5:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/6:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/8:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/9:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/10:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/11:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/12:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/13:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/14:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/15:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/16:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/17:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/18:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/19:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/20:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/21:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/22:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/23:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/24:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/25:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/26:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/27:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/28:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/29:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/30:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/31:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/32:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/33:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/34:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/35:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/36:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/37:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/38:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/39:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/40:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/41:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/42:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/43:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/44:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/45:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/46:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/47:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/0:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/1:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/2:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/3:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/4:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/5:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/6:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/7:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/8:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/9:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/10:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/11:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/12:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/13:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/14:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/15:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/16:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/17:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/18:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/19:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/20:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/21:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/22:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/23:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/24:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/25:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/26:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/27:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/28:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/29:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/30:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/31:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/32:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/33:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/34:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/35:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/36:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/37:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/38:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/39:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/40:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/41:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/42:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/43:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/44:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/45:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/46:SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.Metrics/47:ROOT_MountDiskResource/PersistentVolumeTest.CreateAndDestroyPersistentVolumes/0:ROOT_MountDiskResource/PersistentVolumeTest.ResourcesCheckpointing/0:ROOT_MountDiskResource/PersistentVolumeTest.PreparePersistentVolume/0:ROOT_MountDiskResource/PersistentVolumeTest.MasterFailover/0:ROOT_MountDiskResource/PersistentVolumeTest.IncompatibleCheckpointedResources/0:ROOT_MountDiskResource/PersistentVolumeTest.AccessPersistentVolume/0:ROOT_MountDiskResource/PersistentVolumeTest.SharedPersistentVolumeMultipleTasks/0:ROOT_MountDiskResource/PersistentVolumeTest.SharedPersistentVolumeRescindOnDestroy/0:ROOT_MountDiskResource/PersistentVolumeTest.SharedPersistentVolumeMasterFailover/0:ROOT_MountDiskResource/PersistentVolumeTest.SlaveRecovery/0:ROOT_MountDiskResource/PersistentVolumeTest.GoodACLCreateThenDestroy/0:ROOT_MountDiskResource/PersistentVolumeTest.GoodACLNoPrincipal/0:ROOT_MountDiskResource/PersistentVolumeTest.BadACLNoPrincipal/0:ROOT_MountDiskResource/PersistentVolumeTest.BadACLDropCreateAndDestroy/0:SlaveCount/Registrar_BENCHMARK_Test.Performance/0:SlaveCount/Registrar_BENCHMARK_Test.Performance/1:SlaveCount/Registrar_BENCHMARK_Test.Performance/2:SlaveCount/Registrar_BENCHMARK_Test.Performance/3:SlaveCount/Registrar_BENCHMARK_Test.MarkUnreachableThenReachable/0:SlaveCount/Registrar_BENCHMARK_Test.MarkUnreachableThenReachable/1:SlaveCount/Registrar_BENCHMARK_Test.MarkUnreachableThenReachable/2:SlaveCount/Registrar_BENCHMARK_Test.MarkUnreachableThenReachable/3:SlaveCount/Registrar_BENCHMARK_Test.GcManyAgents/0:SlaveCount/Registrar_BENCHMARK_Test.GcManyAgents/1:SlaveCount/Registrar_BENCHMARK_Test.GcManyAgents/2:SlaveCount/Registrar_BENCHMARK_Test.GcManyAgents/3:ResourcesScalarArithmeticOperators/Resources_Scalar_Arithmetic_BENCHMARK_Test.Arithmetic/0:ResourcesScalarArithmeticOperators/Resources_Scalar_Arithmetic_BENCHMARK_Test.Arithmetic/1:ResourcesScalarArithmeticOperators/Resources_Scalar_Arithmetic_BENCHMARK_Test.Arithmetic/2:ResourcesScalarArithmeticOperators/Resources_Scalar_Arithmetic_BENCHMARK_Test.Arithmetic/3:ResourcesContains/Resources_Contains_BENCHMARK_Test.Contains/0:ResourcesContains/Resources_Contains_BENCHMARK_Test.Contains/1:ResourcesContains/Resources_Contains_BENCHMARK_Test.Contains/2:ResourcesContains/Resources_Contains_BENCHMARK_Test.Contains/3:ResourcesContains/Resources_Contains_BENCHMARK_Test.Contains/4:ResourcesContains/Resources_Contains_BENCHMARK_Test.Contains/5:ResourcesContains/Resources_Contains_BENCHMARK_Test.Contains/6:ResourcesContains/Resources_Contains_BENCHMARK_Test.Contains/7:ResourcesContains/Resources_Contains_BENCHMARK_Test.Contains/8:Tasks/SchedulerReconcileTasks_BENCHMARK_Test.SchedulerLibrary/0:Tasks/SchedulerReconcileTasks_BENCHMARK_Test.SchedulerLibrary/1:Tasks/SchedulerReconcileTasks_BENCHMARK_Test.SchedulerLibrary/2:Tasks/SchedulerReconcileTasks_BENCHMARK_Test.SchedulerLibrary/3:Tasks/SchedulerReconcileTasks_BENCHMARK_Test.SchedulerDriver/0:Tasks/SchedulerReconcileTasks_BENCHMARK_Test.SchedulerDriver/1:Tasks/SchedulerReconcileTasks_BENCHMARK_Test.SchedulerDriver/2:Tasks/SchedulerReconcileTasks_BENCHMARK_Test.SchedulerDriver/3:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/0:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/1:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/2:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/3:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/4:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/5:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/6:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/7:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/8:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/9:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/10:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/11:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/12:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/13:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/14:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/15:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/16:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/17:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/18:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/19:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/20:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/21:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/22:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/23:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/24:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/25:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/26:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/27:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/28:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/29:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/30:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/31:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/32:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/33:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/34:AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/35:IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0:IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/1:IsolationFlag/CpuIsolatorTest.ROOT_SystemCpuUsage/0:IsolationFlag/CpuIsolatorTest.ROOT_SystemCpuUsage/1:IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0:IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/1:BackendFlag/ProvisionerDockerWhiteoutTest.ROOT_INTERNET_CURL_Whiteout/0:Nesting/VolumeImageIsolatorTest.ROOT_ImageInVolumeWithoutRootFilesystem/0:Nesting/VolumeImageIsolatorTest.ROOT_ImageInVolumeWithoutRootFilesystem/1:Nesting/VolumeImageIsolatorTest.ROOT_ImageInVolumeWithRootFilesystem/0:Nesting/VolumeImageIsolatorTest.ROOT_ImageInVolumeWithRootFilesystem/1:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/0:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/1:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/2:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/3:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/4:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/5:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/6:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/7:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/8:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/9:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/10:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/11:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/12:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/13:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/14:TestParam/LinuxCapabilitiesIsolatorTest.ROOT_Ping/15
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from MasterSlaveReconciliationTest
[ RUN ] MasterSlaveReconciliationTest.SlaveReregisterTerminatedExecutor
I1214 04:41:11.559672 2005 cluster.cpp:160] Creating default 'local' authorizer
I1214 04:41:11.560848 2045 master.cpp:380] Master 87dd8179-dd7d-4270-ace2-ea771b57371c (gru1.hw.ca1.mesosphere.com) started on 192.99.40.208:37659
I1214 04:41:11.560878 2045 master.cpp:382] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="true" --authenticate_frameworks="true" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticators="crammd5" --authorizers="local" --credentials="/tmp/cXHI89/credentials" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memory" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/home/bbannier/src/mesos/build/P/share/mesos/webui" --work_dir="/tmp/cXHI89/master" --zk_session_timeout="10secs"
I1214 04:41:11.561079 2045 master.cpp:432] Master only allowing authenticated frameworks to register
I1214 04:41:11.561089 2045 master.cpp:446] Master only allowing authenticated agents to register
I1214 04:41:11.561095 2045 master.cpp:459] Master only allowing authenticated HTTP frameworks to register
I1214 04:41:11.561101 2045 credentials.hpp:39] Loading credentials for authentication from '/tmp/cXHI89/credentials'
I1214 04:41:11.561194 2045 master.cpp:504] Using default 'crammd5' authenticator
I1214 04:41:11.561236 2045 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-master-readonly'
I1214 04:41:11.561274 2045 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-master-readwrite'
I1214 04:41:11.561301 2045 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-master-scheduler'
I1214 04:41:11.561326 2045 master.cpp:584] Authorization enabled
I1214 04:41:11.562155 2039 master.cpp:2045] Elected as the leading master!
I1214 04:41:11.562173 2039 master.cpp:1568] Recovering from registrar
I1214 04:41:11.562347 2045 registrar.cpp:362] Successfully fetched the registry (0B) in 114944ns
I1214 04:41:11.562441 2045 registrar.cpp:461] Applied 1 operations in 7920ns; attempting to update the registry
I1214 04:41:11.562621 2048 registrar.cpp:506] Successfully updated the registry in 155136ns
I1214 04:41:11.562664 2048 registrar.cpp:392] Successfully recovered registrar
I1214 04:41:11.562832 2044 master.cpp:1684] Recovered 0 agents from the registry (166B); allowing 10mins for agents to re-register
I1214 04:41:11.568444 2005 cluster.cpp:446] Creating default 'local' authorizer
I1214 04:41:11.569344 2005 sched.cpp:232] Version: 1.2.0
I1214 04:41:11.569842 2035 slave.cpp:209] Mesos agent started on (912)@192.99.40.208:37659
I1214 04:41:11.570080 2040 sched.cpp:336] New master detected at master@192.99.40.208:37659
I1214 04:41:11.570117 2040 sched.cpp:402] Authenticating with master master@192.99.40.208:37659
I1214 04:41:11.570127 2040 sched.cpp:409] Using default CRAM-MD5 authenticatee
I1214 04:41:11.570220 2040 authenticatee.cpp:121] Creating new client SASL connection
I1214 04:41:11.570000 2035 slave.cpp:210] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/http_credentials" --http_heartbeat_interval="30secs" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/home/bbannier/src/mesos/build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc"
I1214 04:41:11.570297 2035 credentials.hpp:88] Loading credential for authentication from '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/credential'
I1214 04:41:11.570307 2040 master.cpp:6748] Authenticating scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.570351 2035 slave.cpp:352] Agent using credential for: test-principal
I1214 04:41:11.570363 2035 credentials.hpp:39] Loading credentials for authentication from '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/http_credentials'
I1214 04:41:11.570405 2040 authenticator.cpp:98] Creating new server SASL connection
I1214 04:41:11.570439 2035 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
I1214 04:41:11.570461 2040 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
I1214 04:41:11.570478 2035 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
I1214 04:41:11.570752 2035 slave.cpp:539] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I1214 04:41:11.570791 2035 slave.cpp:547] Agent attributes: [ ]
I1214 04:41:11.570801 2035 slave.cpp:552] Agent hostname: gru1.hw.ca1.mesosphere.com
I1214 04:41:11.571265 2035 state.cpp:57] Recovering state from '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc/meta'
I1214 04:41:11.571367 2035 status_update_manager.cpp:203] Recovering status update manager
I1214 04:41:11.571449 2035 slave.cpp:5420] Finished recovery
I1214 04:41:11.571684 2035 slave.cpp:924] New master detected at master@192.99.40.208:37659
I1214 04:41:11.571702 2035 slave.cpp:983] Authenticating with master master@192.99.40.208:37659
I1214 04:41:11.571720 2035 slave.cpp:994] Using default CRAM-MD5 authenticatee
I1214 04:41:11.571751 2035 slave.cpp:956] Detecting new master
I1214 04:41:11.571807 2035 status_update_manager.cpp:177] Pausing sending status updates
I1214 04:41:11.571841 2035 authenticatee.cpp:121] Creating new client SASL connection
I1214 04:41:11.571909 2035 master.cpp:6748] Authenticating slave(912)@192.99.40.208:37659
I1214 04:41:11.571982 2035 authenticator.cpp:98] Creating new server SASL connection
I1214 04:41:11.572031 2035 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
I1214 04:41:11.572052 2035 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
I1214 04:41:11.572078 2035 authenticator.cpp:204] Received SASL authentication start
I1214 04:41:11.572108 2035 authenticator.cpp:326] Authentication requires more steps
I1214 04:41:11.572135 2035 authenticatee.cpp:259] Received SASL authentication step
I1214 04:41:11.572171 2035 authenticator.cpp:232] Received SASL authentication step
I1214 04:41:11.572201 2035 authenticator.cpp:318] Authentication success
I1214 04:41:11.572234 2035 authenticatee.cpp:299] Authentication success
I1214 04:41:11.572257 2035 master.cpp:6778] Successfully authenticated principal 'test-principal' at slave(912)@192.99.40.208:37659
I1214 04:41:11.572331 2035 slave.cpp:1078] Successfully authenticated with master master@192.99.40.208:37659
I1214 04:41:11.572430 2035 master.cpp:5161] Registering agent at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with id 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.572541 2035 registrar.cpp:461] Applied 1 operations in 18957ns; attempting to update the registry
I1214 04:41:11.572929 2035 registrar.cpp:506] Successfully updated the registry in 363008ns
I1214 04:41:11.573122 2035 master.cpp:5232] Registered agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I1214 04:41:11.573240 2035 hierarchical.cpp:490] Added agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (allocated: {})
I1214 04:41:11.573312 2035 slave.cpp:1124] Registered with master master@192.99.40.208:37659; given agent ID 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.573545 2035 slave.cpp:1184] Forwarding total oversubscribed resources {}
I1214 04:41:11.573599 2035 status_update_manager.cpp:184] Resuming sending status updates
I1214 04:41:11.573631 2035 master.cpp:5633] Received update of agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with total oversubscribed resources {}
I1214 04:41:11.573696 2035 hierarchical.cpp:560] Agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com) updated with oversubscribed resources {} (total: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000], allocated: {})
I1214 04:41:11.570480 2040 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
I1214 04:41:11.573894 2040 authenticator.cpp:204] Received SASL authentication start
I1214 04:41:11.573921 2040 authenticator.cpp:326] Authentication requires more steps
I1214 04:41:11.573951 2040 authenticatee.cpp:259] Received SASL authentication step
I1214 04:41:11.573988 2040 authenticator.cpp:232] Received SASL authentication step
I1214 04:41:11.574015 2040 authenticator.cpp:318] Authentication success
I1214 04:41:11.574054 2040 authenticatee.cpp:299] Authentication success
I1214 04:41:11.574079 2040 master.cpp:6778] Successfully authenticated principal 'test-principal' at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.574154 2040 sched.cpp:508] Successfully authenticated with master master@192.99.40.208:37659
I1214 04:41:11.574247 2040 master.cpp:2633] Received SUBSCRIBE call for framework 'default' at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.574265 2040 master.cpp:2081] Authorizing framework principal 'test-principal' to receive offers for role '*'
I1214 04:41:11.574359 2040 master.cpp:2709] Subscribing framework default with checkpointing disabled and capabilities [ ]
I1214 04:41:11.574496 2040 hierarchical.cpp:276] Added framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.574750 2040 sched.cpp:749] Framework registered with 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.574884 2040 master.cpp:6577] Sending 1 offers to framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.575361 2040 master.cpp:3588] Processing ACCEPT call for offers: [ 87dd8179-dd7d-4270-ace2-ea771b57371c-O0 ] on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.575392 2040 master.cpp:3175] Authorizing framework principal 'test-principal' to launch task 0
W1214 04:41:11.575858 2040 validation.cpp:1001] Executor 'default' for task '0' uses less CPUs (None) than the minimum required (0.01). Please update your executor, as this will be mandatory in future releases.
W1214 04:41:11.575880 2040 validation.cpp:1013] Executor 'default' for task '0' uses less memory (None) than the minimum required (32MB). Please update your executor, as this will be mandatory in future releases.
I1214 04:41:11.575949 2040 master.cpp:8501] Adding task 0 with resources cpus(*):1; mem(*):512 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.575989 2040 master.cpp:4240] Launching task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659 with resources cpus(*):1; mem(*):512 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.576154 2040 slave.cpp:1556] Got assigned task '0' for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.576323 2040 slave.cpp:1718] Launching task '0' for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.576510 2040 paths.cpp:530] Trying to chown '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc/slaves/87dd8179-dd7d-4270-ace2-ea771b57371c-S0/frameworks/87dd8179-dd7d-4270-ace2-ea771b57371c-0000/executors/default/runs/d6293832-3a4c-4731-b4de-da91f64c1c09' to user 'bbannier'
I1214 04:41:11.586915 2040 slave.cpp:6347] Launching executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 with resources {} in work directory '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc/slaves/87dd8179-dd7d-4270-ace2-ea771b57371c-S0/frameworks/87dd8179-dd7d-4270-ace2-ea771b57371c-0000/executors/default/runs/d6293832-3a4c-4731-b4de-da91f64c1c09'
I1214 04:41:11.587187 2040 slave.cpp:2040] Queued task '0' for executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.588081 2034 exec.cpp:162] Version: 1.2.0
I1214 04:41:11.588315 2035 slave.cpp:3314] Got registration for executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from executor(912)@192.99.40.208:37659
I1214 04:41:11.588631 2046 exec.cpp:237] Executor registered on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.588687 2035 slave.cpp:2256] Sending queued task '0' to executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 at executor(912)@192.99.40.208:37659
I1214 04:41:11.588961 2034 slave.cpp:3749] Handling status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from executor(912)@192.99.40.208:37659
I1214 04:41:11.589154 2044 status_update_manager.cpp:323] Received status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.589452 2044 slave.cpp:4190] Forwarding the update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to master@192.99.40.208:37659
I1214 04:41:11.589516 2044 slave.cpp:4100] Sending acknowledgement for status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to executor(912)@192.99.40.208:37659
I1214 04:41:11.589596 2044 master.cpp:5769] Status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.589625 2044 master.cpp:5831] Forwarding status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.589680 2044 master.cpp:7867] Updating the state of task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (latest state: TASK_RUNNING, status update state: TASK_RUNNING)
I1214 04:41:11.589999 2034 master.cpp:4877] Processing ACKNOWLEDGE call c9738162-8aa4-40f7-9be8-ec1d4877a94c for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.590206 2045 status_update_manager.cpp:395] Received status update acknowledgement (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.591419 2043 slave.cpp:3749] Handling status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from executor(912)@192.99.40.208:37659
I1214 04:41:11.591619 2043 status_update_manager.cpp:323] Received status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.591711 2043 slave.cpp:4190] Forwarding the update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to master@192.99.40.208:37659
I1214 04:41:11.591760 2043 slave.cpp:4100] Sending acknowledgement for status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to executor(912)@192.99.40.208:37659
I1214 04:41:11.592257 2043 slave.cpp:924] New master detected at master@192.99.40.208:37659
I1214 04:41:11.592280 2043 slave.cpp:983] Authenticating with master master@192.99.40.208:37659
I1214 04:41:11.592298 2043 slave.cpp:994] Using default CRAM-MD5 authenticatee
I1214 04:41:11.592313 2039 status_update_manager.cpp:177] Pausing sending status updates
I1214 04:41:11.592341 2043 slave.cpp:956] Detecting new master
I1214 04:41:11.592363 2039 authenticatee.cpp:121] Creating new client SASL connection
I1214 04:41:11.592383 2043 slave.cpp:4318] Got exited event for executor(912)@192.99.40.208:37659
I1214 04:41:11.592438 2039 master.cpp:6748] Authenticating slave(912)@192.99.40.208:37659
I1214 04:41:11.592550 2039 authenticator.cpp:98] Creating new server SASL connection
I1214 04:41:11.592614 2039 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
I1214 04:41:11.592635 2039 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
I1214 04:41:11.592672 2039 authenticator.cpp:204] Received SASL authentication start
I1214 04:41:11.592707 2039 authenticator.cpp:326] Authentication requires more steps
I1214 04:41:11.592741 2039 authenticatee.cpp:259] Received SASL authentication step
I1214 04:41:11.592779 2039 authenticator.cpp:232] Received SASL authentication step
I1214 04:41:11.592819 2039 authenticator.cpp:318] Authentication success
I1214 04:41:11.592850 2039 authenticatee.cpp:299] Authentication success
I1214 04:41:11.592870 2039 master.cpp:6778] Successfully authenticated principal 'test-principal' at slave(912)@192.99.40.208:37659
I1214 04:41:11.592921 2039 slave.cpp:1078] Successfully authenticated with master master@192.99.40.208:37659
I1214 04:41:11.593102 2039 master.cpp:5311] Re-registering agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.593189 2039 master.cpp:5572] Sending updated checkpointed resources {} to agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.593250 2039 slave.cpp:1226] Re-registered with master master@192.99.40.208:37659
I1214 04:41:11.593283 2039 slave.cpp:1262] Forwarding total oversubscribed resources {}
I1214 04:41:11.593329 2039 status_update_manager.cpp:184] Resuming sending status updates
W1214 04:41:11.593340 2039 status_update_manager.cpp:191] Resending status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.593410 2039 master.cpp:5633] Received update of agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with total oversubscribed resources {}
I1214 04:41:11.593451 2039 slave.cpp:4190] Forwarding the update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to master@192.99.40.208:37659
I1214 04:41:11.593497 2039 hierarchical.cpp:560] Agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com) updated with oversubscribed resources {} (total: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000], allocated: cpus(*):1; mem(*):512)
I1214 04:41:11.593605 2039 master.cpp:5769] Status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.593619 2039 master.cpp:5831] Forwarding status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.593657 2039 master.cpp:7867] Updating the state of task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (latest state: TASK_FINISHED, status update state: TASK_FINISHED)
I1214 04:41:11.593978 2041 master.cpp:4877] Processing ACKNOWLEDGE call 79f94437-6066-412e-9de6-6d9a1f256539 for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
I1214 04:41:11.594013 2041 master.cpp:7963] Removing task 0 with resources cpus(*):1; mem(*):512 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.594048 2005 sched.cpp:2008] Asked to stop the driver
I1214 04:41:11.594158 2048 sched.cpp:1193] Stopping framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594192 2041 status_update_manager.cpp:395] Received status update acknowledgement (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594251 2048 master.cpp:7287] Processing TEARDOWN call for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.594272 2048 master.cpp:7299] Removing framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
I1214 04:41:11.594336 2048 master.cpp:7992] Removing executor 'default' with resources {} of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
I1214 04:41:11.594383 2041 hierarchical.cpp:391] Deactivated framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594450 2041 hierarchical.cpp:342] Removed framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594570 2042 slave.cpp:2609] Shutting down framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
I1214 04:41:11.594593 2042 slave.cpp:4999] Shutting down executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 at executor(912)@192.99.40.208:37659
I1214 04:41:11.594758 2040 slave.cpp:4681] Executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 exited with status 0
I1214 04:41:11.594861 2040 slave.cpp:4785] Cleaning up executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 at executor(912)@192.99.40.208:37659
W1214 04:41:11.594920 2041 master.cpp:5887] Ignoring unknown exited executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
../../src/tests/master_slave_reconciliation_tests.cpp:145: Failure
Actual function call count doesn't match EXPECT_CALL(sched, executorLost(&driver, createExecutorInfo("default", "exit 1").executor_id(), _, _))...
Expected: to be called once
Actual: never called - unsatisfied and active
*** Aborted at 1481708471 (unix time) try "date -d @1481708471" if you are using GNU date ***
I1214 04:41:11.595078 2040 slave.cpp:4873] Cleaning up framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
PC: @ 0x1d5c445 testing::UnitTest::AddTestPartResult()
*** SIGSEGV (@0x0) received by PID 2005 (TID 0x7f04125ff8c0) from PID 0; stack trace: ***
@ 0x7f040d5c95c0 (unknown)
@ 0x1d5c445 testing::UnitTest::AddTestPartResult()
@ 0x1d5c0af testing::internal::AssertHelper::operator=()
@ 0x1d8b61f testing::internal::GoogleTestFailureReporter::ReportFailure()
@ 0x1d80511 testing::internal::UntypedFunctionMockerBase::VerifyAndClearExpectationsLocked()
@ 0x1136031 testing::internal::FunctionMockerBase<>::~FunctionMockerBase()
@ 0x112e023 mesos::internal::tests::MockScheduler::~MockScheduler()
@ 0xf66780 mesos::internal::tests::MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Test::TestBody()
@ 0x1d84a46 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x1d6450d testing::Test::Run()
@ 0x1d65520 testing::TestInfo::Run()
@ 0x1d65c23 testing::TestCase::Run()
@ 0x1d6d8b9 testing::internal::UnitTestImpl::RunAllTests()
@ 0x1d8539f testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x1d6d55f testing::UnitTest::Run()
@ 0xd5d8bb main
@ 0x7f040c348401 __libc_start_main
@ 0x51a81a _start
{code}
> MasterSlaveReconciliationTest.ReconcileLostTask test is flaky
> -------------------------------------------------------------
>
> Key: MESOS-6792
> URL: https://issues.apache.org/jira/browse/MESOS-6792
> Project: Mesos
> Issue Type: Bug
> Environment: Fedora 25, clang, w/ optimizations, SSL build
> Reporter: Benjamin Bannier
> Labels: tech-debt
>
> The test {{MasterSlaveReconciliationTest.ReconcileLostTask}} is flaky for me as of {{e99ea9ce8b1de01dd8b3cac6675337edb6320f38}},
> {code}
> Repeating all tests (iteration 912) . . .
> Note: Google Test filter = <...>
> [==========] Running 1 test from 1 test case.
> [----------] Global test environment set-up.
> [----------] 1 test from MasterSlaveReconciliationTest
> [ RUN ] MasterSlaveReconciliationTest.SlaveReregisterTerminatedExecutor
> I1214 04:41:11.559672 2005 cluster.cpp:160] Creating default 'local' authorizer
> I1214 04:41:11.560848 2045 master.cpp:380] Master 87dd8179-dd7d-4270-ace2-ea771b57371c (gru1.hw.ca1.mesosphere.com) started on 192.99.40.208:37659
> I1214 04:41:11.560878 2045 master.cpp:382] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="true" --authenticate_frameworks="true" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticators="crammd5" --authorizers="local" --credentials="/tmp/cXHI89/credentials" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memory" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/home/bbannier/src/mesos/build/P/share/mesos/webui" --work_dir="/tmp/cXHI89/master" --zk_session_timeout="10secs"
> I1214 04:41:11.561079 2045 master.cpp:432] Master only allowing authenticated frameworks to register
> I1214 04:41:11.561089 2045 master.cpp:446] Master only allowing authenticated agents to register
> I1214 04:41:11.561095 2045 master.cpp:459] Master only allowing authenticated HTTP frameworks to register
> I1214 04:41:11.561101 2045 credentials.hpp:39] Loading credentials for authentication from '/tmp/cXHI89/credentials'
> I1214 04:41:11.561194 2045 master.cpp:504] Using default 'crammd5' authenticator
> I1214 04:41:11.561236 2045 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-master-readonly'
> I1214 04:41:11.561274 2045 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-master-readwrite'
> I1214 04:41:11.561301 2045 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-master-scheduler'
> I1214 04:41:11.561326 2045 master.cpp:584] Authorization enabled
> I1214 04:41:11.562155 2039 master.cpp:2045] Elected as the leading master!
> I1214 04:41:11.562173 2039 master.cpp:1568] Recovering from registrar
> I1214 04:41:11.562347 2045 registrar.cpp:362] Successfully fetched the registry (0B) in 114944ns
> I1214 04:41:11.562441 2045 registrar.cpp:461] Applied 1 operations in 7920ns; attempting to update the registry
> I1214 04:41:11.562621 2048 registrar.cpp:506] Successfully updated the registry in 155136ns
> I1214 04:41:11.562664 2048 registrar.cpp:392] Successfully recovered registrar
> I1214 04:41:11.562832 2044 master.cpp:1684] Recovered 0 agents from the registry (166B); allowing 10mins for agents to re-register
> I1214 04:41:11.568444 2005 cluster.cpp:446] Creating default 'local' authorizer
> I1214 04:41:11.569344 2005 sched.cpp:232] Version: 1.2.0
> I1214 04:41:11.569842 2035 slave.cpp:209] Mesos agent started on (912)@192.99.40.208:37659
> I1214 04:41:11.570080 2040 sched.cpp:336] New master detected at master@192.99.40.208:37659
> I1214 04:41:11.570117 2040 sched.cpp:402] Authenticating with master master@192.99.40.208:37659
> I1214 04:41:11.570127 2040 sched.cpp:409] Using default CRAM-MD5 authenticatee
> I1214 04:41:11.570220 2040 authenticatee.cpp:121] Creating new client SASL connection
> I1214 04:41:11.570000 2035 slave.cpp:210] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/http_credentials" --http_heartbeat_interval="30secs" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/home/bbannier/src/mesos/build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc"
> I1214 04:41:11.570297 2035 credentials.hpp:88] Loading credential for authentication from '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/credential'
> I1214 04:41:11.570307 2040 master.cpp:6748] Authenticating scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
> I1214 04:41:11.570351 2035 slave.cpp:352] Agent using credential for: test-principal
> I1214 04:41:11.570363 2035 credentials.hpp:39] Loading credentials for authentication from '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Y63Hjf/http_credentials'
> I1214 04:41:11.570405 2040 authenticator.cpp:98] Creating new server SASL connection
> I1214 04:41:11.570439 2035 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
> I1214 04:41:11.570461 2040 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
> I1214 04:41:11.570478 2035 http.cpp:922] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
> I1214 04:41:11.570752 2035 slave.cpp:539] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1214 04:41:11.570791 2035 slave.cpp:547] Agent attributes: [ ]
> I1214 04:41:11.570801 2035 slave.cpp:552] Agent hostname: gru1.hw.ca1.mesosphere.com
> I1214 04:41:11.571265 2035 state.cpp:57] Recovering state from '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc/meta'
> I1214 04:41:11.571367 2035 status_update_manager.cpp:203] Recovering status update manager
> I1214 04:41:11.571449 2035 slave.cpp:5420] Finished recovery
> I1214 04:41:11.571684 2035 slave.cpp:924] New master detected at master@192.99.40.208:37659
> I1214 04:41:11.571702 2035 slave.cpp:983] Authenticating with master master@192.99.40.208:37659
> I1214 04:41:11.571720 2035 slave.cpp:994] Using default CRAM-MD5 authenticatee
> I1214 04:41:11.571751 2035 slave.cpp:956] Detecting new master
> I1214 04:41:11.571807 2035 status_update_manager.cpp:177] Pausing sending status updates
> I1214 04:41:11.571841 2035 authenticatee.cpp:121] Creating new client SASL connection
> I1214 04:41:11.571909 2035 master.cpp:6748] Authenticating slave(912)@192.99.40.208:37659
> I1214 04:41:11.571982 2035 authenticator.cpp:98] Creating new server SASL connection
> I1214 04:41:11.572031 2035 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
> I1214 04:41:11.572052 2035 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
> I1214 04:41:11.572078 2035 authenticator.cpp:204] Received SASL authentication start
> I1214 04:41:11.572108 2035 authenticator.cpp:326] Authentication requires more steps
> I1214 04:41:11.572135 2035 authenticatee.cpp:259] Received SASL authentication step
> I1214 04:41:11.572171 2035 authenticator.cpp:232] Received SASL authentication step
> I1214 04:41:11.572201 2035 authenticator.cpp:318] Authentication success
> I1214 04:41:11.572234 2035 authenticatee.cpp:299] Authentication success
> I1214 04:41:11.572257 2035 master.cpp:6778] Successfully authenticated principal 'test-principal' at slave(912)@192.99.40.208:37659
> I1214 04:41:11.572331 2035 slave.cpp:1078] Successfully authenticated with master master@192.99.40.208:37659
> I1214 04:41:11.572430 2035 master.cpp:5161] Registering agent at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with id 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
> I1214 04:41:11.572541 2035 registrar.cpp:461] Applied 1 operations in 18957ns; attempting to update the registry
> I1214 04:41:11.572929 2035 registrar.cpp:506] Successfully updated the registry in 363008ns
> I1214 04:41:11.573122 2035 master.cpp:5232] Registered agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1214 04:41:11.573240 2035 hierarchical.cpp:490] Added agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (allocated: {})
> I1214 04:41:11.573312 2035 slave.cpp:1124] Registered with master master@192.99.40.208:37659; given agent ID 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
> I1214 04:41:11.573545 2035 slave.cpp:1184] Forwarding total oversubscribed resources {}
> I1214 04:41:11.573599 2035 status_update_manager.cpp:184] Resuming sending status updates
> I1214 04:41:11.573631 2035 master.cpp:5633] Received update of agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with total oversubscribed resources {}
> I1214 04:41:11.573696 2035 hierarchical.cpp:560] Agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com) updated with oversubscribed resources {} (total: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000], allocated: {})
> I1214 04:41:11.570480 2040 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
> I1214 04:41:11.573894 2040 authenticator.cpp:204] Received SASL authentication start
> I1214 04:41:11.573921 2040 authenticator.cpp:326] Authentication requires more steps
> I1214 04:41:11.573951 2040 authenticatee.cpp:259] Received SASL authentication step
> I1214 04:41:11.573988 2040 authenticator.cpp:232] Received SASL authentication step
> I1214 04:41:11.574015 2040 authenticator.cpp:318] Authentication success
> I1214 04:41:11.574054 2040 authenticatee.cpp:299] Authentication success
> I1214 04:41:11.574079 2040 master.cpp:6778] Successfully authenticated principal 'test-principal' at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
> I1214 04:41:11.574154 2040 sched.cpp:508] Successfully authenticated with master master@192.99.40.208:37659
> I1214 04:41:11.574247 2040 master.cpp:2633] Received SUBSCRIBE call for framework 'default' at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
> I1214 04:41:11.574265 2040 master.cpp:2081] Authorizing framework principal 'test-principal' to receive offers for role '*'
> I1214 04:41:11.574359 2040 master.cpp:2709] Subscribing framework default with checkpointing disabled and capabilities [ ]
> I1214 04:41:11.574496 2040 hierarchical.cpp:276] Added framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.574750 2040 sched.cpp:749] Framework registered with 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.574884 2040 master.cpp:6577] Sending 1 offers to framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
> I1214 04:41:11.575361 2040 master.cpp:3588] Processing ACCEPT call for offers: [ 87dd8179-dd7d-4270-ace2-ea771b57371c-O0 ] on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
> I1214 04:41:11.575392 2040 master.cpp:3175] Authorizing framework principal 'test-principal' to launch task 0
> W1214 04:41:11.575858 2040 validation.cpp:1001] Executor 'default' for task '0' uses less CPUs (None) than the minimum required (0.01). Please update your executor, as this will be mandatory in future releases.
> W1214 04:41:11.575880 2040 validation.cpp:1013] Executor 'default' for task '0' uses less memory (None) than the minimum required (32MB). Please update your executor, as this will be mandatory in future releases.
> I1214 04:41:11.575949 2040 master.cpp:8501] Adding task 0 with resources cpus(*):1; mem(*):512 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com)
> I1214 04:41:11.575989 2040 master.cpp:4240] Launching task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659 with resources cpus(*):1; mem(*):512 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
> I1214 04:41:11.576154 2040 slave.cpp:1556] Got assigned task '0' for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.576323 2040 slave.cpp:1718] Launching task '0' for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.576510 2040 paths.cpp:530] Trying to chown '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc/slaves/87dd8179-dd7d-4270-ace2-ea771b57371c-S0/frameworks/87dd8179-dd7d-4270-ace2-ea771b57371c-0000/executors/default/runs/d6293832-3a4c-4731-b4de-da91f64c1c09' to user 'bbannier'
> I1214 04:41:11.586915 2040 slave.cpp:6347] Launching executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 with resources {} in work directory '/tmp/MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_7badJc/slaves/87dd8179-dd7d-4270-ace2-ea771b57371c-S0/frameworks/87dd8179-dd7d-4270-ace2-ea771b57371c-0000/executors/default/runs/d6293832-3a4c-4731-b4de-da91f64c1c09'
> I1214 04:41:11.587187 2040 slave.cpp:2040] Queued task '0' for executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.588081 2034 exec.cpp:162] Version: 1.2.0
> I1214 04:41:11.588315 2035 slave.cpp:3314] Got registration for executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from executor(912)@192.99.40.208:37659
> I1214 04:41:11.588631 2046 exec.cpp:237] Executor registered on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
> I1214 04:41:11.588687 2035 slave.cpp:2256] Sending queued task '0' to executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 at executor(912)@192.99.40.208:37659
> I1214 04:41:11.588961 2034 slave.cpp:3749] Handling status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from executor(912)@192.99.40.208:37659
> I1214 04:41:11.589154 2044 status_update_manager.cpp:323] Received status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.589452 2044 slave.cpp:4190] Forwarding the update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to master@192.99.40.208:37659
> I1214 04:41:11.589516 2044 slave.cpp:4100] Sending acknowledgement for status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to executor(912)@192.99.40.208:37659
> I1214 04:41:11.589596 2044 master.cpp:5769] Status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
> I1214 04:41:11.589625 2044 master.cpp:5831] Forwarding status update TASK_RUNNING (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.589680 2044 master.cpp:7867] Updating the state of task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (latest state: TASK_RUNNING, status update state: TASK_RUNNING)
> I1214 04:41:11.589999 2034 master.cpp:4877] Processing ACKNOWLEDGE call c9738162-8aa4-40f7-9be8-ec1d4877a94c for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
> I1214 04:41:11.590206 2045 status_update_manager.cpp:395] Received status update acknowledgement (UUID: c9738162-8aa4-40f7-9be8-ec1d4877a94c) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.591419 2043 slave.cpp:3749] Handling status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from executor(912)@192.99.40.208:37659
> I1214 04:41:11.591619 2043 status_update_manager.cpp:323] Received status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.591711 2043 slave.cpp:4190] Forwarding the update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to master@192.99.40.208:37659
> I1214 04:41:11.591760 2043 slave.cpp:4100] Sending acknowledgement for status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to executor(912)@192.99.40.208:37659
> I1214 04:41:11.592257 2043 slave.cpp:924] New master detected at master@192.99.40.208:37659
> I1214 04:41:11.592280 2043 slave.cpp:983] Authenticating with master master@192.99.40.208:37659
> I1214 04:41:11.592298 2043 slave.cpp:994] Using default CRAM-MD5 authenticatee
> I1214 04:41:11.592313 2039 status_update_manager.cpp:177] Pausing sending status updates
> I1214 04:41:11.592341 2043 slave.cpp:956] Detecting new master
> I1214 04:41:11.592363 2039 authenticatee.cpp:121] Creating new client SASL connection
> I1214 04:41:11.592383 2043 slave.cpp:4318] Got exited event for executor(912)@192.99.40.208:37659
> I1214 04:41:11.592438 2039 master.cpp:6748] Authenticating slave(912)@192.99.40.208:37659
> I1214 04:41:11.592550 2039 authenticator.cpp:98] Creating new server SASL connection
> I1214 04:41:11.592614 2039 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
> I1214 04:41:11.592635 2039 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
> I1214 04:41:11.592672 2039 authenticator.cpp:204] Received SASL authentication start
> I1214 04:41:11.592707 2039 authenticator.cpp:326] Authentication requires more steps
> I1214 04:41:11.592741 2039 authenticatee.cpp:259] Received SASL authentication step
> I1214 04:41:11.592779 2039 authenticator.cpp:232] Received SASL authentication step
> I1214 04:41:11.592819 2039 authenticator.cpp:318] Authentication success
> I1214 04:41:11.592850 2039 authenticatee.cpp:299] Authentication success
> I1214 04:41:11.592870 2039 master.cpp:6778] Successfully authenticated principal 'test-principal' at slave(912)@192.99.40.208:37659
> I1214 04:41:11.592921 2039 slave.cpp:1078] Successfully authenticated with master master@192.99.40.208:37659
> I1214 04:41:11.593102 2039 master.cpp:5311] Re-registering agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
> I1214 04:41:11.593189 2039 master.cpp:5572] Sending updated checkpointed resources {} to agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
> I1214 04:41:11.593250 2039 slave.cpp:1226] Re-registered with master master@192.99.40.208:37659
> I1214 04:41:11.593283 2039 slave.cpp:1262] Forwarding total oversubscribed resources {}
> I1214 04:41:11.593329 2039 status_update_manager.cpp:184] Resuming sending status updates
> W1214 04:41:11.593340 2039 status_update_manager.cpp:191] Resending status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.593410 2039 master.cpp:5633] Received update of agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com) with total oversubscribed resources {}
> I1214 04:41:11.593451 2039 slave.cpp:4190] Forwarding the update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 to master@192.99.40.208:37659
> I1214 04:41:11.593497 2039 hierarchical.cpp:560] Agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 (gru1.hw.ca1.mesosphere.com) updated with oversubscribed resources {} (total: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000], allocated: cpus(*):1; mem(*):512)
> I1214 04:41:11.593605 2039 master.cpp:5769] Status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 from agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
> I1214 04:41:11.593619 2039 master.cpp:5831] Forwarding status update TASK_FINISHED (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.593657 2039 master.cpp:7867] Updating the state of task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (latest state: TASK_FINISHED, status update state: TASK_FINISHED)
> I1214 04:41:11.593978 2041 master.cpp:4877] Processing ACKNOWLEDGE call 79f94437-6066-412e-9de6-6d9a1f256539 for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0
> I1214 04:41:11.594013 2041 master.cpp:7963] Removing task 0 with resources cpus(*):1; mem(*):512 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
> I1214 04:41:11.594048 2005 sched.cpp:2008] Asked to stop the driver
> I1214 04:41:11.594158 2048 sched.cpp:1193] Stopping framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.594192 2041 status_update_manager.cpp:395] Received status update acknowledgement (UUID: 79f94437-6066-412e-9de6-6d9a1f256539) for task 0 of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.594251 2048 master.cpp:7287] Processing TEARDOWN call for framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
> I1214 04:41:11.594272 2048 master.cpp:7299] Removing framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 (default) at scheduler-fa6ed654-9991-4bce-ab02-95d8530f96b3@192.99.40.208:37659
> I1214 04:41:11.594336 2048 master.cpp:7992] Removing executor 'default' with resources {} of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
> I1214 04:41:11.594383 2041 hierarchical.cpp:391] Deactivated framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.594450 2041 hierarchical.cpp:342] Removed framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.594570 2042 slave.cpp:2609] Shutting down framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> I1214 04:41:11.594593 2042 slave.cpp:4999] Shutting down executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 at executor(912)@192.99.40.208:37659
> I1214 04:41:11.594758 2040 slave.cpp:4681] Executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 exited with status 0
> I1214 04:41:11.594861 2040 slave.cpp:4785] Cleaning up executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 at executor(912)@192.99.40.208:37659
> W1214 04:41:11.594920 2041 master.cpp:5887] Ignoring unknown exited executor 'default' of framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000 on agent 87dd8179-dd7d-4270-ace2-ea771b57371c-S0 at slave(912)@192.99.40.208:37659 (gru1.hw.ca1.mesosphere.com)
> ../../src/tests/master_slave_reconciliation_tests.cpp:145: Failure
> Actual function call count doesn't match EXPECT_CALL(sched, executorLost(&driver, createExecutorInfo("default", "exit 1").executor_id(), _, _))...
> Expected: to be called once
> Actual: never called - unsatisfied and active
> *** Aborted at 1481708471 (unix time) try "date -d @1481708471" if you are using GNU date ***
> I1214 04:41:11.595078 2040 slave.cpp:4873] Cleaning up framework 87dd8179-dd7d-4270-ace2-ea771b57371c-0000
> PC: @ 0x1d5c445 testing::UnitTest::AddTestPartResult()
> *** SIGSEGV (@0x0) received by PID 2005 (TID 0x7f04125ff8c0) from PID 0; stack trace: ***
> @ 0x7f040d5c95c0 (unknown)
> @ 0x1d5c445 testing::UnitTest::AddTestPartResult()
> @ 0x1d5c0af testing::internal::AssertHelper::operator=()
> @ 0x1d8b61f testing::internal::GoogleTestFailureReporter::ReportFailure()
> @ 0x1d80511 testing::internal::UntypedFunctionMockerBase::VerifyAndClearExpectationsLocked()
> @ 0x1136031 testing::internal::FunctionMockerBase<>::~FunctionMockerBase()
> @ 0x112e023 mesos::internal::tests::MockScheduler::~MockScheduler()
> @ 0xf66780 mesos::internal::tests::MasterSlaveReconciliationTest_SlaveReregisterTerminatedExecutor_Test::TestBody()
> @ 0x1d84a46 testing::internal::HandleExceptionsInMethodIfSupported<>()
> @ 0x1d6450d testing::Test::Run()
> @ 0x1d65520 testing::TestInfo::Run()
> @ 0x1d65c23 testing::TestCase::Run()
> @ 0x1d6d8b9 testing::internal::UnitTestImpl::RunAllTests()
> @ 0x1d8539f testing::internal::HandleExceptionsInMethodIfSupported<>()
> @ 0x1d6d55f testing::UnitTest::Run()
> @ 0xd5d8bb main
> @ 0x7f040c348401 __libc_start_main
> @ 0x51a81a _start
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)